CN117591835B - Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model - Google Patents

Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model Download PDF

Info

Publication number
CN117591835B
CN117591835B CN202410073112.3A CN202410073112A CN117591835B CN 117591835 B CN117591835 B CN 117591835B CN 202410073112 A CN202410073112 A CN 202410073112A CN 117591835 B CN117591835 B CN 117591835B
Authority
CN
China
Prior art keywords
model
data
layer
lstm
covariance matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410073112.3A
Other languages
Chinese (zh)
Other versions
CN117591835A (en
Inventor
谢雨晨
翁富忠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Information Science and Technology
Original Assignee
Nanjing University of Information Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Information Science and Technology filed Critical Nanjing University of Information Science and Technology
Priority to CN202410073112.3A priority Critical patent/CN117591835B/en
Publication of CN117591835A publication Critical patent/CN117591835A/en
Application granted granted Critical
Publication of CN117591835B publication Critical patent/CN117591835B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Analysis (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Computational Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Algebra (AREA)
  • Probability & Statistics with Applications (AREA)
  • Operations Research (AREA)
  • Complex Calculations (AREA)

Abstract

The invention discloses an atmospheric temperature profile generating method based on a covariance matrix and a DCN-LSTM model, which comprises the following steps: (1) Acquiring ATMS remote sensing data and ERA5 re-analysis data of corresponding date; (2) Performing space-time matching processing on ATMS and ERA5 data, and performing normalization processing; (3) Counting the average value of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among all air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B; (4) Constructing a data set, and dividing the data set into a test sample and a training sample; (5) Constructing a DCN-LSTM network model introducing a covariance matrix; (6) verifying the model accuracy by testing the sample; the invention can effectively solve the technical problem of inversion of the profile data under the condition of sparse background field information, and improve the timeliness and accuracy of inversion of the atmospheric profile.

Description

Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model
Technical Field
The invention belongs to the field of atmospheric parameter inversion, and particularly relates to an atmospheric temperature profile generating method based on a covariance matrix and a DCN-LSTM model.
Background
The temperature profile is an important parameter reflecting the vertical state of the atmosphere, and the accurate acquisition of the atmospheric temperature has important significance on climate change, environmental monitoring, numerical weather forecast and the like. At present, researchers mainly generate an atmospheric temperature profile through data inversion of a hyperspectral infrared instrument and a microwave instrument. The hyperspectral instrument has higher spectral resolution in the infrared band, but the penetration capability of the microwave instrument enables the hyperspectral instrument to have wider weight distribution. The atmospheric temperature profile product inverted by the microwave instrument can be used for analyzing thermodynamic structures of disaster weather such as typhoons and the like. The existing atmospheric temperature profile inversion model depends on a physical model, such as a variational inversion model, needs auxiliary external data and consumes a large amount of machine time. The variance inversion algorithm is based on a background covariance matrix or a background error covariance matrix, and control information is transmitted in a vertical direction or a horizontal direction. In addition, the machine learning and depth method such as ANN, DNN, LSTM and the like which rely on statistics also have wider application in the inversion of the atmospheric temperature profile. However, the statistical algorithm has poor fitting effect in areas with less observation information (such as the high-level areas of the atmosphere). The method has the advantages that the method is used for researching how to add physical constraint on a statistical model under the condition that the method only depends on ATMS microwave instrument data, extracting the vertical characteristics of the atmospheric temperature from massive remote sensing data, realizing the high-precision inversion of the atmospheric temperature profile, and having certain important theoretical and technical significance.
Disclosure of Invention
The invention aims to: the invention aims to provide an atmospheric temperature profile generating method based on a covariance matrix and a DCN-LSTM model, which can efficiently generate an atmospheric temperature profile product without additional auxiliary data and can extract the atmospheric vertical thermodynamic structures of various weather phenomena.
The technical scheme is as follows: the invention discloses an atmospheric temperature profile generating method based on a covariance matrix and a DCN-LSTM model, which comprises the following steps:
the atmosphere temperature profile generating method based on the covariance matrix and the DCN-LSTM model is characterized by comprising the following steps of:
(1) Acquiring ATMS remote sensing data and ERA5 re-analysis data of corresponding date, and preprocessing the ATMS data;
(2) Performing space-time matching treatment on ATMS and ERA5 data, and counting the maximum value and the minimum value of each parameter, and performing normalization treatment;
(3) Counting the average value of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among all air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
(4) Taking normalized ERA5 temperature profile data as a true value, taking normalized ATMS brightness temperature, angle and geographic positioning information as input, constructing a data set, and taking the data set as 1:9, dividing the ratio into a test sample and a training sample;
(5) Constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by calculating a brand new loss function and utilizing a back propagation algorithm;
(6) Verifying the model accuracy through a test sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
Further, in the step 1, the ERA5 re-analysis data includes: temperature information at 37 layers of atmospheric altitude between 1000hpa and 1 hpa; ATMS data preprocessing includes: and (3) scaling processing of bright temperature observation, extracting observation angle information including satellite zenith angle, satellite azimuth angle, solar zenith angle and solar azimuth angle, and extracting geographic positioning information including longitude and latitude.
Further, in the step 2, normalization processing is performed on the statistical maximum value and the statistical minimum value of each parameter; the formula is as follows:
；
Wherein C represents a parameter in the dataset, And/>Respectively statistically obtained maximum and minimum,/>Is a normalized parameter.
Further, in the step 3, the covariance matrix B is a symmetric matrix with 37×37 dimensions, and the diagonal line is 1.
Further, in the step 4, after the training samples are obtained, 17280 samples are taken as a group and input into the network model for training, i.e. batch_size=17280.
Further, the step 5 includes the following steps:
(51) The first layer of the DCN-LSTM model is an input layer, and the input dimension is 17280 multiplied by 28; the model body can be divided into an encoder structure and a decoder structure, wherein a DNN model is used as an encoder, and an LSTM model is used as a decoder;
(52) The DNN model comprises 3 hidden layers, the input dimension of the hidden layer of the first layer is 17280×28, the post-hidden layer connection LeakyReLU activates the function and Dropout function, and the output data dimension is 17280×64; the following 2 hidden layers each take the output of the previous layer as input and each contain LeakyReLU activation functions and Dropout functions; the output dimension of the second layer is 17280×128, and the output dimension of the third layer is 17280×256; after DNN characteristic extraction, the data size is 17280 multiplied by 256;
(53) The DNN encoder and LSTM decoder are connected by a 1D CNN layer with input dimensions 17280 x 256 and output dimensions 17280 x 512, and the layer post-connection LeakyReLU activates the function. The convolution kernel Size of the 1D CNN layer is set to 3, the edge Padding Size (Padding Size) is set to 1, and the convolution kernel step Size is set to 1;
(54) The LSTM module acts as a decoder of the model, and also contains 3 hidden layers, each layer containing 512 LSTM neurons. After the data is decoded by LSTM, the dimension is unchanged;
(55) Finally, the model takes a full connection layer as an output layer, adjusts the output dimension to 17280 multiplied by 37, and takes a ReLU as an activation function to obtain a simulated atmospheric temperature profile;
(56) The model loss function is designed, and the model loss function comprises two parts, and the formula is as follows:
；
wherein,
；
；
Wherein w is a scaling factor, controlAnd/>Of the order of magnitude of (3); /(I)Representing MAE error, y represents ERA5 temperature data,/>Representing the generated value of the DCN-LSTM model; /(I)Representing the covariance matrix of y,Representative/>Is a covariance matrix of (a). /(I)The calculation should not be performed on a certain number of samples, i.e. the Batch Size when training the model is too small.
The invention relates to an atmospheric temperature profile generating system based on a covariance matrix and a DCN-LSTM model, which comprises the following components:
and a data acquisition module: the ERA5 re-analysis data is used for acquiring ATMS remote sensing data and corresponding dates, and preprocessing the ATMS data;
And a matching module: the method is used for carrying out space-time matching processing on ATMS and ERA5 data, counting the maximum value and the minimum value of each parameter and carrying out normalization processing;
Covariance matrix module: the method comprises the steps of counting average values of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
And (3) constructing a data set module: the method is used for constructing a data set by taking normalized ERA5 temperature profile data as a true value and normalized ATMS brightness temperature, angle and geographic positioning information as input, and the data set is calculated according to 1:9, dividing the ratio into a test sample and a training sample;
DCN-LSTM module: the method comprises the steps of constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by using a back propagation algorithm through calculating a brand new loss function;
and (3) a tuning module: for verifying model accuracy by testing the sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
An apparatus according to the present invention comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program implementing any one of the atmospheric temperature profile generation systems based on covariance matrix and DCN-LSTM model when loaded into the processor.
A storage medium according to the present invention stores a computer program which, when executed by a processor, implements an atmospheric temperature profile generation system based on a covariance matrix and a DCN-LSTM model according to any one of the above.
The beneficial effects are that: compared with the prior art, the invention has the following remarkable advantages: the method can generate the atmospheric temperature profile with high aging and high precision under the condition of no need of auxiliary data (such as ground surface temperature, ground surface emissivity, humidity profile and other background information), and the DNN-LSTM model introduced with the covariance matrix can be better fitted under the same configuration and data to obtain more accurate results; the technical problem of inversion of the profile data under the condition of no background field information can be effectively solved, and the timeliness and accuracy of inversion of the atmospheric profile are improved.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
FIG. 2 is a schematic diagram of a DCN-LSTM network model of the present invention.
Description of the embodiments
The technical scheme of the invention is further described below with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present invention provides a method for generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model, including the following steps:
(1) Acquiring ATMS remote sensing data and ERA5 re-analysis data of corresponding date, and preprocessing the ATMS data; ERA5 analysis data includes: temperature information at 37 layers of atmospheric altitude between 1000hpa and 1 hpa; ATMS data preprocessing includes: and (3) scaling processing of bright temperature observation, extracting observation angle information including satellite zenith angle, satellite azimuth angle, solar zenith angle and solar azimuth angle, and extracting geographic positioning information including longitude and latitude.
(2) Performing space-time matching treatment on ATMS and ERA5 data, and counting the maximum value and the minimum value of each parameter, and performing normalization treatment; the formula is as follows:
；
Wherein C represents a parameter in the dataset, And/>Respectively statistically obtained maximum and minimum,/>Is a normalized parameter.
(3) And counting the average value of the ERA5 temperature profile data to obtain an average value data set, and calculating covariance among all air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B. The covariance matrix B is a symmetric matrix of 37×37 dimensions, and the diagonal is 1.
(4) Taking normalized ERA5 temperature profile data as a true value, taking normalized ATMS brightness temperature, angle and geographic positioning information as input, constructing a data set, and taking the data set as 1:9, dividing the ratio into a test sample and a training sample; after the training samples are obtained, 17280 samples are input into the network model for training, i.e., batch_size=17280.
(5) As shown in fig. 2, constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by using a back propagation algorithm through calculating a brand new loss function; the method comprises the following steps:
(51) The first layer of the DCN-LSTM model is an input layer, and the input dimension is 17280 multiplied by 28; the model body can be divided into an encoder structure and a decoder structure, with the DNN model as the encoder and the LSTM model as the decoder.
(52) The DNN model comprises 3 hidden layers, the input dimension of the hidden layer of the first layer is 17280×28, the post-hidden layer connection LeakyReLU activates the function and Dropout function, and the output data dimension is 17280×64; the following 2 hidden layers each take the output of the previous layer as input and each contain LeakyReLU activate functions and Dropout functions. The output dimension of the second layer is 17280×128 and the output dimension of the third layer is 17280×256. After DNN feature extraction, the data size was 17280×256.
(53) The DNN encoder and LSTM decoder are connected by a 1D CNN layer with input dimensions 17280 x 256 and output dimensions 17280 x 512, and the layer post-connection LeakyReLU activates the function. The convolution kernel Size of the 1D CNN layer is set to 3, the edge Padding Size (Padding Size) is set to 1, and the convolution kernel step Size is set to 1.
(54) The LSTM module acts as a decoder of the model, and also contains 3 hidden layers, each layer containing 512 LSTM neurons. After LSTM decoding, the data has unchanged dimension.
(55) Finally, the model takes a full connection layer as an output layer, adjusts the output dimension to 17280 multiplied by 37, and takes the ReLU as an activation function to obtain a simulated atmospheric temperature profile.
(56) The model loss function is designed, and the model loss function comprises two parts, and the formula is as follows:
；
wherein,
；
；
Wherein w is a scaling factor, controlAnd/>Of the order of magnitude of (3); /(I)Representing MAE error, y represents ERA5 temperature data,/>Representing the generated value of the DCN-LSTM model; /(I)Representing the covariance matrix of y,Representative/>Is a covariance matrix of (a). /(I)The calculation should not be performed on a certain number of samples, i.e. the Batch Size when training the model is too small.
(6) Verifying the model accuracy through a test sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
Specific data:
Step 1: ATMS remote sensing data and ERA5 analysis data acquisition. Downloading SDR data of days 1 and 15 of 12 months of 2022 from NOAA CLASS officially; ERA5 layer hour by hour atmospheric temperature data of the same date was obtained through CDS API. The ATMS and ERA5 data are decoded.
Step 2: and (5) data space-time matching and normalization. And simultaneously reading ATMS data and ERA5 data at corresponding moments, and performing space matching by using a nearest neighbor method. And (5) counting the maximum value and the minimum value of all parameters, and normalizing.
Step 3: a background covariance matrix B is calculated.
Step 4: and constructing a data set and a DCN-LSTM atmospheric temperature profile generating model. And taking ATMS observation information, angle information and geographic positioning information as x and ERA5 temperature profile information as y to construct a data set. 1, the method comprises the following steps: the scale of 9 randomly divides the dataset into test and training samples. And constructing a DCN-LSTM atmospheric temperature profile generating model based on the pytorch framework.
Step 5: model training and tuning. The model was trained in the GPU environment while 4 GPUs were invoked for distributed training, adam optimizers were used for error back propagation and optimization of model parameters, learning rate was set to 0.0001, β 1 was set to 0.9, β 2 was set to 0.999, training period was 200 total, and after 100 periods, learning rate was attenuated. During model training, the model is tested once every 5 periods based on a test sample, and if the precision on a test set meets the requirement, training is skipped in advance; if the accuracy still does not meet the requirement after the model is trained for 200 periods, the learning parameters are adjusted for retraining.
Step 6: and outputting and applying results. And carrying out inverse normalization on the model output result to obtain an atmospheric temperature profile product with physical significance.
The embodiment of the invention also provides an atmospheric temperature profile generating system based on the covariance matrix and the DCN-LSTM model, which comprises the following steps:
and a data acquisition module: the ERA5 re-analysis data is used for acquiring ATMS remote sensing data and corresponding dates, and preprocessing the ATMS data;
And a matching module: the method is used for carrying out space-time matching processing on ATMS and ERA5 data, counting the maximum value and the minimum value of each parameter and carrying out normalization processing;
Covariance matrix module: the method comprises the steps of counting average values of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
And (3) constructing a data set module: the method is used for constructing a data set by taking normalized ERA5 temperature profile data as a true value and normalized ATMS brightness temperature, angle and geographic positioning information as input, and the data set is calculated according to 1:9, dividing the ratio into a test sample and a training sample;
DCN-LSTM module: the method comprises the steps of constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by using a back propagation algorithm through calculating a brand new loss function;
and (3) a tuning module: for verifying model accuracy by testing the sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
The embodiment of the invention also provides equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the atmospheric temperature profile generating system based on the covariance matrix and the DCN-LSTM model when being loaded to the processor.
The embodiment of the invention also provides a storage medium, which stores a computer program, and the computer program realizes the atmospheric temperature profile generating system based on the covariance matrix and the DCN-LSTM model when being executed by a processor.

Claims (8)

1. The atmosphere temperature profile generating method based on the covariance matrix and the DCN-LSTM model is characterized by comprising the following steps of:
(1) Acquiring ATMS remote sensing data and ERA5 re-analysis data of corresponding date, and preprocessing the ATMS data;
(2) Performing space-time matching treatment on ATMS and ERA5 data, and counting the maximum value and the minimum value of each parameter, and performing normalization treatment;
(3) Counting the average value of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among all air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
(4) Taking normalized ERA5 temperature profile data as a true value, taking normalized ATMS brightness temperature, angle and geographic positioning information as input, constructing a data set, and taking the data set as M: the proportion of N is divided into a test sample and a training sample;
(5) Constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by calculating a brand new loss function and utilizing a back propagation algorithm; the method comprises the following steps:
(51) The first layer of the DCN-LSTM model is an input layer, and the input dimension is 17280 multiplied by 28; the model main body is divided into an encoder structure and a decoder structure, a DNN model is used as an encoder, and an LSTM model is used as a decoder;
(52) The DNN model comprises 3 hidden layers, the input dimension of the hidden layer of the first layer is 17280×28, the post-hidden layer connection LeakyReLU activates the function and Dropout function, and the output data dimension is 17280×64; the following 2 hidden layers each take the output of the previous layer as input and each contain LeakyReLU activation functions and Dropout functions; the output dimension of the second layer is 17280×128, and the output dimension of the third layer is 17280×256; after DNN characteristic extraction, the data size is 17280 multiplied by 256;
(53) Connecting the DNN encoder and the LSTM decoder by using a 1D CNN layer, wherein the input dimension of the 1D CNN layer is 17280 multiplied by 256, the output dimension is 17280 multiplied by 512, and the post-layer connection LeakyReLU activates the function; the convolution kernel Size of the 1D CNN layer is set to 3, the edge Padding Size (Padding Size) is set to 1, and the convolution kernel step Size is set to 1;
(54) The LSTM module is used as a decoder of the model, and also comprises 3 hidden layers, wherein each layer comprises 512 LSTM neurons; after the data is decoded by LSTM, the dimension is unchanged;
(55) Finally, the model takes a full connection layer as an output layer, adjusts the output dimension to 17280 multiplied by 37, and takes a ReLU as an activation function to obtain a simulated atmospheric temperature profile;
(56) The model loss function is designed, and the model loss function comprises two parts, and the formula is as follows:
；
wherein,
；
；
Wherein w is a scaling factor, controlAnd/>Of the order of magnitude of (3); /(I)Representing MAE error, y represents ERA5 temperature data,/>Representing the generated value of the DCN-LSTM model; /(I)Representing the covariance matrix of y,Representative/>Is a covariance matrix of (a);
(6) Verifying the model accuracy through a test sample; and (3) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
2. The method of generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model according to claim 1, wherein in the step (1), ERA5 re-analysis data comprises: temperature information at 37 layers of atmospheric altitude between 1000hpa and 1 hpa; ATMS data preprocessing includes: and (3) scaling processing of bright temperature observation, extracting observation angle information including satellite zenith angle, satellite azimuth angle, solar zenith angle and solar azimuth angle, and extracting geographic positioning information including longitude and latitude.
3. The method for generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model according to claim 1, wherein in the step (2), normalization processing is performed on a statistical maximum value and a statistical minimum value of each parameter; the formula is as follows:
；
Wherein C represents a parameter in the dataset, And/>Respectively obtaining a maximum value and a minimum value by statistics; /(I)Is a normalized parameter.
4. The method of generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model according to claim 1, wherein in the step (3), the covariance matrix B is a symmetric matrix with 37×37 dimensions, and the diagonal is 1.
5. The method of claim 1, wherein in the step (4), after obtaining training samples, the training samples are input into the network model as a group of 17280 samples, i.e. batch_size=17280.
6. An atmospheric temperature profile generation system based on a covariance matrix and a DCN-LSTM model, comprising:
and a data acquisition module: the ERA5 re-analysis data is used for acquiring ATMS remote sensing data and corresponding dates, and preprocessing the ATMS data;
And a matching module: the method is used for carrying out space-time matching processing on ATMS and ERA5 data, counting the maximum value and the minimum value of each parameter and carrying out normalization processing;
Covariance matrix module: the method comprises the steps of counting average values of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
And (3) constructing a data set module: the method is used for constructing a data set by taking normalized ERA5 temperature profile data as a true value and normalized ATMS brightness temperature, angle and geographic positioning information as input, and the data set is calculated according to 1:9, dividing the ratio into a test sample and a training sample;
DCN-LSTM module: the method comprises the steps of constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by using a back propagation algorithm through calculating a brand new loss function; the method comprises the following steps:
(51) The first layer of the DCN-LSTM model is an input layer, and the input dimension is 17280 multiplied by 28; the model main body is divided into an encoder structure and a decoder structure, a DNN model is used as an encoder, and an LSTM model is used as a decoder;
(52) The DNN model comprises 3 hidden layers, the input dimension of the hidden layer of the first layer is 17280×28, the post-hidden layer connection LeakyReLU activates the function and Dropout function, and the output data dimension is 17280×64; the following 2 hidden layers each take the output of the previous layer as input and each contain LeakyReLU activation functions and Dropout functions; the output dimension of the second layer is 17280×128, and the output dimension of the third layer is 17280×256; after DNN characteristic extraction, the data size is 17280 multiplied by 256;
(53) Connecting the DNN encoder and the LSTM decoder by using a 1D CNN layer, wherein the input dimension of the 1D CNN layer is 17280 multiplied by 256, the output dimension is 17280 multiplied by 512, and the post-layer connection LeakyReLU activates the function; the convolution kernel Size of the 1D CNN layer is set to 3, the edge Padding Size (Padding Size) is set to 1, and the convolution kernel step Size is set to 1;
(54) The LSTM module is used as a decoder of the model, and also comprises 3 hidden layers, wherein each layer comprises 512 LSTM neurons; after the data is decoded by LSTM, the dimension is unchanged;
(55) Finally, the model takes a full connection layer as an output layer, adjusts the output dimension to 17280 multiplied by 37, and takes a ReLU as an activation function to obtain a simulated atmospheric temperature profile;
(56) The model loss function is designed, and the model loss function comprises two parts, and the formula is as follows:
；
wherein,
；
；
Wherein w is a scaling factor, controlAnd/>Of the order of magnitude of (3); /(I)Representing MAE error, y represents ERA5 temperature data,/>Representing the generated value of the DCN-LSTM model; /(I)Representing the covariance matrix of y,Representative/>Is a covariance matrix of (a);
And (3) a tuning module: for verifying model accuracy by testing the sample; if the preset precision is not met, returning to the DCN-LSTM module for re-tuning, and if the preset precision is met, the trained model is the target model.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the computer program when loaded to the processor implements a method for generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model according to any one of claims 1-6.
8. A computer storage medium storing a computer program, characterized in that the computer program when executed by a processor implements a method for generating an atmospheric temperature profile based on a covariance matrix and DCN-LSTM model according to any one of claims 1-6.
CN202410073112.3A 2024-01-18 2024-01-18 Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model Active CN117591835B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410073112.3A CN117591835B (en) 2024-01-18 2024-01-18 Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410073112.3A CN117591835B (en) 2024-01-18 2024-01-18 Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model

Publications (2)

Publication Number Publication Date
CN117591835A CN117591835A (en) 2024-02-23
CN117591835B true CN117591835B (en) 2024-04-19

Family

ID=89910240

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410073112.3A Active CN117591835B (en) 2024-01-18 2024-01-18 Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model

Country Status (1)

Country Link
CN (1) CN117591835B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111339665A (en) * 2020-02-27 2020-06-26 中国科学院空天信息创新研究院 Troposphere ozone profile calculation method
CN115859789A (en) * 2022-11-23 2023-03-28 北京信息科技大学 Method for improving inversion accuracy of polar atmosphere temperature profile
CN116796291A (en) * 2023-04-26 2023-09-22 安徽大学 LSTM-MEA-SVR-based air quality forecasting system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10871594B2 (en) * 2019-04-30 2020-12-22 ClimateAI, Inc. Methods and systems for climate forecasting using artificial neural networks
US10909446B2 (en) * 2019-05-09 2021-02-02 ClimateAI, Inc. Systems and methods for selecting global climate simulation models for training neural network climate forecasting models

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111339665A (en) * 2020-02-27 2020-06-26 中国科学院空天信息创新研究院 Troposphere ozone profile calculation method
CN115859789A (en) * 2022-11-23 2023-03-28 北京信息科技大学 Method for improving inversion accuracy of polar atmosphere temperature profile
CN116796291A (en) * 2023-04-26 2023-09-22 安徽大学 LSTM-MEA-SVR-based air quality forecasting system

Also Published As

Publication number Publication date
CN117591835A (en) 2024-02-23

Similar Documents

Publication Publication Date Title
Wargan et al. The global structure of upper troposphere‐lower stratosphere ozone in GEOS‐5: A multiyear assimilation of EOS Aura data
Kotsuki et al. Assimilating the global satellite mapping of precipitation data with the Nonhydrostatic Icosahedral Atmospheric Model (NICAM)
Järvinen et al. Variational assimilation of time sequences of surface observations with serially correlated errors
Liu et al. A new statistical downscaling model for autumn precipitation in China
CN112052627A (en) Method, device, medium and equipment for estimating near-surface ozone space distribution
Ferrara et al. Large‐scale control of the lower stratosphere on variability of tropical cyclone intensity
Xian et al. All‐sky assimilation of the MWHS‐2 observations and evaluation the impacts on the analyses and forecasts of binary typhoons
Kotsuki et al. Online model parameter estimation with ensemble data assimilation in the real global atmosphere: A case with the nonhydrostatic icosahedral atmospheric model (NICAM) and the global satellite mapping of precipitation data
Vogel et al. Uncertainty in aerosol optical depth from modern aerosol‐climate models, reanalyses, and satellite products
Shastri et al. Improving Global Forecast System of extreme precipitation events with regional statistical model: Application of quantile‐based probabilistic forecasts
Privé et al. The role of model and initial condition error in numerical weather forecasting investigated with an observing system simulation experiment
Lee et al. Improvements of 6S look-up-table based surface reflectance employing minimum curvature surface method
Hoffman et al. Progress in forecast skill at three leading global operational NWP centers during 2015–17 as seen in summary assessment metrics (SAMs)
Koh et al. Subgrid‐scale cloud–radiation feedback for the B etts–M iller–J anjić convection scheme
Wang et al. Customized deep learning for precipitation bias correction and downscaling
Terasaki et al. Multi-year analysis using the NICAM-LETKF data assimilation system
CN114330110A (en) Model training method, photovoltaic power generation power prediction method and system
Chi et al. Cloud macrophysical characteristics in China mainland and east coast from 2006 to 2017 using satellite active remote sensing observations
CN117591835B (en) Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model
Li et al. α‐Chapman scale height: Longitudinal variation and global modeling
Tim et al. Oceanic influence on the precipitation in Venezuela under current and future climate
Ding et al. A feasible approach to improve forecast skill of summer precipitation in northeast China by statistical regression of the northeast China cold vortex in the multimodel ensemble
CN112926625A (en) Method for analyzing deviation influence factors of satellite radiation data
Zhu et al. A 4DEnVar‐Based Ensemble Four‐Dimensional Variational (En4DVar) Hybrid Data Assimilation System for Global NWP: System Description and Primary Tests
Kotsuki et al. Weight structure of the local ensemble transform Kalman filter: A case with an intermediate atmospheric general circulation model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant