CN117591835A - Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model - Google Patents

Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model Download PDF

Info

Publication number
CN117591835A
CN117591835A CN202410073112.3A CN202410073112A CN117591835A CN 117591835 A CN117591835 A CN 117591835A CN 202410073112 A CN202410073112 A CN 202410073112A CN 117591835 A CN117591835 A CN 117591835A
Authority
CN
China
Prior art keywords
model
data
covariance matrix
lstm
dcn
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410073112.3A
Other languages
Chinese (zh)
Other versions
CN117591835B (en
Inventor
谢雨晨
翁富忠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Information Science and Technology
Original Assignee
Nanjing University of Information Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Information Science and Technology filed Critical Nanjing University of Information Science and Technology
Priority to CN202410073112.3A priority Critical patent/CN117591835B/en
Publication of CN117591835A publication Critical patent/CN117591835A/en
Application granted granted Critical
Publication of CN117591835B publication Critical patent/CN117591835B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The invention discloses an atmospheric temperature profile generating method based on a covariance matrix and a DCN-LSTM model, which comprises the following steps: (1) Acquiring ATMS remote sensing data and ERA5 re-analysis data of corresponding date; (2) Performing space-time matching processing on ATMS and ERA5 data, and performing normalization processing; (3) Counting the average value of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among all air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B; (4) Constructing a data set, and dividing the data set into a test sample and a training sample; (5) Constructing a DCN-LSTM network model introducing a covariance matrix; (6) verifying the model accuracy by testing the sample; the invention can effectively solve the technical problem of inversion of the profile data under the condition of sparse background field information, and improve the timeliness and accuracy of inversion of the atmospheric profile.

Description

Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model
Technical Field
The invention belongs to the field of atmospheric parameter inversion, and particularly relates to an atmospheric temperature profile generating method based on a covariance matrix and a DCN-LSTM model.
Background
The temperature profile is an important parameter reflecting the vertical state of the atmosphere, and the accurate acquisition of the atmospheric temperature has important significance on climate change, environmental monitoring, numerical weather forecast and the like. At present, researchers mainly generate an atmospheric temperature profile through data inversion of a hyperspectral infrared instrument and a microwave instrument. The hyperspectral instrument has higher spectral resolution in the infrared band, but the penetration capability of the microwave instrument enables the hyperspectral instrument to have wider weight distribution. The atmospheric temperature profile product inverted by the microwave instrument can be used for analyzing thermodynamic structures of disaster weather such as typhoons and the like. The existing atmospheric temperature profile inversion model depends on a physical model, such as a variational inversion model, needs auxiliary external data and consumes a large amount of machine time. The variance inversion algorithm is based on a background covariance matrix or a background error covariance matrix, and control information is transmitted in a vertical direction or a horizontal direction. In addition, the machine learning and depth method based on statistics, such as ANN, DNN, LSTM, has wide application in the inversion of the atmospheric temperature profile. However, the statistical algorithm has poor fitting effect in areas with less observation information (such as the high-level areas of the atmosphere). The method has the advantages that the method is used for researching how to add physical constraint on a statistical model under the condition that the method only depends on ATMS microwave instrument data, extracting the vertical characteristics of the atmospheric temperature from massive remote sensing data, realizing the high-precision inversion of the atmospheric temperature profile, and having certain important theoretical and technical significance.
Disclosure of Invention
The invention aims to: the invention aims to provide an atmospheric temperature profile generating method based on a covariance matrix and a DCN-LSTM model, which can efficiently generate an atmospheric temperature profile product without additional auxiliary data and can extract the atmospheric vertical thermodynamic structures of various weather phenomena.
The technical scheme is as follows: the invention discloses an atmospheric temperature profile generating method based on a covariance matrix and a DCN-LSTM model, which comprises the following steps:
the atmosphere temperature profile generating method based on the covariance matrix and the DCN-LSTM model is characterized by comprising the following steps of:
(1) Acquiring ATMS remote sensing data and ERA5 re-analysis data of corresponding date, and preprocessing the ATMS data;
(2) Performing space-time matching treatment on ATMS and ERA5 data, and counting the maximum value and the minimum value of each parameter, and performing normalization treatment;
(3) Counting the average value of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among all air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
(4) Taking normalized ERA5 temperature profile data as a true value, taking normalized ATMS brightness temperature, angle and geographic positioning information as input, constructing a data set, and taking the data set as 1:9, dividing the ratio into a test sample and a training sample;
(5) Constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by calculating a brand new loss function and utilizing a back propagation algorithm;
(6) Verifying the model accuracy through a test sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
Further, in the step 1, the ERA5 re-analysis data includes: temperature information at 37 layers of atmospheric altitude between 1000hpa and 1 hpa; ATMS data preprocessing includes: and (3) scaling processing of bright temperature observation, extracting observation angle information including satellite zenith angle, satellite azimuth angle, solar zenith angle and solar azimuth angle, and extracting geographic positioning information including longitude and latitude.
Further, in the step 2, normalization processing is performed on the statistical maximum value and the statistical minimum value of each parameter; the formula is as follows:
；
wherein C represents a parameter in the dataset,and->Respectively statistically obtained maximum and minimum, < ->Is a normalized parameter.
Further, in the step 3, the covariance matrix B is a symmetric matrix with 37×37 dimensions, and the diagonal line is 1.
Further, in the step 4, after the training samples are obtained, 17280 samples are taken as a group and input into the network model for training, i.e. batch_size=17280.
Further, the step 5 includes the following steps:
(51) The first layer of the DCN-LSTM model is an input layer, and the input dimension is 17280 multiplied by 28; the model body can be divided into an encoder structure and a decoder structure, wherein a DNN model is used as an encoder, and an LSTM model is used as a decoder;
(52) The DNN model comprises 3 hidden layers, wherein the input dimension of the hidden layer of the first layer is 17280 multiplied by 28, the hidden layer is connected with a LeakyReLU activation function and a Dropout function, and the output data dimension is 17280 multiplied by 64; the following 2 hidden layers each take the output of the previous layer as input and each contains a LeakyReLU activation function and a Dropout function; the output dimension of the second layer is 17280×128, and the output dimension of the third layer is 17280×256; after DNN characteristic extraction, the data size is 17280 multiplied by 256;
(53) The DNN encoder and LSTM decoder are connected by a 1D CNN layer, the input dimension of the 1D CNN layer is 17280×256, the output dimension is 17280×512, and the layer is connected with the LeakyReLU activation function. The convolution kernel Size of the 1D CNN layer is set to 3, the edge Padding Size (Padding Size) is set to 1, and the convolution kernel step Size is set to 1;
(54) The LSTM module acts as a decoder of the model, and also contains 3 hidden layers, each layer containing 512 LSTM neurons. After the data is decoded by LSTM, the dimension is unchanged;
(55) Finally, the model takes a full connection layer as an output layer, adjusts the output dimension to 17280 multiplied by 37, and takes a ReLU as an activation function to obtain a simulated atmospheric temperature profile;
(56) The model loss function is designed, and the model loss function comprises two parts, and the formula is as follows:
；
wherein,
；
；
wherein w is a scaling factor, controlAnd->Of the order of magnitude of (3); />Representing MAE error, y represents ERA5 temperature data,/->Representing the generated value of the DCN-LSTM model; />Covariance matrix representing y,>representative ofIs a covariance matrix of (a). />The calculation should not be performed on a certain number of samples, i.e. the Batch Size when training the model is too small.
The invention relates to an atmospheric temperature profile generating system based on a covariance matrix and a DCN-LSTM model, which comprises the following components:
and a data acquisition module: the ERA5 re-analysis data is used for acquiring ATMS remote sensing data and corresponding dates, and preprocessing the ATMS data;
and a matching module: the method is used for carrying out space-time matching processing on ATMS and ERA5 data, counting the maximum value and the minimum value of each parameter and carrying out normalization processing;
covariance matrix module: the method comprises the steps of counting average values of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
and (3) constructing a data set module: the method is used for constructing a data set by taking normalized ERA5 temperature profile data as a true value and normalized ATMS brightness temperature, angle and geographic positioning information as input, and the data set is calculated according to 1:9, dividing the ratio into a test sample and a training sample;
DCN-LSTM module: the method comprises the steps of constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by using a back propagation algorithm through calculating a brand new loss function;
and (3) a tuning module: for verifying model accuracy by testing the sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
An apparatus according to the present invention comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program implementing any one of the atmospheric temperature profile generation systems based on covariance matrix and DCN-LSTM model when loaded into the processor.
A storage medium according to the present invention stores a computer program which, when executed by a processor, implements an atmospheric temperature profile generation system based on a covariance matrix and a DCN-LSTM model according to any one of the above.
The beneficial effects are that: compared with the prior art, the invention has the following remarkable advantages: the method can generate the atmospheric temperature profile with high aging and high precision under the condition of no need of auxiliary data (such as ground surface temperature, ground surface emissivity, humidity profile and other background information), and the DNN-LSTM model introduced with the covariance matrix can be better fitted under the same configuration and data to obtain more accurate results; the technical problem of inversion of the profile data under the condition of no background field information can be effectively solved, and the timeliness and accuracy of inversion of the atmospheric profile are improved.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
FIG. 2 is a schematic diagram of a DCN-LSTM network model of the present invention.
Description of the embodiments
The technical scheme of the invention is further described below with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present invention provides a method for generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model, including the following steps:
(1) Acquiring ATMS remote sensing data and ERA5 re-analysis data of corresponding date, and preprocessing the ATMS data; ERA5 analysis data includes: temperature information at 37 layers of atmospheric altitude between 1000hpa and 1 hpa; ATMS data preprocessing includes: and (3) scaling processing of bright temperature observation, extracting observation angle information including satellite zenith angle, satellite azimuth angle, solar zenith angle and solar azimuth angle, and extracting geographic positioning information including longitude and latitude.
(2) Performing space-time matching treatment on ATMS and ERA5 data, and counting the maximum value and the minimum value of each parameter, and performing normalization treatment; the formula is as follows:
；
wherein C represents a parameter in the dataset,and->Respectively statistically obtained maximum and minimum, < ->Is a normalized parameter.
(3) And counting the average value of the ERA5 temperature profile data to obtain an average value data set, and calculating covariance among all air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B. The covariance matrix B is a symmetric matrix of 37×37 dimensions, and the diagonal is 1.
(4) Taking normalized ERA5 temperature profile data as a true value, taking normalized ATMS brightness temperature, angle and geographic positioning information as input, constructing a data set, and taking the data set as 1:9, dividing the ratio into a test sample and a training sample; after the training samples are obtained, 17280 samples are input into the network model for training, i.e., batch_size=17280.
(5) As shown in fig. 2, constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by using a back propagation algorithm through calculating a brand new loss function; the method comprises the following steps:
(51) The first layer of the DCN-LSTM model is an input layer, and the input dimension is 17280 multiplied by 28; the model body can be divided into an encoder structure and a decoder structure, with the DNN model as the encoder and the LSTM model as the decoder.
(52) The DNN model comprises 3 hidden layers, wherein the input dimension of the hidden layer of the first layer is 17280 multiplied by 28, the hidden layer is connected with a LeakyReLU activation function and a Dropout function, and the output data dimension is 17280 multiplied by 64; the following 2 hidden layers each take the output of the previous layer as input and each contain a LeakyReLU activation function and a Dropout function. The output dimension of the second layer is 17280×128 and the output dimension of the third layer is 17280×256. After DNN feature extraction, the data size was 17280×256.
(53) The DNN encoder and LSTM decoder are connected by a 1D CNN layer, the input dimension of the 1D CNN layer is 17280×256, the output dimension is 17280×512, and the layer is connected with the LeakyReLU activation function. The convolution kernel Size of the 1D CNN layer is set to 3, the edge Padding Size (Padding Size) is set to 1, and the convolution kernel step Size is set to 1.
(54) The LSTM module acts as a decoder of the model, and also contains 3 hidden layers, each layer containing 512 LSTM neurons. After LSTM decoding, the data has unchanged dimension.
(55) Finally, the model takes a full connection layer as an output layer, adjusts the output dimension to 17280 multiplied by 37, and takes the ReLU as an activation function to obtain a simulated atmospheric temperature profile.
(56) The model loss function is designed, and the model loss function comprises two parts, and the formula is as follows:
；
wherein,
；
；
wherein w is a scaling factor, controlAnd->Of the order of magnitude of (3); />Representing MAE error, y represents ERA5 temperature data,/->Representing the generated value of the DCN-LSTM model; />Covariance matrix representing y,>representative ofIs a covariance matrix of (a). />The calculation should not be performed on a certain number of samples, i.e. the Batch Size when training the model is too small.
(6) Verifying the model accuracy through a test sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
Specific data:
step 1: ATMS remote sensing data and ERA5 analysis data acquisition. Downloading SDR data of days 1 and 15 of 12 months of 2022 from NOAA CLASS officially; ERA5 layer hour by hour atmospheric temperature data of the same date was obtained through CDS API. The ATMS and ERA5 data are decoded.
Step 2: and (5) data space-time matching and normalization. And simultaneously reading ATMS data and ERA5 data at corresponding moments, and performing space matching by using a nearest neighbor method. And (5) counting the maximum value and the minimum value of all parameters, and normalizing.
Step 3: a background covariance matrix B is calculated.
Step 4: and constructing a data set and a DCN-LSTM atmospheric temperature profile generating model. And taking ATMS observation information, angle information and geographic positioning information as x and ERA5 temperature profile information as y to construct a data set. 1, the method comprises the following steps: the scale of 9 randomly divides the dataset into test and training samples. And constructing a DCN-LSTM atmospheric temperature profile generating model based on the pytorch framework.
Step 5: model training and tuning. Training the model in a GPU environment, simultaneously calling 4 GPUs for distributed training, and using Adam's optimizationThe error back propagation and model parameter optimization are carried out by the converter, and the learning rate is set to be 0.0001 and beta 1 Set to 0.9, beta 2 Set to 0.999, training period is 200 in total, and after 100 periods, learning rate is attenuated. During model training, the model is tested once every 5 periods based on a test sample, and if the precision on a test set meets the requirement, training is skipped in advance; if the accuracy still does not meet the requirement after the model is trained for 200 periods, the learning parameters are adjusted for retraining.
Step 6: and outputting and applying results. And carrying out inverse normalization on the model output result to obtain an atmospheric temperature profile product with physical significance.
The embodiment of the invention also provides an atmospheric temperature profile generating system based on the covariance matrix and the DCN-LSTM model, which comprises the following steps:
and a data acquisition module: the ERA5 re-analysis data is used for acquiring ATMS remote sensing data and corresponding dates, and preprocessing the ATMS data;
and a matching module: the method is used for carrying out space-time matching processing on ATMS and ERA5 data, counting the maximum value and the minimum value of each parameter and carrying out normalization processing;
covariance matrix module: the method comprises the steps of counting average values of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
and (3) constructing a data set module: the method is used for constructing a data set by taking normalized ERA5 temperature profile data as a true value and normalized ATMS brightness temperature, angle and geographic positioning information as input, and the data set is calculated according to 1:9, dividing the ratio into a test sample and a training sample;
DCN-LSTM module: the method comprises the steps of constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by using a back propagation algorithm through calculating a brand new loss function;
and (3) a tuning module: for verifying model accuracy by testing the sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
The embodiment of the invention also provides equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the atmospheric temperature profile generating system based on the covariance matrix and the DCN-LSTM model when being loaded to the processor.
The embodiment of the invention also provides a storage medium, which stores a computer program, and the computer program realizes the atmospheric temperature profile generating system based on the covariance matrix and the DCN-LSTM model when being executed by a processor.

Claims (9)

1. The atmosphere temperature profile generating method based on the covariance matrix and the DCN-LSTM model is characterized by comprising the following steps of:
(1) Acquiring ATMS remote sensing data and ERA5 re-analysis data of corresponding date, and preprocessing the ATMS data;
(2) Performing space-time matching treatment on ATMS and ERA5 data, and counting the maximum value and the minimum value of each parameter, and performing normalization treatment;
(3) Counting the average value of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among all air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
(4) Taking normalized ERA5 temperature profile data as a true value, taking normalized ATMS brightness temperature, angle and geographic positioning information as input, constructing a data set, and taking the data set as M: the proportion of N is divided into a test sample and a training sample;
(5) Constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by calculating a brand new loss function and utilizing a back propagation algorithm;
(6) Verifying the model accuracy through a test sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
2. The method for generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model according to claim 1, wherein in step 1, ERA5 re-analysis data comprises: temperature information at 37 layers of atmospheric altitude between 1000hpa and 1 hpa; ATMS data preprocessing includes: and (3) scaling processing of bright temperature observation, extracting observation angle information including satellite zenith angle, satellite azimuth angle, solar zenith angle and solar azimuth angle, and extracting geographic positioning information including longitude and latitude.
3. The method for generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model according to claim 1, wherein in the step 2, normalization processing is performed on a statistical maximum value and a statistical minimum value of each parameter; the formula is as follows:
；
wherein C represents a parameter in the dataset,and->Respectively obtaining a maximum value and a minimum value by statistics; />Is a normalized parameter.
4. The method for generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model according to claim 1, wherein in the step 3, the covariance matrix B is a symmetric matrix with 37×37 dimensions, and the diagonal is 1.
5. The method of claim 1, wherein in step 4, after obtaining training samples, the training samples are input into the network model as a group of 17280 samples, i.e. batch_size=17280.
6. The method for generating an atmospheric temperature profile based on a covariance matrix and a DCN-LSTM model according to claim 1, wherein the step 5 comprises the following steps:
(51) The first layer of the DCN-LSTM model is an input layer, and the input dimension is 17280 multiplied by 28; the model body can be divided into an encoder structure and a decoder structure, wherein a DNN model is used as an encoder, and an LSTM model is used as a decoder;
(52) The DNN model comprises 3 hidden layers, wherein the input dimension of the hidden layer of the first layer is 17280 multiplied by 28, the hidden layer is connected with a LeakyReLU activation function and a Dropout function, and the output data dimension is 17280 multiplied by 64; the following 2 hidden layers each take the output of the previous layer as input and each contains a LeakyReLU activation function and a Dropout function; the output dimension of the second layer is 17280×128, and the output dimension of the third layer is 17280×256; after DNN characteristic extraction, the data size is 17280 multiplied by 256;
(53) Connecting the DNN encoder and the LSTM decoder by using a 1D CNN layer, wherein the input dimension of the 1D CNN layer is 17280 multiplied by 256, the output dimension is 17280 multiplied by 512, and the layer is connected with a LeakyReLU activation function; the convolution kernel Size of the 1D CNN layer is set to 3, the edge Padding Size (Padding Size) is set to 1, and the convolution kernel step Size is set to 1;
(54) The LSTM module is used as a decoder of the model, and also comprises 3 hidden layers, wherein each layer comprises 512 LSTM neurons; after the data is decoded by LSTM, the dimension is unchanged;
(55) Finally, the model takes a full connection layer as an output layer, adjusts the output dimension to 17280 multiplied by 37, and takes a ReLU as an activation function to obtain a simulated atmospheric temperature profile;
(56) The model loss function is designed, and the model loss function comprises two parts, and the formula is as follows:
；
wherein,
；
；
wherein w is a scaling factor, controlAnd->Of the order of magnitude of (3); />Representing MAE error, y represents ERA5 temperature data,/->Representing the generated value of the DCN-LSTM model; />Covariance matrix representing y,>represents->Is a covariance matrix of (a).
7. An atmospheric temperature profile generation system based on a covariance matrix and a DCN-LSTM model, comprising:
and a data acquisition module: the ERA5 re-analysis data is used for acquiring ATMS remote sensing data and corresponding dates, and preprocessing the ATMS data;
and a matching module: the method is used for carrying out space-time matching processing on ATMS and ERA5 data, counting the maximum value and the minimum value of each parameter and carrying out normalization processing;
covariance matrix module: the method comprises the steps of counting average values of ERA5 temperature profile data to obtain an average value data set, and calculating covariance among air pressure layers by using the data set to obtain a covariance matrix, namely a background covariance matrix B;
and (3) constructing a data set module: the method is used for constructing a data set by taking normalized ERA5 temperature profile data as a true value and normalized ATMS brightness temperature, angle and geographic positioning information as input, and the data set is calculated according to 1:9, dividing the ratio into a test sample and a training sample;
DCN-LSTM module: the method comprises the steps of constructing a DCN-LSTM network model introducing a covariance matrix, training the model, and optimizing parameters of the model by using a back propagation algorithm through calculating a brand new loss function;
and (3) a tuning module: for verifying model accuracy by testing the sample; and (5) returning to the step (5) for re-tuning if the preset precision is not met, and if the preset precision is met, the trained model is the target model.
8. An apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program when loaded into the processor implements an atmospheric temperature profile generation system based on a covariance matrix and DCN-LSTM model according to any one of claims 1-7.
9. A storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements an atmospheric temperature profile generation system based on a covariance matrix and DCN-LSTM model according to any one of claims 1-7.
CN202410073112.3A 2024-01-18 2024-01-18 Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model Active CN117591835B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410073112.3A CN117591835B (en) 2024-01-18 2024-01-18 Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410073112.3A CN117591835B (en) 2024-01-18 2024-01-18 Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model

Publications (2)

Publication Number Publication Date
CN117591835A true CN117591835A (en) 2024-02-23
CN117591835B CN117591835B (en) 2024-04-19

Family

ID=89910240

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410073112.3A Active CN117591835B (en) 2024-01-18 2024-01-18 Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model

Country Status (1)

Country Link
CN (1) CN117591835B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111339665A (en) * 2020-02-27 2020-06-26 中国科学院空天信息创新研究院 Troposphere ozone profile calculation method
US20200348448A1 (en) * 2019-04-30 2020-11-05 ClimateAI, Inc. Methods and systems for climate forecasting using artificial neural networks
US20200356839A1 (en) * 2019-05-09 2020-11-12 ClimateAI, Inc. Systems and methods for selecting global climate simulation models for training neural network climate forecasting models
CN115859789A (en) * 2022-11-23 2023-03-28 北京信息科技大学 Method for improving inversion accuracy of polar atmosphere temperature profile
CN116796291A (en) * 2023-04-26 2023-09-22 安徽大学 LSTM-MEA-SVR-based air quality forecasting system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200348448A1 (en) * 2019-04-30 2020-11-05 ClimateAI, Inc. Methods and systems for climate forecasting using artificial neural networks
US20200356839A1 (en) * 2019-05-09 2020-11-12 ClimateAI, Inc. Systems and methods for selecting global climate simulation models for training neural network climate forecasting models
CN111339665A (en) * 2020-02-27 2020-06-26 中国科学院空天信息创新研究院 Troposphere ozone profile calculation method
CN115859789A (en) * 2022-11-23 2023-03-28 北京信息科技大学 Method for improving inversion accuracy of polar atmosphere temperature profile
CN116796291A (en) * 2023-04-26 2023-09-22 安徽大学 LSTM-MEA-SVR-based air quality forecasting system

Also Published As

Publication number Publication date
CN117591835B (en) 2024-04-19

Similar Documents

Publication Publication Date Title
Gauthier et al. Extension of 3DVAR to 4DVAR: Implementation of 4DVAR at the Meteorological Service of Canada
Liu et al. A new statistical downscaling model for autumn precipitation in China
Ferrara et al. Large‐scale control of the lower stratosphere on variability of tropical cyclone intensity
Shastri et al. Improving Global Forecast System of extreme precipitation events with regional statistical model: Application of quantile‐based probabilistic forecasts
Cheng et al. Development of high resolution and homogenized gridded land surface air temperature data: a case study over Pan-East Asia
Privé et al. The role of model and initial condition error in numerical weather forecasting investigated with an observing system simulation experiment
Xian et al. All‐sky assimilation of the MWHS‐2 observations and evaluation the impacts on the analyses and forecasts of binary typhoons
Kotsuki et al. Online model parameter estimation with ensemble data assimilation in the real global atmosphere: A case with the nonhydrostatic icosahedral atmospheric model (NICAM) and the global satellite mapping of precipitation data
Yang et al. Probabilistic post-processing of gridded atmospheric variables and its application to site adaptation of shortwave solar radiation
Hoffman et al. Progress in forecast skill at three leading global operational NWP centers during 2015–17 as seen in summary assessment metrics (SAMs)
CN114595876A (en) Regional wind field prediction model generation method and device and electronic equipment
Hyman et al. Probabilistic retrieval of volcanic SO 2 layer height and partial column density using the Cross-track Infrared Sounder (CrIS)
Terasaki et al. Multi-year analysis using the NICAM-LETKF data assimilation system
Koshin et al. An ensemble Kalman filter data assimilation system for the whole neutral atmosphere
Sun et al. A Deep Learning‐Based Bias Correction Method for Predicting Ocean Surface Waves in the Northwest Pacific Ocean
Chi et al. Cloud macrophysical characteristics in China mainland and east coast from 2006 to 2017 using satellite active remote sensing observations
CN117591835B (en) Atmospheric temperature profile generation method based on covariance matrix and DCN-LSTM model
Li et al. α‐Chapman scale height: Longitudinal variation and global modeling
Ding et al. A feasible approach to improve forecast skill of summer precipitation in northeast China by statistical regression of the northeast China cold vortex in the multimodel ensemble
Prates et al. Assimilation of transformed retrievals obtained from clear‐sky IASI measurements
Tim et al. Oceanic influence on the precipitation in Venezuela under current and future climate
Kotsuki et al. Weight structure of the local ensemble transform Kalman filter: A case with an intermediate atmospheric general circulation model
CN112149349A (en) Typhoon path forecasting method based on deep neural network
Greenslade et al. The impact of the assimilation of scatterometer winds on surface wind and wave forecasts
CN116295853B (en) Multichannel thermal infrared data surface temperature inversion method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant