CN113624694A - Inversion method and device for atmospheric methane concentration - Google Patents
Inversion method and device for atmospheric methane concentration Download PDFInfo
- Publication number
- CN113624694A CN113624694A CN202111178987.2A CN202111178987A CN113624694A CN 113624694 A CN113624694 A CN 113624694A CN 202111178987 A CN202111178987 A CN 202111178987A CN 113624694 A CN113624694 A CN 113624694A
- Authority
- CN
- China
- Prior art keywords
- sample
- input data
- data
- target
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Biochemistry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Immunology (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Pathology (AREA)
- Algebra (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The invention provides an inversion method and device of atmospheric methane concentration, which relate to the technical field of environmental monitoring and comprise the following steps: acquiring sample input data and sample calibration data of a region to be monitored; carrying out dimensionality reduction on the sample input data by utilizing a PCA algorithm to obtain dimensionality reduction sample input data; training a preset XGboost model by using dimension-reducing sample input data and sample calibration data to obtain a target XGboost model; after the current input data of the area to be monitored is obtained, carrying out dimensionality reduction on the current input data by utilizing a PCA algorithm to obtain dimensionality reduction current input data; the dimensionality reduction current input data are input into a target XGboost model to obtain the current atmospheric methane column concentration of the region to be monitored, and the technical problems that the inversion error is large and the inversion efficiency is low in the existing atmospheric methane concentration inversion method are solved.
Description
Technical Field
The invention relates to the technical field of environmental monitoring, in particular to an inversion method and device for atmospheric methane concentration.
Background
The traditional method for monitoring the CH4 concentration change generally adopts a foundation detection method, the foundation detection method is high in precision and reliability, but the construction cost and the maintenance cost of a foundation monitoring site are high, and large-scale construction cannot be carried out at present, so that the ground monitoring mode is limited by space, the global large-range real-time detection capability is lacked, and the observation result is difficult to be used for global CH4 concentration distribution research.
The satellite remote sensing monitoring mode can carry out observation covering the whole world and can provide stable and continuous data with good global scale and space-time consistency. However, the influence of the satellite observation surface on the atmosphere and the earth surface is complex, the accuracy of the obtained atmosphere and earth surface parameters is difficult to guarantee, the inversion of the atmosphere CH4 is easy to be interfered by external factors, the inversion accuracy is directly influenced, and the performance of an instrument, the atmosphere parameters and the earth surface characteristics are three main error sources for the inversion.
In the inversion process by using remote sensing data, the measurement error of the data and the uncertainty of various parameters inevitably affect the inversion process. The influence of the satellite observation surface on the atmosphere and the earth surface is complex, and the accuracy of the obtained atmosphere and earth surface parameters is difficult to ensure, so that the inversion of the atmosphere CH4 is easy to be interfered by external factors, and the inversion accuracy is directly influenced. Satellite remote sensing inversion experiments show that instrument performance, atmospheric parameters and earth surface characteristics are three main error sources for inversion. The signals received by the satellite remote sensors are the result of the combined action of the atmosphere and the earth surface, so the earth surface reflectivity is an important factor influencing the inversion accuracy of the atmosphere CH 4. The uncertainty of the atmospheric state parameters is another important source of inversion errors, so that systematic errors are brought to the inversion result, and the errors are difficult to eliminate by using the CH4 absorption band alone for inversion. The problems of inversion error, low inversion time resolution and the like exist in the existing inversion methods in different degrees.
No effective solution has been proposed to the above problems.
Disclosure of Invention
In view of this, the present invention provides an inversion method and an inversion apparatus for atmospheric methane concentration, so as to alleviate the technical problems of large inversion error and low inversion efficiency of the existing inversion method for atmospheric methane concentration.
In a first aspect, an embodiment of the present invention provides an inversion method of atmospheric methane concentration, including: acquiring sample input data and sample calibration data of a region to be monitored, wherein the sample input data comprise remote sensing spectrum data of a preset number of different wave bands, and the sample calibration data comprise: sample atmospheric methane column concentration and sample dry air column concentration; performing dimensionality reduction on the sample input data by using a PCA algorithm to obtain dimensionality reduction sample input data; training a preset XGboost model by using the dimensionality reduction sample input data and the sample calibration data to obtain a target XGboost model; after the current input data of the area to be monitored is obtained, carrying out dimensionality reduction on the current input data by utilizing a PCA algorithm to obtain dimensionality reduction current input data; and inputting the dimensionality reduction current input data into the target XGboost model to obtain the current atmospheric methane column concentration of the area to be monitored.
Further, using a PCA algorithm to perform dimensionality reduction processing on the sample input data to obtain dimensionality reduced sample input data, including: constructing a sample matrix based on the remote sensing spectral data, wherein the sample matrix is a k multiplied by n matrix, k is the number of position points contained in the remote sensing spectral data, and n is the number of spectral channels in the remote sensing spectral data; calculating a covariance matrix of the sample matrix, and calculating an eigenvalue of the covariance matrix and an eigenvector corresponding to the eigenvalue; determining a target characteristic value in the characteristic values, and constructing a transformation matrix based on a characteristic vector corresponding to the target characteristic value; calculating the product between the transformation matrix and the sample matrix to obtain a target matrix; and determining the target matrix as the dimension reduction sample input data.
Further, determining a target feature value in the feature values includes: sequencing the characteristic values, and calculating the accumulated contribution rate of the front target number of main components, wherein the accumulated contribution rate is the ratio of the sum of the front target number of maximum characteristic values to the sum of the characteristic values of the remote sensing spectrum data; and determining the corresponding maximum characteristic value of the number of the previous targets as the target characteristic value when the accumulated contribution rate is greater than a preset threshold value.
Further, training a preset XGboost model by using the dimensionality reduction sample input data and the sample calibration data to obtain a target XGboost model, comprising the following steps of: segmenting the dimension reduction sample input data and the sample calibration data according to a preset proportion to obtain a training data set, a test data set and a check data set; and training the preset XGboost model by using the training data set, the testing data set and the checking data set to obtain the target XGboost model.
In a second aspect, an embodiment of the present invention further provides an apparatus for inverting atmospheric methane concentration, including: the system comprises an acquisition unit, a first dimension reduction unit, a training unit, a second dimension reduction unit and an inversion unit, wherein the acquisition unit is used for acquiring sample input data and sample calibration data of an area to be monitored, the sample input data comprises a preset number of remote sensing spectrum data of different wave bands, and the sample calibration data comprises: sample atmospheric methane column concentration and sample dry air column concentration; the first dimension reduction unit is used for performing dimension reduction processing on the sample input data by utilizing a PCA algorithm to obtain dimension reduction sample input data; the training unit is used for training a preset XGboost model by using the dimensionality reduction sample input data and the sample calibration data to obtain a target XGboost model; the second dimension reduction unit is used for performing dimension reduction processing on the current input data by utilizing a PCA algorithm after the current input data of the area to be monitored is acquired, so that dimension reduction current input data are obtained; and the inversion unit is used for inputting the dimensionality reduction current input data into the target XGboost model to obtain the current atmospheric methane column concentration of the area to be monitored.
Further, the dimension reduction unit is configured to: constructing a sample matrix based on the remote sensing spectral data, wherein the sample matrix is a k multiplied by n matrix, k is the number of position points contained in the remote sensing spectral data, and n is the number of spectral channels in the remote sensing spectral data; calculating a covariance matrix of the sample matrix, and calculating an eigenvalue of the covariance matrix and an eigenvector corresponding to the eigenvalue; determining a target characteristic value in the characteristic values, and constructing a transformation matrix based on a characteristic vector corresponding to the target characteristic value; calculating the product between the transformation matrix and the sample matrix to obtain a target matrix; and determining the target matrix as the dimension reduction sample input data.
Further, the dimension reduction unit is configured to: sequencing the characteristic values, and calculating the accumulated contribution rate of the front target number of main components, wherein the accumulated contribution rate is the ratio of the sum of the front target number of maximum characteristic values to the sum of the characteristic values of the remote sensing spectrum data; and determining the corresponding maximum characteristic value of the number of the previous targets as the target characteristic value when the accumulated contribution rate is greater than a preset threshold value.
Further, the training unit is configured to: segmenting the dimension reduction sample input data and the sample calibration data according to a preset proportion to obtain a training data set, a test data set and a check data set; and training the preset XGboost model by using the training data set, the testing data set and the checking data set to obtain the target XGboost model.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory is used to store a program that supports the processor to execute the method in the first aspect, and the processor is configured to execute the program stored in the memory.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored.
In the embodiment of the invention, sample input data and sample calibration data of a region to be monitored are obtained, wherein the sample input data comprise remote sensing spectrum data of a preset number of different wave bands, and the sample calibration data comprise: sample atmospheric methane column concentration and sample dry air column concentration; carrying out dimensionality reduction on the sample input data by utilizing a PCA algorithm to obtain dimensionality reduction sample input data; training a preset XGboost model by using dimension-reducing sample input data and sample calibration data to obtain a target XGboost model; after the current input data of the area to be monitored is obtained, carrying out dimensionality reduction on the current input data by utilizing a PCA algorithm to obtain dimensionality reduction current input data; and inputting the dimensionality reduction current input data into a target XGboost model to obtain the current atmospheric methane column concentration of the area to be monitored. According to the method and the device, the dry air concentration which has high correlation with the methane concentration is added to the construction process of the inversion model, and the remote sensing spectral data are subjected to dimensionality reduction processing, so that the aim of accurate and efficient inversion of the atmospheric methane concentration is fulfilled, the technical problems that the inversion error of the existing atmospheric methane concentration inversion method is large and the inversion efficiency is low are solved, and therefore the technical effects of improving the inversion efficiency of the atmospheric methane concentration and reducing the inversion error of the atmospheric methane concentration are achieved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a method for inversion of atmospheric methane concentration according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an apparatus for inverting atmospheric methane concentration according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
in accordance with an embodiment of the present invention, there is provided an embodiment of a method for inversion of atmospheric methane concentration, it being noted that the steps illustrated in the flowchart of the figure may be performed in a computer system, such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Fig. 1 is a flow chart of a method for inverting atmospheric methane concentration according to an embodiment of the present invention, as shown in fig. 1, the method comprising the steps of:
step S102, obtaining sample input data and sample calibration data of a region to be monitored, wherein the sample input data comprise remote sensing spectrum data of a preset number of different wave bands, and the sample calibration data comprise: sample atmospheric methane column concentration and sample dry air column concentration;
it should be noted that, the sample input data is typically remote sensing spectral data of different bands acquired by the gossat sensor TANSO-FTS L1B, the number of bands of the remote sensing spectral data of different bands is 4, and in general, the preset number is set to 3.
The sample atmospheric methane column concentration is generally the atmospheric methane column concentration collected by TANSO-FTS SWIR Level 2.
Step S104, using PCA algorithm to perform dimensionality reduction processing on the sample input data to obtain dimensionality reduction sample input data;
step S106, training a preset XGboost model by using the dimensionality reduction sample input data and the sample calibration data to obtain a target XGboost model;
step S108, after the current input data of the area to be monitored is obtained, carrying out dimensionality reduction on the current input data by utilizing a PCA algorithm to obtain dimensionality reduction current input data;
and step S110, inputting the dimensionality reduction current input data into the target XGboost model to obtain the current atmospheric methane column concentration of the area to be monitored.
In the embodiment of the invention, sample input data and sample calibration data of a region to be monitored are obtained, wherein the sample input data comprise remote sensing spectrum data of a preset number of different wave bands, and the sample calibration data comprise: sample atmospheric methane column concentration and sample dry air column concentration; carrying out dimensionality reduction on the sample input data by utilizing a PCA algorithm to obtain dimensionality reduction sample input data; training a preset XGboost model by using dimension-reducing sample input data and sample calibration data to obtain a target XGboost model; after the current input data of the area to be monitored is obtained, carrying out dimensionality reduction on the current input data by utilizing a PCA algorithm to obtain dimensionality reduction current input data; and inputting the dimensionality reduction current input data into a target XGboost model to obtain the current atmospheric methane column concentration of the area to be monitored. According to the method and the device, the dry air concentration which has high correlation with the methane concentration is added to the construction process of the inversion model, and the remote sensing spectral data are subjected to dimensionality reduction processing, so that the aim of accurate and efficient inversion of the atmospheric methane concentration is fulfilled, the technical problems that the inversion error of the existing atmospheric methane concentration inversion method is large and the inversion efficiency is low are solved, and therefore the technical effects of improving the inversion efficiency of the atmospheric methane concentration and reducing the inversion error of the atmospheric methane concentration are achieved.
In the embodiment of the present invention, step S104 includes the following steps:
step S11, constructing a sample matrix based on the remote sensing spectral data, wherein the sample matrix is a k multiplied by n matrix, k is the number of position points contained in the remote sensing spectral data, and n is the number of spectral channels in the remote sensing spectral data;
step S12, calculating a covariance matrix of the sample matrix, and calculating an eigenvalue of the covariance matrix and an eigenvector corresponding to the eigenvalue;
step S13, determining a target characteristic value in the characteristic values, and constructing a transformation matrix based on the characteristic vector corresponding to the target characteristic value;
step S14: calculating the product between the transformation matrix and the sample matrix to obtain a target matrix;
step S15, determining the target matrix as the dimension reduction sample input data.
In the embodiment of the present invention, the dimension reduction process is described by taking the example that the sample input data includes remote sensing spectral data of 3 different bands.
The remote sensing spectral data of 3 different wave bands are respectively band1, band2 and band3, which is explained as band 1. band1 data Single Spectrum data of
The data of k position points in total and the band1 data in total have n =1755 spectrum channels, and form the structure
Matrix array
And carrying out dimensionality reduction on 1755 spectral channels by using a PCA algorithm. The method comprises the following specific steps:
first, based on remote sensingSpectral data band1, construction
Sample matrix
Sample matrix X, one spectral sample X per row, each column representing the spectral data of a point location.
Then, the covariance matrix of the sample matrix is calculated by the following formula
，
Is a transposed matrix of the sample matrix.
Then, the eigenvalue of the covariance matrix and the eigenvector corresponding to the eigenvalue are calculated using the following formula.
Then, a target characteristic value in the characteristic values is determined, and a transformation matrix is constructed based on the characteristic vector corresponding to the target characteristic value.
Specifically, the eigenvalues are arranged in descending order, and the cumulative contribution ratio of the first m principal components (i.e., the maximum eigenvalues of the previous target number) is calculated according to the following formula:
。
and when the accumulated contribution rate is greater than 85%, the value m (namely the target quantity) is the dimensionality of the dimensionality reduction sample input data, wherein the accumulated contribution rate is used for representing the information preservation degree of the dimensionality reduction sample input data on the remote sensing spectral data.
Forming a transformation matrix by using eigenvectors corresponding to the maximum eigenvalues of the number (m) of previous targets
，
，
。
And finally, calculating the product between the transformation matrix and the sample matrix to obtain a target matrix, wherein the 1755-dimensional band1 is reduced to m-dimensional reduced-dimension sample input data.
The same method is used for performing dimension reduction processing on the band2 and the band3, and the specific process is not described again. Data Y1, Y2 and Y3 of the dimensionality reduced by the Band1, the Band2 and the Band3 are obtained by the dimensionality reduction method.
In the embodiment of the present invention, step S106 includes the following steps:
step S21, segmenting the dimension reduction sample input data and the sample calibration data according to a preset proportion to obtain a training data set, a test data set and a check data set;
step S22, training the preset XGboost model by using the training data set, the test data set and the verification data set to obtain the target XGboost model.
In the embodiment of the present invention, Y1, Y2, Y3, the sample atmospheric methane column concentration, and the sample dry air column concentration are divided according to a preset ratio, and in general, the preset ratio is 60%, 20%, and 20%, that is, data of Y1, Y2, Y3, 60% of the sample atmospheric methane column concentration, and the sample dry air column concentration is used as a training data set, data of Y1, Y2, Y3, 20% of the sample atmospheric methane column concentration, and the sample dry air column concentration is used as a test data set, and data of Y1, Y2, Y3, 20% of the sample atmospheric methane column concentration, and the sample dry air column concentration is used as a verification data set.
And then, training a preset XGboost model by using the training data set, the testing data set and the verifying data set so as to obtain a target XGboost model.
It should be noted that, in the training process, parameters in the preset XGBoost model need to be set, and the specific settings are as follows:
the parameters of the XGBoost are divided into three categories: general parameters (macroscopic function control), Booster parameters (controlling boost (tree/regression) at each step), and learning objective parameters (controlling the performance of the training objective). And optimizing and adjusting three types of parameters in model training:
adjusting and setting general parameters:
boost: dart (using dart decision tree);
nthread: (algorithm auto detect, compute using all cores of the CPU).
boost parameter adjustment setting:
learning_rate ：0.1；
min_child_weight：9；
max _ depth: 9; (maximum depth of tree, this value is also used to avoid overfitting, the larger max _ depth, the more specific and local samples the model will learn, typical values: 3-10);
gamma is 0.0 (the minimum loss function reduction value required by node splitting is specified, the larger the value of the parameter is, the more conservative the algorithm is, the value of the parameter is closely related to the loss function, so that the parameter needs to be adjusted);
subsample: 0.9 (control the ratio of random sampling for each tree, reduce the value of this parameter, the algorithm is more conservative, avoid overfitting. however, if this value is set too small, it may result in under-fitting, typical values: 0.5-1);
colsample _ byte: 0.8 (to control the fraction of the number of columns per random sample (each column is a feature, typical value: 0.5-1);
alpha: 1e-05 (which can be applied in case of very high dimensionality, making the algorithm faster);
importance_type: gain；
missing: np.nan；
n_estimators: 138；
subsample: 0.9。
adjusting and setting learning target parameters:
objective ：reg:squarederror。
in the embodiment of the invention, the inversion method of the atmospheric methane concentration has the following advantages:
the method has the advantages that 1, the PCA is used for reducing the dimension of the data, reducing the data and reducing the number of characteristic values, so that not only can overfitting be prevented, but also the data storage space is saved, and the operation speed of the algorithm is improved.
2, the XGboost algorithm is used for model training, and the training speed is higher due to the support of parallel processing.
3, the inversion of the dry air concentration is added to improve the inversion accuracy, because the dry air concentration and the CH4 concentration have high correlation, and the inversion can improve the accuracy of the inversion result of CH 4.
4, atmospheric methane concentration products with time resolution can be produced, atmospheric methane concentration products with time resolution of 3 days can be provided through a trained model algorithm, and only atmospheric methane concentration products with time resolution of 1 month are provided by GOSAT at present.
Example two:
the embodiment of the invention also provides an inversion device of the atmospheric methane concentration, which is used for executing the inversion method of the atmospheric methane concentration provided by the embodiment of the invention, and the following is a specific introduction of the inversion device of the atmospheric methane concentration provided by the embodiment of the invention.
As shown in fig. 2, fig. 2 is a schematic diagram of the apparatus for inverting the atmospheric methane concentration, and the apparatus for inverting the atmospheric methane concentration includes: the system comprises an acquisition unit 10, a first dimension reduction unit 20, a training unit 30, a second dimension reduction unit 40 and an inversion unit 50.
The acquiring unit 10 is configured to acquire sample input data and sample calibration data of an area to be monitored, where the sample input data includes a preset number of remote sensing spectrum data in different bands, and the sample calibration data includes: sample atmospheric methane column concentration and sample dry air column concentration;
the first dimension reduction unit 20 is configured to perform dimension reduction processing on the sample input data by using a PCA algorithm to obtain dimension reduced sample input data;
the training unit 30 is configured to train a preset XGBoost model by using the dimensionality reduction sample input data and the sample calibration data to obtain a target XGBoost model;
the second dimension reduction unit 40 is configured to, after obtaining the current input data of the area to be monitored, perform dimension reduction processing on the current input data by using a PCA algorithm to obtain dimension reduction current input data;
and the inversion unit 50 is configured to input the dimensionality reduction current input data into the target XGBoost model to obtain the current atmospheric methane column concentration of the area to be monitored.
In the embodiment of the invention, sample input data and sample calibration data of a region to be monitored are obtained, wherein the sample input data comprise remote sensing spectrum data of a preset number of different wave bands, and the sample calibration data comprise: sample atmospheric methane column concentration and sample dry air column concentration; carrying out dimensionality reduction on the sample input data by utilizing a PCA algorithm to obtain dimensionality reduction sample input data; training a preset XGboost model by using dimension-reducing sample input data and sample calibration data to obtain a target XGboost model; after the current input data of the area to be monitored is obtained, carrying out dimensionality reduction on the current input data by utilizing a PCA algorithm to obtain dimensionality reduction current input data; and inputting the dimensionality reduction current input data into a target XGboost model to obtain the current atmospheric methane column concentration of the area to be monitored. According to the method and the device, the dry air concentration which has high correlation with the methane concentration is added to the construction process of the inversion model, and the remote sensing spectral data are subjected to dimensionality reduction processing, so that the aim of accurate and efficient inversion of the atmospheric methane concentration is fulfilled, the technical problems that the inversion error of the existing atmospheric methane concentration inversion method is large and the inversion efficiency is low are solved, and therefore the technical effects of improving the inversion efficiency of the atmospheric methane concentration and reducing the inversion error of the atmospheric methane concentration are achieved.
Preferably, the dimension reduction unit is configured to: constructing a sample matrix based on the remote sensing spectral data, wherein the sample matrix is a k multiplied by n matrix, k is the number of position points contained in the remote sensing spectral data, and n is the number of spectral channels in the remote sensing spectral data; calculating a covariance matrix of the sample matrix, and calculating an eigenvalue of the covariance matrix and an eigenvector corresponding to the eigenvalue; determining a target characteristic value in the characteristic values, and constructing a transformation matrix based on a characteristic vector corresponding to the target characteristic value; and determining the target matrix as the dimension reduction sample input data.
Preferably, the dimension reduction unit is configured to: sequencing the characteristic values, and calculating the accumulated contribution rate of the front target number of main components, wherein the accumulated contribution rate is the ratio of the sum of the front target number of maximum characteristic values to the sum of the characteristic values of the remote sensing spectrum data; and determining the corresponding maximum characteristic value of the number of the previous targets as the target characteristic value when the accumulated contribution rate is greater than a preset threshold value.
Preferably, the training unit is configured to: segmenting the dimension reduction sample input data and the sample calibration data according to a preset proportion to obtain a training data set, a test data set and a check data set; and training the preset XGboost model by using the training data set, the testing data set and the checking data set to obtain the target XGboost model.
Example three:
an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory is used to store a program that supports the processor to execute the method described in the first embodiment, and the processor is configured to execute the program stored in the memory.
Referring to fig. 3, an embodiment of the present invention further provides an electronic device 100, including: a processor 60, a memory 61, a bus 62 and a communication interface 63, wherein the processor 60, the communication interface 63 and the memory 61 are connected through the bus 62; the processor 60 is arranged to execute executable modules, such as computer programs, stored in the memory 61.
The Memory 61 may include a high-speed Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 63 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.
The bus 62 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 3, but this does not indicate only one bus or one type of bus.
The memory 61 is used for storing a program, the processor 60 executes the program after receiving an execution instruction, and the method executed by the apparatus defined by the flow process disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 60, or implemented by the processor 60.
The processor 60 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 60. The Processor 60 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory 61, and the processor 60 reads the information in the memory 61 and, in combination with its hardware, performs the steps of the above method.
Example four:
the embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the method in the first embodiment.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. An inversion method of atmospheric methane concentration, comprising:
acquiring sample input data and sample calibration data of a region to be monitored, wherein the sample input data comprise remote sensing spectrum data of a preset number of different wave bands, and the sample calibration data comprise: sample atmospheric methane column concentration and sample dry air column concentration;
performing dimensionality reduction on the sample input data by using a PCA algorithm to obtain dimensionality reduction sample input data;
training a preset XGboost model by using the dimensionality reduction sample input data and the sample calibration data to obtain a target XGboost model;
after the current input data of the area to be monitored is obtained, carrying out dimensionality reduction on the current input data by utilizing a PCA algorithm to obtain dimensionality reduction current input data;
and inputting the dimensionality reduction current input data into the target XGboost model to obtain the current atmospheric methane column concentration of the area to be monitored.
2. The method of claim 1, wherein performing a dimensionality reduction process on the sample input data using a PCA algorithm to obtain reduced-dimensionality sample input data, comprises:
constructing a sample matrix based on the remote sensing spectral data, wherein the sample matrix is a k multiplied by n matrix, k is the number of position points contained in the remote sensing spectral data, and n is the number of spectral channels in the remote sensing spectral data;
calculating a covariance matrix of the sample matrix, and calculating an eigenvalue of the covariance matrix and an eigenvector corresponding to the eigenvalue;
determining a target characteristic value in the characteristic values, and constructing a transformation matrix based on a characteristic vector corresponding to the target characteristic value;
calculating the product between the transformation matrix and the sample matrix to obtain a target matrix;
and determining the target matrix as the dimension reduction sample input data.
3. The method of claim 2, wherein determining the target one of the feature values comprises:
sequencing the characteristic values, and calculating the accumulated contribution rate of the front target number of main components, wherein the accumulated contribution rate is the ratio of the sum of the front target number of maximum characteristic values to the sum of the characteristic values of the remote sensing spectrum data;
and determining the corresponding maximum characteristic value of the number of the previous targets as the target characteristic value when the accumulated contribution rate is greater than a preset threshold value.
4. The method of claim 1, wherein training a preset XGboost model using the dimensionality reduction sample input data and the sample calibration data to obtain a target XGboost model comprises:
segmenting the dimension reduction sample input data and the sample calibration data according to a preset proportion to obtain a training data set, a test data set and a check data set;
and training the preset XGboost model by using the training data set, the testing data set and the checking data set to obtain the target XGboost model.
5. An apparatus for inverting atmospheric methane concentration, comprising: an obtaining unit, a first dimension reduction unit, a training unit, a second dimension reduction unit and an inversion unit, wherein,
the acquisition unit is used for acquiring sample input data and sample calibration data of an area to be monitored, wherein the sample input data comprises remote sensing spectrum data of a preset number of different wave bands, and the sample calibration data comprises: sample atmospheric methane column concentration and sample dry air column concentration;
the first dimension reduction unit is used for performing dimension reduction processing on the sample input data by utilizing a PCA algorithm to obtain dimension reduction sample input data;
the training unit is used for training a preset XGboost model by using the dimensionality reduction sample input data and the sample calibration data to obtain a target XGboost model;
the second dimension reduction unit is used for performing dimension reduction processing on the current input data by utilizing a PCA algorithm after the current input data of the area to be monitored is acquired, so that dimension reduction current input data are obtained;
and the inversion unit is used for inputting the dimensionality reduction current input data into the target XGboost model to obtain the current atmospheric methane column concentration of the area to be monitored.
6. The apparatus of claim 5, wherein the dimension reduction unit is configured to:
constructing a sample matrix based on the remote sensing spectral data, wherein the sample matrix is a k multiplied by n matrix, k is the number of position points contained in the remote sensing spectral data, and n is the number of spectral channels in the remote sensing spectral data;
calculating a covariance matrix of the sample matrix, and calculating an eigenvalue of the covariance matrix and an eigenvector corresponding to the eigenvalue;
determining a target characteristic value in the characteristic values, and constructing a transformation matrix based on a characteristic vector corresponding to the target characteristic value;
calculating the product between the transformation matrix and the sample matrix to obtain a target matrix;
and determining the target matrix as the dimension reduction sample input data.
7. The apparatus of claim 6, wherein the dimension reduction unit is configured to:
sequencing the characteristic values, and calculating the accumulated contribution rate of the front target number of main components, wherein the accumulated contribution rate is the ratio of the sum of the front target number of maximum characteristic values to the sum of the characteristic values of the remote sensing spectrum data;
and determining the corresponding maximum characteristic value of the number of the previous targets as the target characteristic value when the accumulated contribution rate is greater than a preset threshold value.
8. The apparatus of claim 5, wherein the training unit is configured to:
segmenting the dimension reduction sample input data and the sample calibration data according to a preset proportion to obtain a training data set, a test data set and a check data set;
and training the preset XGboost model by using the training data set, the testing data set and the checking data set to obtain the target XGboost model.
9. An electronic device comprising a memory for storing a program that enables a processor to perform the method of any of claims 1 to 4 and a processor configured to execute the program stored in the memory.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of the claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111178987.2A CN113624694A (en) | 2021-10-11 | 2021-10-11 | Inversion method and device for atmospheric methane concentration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111178987.2A CN113624694A (en) | 2021-10-11 | 2021-10-11 | Inversion method and device for atmospheric methane concentration |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113624694A true CN113624694A (en) | 2021-11-09 |
Family
ID=78390821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111178987.2A Pending CN113624694A (en) | 2021-10-11 | 2021-10-11 | Inversion method and device for atmospheric methane concentration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113624694A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116486931A (en) * | 2023-06-21 | 2023-07-25 | 上海航天空间技术有限公司 | Full-coverage atmospheric methane concentration data production method and system coupled with physical mechanism |
| CN116738232A (en) * | 2023-06-16 | 2023-09-12 | 中国科学院空天信息创新研究院 | Urban atmospheric carbon emission distribution detection method based on FTIR spectrum |
| CN117556262A (en) * | 2024-01-08 | 2024-02-13 | 中国科学院空天信息创新研究院 | Atmospheric methane emission intensity monitoring method and device, electronic equipment and storage medium |
| CN116738232B (en) * | 2023-06-16 | 2024-05-31 | 中国科学院空天信息创新研究院 | Urban atmospheric carbon emission distribution detection method based on FTIR spectrum |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103940767A (en) * | 2014-05-07 | 2014-07-23 | 重庆大学 | Gas concentration inversion method based on multi-manifold learning |
| CN109387487A (en) * | 2018-12-18 | 2019-02-26 | 成都信息工程大学 | Short-wave infrared high-spectral data atmospheric methane fast inversion method under the conditions of cirrus |
| WO2019246280A1 (en) * | 2018-06-19 | 2019-12-26 | Seekops Inc. | Emissions estimate model algorithms and methods |
| CN111859800A (en) * | 2020-07-15 | 2020-10-30 | 河海大学 | Method for spatio-temporal estimation and prediction of PM2.5 concentration distribution |
| CN113379146A (en) * | 2021-06-24 | 2021-09-10 | 合肥工业大学智能制造技术研究院 | Pollutant concentration inversion method based on multi-feature selection algorithm |
-
2021
- 2021-10-11 CN CN202111178987.2A patent/CN113624694A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103940767A (en) * | 2014-05-07 | 2014-07-23 | 重庆大学 | Gas concentration inversion method based on multi-manifold learning |
| WO2019246280A1 (en) * | 2018-06-19 | 2019-12-26 | Seekops Inc. | Emissions estimate model algorithms and methods |
| CN109387487A (en) * | 2018-12-18 | 2019-02-26 | 成都信息工程大学 | Short-wave infrared high-spectral data atmospheric methane fast inversion method under the conditions of cirrus |
| CN111859800A (en) * | 2020-07-15 | 2020-10-30 | 河海大学 | Method for spatio-temporal estimation and prediction of PM2.5 concentration distribution |
| CN113379146A (en) * | 2021-06-24 | 2021-09-10 | 合肥工业大学智能制造技术研究院 | Pollutant concentration inversion method based on multi-feature selection algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 田园 等: "地基高分辨率傅里叶变换红外光谱反演环境大气中的CH4浓度变化", 《物理学报》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116738232A (en) * | 2023-06-16 | 2023-09-12 | 中国科学院空天信息创新研究院 | Urban atmospheric carbon emission distribution detection method based on FTIR spectrum |
| CN116738232B (en) * | 2023-06-16 | 2024-05-31 | 中国科学院空天信息创新研究院 | Urban atmospheric carbon emission distribution detection method based on FTIR spectrum |
| CN116486931A (en) * | 2023-06-21 | 2023-07-25 | 上海航天空间技术有限公司 | Full-coverage atmospheric methane concentration data production method and system coupled with physical mechanism |
| CN116486931B (en) * | 2023-06-21 | 2023-08-29 | 上海航天空间技术有限公司 | Full-coverage atmospheric methane concentration data production method and system coupled with physical mechanism |
| CN117556262A (en) * | 2024-01-08 | 2024-02-13 | 中国科学院空天信息创新研究院 | Atmospheric methane emission intensity monitoring method and device, electronic equipment and storage medium |
| CN117556262B (en) * | 2024-01-08 | 2024-05-14 | 中国科学院空天信息创新研究院 | Atmospheric methane emission intensity monitoring method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9310517B2 (en) | Method and system for determining accuracy of a weather prediction model | |
| CN113624694A (en) | Inversion method and device for atmospheric methane concentration | |
| CN106875037B (en) | Wind power prediction method and device | |
| US20220036223A1 (en) | Processing apparatus, processing method, and non-transitory storage medium | |
| CN111288973B (en) | Method and device for obtaining flow rate of sea surface, computer equipment and storage medium | |
| CN112069955B (en) | Typhoon intensity remote sensing inversion method based on deep learning | |
| CN116046692B (en) | Soil heavy metal pollution monitoring method and device based on hyperspectrum | |
| CN112446397A (en) | Grass yield estimation method and device based on remote sensing and random forest and storage medium | |
| CN113361194B (en) | Sensor drift calibration method based on deep learning, electronic equipment and storage medium | |
| CN112326882B (en) | Air quality sensor processing method and device | |
| CN115841193A (en) | Method and device for predicting forest pests | |
| CN115687854B (en) | High-precision soil sample parameter measuring method and system thereof | |
| CN114691661B (en) | Assimilation-based cloud air guide and temperature and humidity profile pretreatment analysis method and system | |
| CN109187898B (en) | Soft measurement method and device for ammonia nitrogen content of water in aquaculture environment | |
| CN117153297A (en) | Cement concrete compressive strength detection method, system and electronic equipment | |
| CN115980755A (en) | Storm approach extrapolation method and device based on ConvLSTM | |
| CN116415488A (en) | Soil humidity prediction method and device, electronic equipment and storage medium | |
| CN105954206B (en) | The measurement method and system of purple maize leaf anthocyanin content | |
| CN116027459A (en) | Calculation method of atmospheric weighted average temperature based on numerical weather forecast data | |
| CN115855860A (en) | Method for predicting moisture content in tea drying process and processing terminal | |
| CN114509390A (en) | Soil organic matter prediction method, device, equipment and storage medium | |
| Kesavavarthini et al. | Bias correction of CMIP6 simulations of precipitation over Indian monsoon core region using deep learning algorithms | |
| CN113779837B (en) | Structural modal sensitivity analysis method based on novel actual measurement normalization technology | |
| CN117670873B (en) | Thermal comfort index detection method, thermal comfort index detection device, computer equipment and storage medium | |
| CN117391221B (en) | NDVI prediction integrated optimization method and system based on machine learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211109 |