CN117540201A - Carbon emission estimation method, neural network model, training method and sky ground three-dimensional carbon monitoring method - Google Patents
Carbon emission estimation method, neural network model, training method and sky ground three-dimensional carbon monitoring method Download PDFInfo
- Publication number
- CN117540201A CN117540201A CN202310262728.0A CN202310262728A CN117540201A CN 117540201 A CN117540201 A CN 117540201A CN 202310262728 A CN202310262728 A CN 202310262728A CN 117540201 A CN117540201 A CN 117540201A
- Authority
- CN
- China
- Prior art keywords
- data
- earth surface
- characteristic data
- carbon emission
- province
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 title claims abstract description 92
- 229910052799 carbon Inorganic materials 0.000 title claims abstract description 92
- 238000012544 monitoring process Methods 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000012549 training Methods 0.000 title claims abstract description 53
- 238000003062 neural network model Methods 0.000 title claims abstract description 22
- 210000002569 neuron Anatomy 0.000 claims abstract description 11
- 230000001174 ascending effect Effects 0.000 claims abstract description 3
- 238000012360 testing method Methods 0.000 claims description 24
- 238000002310 reflectometry Methods 0.000 claims description 22
- 238000007781 pre-processing Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 12
- 238000011156 evaluation Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 12
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical compound O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 description 10
- 230000008859 change Effects 0.000 description 10
- 238000013528 artificial neural network Methods 0.000 description 7
- 239000007789 gas Substances 0.000 description 6
- 229910002092 carbon dioxide Inorganic materials 0.000 description 5
- 238000010276 construction Methods 0.000 description 5
- 238000009826 distribution Methods 0.000 description 5
- 230000002194 synthesizing effect Effects 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 239000003344 environmental pollutant Substances 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 239000005431 greenhouse gas Substances 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000012876 topography Methods 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 239000001569 carbon dioxide Substances 0.000 description 2
- 239000004568 cement Substances 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000002803 fossil fuel Substances 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000004298 light response Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012806 monitoring device Methods 0.000 description 2
- 231100000719 pollutant Toxicity 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 229910052760 oxygen Inorganic materials 0.000 description 1
- 239000001301 oxygen Substances 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Abstract
The invention discloses a carbon emission estimation method, a neural network model, a training method and a sky-ground three-dimensional carbon monitoring method, which relate to the field of carbon emission estimation and have the technical scheme that: the carbon emission estimation model training method is characterized in that the deep neural network model comprises a plurality of layers and is built in a mode of ascending dimension and descending dimension; the number of neurons per layer is a power of 2. The purpose of improving training efficiency is achieved.
Description
Technical Field
The invention relates to the field of carbon emission evaluation, in particular to a carbon emission evaluation method, a neural network model, a training method and a sky-ground three-dimensional carbon monitoring method.
Background
The method is necessary to carry out sky ground carbon monitoring and long time sequence China carbon change analysis, and provide grid scale CO2 emission data of global and whole China areas and key carbon emission monitoring areas for monitoring and controlling greenhouse gases. In the prior art, a method for estimating the carbon emission by using satellite remote sensing data is not available.
Disclosure of Invention
In a first aspect, the present invention aims to provide a method for training a carbon emission estimation model.
The technical aim of the invention is realized by the following technical scheme: the carbon emission estimation model training method is characterized in that the deep neural network model comprises a plurality of layers and is built in a mode of ascending dimension and descending dimension; the number of neurons per layer is a power of 2.
Further, the method comprises the step of preprocessing the data, wherein the data preprocessing method comprises the following steps: a, acquiring multi-source data, wherein the multi-source data are all provincial data, and comprise night light data, a digital elevation model, earth surface coverage type data, earth surface reflectivity data, earth surface temperature data, vegetation data and carbon emission data; b, preprocessing data, namely processing multi-source data to obtain characteristic data values of each province; the characteristic data comprise light index characteristic data, elevation characteristic data, terrain complexity characteristic data, earth surface coverage type characteristic data, earth surface reflectivity characteristic data, earth surface temperature characteristic data, vegetation index characteristic data and carbon emission characteristic data; taking the average value of the night light data of each province as the characteristic data of the light index of each province; calculating the average elevation value of each province through a digital elevation model to be used as elevation characteristic data of each province; taking each pixel point of the digital elevation model as a center, taking 2-5 pixel points as radiuses, calculating entropy values of pixel values as terrain complexity characteristics of the pixel points, and taking an average value of the terrain complexity characteristics as terrain complexity characteristic data of each province; after the earth surface coverage type data are acquired, assigning values to different earth surface coverage types, and calculating the average value of pixel values of the earth surface coverage types of each province to be used as earth surface coverage characteristic data of each province; after the earth surface reflectivity data are obtained, cloud pixels are removed, the annual average value of the wave band is synthesized, and characteristic data of average reflectivity of each province are obtained; after the earth surface temperature data are acquired, calculating an earth surface temperature average value of each province of the year to be used as earth surface temperature characteristic data of each province; after vegetation data are acquired, calculating average pixel values of all the provinces as vegetation index characteristic data of all the provinces; and calculating the annual carbon emission data of each province by using the published carbon emission data statistics set of each province, and taking the data as characteristic data of the carbon emission data of each province.
Further, the nationwide characteristic data is divided into four data sets by taking province as a unit; training by using the four data sets respectively; obtaining four deep neural network models; the four models are then integrated in an averaged fashion.
Further, the total of seven levels, five of which are hidden layers; from the input layer to the output layer, the number of neurons per layer is 18, 32, 16, 8, and 1, respectively.
Further, the cost function of the model is a mean square error; the activation function of each layer is ReLU; the optimization algorithm is an Adam optimization algorithm; the initial value of the learning rate is 0.001; the learning rate is reduced to 95% of the original learning rate through self-adaptive adjustment of the learning rate, namely when the cost function of the test set is not reduced for 15 times.
Further, the batch size is set to be the size of the whole training set, the iteration round number is 10000, parameters at the end of each round are saved, and final parameters of the model are determined according to the parameters corresponding to the lowest test set cost function.
In a second aspect, the invention provides a neural network model for estimating carbon emissions, which is obtained by training by any one of the methods.
In a third aspect, the present invention provides a carbon emission estimation method comprising the steps of: a, acquiring multi-source data, wherein the multi-source data are all provincial data, and comprise night light data, a digital elevation model, earth surface coverage type data, earth surface reflectivity data, earth surface temperature data, vegetation data and carbon emission data; b, preprocessing data, namely processing multi-source data to obtain characteristic data values of each province; the characteristic data comprise light index characteristic data, elevation characteristic data, terrain complexity characteristic data, earth surface coverage type characteristic data, earth surface reflectivity characteristic data, earth surface temperature characteristic data and vegetation index characteristic data; taking the average value of the annual night light data of the area as the annual light index characteristic data of the area; calculating an average elevation value of the region through a digital elevation model, and taking the average elevation value as elevation characteristic data of the region; taking each pixel point of the digital elevation model as a center, taking 2-5 pixel points as radiuses, calculating entropy values of pixel values as terrain complexity characteristics of the pixel points, and taking an average value of the terrain complexity characteristics as terrain complexity characteristic data of the region; after the earth surface coverage type data are obtained, assigning values to different earth surface coverage types, and calculating the average value of pixel values of the earth surface coverage types of the region to be used as earth surface coverage characteristic data of the region; after the earth surface reflectivity data are obtained, cloud pixels are removed, the annual average value of the wave band is synthesized, and the characteristic data of the regional average reflectivity are obtained; after the earth surface temperature data are acquired, calculating an earth surface temperature average value of the area in the year to be used as earth surface temperature characteristic data of the area; after vegetation data are acquired, calculating an average pixel value of the region as vegetation index characteristic data of the region; c model estimation, inputting characteristic data values of each province into the carbon emission estimation neural network model according to claim 7, and obtaining carbon emission prediction data of the region in the year.
In a fourth aspect, the present invention provides a carbon emission estimation device, configured to implement a carbon emission estimation method as described above, including a data acquisition module, a data preprocessing module, and a model estimation module; the data acquisition module is used for acquiring multi-source data; the data preprocessing module is used for realizing data preprocessing; the model estimation module is used for receiving the preprocessed data and realizing the estimation of the carbon emission data.
In a fifth aspect, the present invention provides a carbon emission evaluation method, including comprehensive evaluation of multiple source data, where the multiple source data includes evaluation data, unmanned aerial vehicle monitoring data, and foundation monitoring data, and the method includes the following steps: a, estimating the carbon emission amount of each region by the carbon emission estimation method as described above; b, planning a route of the unmanned aerial vehicle through estimated data, and realizing multipoint sampling through monitoring equipment on the unmanned aerial vehicle; and C, selecting a monitoring site through the estimated data, and realizing fixed-point sampling by using equipment of the ground monitoring site.
In summary, the invention has at least one of the following advantages:
1. high precision: by training the deep neural network model, complex nonlinear relations can be learned and simulated, carbon emission can be estimated more accurately, and estimation accuracy is improved.
2. Capacity of strengthening: since the deep neural network model has a strong generalization capability, more data and variables can be processed, and a better predictive capability can be exhibited when facing new data.
3. High efficiency: the model is built in a dimension lifting and reducing mode, so that the feature quantity can be reduced, and the training efficiency of the model is improved.
4. Self-adaption: the number of neurons in each layer is the power of 2, and the design can enable the model to adapt to the scale of input data, so that the performance of the model is prevented from being reduced due to excessive or insufficient data.
5. The accuracy of the estimation can be improved by the multi-source data.
Drawings
Deep neural network model shown in FIG. 1
FIG. 2 shows the change in learning rate of 4 models during training
FIG. 3 shows the change in training process of training set and test set of 4 models
FIG. 4 is a schematic diagram illustrating a carbon emission monitoring method according to an embodiment of the present invention
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects to be solved more clear, the invention is further described in detail below with reference to the accompanying drawings and embodiments.
Example 1:
the embodiment provides a carbon emission deep neural network estimation model and a training method thereof. And (5) performing model training and super-parameter adjustment by adopting a deep neural network integrated model. In order to obtain the optimal model performance, experiments are continuously carried out and the results under different super-parameter conditions are compared. Experiments have determined that the final model has 6 levels. After the model receives the input data, the data is first subjected to an up-scaling operation to extract 32 features. The dimensions are then gradually reduced until the features are finally compressed into 1 dimension to obtain the final output result.
Model construction
The deep neural network structure adopted in the embodiment is a fully-connected network structure, and because the plasticity of the fully-connected network structure is very high, no absolute rules for network construction and super-parameter adjustment exist so far. The present embodiment follows the following rules when building a network structure: 1. firstly, increasing the dimension and then decreasing the dimension, namely, increasing the number of neurons of each layer by layer from the original characteristic number at the front part of the network, and decreasing the number of neurons of each layer by layer at the rear part of the network until the shrinkage is 1; 2. the number of neurons of the adjacent layers is prevented from suddenly changing, and the slow change process is kept as much as possible; 3. to avoid blindly searching the number of neurons, the number of neurons per layer is set to the power of 2. The cost function is set to mean square error (Mean Square Error, MSE) and the activation function for each layer is set to ReLU (Rectified LinearUnit). To improve training efficiency, adam optimization algorithm is used instead of standard gradient descent algorithm.
The super-parameter is the learning rate, and the embodiment takes 0.001 as the initial learning rate, and then adaptively changes the learning rate, that is, if the cost function of the test set is not reduced for 15 times in the training process, the learning rate is reduced to 95% of the original learning rate. In addition, the batch size is set to be the size of the whole training set, the iteration round number is 10000, parameters of each round of ending are saved in the training process, and finally, the corresponding parameters of the test set with the lowest cost function are determined to be final parameters of the model.
Training rules
In order to avoid the dependence of the model on single data segmentation and at the same time increase the stability of the model, the present embodiment first generates 4 sets of data sets, each set of test sets not overlapping, using the rules of cross-validation. And then training four deep neural network models by using the four different data sets respectively according to the characteristic standardization, model construction and super-parameter adjustment methods introduced above. The four optimal models are then integrated in an average manner. In this way, the stability of the model is enhanced.
The deep neural network model of the study was written using a TensorFlow framework and accelerated by a Graphics Processor (GPU) NVIDIATeslaV 100.
Sources of data sets
The data set in this embodiment includes feature data of each province in a province as a basic unit. The characteristic data are obtained by preprocessing multi-source data of each province. The multi-source data includes: including night light data, digital elevation model, surface coverage type data, surface reflectivity data, surface temperature data, vegetation data, and carbon emission data. The characteristic data comprises light index characteristic data, elevation characteristic data, terrain complexity characteristic data, earth surface coverage type characteristic data, earth surface reflectivity characteristic data, earth surface temperature characteristic data, vegetation index characteristic data and carbon emission characteristic data.
The night light data source and the pretreatment method are as follows: the widely used sources of night light data are two satellite remote sensing sensors of DMSP-OLS and NPP-VIIRS, and the two sensors have differences in spatial resolution and light response. Calibration is required to obtain continuous night light data of the same resolution. Specifically, the correction process includes the steps of:
1. DMSP-OLS and NPP-VIIRS data from 2013 were converted into Arabidopsis-equal-product cone projections in order to place the data from both sensors in the same projection coordinate system for subsequent processing. Meanwhile, the data of the vcm version of NPP-VIIRS (namely pixels affected by stray light are directly removed), and the effective observation count (cf_ cvg file) is used for assisting in calculating the annual average product.
2. Because of the different spatial resolutions of the two sensors, it is necessary to resample the NPP-VIIRS data to 1km resolution to be consistent with the DMSP-OLS data. To remove temporal illumination and background noise in the NPP-VIIRS data, DMSP-OLS lamplight masks are used to extract the NPP-VIIRS data.
3. And using the variation coefficient as a tool to select a calibration field, and solving pixels with uniform light radiation in space and stable time during the passing of the two sensors through traversal calculation. Then, based on the model, a mutual calibration model of the two sensors is established, and mutual calibration and calibration of night lamplight data are completed, so that lamplight change conditions of different areas can be reflected more accurately. The average value of night light data of each province in China in 2001-2021 is extracted and used as the characteristic data of the light index of each province in China.
The method for acquiring and preprocessing the elevation feature data and the terrain complexity feature data comprises the following steps: and extracting the high Cheng Ji terrain complexity of each province in China through the administrative division and the SRTM DEM data in the province in China. Firstly, calculating the average elevation of each province in China, so as to obtain corresponding elevation characteristic data. Secondly, using each pixel as a middle point, using three pixel lengths as radiuses to draw circles, calculating the entropy (entropy) of the pixel value in each circle, and obtaining the confusion degree of the elevation value of the local area so as to represent the terrain complexity characteristic data of the area where the central pixel is located. The terrain complexity calculation formula is as follows:
wherein entropy is the entropy of the pixels in the circle, I is the number of pixels, P i Is the pixel value.
The surface coverage type characteristic data acquisition and preprocessing method comprises the following steps: in order to prevent the model from being easy to fit due to excessive characteristics, MCD12Q1.006 products are selected, and the pixel duty ratio of various ground object coverage types in each province of China is obtained by reclassifying. And 6 types of surface coverage types (water, forest, grassland, crops, city and construction land and others) are selected for feature extraction. Respectively assigning values (0-5) to six types of ground object coverage types, and finally counting pixel values of various types of ground surface coverage types in each province through administrative division in each province of China and calculating the proportion of the ground surface coverage types; and calculating the average value of the pixel values of the earth coverage types of each province as earth coverage characteristic data of each province.
The surface reflectivity characteristic data is obtained and preprocessed as follows: and selecting MOD09A1.006 products in 2001-2021, removing cloud pixels on the image, synthesizing an annual average value of a wave band, and obtaining average surface reflectivity characteristic data of each province.
The surface temperature characteristic data is obtained and preprocessed as follows: MOD11A2.006 product is selected, and the update period of the product is 8 days. Firstly, synthesizing the data in an annual scale, and secondly, counting annual average surface temperature characteristics of each province through administrative division of each province in China; the average value of the surface temperatures of each province in the year is calculated to be the surface temperature characteristic data of each province.
The vegetation index characteristic data is obtained and preprocessed as follows: MOD13A2.006 product was selected and the refresh period of the product was 16 days. Firstly, synthesizing the data in an annual scale, and calculating the average pixel value of each province as vegetation index characteristic data of each province.
The carbon emission characteristic data finally draws Chinese provincial carbon emission statistics of 2001-2021 by using published provincial carbon emission statistics data sets including 17 fossil fuel emissions burned by 47 socioeconomic departments and emissions of cement production industry; the data are used as the characteristic data of carbon emission in each province.
All data will be divided into four data sets in units of provinces. And respectively inputting the four data sets into a model for training according to the method.
Training effect:
according to the method for configuring the deep neural network and the thought of super-parameter adjustment, the deep neural network model shown in fig. 1 is finally constructed by continuously comparing the performances of the model under different super-parameters. The model has 6 layers, after the data of the input layer are obtained, the model firstly performs dimension-increasing operation, 32 features are extracted, and then the dimension is gradually reduced until the dimension is finally compressed into 1 dimension. The corresponding configuration and super parameters are shown in the table.
Configuration and super-parameters of finally determined deep neural network model
The self-adaptive learning rate is dynamically changed in the training process, the self-adaptive learning rate adjustment endurance and the self-adaptive learning rate adjustment percentage are that when the RMSE on the test set is continuously reduced by 15 rounds, the learning rate is reduced to 95% and the deep neural network model with the configuration and the super parameters is applied to 4 groups of different data sets generated before for training and testing, and in the iteration process of 10000 rounds, the learning rate is automatically reduced to 95% when the RMSE on the test set is difficult to be reduced due to the adoption of the self-adaptive learning rate strategy, so that the learning rate is more approximate to the local optimal point. Fig. 2 shows the change of learning rate of the 4 models in the training process, and the first significant change of learning rate of the four models occurs between [1500,4000], and then the learning rate gradually decreases. Taking model 1 as an example, when the number of iteration rounds is between [1,1832], the learning rate is 0.001 as an initial value, and the initial learning rate is unchanged, which indicates that the initial learning rate can support efficient learning of the model at this stage. However, when the number of iteration rounds exceeds 1832, the learning rate begins to adaptively drop, indicating that the model is now approaching a local optimum, but the initial learning rate is relatively too large to be suitable for further performing the gradient descent, and thus the initial learning rate is reduced. And then the model continuously and simultaneously executes gradient descent and self-adaptive change of the learning rate, and gradually approaches to the local optimal point. Compared with the fixed learning rate, the learning rate self-adaptive strategy adopted in the study can enable the model to learn better, and has obvious advantages.
Fig. 3 shows the change in training between training and testing sets of 4 models. It can be seen from the graph that all curves are smooth and have no obvious oscillation, and when the model is trained to the optimal position, the difference between the performance on the training set and the performance on the test set is not very remarkable, so that the training process of the model is ideal.
Fig. 3 shows model 1 training and testing curves (a, b), model 2 training and testing curves (c, d), model 3 training and testing curves (e, f), and model 4 training and testing curves (g, h). Train represents performance on the training set, test represents performance on the test set; the points identify the locations of the lowest and highest occurrences on the test set.
The following table shows R of 4 optimal models on training set and test set 2 And RMSE, and their average performance. Their average R over the training set 2 An average RMSE over training set of 148.184 ± 36.768 and an average R over test set of 0.958±0.017 2 The average RMSE over the test set was 215.933 ± 18.327 at 0.913±0.004, indicating that the neural network is well able to establish a regression relationship between the geochemical characteristics and carbon emissions of this embodiment.
Table 4.84 evaluation index and average value of the optimal neural network on training set and test set
Example 2:
as shown in fig. 4, the present embodiment provides a carbon monitoring method based on sky-ground multi-source data. A carbon monitoring method for performing top-down carbon monitoring, bottom-up carbon monitoring and unmanned aerial vehicle key area monitoring in a three-in-one manner is developed. By utilizing the method of the embodiment, a seamless grid-scale carbon emission and energy intensity data set of the national region is produced, and carbon monitoring of a key gas pollution region in a small region range is realized. Can provide powerful technical support and support for monitoring and controlling greenhouse gases.
S1.1, cloud identification.
In the cloud identification process, the earth's surface is first divided into land and sea, and cloud detection is performed on them separately. For different wave bands above land and sea, such as blue light wave band, near infrared wave band and oxygen A absorption wave band, there is a great difference in cloud detection threshold under different areas and times. Therefore, we have established bright earth reservoirs, both land and sea, including desert, bare soil and ice and snow covered earth reservoirs, to more accurately detect clouds. By performing cloud detection and mask pretreatment on the data and land water, we can acquire Liu Deyun pixels and realize cloud removal.
S1.2, continuously correcting night lamplight data.
The widely used sources of night light data are two satellite remote sensing sensors of DMSP-OLS and NPP-VIIRS, and the two sensors have differences in spatial resolution and light response. Calibration is required to obtain continuous night light data of the same resolution. Specifically, the correction process includes the steps of:
1) DMSP-OLS and NPP-VIIRS data from 2013 were converted into Arabidopsis-equal-product cone projections in order to place the data from both sensors in the same projection coordinate system for subsequent processing. Meanwhile, the data of the vcm version of NPP-VIIRS (namely pixels affected by stray light are directly removed), and the effective observation count (cf_ cvg file) is used for assisting in calculating the annual average product.
2) Because of the different spatial resolutions of the two sensors, it is necessary to resample the NPP-VIIRS data to 1km resolution to be consistent with the DMSP-OLS data. To remove temporal illumination and background noise in the NPP-VIIRS data, DMSP-OLS lamplight masks are used to extract the NPP-VIIRS data.
3) And using the variation coefficient as a tool to select a calibration field, and solving pixels with uniform light radiation in space and stable time during the passing of the two sensors through traversal calculation. Then, based on the model, a mutual calibration model of the two sensors is established, and mutual calibration and calibration of night lamplight data are completed, so that lamplight change conditions of different areas can be reflected more accurately.
S1.3, preprocessing multi-source data.
When acquiring CO2 concentration values for a continuous grid scale, feature extraction is required for a plurality of data. These data include: night light data, a digital elevation model, surface coverage type data of MODIS, surface reflectivity data, surface temperature data and normalized vegetation index data.
a. Night light data: the average value of night light data of each province in China in 2001-2021 is extracted and used as the annual light index characteristic of each province in China.
b. Digital elevation model: and extracting the high Cheng Ji terrain complexity of each province in China through the administrative division and the SRTM DEM data in the province in China. Firstly, calculating the average elevation of each province in China, so as to obtain the corresponding elevation characteristics. Secondly, we draw a circle by taking each pixel as a middle point and three pixel lengths as radiuses, and calculate the entropy (entropy) of the pixel values in each circle to obtain the confusion degree of the elevation values of the local area, so as to represent the feature of the terrain complexity of the area where the central pixel is located.
c. Surface coverage type data. In order to prevent the model from being easy to fit due to excessive characteristics, MCD12Q1.006 products are selected, and the pixel duty ratio of various ground object coverage types in each province of China is obtained by reclassifying. And 6 types of surface coverage types (water, forest, grassland, crops, city and construction land and others) are selected for feature extraction. And (3) respectively assigning values (0-5) for the six types of ground object coverage types, and finally counting pixel values of various ground surface coverage types in each province through administrative division in each province of China and calculating the proportion of the ground surface coverage types.
d. Surface reflectance data. MOD09A1.006 products in 2001-2021 are selected, cloud pixels on the images are removed, the annual average value of each wave band is synthesized, and the characteristic of the earth surface reflectivity of each province is counted by the administrative boundaries of each province in China.
e. Surface temperature data. MOD11A2.006 product is selected, and the update period of the product is 8 days. Firstly, synthesizing the data in the annual scale, and secondly, counting the annual average surface temperature characteristics of each province through administrative division of each province in China.
f. And normalizing the vegetation index data. MOD13A2.006 product was selected, which had a 16 day refresh period. Firstly, synthesizing the data in annual scale, and secondly, counting the normalized vegetation index characteristics of each province through administrative division of each province in China.
Finally, the published provincial carbon emission statistical data set is used to draw Chinese provincial carbon emission statistical data from 2001 to 2021, wherein the published provincial carbon emission statistical data set comprises 17 fossil fuel emissions combusted by 47 social and economic departments and emissions of cement production industries.
S1.4, model training and super-parameter adjustment.
The invention adopts a deep neural network integrated model to carry out model training and super-parameter adjustment. In order to obtain the optimal model performance, experiments are continuously carried out and the results under different super-parameter conditions are compared. Experiments have determined that the final model has 6 levels. After the model receives the input data, the data is first subjected to an up-scaling operation to extract 32 features. The dimensions are then gradually reduced until the features are finally compressed into 1 dimension to obtain the final output result. For a specific method reference is made to example 1.
S1.5, model application.
The invention uses multisource data to preprocess the data, and then uses the administrative boundaries of the ground level city to extract the city characteristics from 2001 to 2021 from the image data. These city features are input into the deep neural network model, which calculates the energy intensity value and CO of each city each year after reasoning 2 Concentration values. Finally, the urban energy intensity and CO in the regions from 2001 to 2021 are obtained 2 Results of concentration values.
S2.1, unmanned aerial vehicle line planning.
After mastering the data of the areas with higher CO2 concentration and the industrial areas with lower energy utilization efficiency, unmanned aerial vehicle route planning is needed to realize the monitoring target. This requires consideration of the distribution of the monitored points and geographical environmental information such as terrain and climatic conditions. When planning a line, sufficient coverage of monitoring points is required to be ensured so as to ensure sufficient monitoring of greenhouse gases in an area and provide accurate data support for subsequent data analysis.
S2.2, unmanned aerial vehicle gas detection.
In order to ensure continuous monitoring of greenhouse atmospheric components at different points and collect data capable of meeting three-dimensional monitoring of greenhouse atmospheric components in an industrial area, a gas monitoring device is arranged on a flexible and mobile unmanned aerial vehicle. The gas monitoring device adopts an electrochemical gas sensor, has high measurement precision, higher gas anti-interference capability, stability and reliability, and can normally work under high altitude and severe weather conditions. By continuous monitoring in the air, enough data can be collected for data analysis, and monitoring of greenhouse atmosphere components is realized, so that powerful support is provided for environmental protection and carbon emission reduction.
S3.1, selecting a carbon monitoring site.
In order to ensure the accuracy of the carbon monitoring result, the selection of the point location is very critical. When selecting the monitoring point, multiple data sources including satellite data, model calculation data and ground measurement need to be comprehensively considered. By analyzing these data sources, the location and number of monitoring points are determined. The selection of the monitoring points should be random and cover areas with high concentration, low concentration and medium concentration. When specifically selecting the monitoring point, various factors such as topography, weather, population distribution and the like are also required to be considered so as to ensure the comprehensiveness and accuracy of the monitoring data.
S3.2, monitoring foundation carbon.
After the monitoring points are selected, the foundation carbon monitoring equipment is required to be transported to the points for instrument erection. In order to ensure the quality and accuracy of the data, an area with flat topography, wide space and no obvious pollution source needs to be selected for instrument erection. Before the instrument is set up, these areas need to be fully investigated and evaluated to ensure the reliability of the monitoring data. After the instrument is erected, the instrument needs to be calibrated and tested to ensure the accuracy and consistency of the monitoring data. And then sun tracking is carried out through control software, after focusing the sun, spectrum data acquisition is started, and information such as longitude and latitude, altitude, monitoring environment, sun surrounding environment in the sky and the like of the monitoring point position is recorded, so that detailed quality control basis information is provided for subsequent data inversion.
S3.3, carbon monitoring results and verification.
And collecting and analyzing the data collected by the monitoring points to finally obtain a complete carbon monitoring result. In the result analysis process, various factors such as topography, weather, population distribution and the like need to be comprehensively considered. Monitoring CO by remote sensing of foundation 2 、CH 4 The CO concentration data is analyzed by high and low values and combined with multiple superposition analysis of conditions such as zone location and the like, and the CO of each foundation measurement point is analyzed 2 Concentration profile superimposed to top-down produced seamless CO 2 On the grid plot of the concentration, a comparison of the results was made. Thus, S1 was verified to produce CO 2 Concentration data accuracy. Thus, the accuracy and reliability of the carbon monitoring result can be ensured, and powerful support is provided for environmental protection.
The embodiment trains the artificial intelligent model by using the provincial level characteristic data, and applies the artificial intelligent model to the estimation of the carbon emission of the city, the district and the county level. A more refined carbon emission estimation can be achieved. Providing a path planning reference opinion for unmanned aerial vehicle monitoring; providing point setting suggestions for foundation monitoring.
The embodiment can form a grid diagram of nationwide area high-precision, full-coverage and seamless carbon estimation by using the sky and ground three-in-one multi-source carbon monitoring data, thereby providing possibility for implementing carbon monitoring. However, the satellite can only monitor carbon emission in a massive region at present, and cannot completely and seamlessly cover a global region. According to the method, the carbon emission and the energy intensity are estimated in a seamless grid scale through multi-source satellite observation data represented by night lamplight, so that the blank of the data is made up, and the carbon emission data of any region can be obtained rapidly through a top-down carbon monitoring method compared with a traditional foundation measurement method. In order to improve and verify the accuracy of the top-down carbon estimation result, the method also adopts the technology of data inversion based on the remote sensing monitoring data of the foundation, and performs data collection and analysis on methane and carbon dioxide with the most obvious regional greenhouse effect, and performs identification and judgment on the methane and the carbon dioxide, thereby grasping the time-space distribution rule. Finally, an unmanned aerial vehicle key area monitoring method is designed aiming at the problems that in the atmospheric pollution monitoring, the vertical distribution of pollutants and the total concentration of areas cannot be monitored by a ground monitoring station. The monitoring terminal can acquire the concentration of the multi-parameter atmospheric pollutants, and is arranged on the flexible unmanned aerial vehicle to continuously monitor the atmospheric environmental pollutants at different points. The method has stable performance and high precision, is complementary with the existing ground monitoring network, and can meet the requirement of three-dimensional monitoring of the atmospheric pollution of an industrial area. Thereby, targeted policy suggestions are proposed to promote the realization of sustainable development targets.
The present embodiment is only for explanation of the present invention and is not to be construed as limiting the present invention, and modifications to the present embodiment, which may not creatively contribute to the present invention as required by those skilled in the art after reading the present specification, are all protected by patent laws within the scope of claims of the present invention.
Claims (10)
1. The carbon emission estimation model training method is characterized in that the deep neural network model comprises a plurality of layers and is built in a mode of ascending dimension and descending dimension; the number of neurons per layer is a power of 2.
2. The carbon emission estimation model training method according to claim 1, characterized in that: the data preprocessing method comprises the following steps:
a, acquiring multi-source data, wherein the multi-source data are all provincial data, and comprise night light data, a digital elevation model, earth surface coverage type data, earth surface reflectivity data, earth surface temperature data, vegetation data and carbon emission data;
b, preprocessing data, namely processing multi-source data to obtain characteristic data values of each province; the characteristic data comprise light index characteristic data, elevation characteristic data, terrain complexity characteristic data, earth surface coverage type characteristic data, earth surface reflectivity characteristic data, earth surface temperature characteristic data, vegetation index characteristic data and carbon emission characteristic data;
taking the average value of the night light data of each province as the characteristic data of the light index of each province;
calculating the average elevation value of each province through a digital elevation model to be used as elevation characteristic data of each province; taking each pixel point of the digital elevation model as a center, taking 2-5 pixel points as radiuses, calculating entropy values of pixel values as terrain complexity characteristics of the pixel points, and taking an average value of the terrain complexity characteristics as terrain complexity characteristic data of each province;
after the earth surface coverage type data are acquired, assigning values to different earth surface coverage types, and calculating the average value of pixel values of the earth surface coverage types of each province to be used as earth surface coverage characteristic data of each province;
after the earth surface reflectivity data are obtained, cloud pixels are removed, the annual average value of the wave band is synthesized, and the average earth surface reflectivity characteristic data of each province are obtained;
after the earth surface temperature data are acquired, calculating an earth surface temperature average value of each province of the year to be used as earth surface temperature characteristic data of each province;
after vegetation data are acquired, calculating average pixel values of all the provinces as vegetation index characteristic data of all the provinces;
and calculating annual carbon emission data of each province by using the published carbon emission data statistics set of each province, and taking the data as carbon emission characteristic data of each province.
3. The carbon emission estimation model training method according to claim 2, characterized in that: dividing the nationwide characteristic data into four data sets by taking province as a unit; training by using the four data sets respectively; obtaining four deep neural network models; the four models are then integrated in an averaged fashion.
4. A carbon emission estimation model training method according to claim 3, characterized in that: the total of seven levels, five of which are hidden layers; from the input layer to the output layer, the number of neurons per layer is 18, 32, 16, 8, and 1, respectively.
5. The carbon emission estimation model training method according to claim 4, characterized in that: the cost function of the model is mean square error; the activation function of each layer is ReLU; the optimization algorithm is an Adam optimization algorithm; the initial value of the learning rate is 0.001; the learning rate is reduced to 95% of the original learning rate through self-adaptive adjustment of the learning rate, namely when the cost function of the test set is not reduced for 15 times.
6. The carbon emission estimation model training method according to claim 5, characterized in that: the batch size is set to be the size of the whole training set, the iteration round number is 10000, parameters at the end of each round are saved, and final parameters of the model are determined by the corresponding parameters of the test set with the lowest cost function.
7. A carbon emission estimation neural network model is characterized in that: training obtained by the method according to any one of claims 2 to 6.
8. A carbon emission estimation method, characterized by comprising the steps of:
a, acquiring multi-source data, wherein the multi-source data are all provincial data, and comprise night light data, a digital elevation model, earth surface coverage type data, earth surface reflectivity data, earth surface temperature data, vegetation data and carbon emission data;
b, preprocessing data, namely processing multi-source data to obtain characteristic data values of each province; the characteristic data comprise light index characteristic data, elevation characteristic data, terrain complexity characteristic data, earth surface coverage type characteristic data, earth surface reflectivity characteristic data, earth surface temperature characteristic data and vegetation index characteristic data;
taking the average value of the annual night light data of the area as the annual light index characteristic data of the area;
calculating an average elevation value of the region through a digital elevation model, and taking the average elevation value as elevation characteristic data of the region; taking each pixel point of the digital elevation model as a center, taking 2-5 pixel points as radiuses, calculating entropy values of pixel values as terrain complexity characteristics of the pixel points, and taking an average value of the terrain complexity characteristics as terrain complexity characteristic data of the region;
after the earth surface coverage type data are obtained, assigning values to different earth surface coverage types, and calculating the average value of pixel values of the earth surface coverage types of the region to be used as earth surface coverage characteristic data of the region;
after the earth surface reflectivity data are obtained, cloud pixels are removed, the annual average value of the wave band is synthesized, and the characteristic data of the regional average reflectivity are obtained;
after the earth surface temperature data are acquired, calculating an earth surface temperature average value of the area in the year to be used as earth surface temperature characteristic data of the area;
after vegetation data are acquired, calculating an average pixel value of the region as vegetation index characteristic data of the region;
c model estimation, inputting characteristic data values of each province into the carbon emission estimation neural network model according to claim 7, and obtaining carbon emission prediction data of the region in the year.
9. A carbon emission estimation apparatus for implementing a carbon emission estimation method as defined in claim 8, characterized in that: the system comprises a data acquisition module, a data preprocessing module and a model estimation module;
the data acquisition module is used for acquiring multi-source data;
the data preprocessing module is used for realizing data preprocessing;
the model estimation module is used for receiving the preprocessed data and realizing the estimation of the carbon emission data.
10. The sky-ground three-dimensional carbon monitoring method is characterized by comprising multiple source data comprehensive evaluation, wherein the multiple source data comprises estimation data, unmanned aerial vehicle monitoring data and foundation monitoring data, and comprises the following steps of:
a, estimating the carbon emission amount of each region by the carbon emission estimation method according to claim 8;
b, planning a route of the unmanned aerial vehicle through estimated data, and realizing multipoint sampling through monitoring equipment on the unmanned aerial vehicle;
and C, selecting a monitoring site through the estimated data, and realizing fixed-point sampling by using equipment of the ground monitoring site.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310262728.0A CN117540201A (en) | 2023-03-17 | 2023-03-17 | Carbon emission estimation method, neural network model, training method and sky ground three-dimensional carbon monitoring method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310262728.0A CN117540201A (en) | 2023-03-17 | 2023-03-17 | Carbon emission estimation method, neural network model, training method and sky ground three-dimensional carbon monitoring method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117540201A true CN117540201A (en) | 2024-02-09 |
Family
ID=89781295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310262728.0A Pending CN117540201A (en) | 2023-03-17 | 2023-03-17 | Carbon emission estimation method, neural network model, training method and sky ground three-dimensional carbon monitoring method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117540201A (en) |
-
2023
- 2023-03-17 CN CN202310262728.0A patent/CN117540201A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108227041B (en) | Horizontal visibility forecasting method based on site measured data and mode result | |
| Asadi et al. | Simulation of green roofs and their potential mitigating effects on the urban heat island using an artificial neural network: A case study in Austin, Texas | |
| CN103063202B (en) | Cyanobacteria biomass spatial-temporal change monitoring and visualization method based on remote sensing image | |
| CN108399399B (en) | Urban range extraction method based on noctilucent remote sensing image | |
| CN110346517B (en) | Smart city industrial atmosphere pollution visual early warning method and system | |
| CN102507586B (en) | Remote sensing monitoring method for carbon emission | |
| Yang et al. | The impact analysis of water body landscape pattern on urban heat island: a case study of Wuhan City | |
| CN108872964B (en) | Ginkgo artificial forest canopy closure degree extraction method based on unmanned aerial vehicle LiDAR data | |
| Liu et al. | RS and GIS supported urban LULC and UHI change simulation and assessment | |
| CN104463168B (en) | A kind of useless house site information automation extraction method of sky based on remote sensing image | |
| CN112070056A (en) | Sensitive land use identification method based on object-oriented and deep learning | |
| CN114005048A (en) | Multi-temporal data-based land cover change and thermal environment influence research method | |
| CN116151610B (en) | Method for simulating risk exposure space of disaster-bearing body on underlying surface of heterogeneous city | |
| CN110765885B (en) | City expansion detection method and device based on heterogeneous luminous remote sensing image | |
| CN113987912A (en) | Pollutant on-line monitoring system based on geographic information | |
| Wu et al. | Influence of underlying surface change caused by urban renewal on land surface temperatures in Central Guangzhou | |
| Lian et al. | Sensitivity to the sources of uncertainties in the modeling of atmospheric CO 2 concentration within and in the vicinity of Paris | |
| CN115115224A (en) | Mining area vegetation carbon sink grading and change trend evaluation and analysis method | |
| CN116148188A (en) | Air-space-ground integrated lake water quality tracing method, system, equipment and storage medium | |
| CN115420690A (en) | Near-surface trace gas concentration inversion model and inversion method | |
| CN111879915B (en) | High-resolution monthly soil salinity monitoring method and system for coastal wetland | |
| CN117219183A (en) | High coverage near ground NO in cloudy rain areas 2 Concentration estimation method and system | |
| Lou et al. | Recognition of thaw slumps based on machine learning and UAVs: A case study in the Qilian Mountains, northeastern Qinghai-Tibet Plateau | |
| CN115983522B (en) | Rural habitat quality assessment and prediction method | |
| CN111723525A (en) | PM2.5 inversion method based on multi-source data and neural network model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |