CN115146181A - Time sequence-based time-space reconstruction method for chlorophyll A concentration product - Google Patents
Time sequence-based time-space reconstruction method for chlorophyll A concentration product Download PDFInfo
- Publication number
- CN115146181A CN115146181A CN202210651174.9A CN202210651174A CN115146181A CN 115146181 A CN115146181 A CN 115146181A CN 202210651174 A CN202210651174 A CN 202210651174A CN 115146181 A CN115146181 A CN 115146181A
- Authority
- CN
- China
- Prior art keywords
- chlorophyll
- concentration
- data
- time
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- ATNHDLDRLWWWCB-AENOIHSZSA-M chlorophyll a Chemical compound C1([C@@H](C(=O)OC)C(=O)C2=C3C)=C2N2C3=CC(C(CC)=C3C)=[N+]4C3=CC3=C(C=C)C(C)=C5N3[Mg-2]42[N+]2=C1[C@@H](CCC(=O)OC\C=C(/C)CCC[C@H](C)CCC[C@H](C)CCCC(C)C)[C@H](C)C2=C5 ATNHDLDRLWWWCB-AENOIHSZSA-M 0.000 title claims abstract description 175
- 229930002868 chlorophyll a Natural products 0.000 title claims abstract description 170
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000003062 neural network model Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims description 62
- 230000006870 function Effects 0.000 claims description 25
- 230000014759 maintenance of location Effects 0.000 claims description 22
- 238000000354 decomposition reaction Methods 0.000 claims description 21
- 238000002790 cross-validation Methods 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 238000012937 correction Methods 0.000 claims description 6
- 230000002123 temporal effect Effects 0.000 claims description 6
- 229930002875 chlorophyll Natural products 0.000 claims description 5
- 235000019804 chlorophyll Nutrition 0.000 claims description 5
- 230000007547 defect Effects 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000000873 masking effect Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 abstract description 2
- 230000008859 change Effects 0.000 description 6
- 230000007613 environmental effect Effects 0.000 description 5
- 238000011160 research Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 239000013535 sea water Substances 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 239000005422 algal bloom Substances 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013481 data capture Methods 0.000 description 2
- 238000013075 data extraction Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 241001420622 Meris Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000001311 chemical methods and process Methods 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000002844 melting Methods 0.000 description 1
- 230000008018 melting Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 229910052760 oxygen Inorganic materials 0.000 description 1
- 239000001301 oxygen Substances 0.000 description 1
- 230000029553 photosynthesis Effects 0.000 description 1
- 238000010672 photosynthesis Methods 0.000 description 1
- 239000000049 pigment Substances 0.000 description 1
- 238000011158 quantitative evaluation Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Investigating Or Analysing Biological Materials (AREA)
Abstract
The invention discloses a time-series-based time-space reconstruction method for chlorophyll A concentration products, which comprises the following steps: s1, downloading a target area long time sequence chlorophyll A concentration data product; s2, determining an initial data set of the chlorophyll A concentration; s3, removing chlorophyll A concentration data with the coverage rate smaller than a preset threshold value; s4, obtaining low-frequency information of a chlorophyll A concentration product; s5, extracting high-frequency information of the long-time-series-scale chlorophyll A concentration data; s6, carrying out data processing on the low-frequency information and the high-frequency information reconstructed in the steps S4 and S5 by using an NARX neural network model; s7, generating reconstructed chlorophyll A concentration data; s8, evaluating the fitting condition of the reconstructed lacking chlorophyll A concentration and an actual measurement value; and S9, generating a long-time chlorophyll A concentration reconstruction product set. The invention realizes the reconstruction of the chlorophyll A concentration product with large range and long time sequence.
Description
Technical Field
The invention relates to the technical field of remote sensing geography application, in particular to a time sequence-based chlorophyll A concentration product space-time reconstruction method.
Background
According to the evaluation report provided by the relevant departments of world climate change, the global air temperature is in a continuous rising state since 1840 years and shows an obvious acceleration trend, and the changes of the global climate cause the melting of ocean glaciers, the reduction of the salinity of seawater, the increase and decrease of the oxygen content of seawater and the abnormal ocean circulation, thereby deeply influencing the condition of the ocean ecological environment and changing the space-time pattern and the physical-to-chemical process of marine organisms. Compared with the marine environmental factors, the phytoplankton is more sensitive to climate change and responds more quickly. Research has shown that the change of marine physical environmental factors can affect the growth of phytoplankton, thus causing the phytoplankton to generate nonlinear response to a plurality of environmental factor variables, and further amplifying the originally weak disturbance effect of the environmental variables. Chlorophyll A is a main pigment for photosynthesis of seawater phytoplankton and is an important parameter for representing the concentration of the phytoplankton. The quantitative evaluation of chlorophyll A can comprehensively understand the change rule and influence factors of the marine phytoplankton and the response mechanism of the phytoplankton to the change of the marine environmental factors, and is very important.
Since the 70 s of the 20 th century, with the application of a series of ocean water color sensors with high signal-to-noise ratio and sensitivity, such as SeaWiFS, MODIS, MERIS, VIIRS and OLCI (optical line imaging), a series of chlorophyll A concentration synthetic products of multi-source remote sensing data are released by domestic and foreign scientific research institutions, and effective data support is provided for the development of phytoplankton high-time-space dynamic change research. However, considering the difference of the on-orbit time, the imaging mode, the time and the pixel resolution of the satellite sensor, the chlorophyll A concentration product of the multi-source remote sensing data has certain defects: the existing remote sensing chlorophyll A concentration synthesis product is influenced by cloud and rain weather, so that large area loss exists in space, and effective data and invalid data are not uniformly and irregularly distributed in time. For example, in the south China sea and the adjacent areas thereof, the current chlorophyll A concentration product is only covered by 60% -80% of data, and chlorophyll data loss exists in a large area, so that the research of a marine ecosystem is greatly limited by the data. Therefore, an efficient missing data reconstruction method needs to be developed, complete data products are supplemented to the maximum extent, and comprehensive utilization of multi-source remote sensing data is achieved.
An effective solution to the problems in the related art has not been proposed yet.
Disclosure of Invention
Aiming at the problems in the related art, the invention provides a time-series-based chlorophyll A concentration product space-time reconstruction method, so as to overcome the technical problems in the prior related art.
Therefore, the invention adopts the following specific technical scheme:
a time-series based chlorophyll A concentration product space-time reconstruction method comprises the following steps:
s1, downloading a target area length time sequence chlorophyll A concentration data product, and masking a target area by adopting an area of interest;
s2, evaluating the coverage and precision conditions of the chlorophyll A concentration data product on a target area, and determining an initial data set of the chlorophyll A concentration;
s3, removing chlorophyll A concentration data with the coverage rate smaller than a preset threshold value;
s4, reconstructing long-time-scale chlorophyll A concentration data by using a preset DINEOF reconstruction algorithm, and obtaining low-frequency information of a chlorophyll A concentration product;
s5, carrying out time series data prediction by using a BAR model, and extracting high-frequency information of chlorophyll A concentration data of a long time series scale;
s6, carrying out data processing on the low-frequency information and the high-frequency information reconstructed in the steps S4 and S5 by using an NARX neural network model;
s7, calling an ntol neural network tool box of the mATLAB to generate reconstructed chlorophyll A concentration data;
s8, comparing the reconstructed chlorophyll A concentration value with the actually measured chlorophyll A concentration, and evaluating the fitting condition of the reconstructed missing chlorophyll A concentration and the actually measured value;
and S9, generating a long-time chlorophyll A concentration reconstruction product set.
Further, the method for evaluating the coverage and accuracy of the chlorophyll A concentration data product on the target area comprises the following steps:
s21, counting the number of invalid values of the chlorophyll A concentration product, and dividing the number of the invalid values with the total number of the picture elements of the chlorophyll product to obtain a chlorophyll A concentration coverage value in each period;
and S22, converting the long time sequence data and the geographical coordinates of the actually-measured chlorophyll A concentration sampling point into pixel coordinates in a chlorophyll A concentration product, and performing linear fitting analysis on the actually-measured concentration value and the product value to obtain a correlation coefficient and a root mean square error.
Further, the step of reconstructing the long-time-scale chlorophyll A concentration data by using a preset DINEOF reconstruction algorithm and obtaining the low-frequency information of the chlorophyll A concentration product comprises the following steps:
s41, constructing a time-space chlorophyll A concentration data matrix X to be reconstructed 0 m*n Calculating a matrix X of pitch values for valid data points of all non-missing data m*n And setting a cross validation set X C ；
S42, performing iterative decomposition on chlorophyll A concentration data;
s43, calculating a minimum cross validation error and an optimal modal retention number;
s44, reconstructing a missing data area and extracting chlorophyll A concentration trend low-frequency information;
where m and n are the numbers in the spatial and temporal dimensions, respectively.
Further, the iterative decomposition of chlorophyll a concentration data comprises the steps of:
s421, pair matrix X m*n Performing first singular value decomposition, and setting the modal number to be 1 to obtain a matrix X;
s422, reconstructing the matrix X to obtain a matrix
And will cross-validate set X C And matrix
Carrying out comparison;
s423, the reconstructed matrix
Carrying out secondary decomposition to obtain the correction value of the missing point matrix
S424, repeating the step S42, setting the modal retention number p to be 1, and calculating the cross validation matrix X C Performing repeated iteration on the RMSE of the secondary reconstruction value and the real value;
s425, until the root mean square error is less than 1e of the previous root mean square error -4 At doubling, the iteration stops and the root mean square error converges.
Further, the calculation formula of the matrix X is:
X＝U m*n S m*n V n*n T
in the formula, matrixes Um m, sm n and Vn n are respectively a diagonal matrix and a time modal component which are formed by a characteristic mode and a singular value of a corresponding space dimension vector after SVD decomposition, and T is matrix transposition;
the matrix
The calculation formula of (2) is as follows:
in the formula, i is the spatial index of the matrix, j is the time index of the matrix, ut and vt are the t-th column of the spatial characteristic mode and the t-th column of the temporal characteristic mode, a t Is the t column singular value;
correction values of the defect point matrix
The calculation formula of (c) is:
in the formula, X re Represented as a new matrix at the first singular value decomposition reconstruction.
Further, the calculating the minimum cross validation error and the optimal modal retention number comprises the following steps:
s431, setting the value of the mode retention number P to 1,2,3, … K max And recording the root mean square error R corresponding to the retention number P values of different modes P ；
S432, repeating the step S42, and performing singular value decomposition and missing value reconstruction until the cross validation condition is met when the mode retention number is k;
and S433, in the results of the past iterations, a minimum RP value is always present, and the P value at the moment is taken as the optimal modal retention number k.
Further, the reconstructing the missing data area and extracting the chlorophyll A concentration trend low-frequency information comprises the following steps:
will cross-validate set X C Matrix values as cross-validation are restored to the original matrix X 0 m*n And reconstructing the data to obtain chlorophyll A concentration trend low-frequency information.
Further, the method for predicting time series data by using the BAR model and extracting high-frequency information of long-time-scale chlorophyll A concentration data comprises the following steps:
s51, evaluating the relevance between adjacent data of time series chlorophyll A concentration products by using a BAR model;
s52, reconstructing missing pixel data by using the mean value of the forward autoregressive and backward autoregressive data, and determining high-frequency information of chlorophyll A concentration;
wherein, the stable time series calculation formula of the BAR model is as follows:
in the formula, y T Is a stable time sequence of the autoregressive model;
p' is the optimal order of the autoregressive model;
ε T is a mean of 0 and a variance of σ 2 White gaussian noise.
Further, the data processing of the low-frequency information and the high-frequency information reconstructed in steps S4 and S5 by using the NARX neural network model includes the following steps:
s61, using a nonlinear fitting process function in the NARX neural network model, and taking the reconstructed chlorophyll A concentration data as a target vector;
s62, reconstructing chlorophyll A concentration data by taking low-frequency information of a DINEOF reconstructed long-time-sequence chlorophyll A concentration result and BAR model reconstructed concentration high-frequency information as input quantities;
wherein, the calculation formula of the target vector is as follows:
y(t)＝f(y(t-1),y(t-2),...y(t-n y ),u(t-1),u(t-2),...,u(t-n u ))
in the formula, f is a nonlinear fitting process function of the NARX neural network;
y (t) is the reconstructed chlorophyll A concentration data as a target vector;
u (t) is DINEOF reconstruction result and BAR reconstruction result data.
Further, the invoking of the ntol neural network toolbox of the mAtlAb to generate the reconstructed chlorophyll a concentration data comprises the following steps:
and generating reconstructed chlorophyll A concentration data by using a hyperboloid tangent S-shaped function as a transmission function of a hidden layer and an output layer, using a gradient descent momentum as a learning function and using a conjugate gradient function as a training function.
The invention has the beneficial effects that:
1. the relevant data extraction and vector conversion processes are realized through MATLAB programming and an ArcGIS platform, so that the manual participation is reduced, and the accuracy of a data reconstruction algorithm is improved.
2. The method is mainly innovated, and the time series low-frequency information of a DINEOF reconstruction algorithm and the high-frequency information of a BAR reconstruction model are effectively integrated. The invention can acquire data which is fully covered and reasonably distributed in space in different seasons, different spatial scales and different areas, and the detailed information of the data is more definite. In the aspect of practical application, the reconstructed data captures the oceanic extreme algal bloom of short time scale which cannot be captured by the original monthly scale data, and the validity of the reconstructed data is also proved.
3. The method combines a DINEOF reconstruction algorithm, a BAR reconstruction model and a NARX neural network model, aims at the problem of data loss of the chlorophyll A concentration product, integrates high-frequency information and low-frequency information of the chlorophyll A concentration product, and realizes the reconstruction of the chlorophyll A concentration product in a large-range and long-time sequence.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flow chart of a time series based spatiotemporal reconstruction method of chlorophyll A concentration products according to an embodiment of the present invention;
FIG. 2 is a product coverage map of chlorophyll A concentration data in south China sea area;
fig. 3 is a flow diagram of NARX neural network training;
FIG. 4 is a graph of the DINEOF algorithm results for 8-day chlorophyll A concentration product reconstitution;
FIG. 5 is a graph of the results of the 8-day chlorophyll A concentration product reconstructed by the present algorithm.
Detailed Description
For further explanation of the various embodiments, the drawings which form a part of the disclosure and which are incorporated in and constitute a part of this specification, illustrate embodiments and, together with the description, serve to explain the principles of operation of the embodiments, and to enable others of ordinary skill in the art to understand the various embodiments and advantages of the invention, and, by reference to these figures, reference is made to the accompanying drawings, which are not to scale and wherein like reference numerals generally refer to like elements.
According to an embodiment of the invention, a time-series based chlorophyll A concentration product space-time reconstruction method is provided.
The present invention will now be further described with reference to the accompanying drawings and specific embodiments, as shown in fig. 1, a time-series based method for spatiotemporal reconstruction of chlorophyll a concentration products according to an embodiment of the present invention comprises the following steps:
s1, downloading a target area long time sequence chlorophyll A concentration data product, reading a vector file by using a shape function of matlab, masking a target area by adopting an interested area, setting a land part as a null value, and reserving target area chlorophyll data;
specifically, the long-time chlorophyll A concentration data products are Chl1-GSM, chl1-AVW and ChlOC5.
S2, evaluating the coverage and precision conditions of the chlorophyll A concentration data product on a target area, and determining an initial data set of the chlorophyll A concentration;
s3, removing chlorophyll A concentration data with the coverage rate smaller than a preset threshold value; ensuring sufficient reconstructed sample data; the predetermined threshold is 5% -10%, preferably 10% -15%.
S4, reconstructing long-time-series scale (8 days in 1998-2018) chlorophyll A concentration data by using a preset DINEOF reconstruction algorithm, and obtaining low-frequency information of a chlorophyll A concentration product; and (4) calculating the DINEOF operation of the chlorophyll A concentration product until the maximum characteristic mode of the step S6 or the step S7.
S5, carrying out time series data prediction by using a BAR model, and extracting high-frequency information of chlorophyll A concentration data in a long time series scale (the scale of 8 days in 1998-2018);
s6, carrying out data processing on the low-frequency information and the high-frequency information reconstructed in the steps S4 and S5 by using an NARX neural network model; when DINEOF and BAR are used for reconstructing chlorophyll a data by using a NARX neural network model, the number of hidden layer nodes is 30. S7, calling an ntool neural network tool box of the mATLAB to generate reconstructed chlorophyll A concentration data; and randomly selecting 70% of the data as model training data, 15% of the data as verification data and 15% of the data as prediction data.
S8, comparing the reconstructed chlorophyll A concentration value with the actually measured chlorophyll A concentration, and evaluating the fitting condition of the reconstructed missing chlorophyll A concentration and the actually measured value; evaluating the space-time continuity and precision of the reconstruction result;
and S9, generating a long-time chlorophyll A concentration reconstruction product set.
In one embodiment, said evaluating the coverage and accuracy of the chlorophyll a concentration data product over the target area, determining an initial data set of chlorophyll a concentrations comprises the steps of:
s21, counting the invalid value number of the chlorophyll A concentration product by using a numel (fine (isnan (a)) command of matlab, and dividing the invalid value number by the total number of pixel elements of the chlorophyll product to obtain a chlorophyll A concentration coverage value in each period;
s22, converting the long time sequence data (NOMAD and Bio Argo) and the geographic coordinates of the actually-measured chlorophyll A concentration sampling points into pixel coordinates in a chlorophyll A concentration product by using a pix2mp function of matlab, and performing linear fitting analysis on the actually-measured concentration values and the product values to obtain a correlation coefficient and a root mean square error.
In one embodiment, the reconstructing long-time-scale chlorophyll a concentration data by using a preset DINEOF reconstruction algorithm and obtaining low-frequency information of a chlorophyll a concentration product includes the following steps:
s41, constructing a time-space chlorophyll A concentration data matrix X to be reconstructed 0 m*n Calculating a matrix X of pitch values for valid data points of all non-missing data m*n And setting a cross validation set X C ；
Specifically, the value of the mask parameter marked as missing data points in all DINEOF is set to 0, and the matrix X is 0 m*n Mean value in the time dimension of
Randomly taking data of 1% -3% of the total amount from the effective value as a cross validation set X C First, X is C The valid values in the matrix become invalid defect points and the values of these pixels are set to 0.
S42, performing iterative decomposition on chlorophyll A concentration data;
s43, calculating a minimum cross validation error and an optimal modal retention number;
s44, reconstructing a missing data area and extracting chlorophyll A concentration trend low-frequency information;
where m and n are the numbers in the spatial and temporal dimensions, respectively.
In one embodiment, the iterative decomposition of chlorophyll a concentration data comprises the steps of:
s421, pair matrix X m*n Performing first singular value decomposition, and setting the modal number to be 1 to obtain a matrix X;
s422, reconstructing the matrix X to obtain a matrix
And will cross-validate set X C And matrix
Carrying out comparison;
s423, the reconstructed matrix
Carrying out secondary decomposition to obtain the correction value of the missing point matrix
S424, repeating the step S42, setting the modal retention number p to be 1, and calculating the cross validation matrix X C Performing repeated iteration on the RMSE of the secondary reconstruction value and the real value;
s425, until the root mean square error is less than 1e of the previous root mean square error -4 At doubling, the iteration stops and the root mean square error converges.
In one embodiment, the matrix X is calculated by the formula:
X＝U m*n S m*n V n*n T
in the formula, matrixes Um m, sm n and Vn n are respectively a diagonal matrix and a time modal component which are formed by a characteristic mode and a singular value of a corresponding space dimension vector after SVD decomposition, and T is matrix transposition;
the matrix
The calculation formula of (2) is as follows:
in the formula, i is the spatial index of the matrix, j is the time index of the matrix, ut and vt are the t-th column of the spatial characteristic mode and the t-th column of the temporal characteristic mode, a t Is the t column singular value;
correction values of the defect point matrix
The calculation formula of (2) is as follows:
in the formula, X re Is shown as firstAnd decomposing a new matrix at the reconstruction position by using the secondary singular value.
In one embodiment, the calculating the minimum cross-validation error and the optimal modal retention number comprises the steps of:
s431, setting the value of the mode retention number P to 1,2,3, … K max And recording the root mean square error R corresponding to the retention number P values of different modes P ；
S432, repeating the step S42, and performing singular value decomposition and missing value reconstruction until the cross validation condition is met when the mode retention number is k;
and S433, in the results of the past iterations, a minimum RP value is always present, and the P value at the moment is taken as the optimal modal retention number k.
In one embodiment, the reconstructing the missing data region and extracting chlorophyll a concentration trending low frequency information comprises the steps of:
will cross-validate set X C Matrix values as cross-validation are restored to the original matrix X 0 m*n And reconstructing the data to obtain chlorophyll A concentration trend low-frequency information.
In one embodiment, the method for performing time series data prediction by using a BAR model and extracting high-frequency information of long-time scale chlorophyll a concentration data comprises the following steps:
s51, evaluating the relevance between adjacent data of time series chlorophyll A concentration products by using a BAR model;
s52, reconstructing missing pixel data by using the mean value of the forward autoregressive data and the backward autoregressive data, and determining high-frequency information of the chlorophyll A concentration;
wherein, the stable time series calculation formula of the BAR model is as follows:
in the formula, y T Is a stable time sequence of the autoregressive model;
p' is the optimal order of the autoregressive model;
ε T is a mean of 0 and a variance of σ 2 White gaussian noise. The correlation coefficient of the values obtained at adjacent times is 0.72, which is much greater than the 0.5 desired value.
In one embodiment, the data processing of the low frequency information and the high frequency information reconstructed in steps S4 and S5 by using the NARX neural network model includes the following steps:
s61, using a nonlinear fitting process function in the NARX neural network model, and taking the reconstructed chlorophyll A concentration data as a target vector;
s62, reconstructing chlorophyll A concentration data by taking low-frequency information of a DINEOF reconstructed long-time-sequence chlorophyll A concentration result and BAR model reconstructed concentration high-frequency information as input quantities;
wherein, the calculation formula of the target vector is as follows:
y(t)＝f(y(t-1),y(t-2),...y(t-n y ),u(t-1),u(t-2),...,u(t-n u ))
wherein f is a nonlinear fitting process function of the NARX neural network;
y (t) is the reconstructed chlorophyll A concentration data as a target vector;
u (t) is DINEOF reconstruction result and BAR reconstruction result data.
In one embodiment, the invoking the ntol neural network toolbox of the martlab to generate the reconstructed chlorophyll a concentration data comprises the steps of:
and generating reconstructed chlorophyll A concentration data by using a hyperboloid tangent S-shaped function as a transmission function of a hidden layer and an output layer, using a gradient descent momentum as a learning function and using a conjugate gradient function as a training function.
For the convenience of understanding the technical solutions of the present invention, the working principle or the operation mode of the present invention in the practical process will be described in detail below.
In practical application, the remote sensing product set of chlorophyll A concentration for 8 days is reconstructed based on multisource long time sequence data including ChlOC5, NOMAD, bio Argo and on-site actual measurement chlorophyll A concentration data. The method mainly comprises three parts: (1) evaluation of chlorophyll A concentration product quality: evaluating the space coverage value of the long-time sequence chlorophyll A concentration synthesis products (Chl 1-GSM, chl1-AVW and ChlOC 5) in the target area; and carrying out correlation analysis of the chlorophyll A concentration synthetic product with the NOMAD, the Bio Argo and the actually measured chlorophyll A concentration, and determining the selected chlorophyll A concentration product. And (2) extracting chlorophyll A concentration frequency information: performing iterative decomposition and synthesis on chlorophyll A concentration Data by using Data Interpolating Empirical Orthogonal Functions (DINEOF), calculating the minimum cross validation error and the optimal modal retention number, reconstructing a missing Data area and extracting chlorophyll A concentration trend low-frequency information; and (3) evaluating the relevance between adjacent data of time series chlorophyll A concentration products by using a Bidirectional Auto Regression (BAR) model, and reconstructing missing pixel data by using the mean value of forward Auto Regression data and backward Auto Regression data to determine the high-frequency information of the chlorophyll A concentration. (3) constructing a chlorophyll A concentration reconstruction product set: using a Levenberg-Marequidart function in a nonlinear autoregressive neural network model (NARX), taking the reconstructed chlorophyll A concentration data as a target quantity, taking low-frequency information of a DINEOF reconstructed long-time-sequence chlorophyll A concentration result and BAR reconstructed concentration high-frequency information as input quantities, and reconstructing a chlorophyll A concentration data product; and comparing the actually measured chlorophyll A concentration data with the reconstructed chlorophyll A concentration result, verifying the chlorophyll A concentration reconstruction algorithm precision, and obtaining a 21-year-old 8-day chlorophyll A concentration reconstruction data product set of 1998-2018.
In summary, by means of the above technical solution of the present invention, the relevant data extraction and vector conversion processes of the present invention are both implemented by MATLAB programming and ArcGIS platform, thereby reducing human involvement and improving the accuracy of the data reconstruction algorithm. The method is mainly innovated, and the time series low-frequency information of a DINEOF reconstruction algorithm and the high-frequency information of a BAR reconstruction model are effectively integrated. The invention can acquire data which is fully covered and reasonably distributed in space in different seasons, different spatial scales and different areas, and the detailed information of the data is more definite. In the aspect of practical application, the reconstructed data captures the oceanic extreme algal bloom of short time scale which cannot be captured by the original monthly scale data, and the validity of the reconstructed data is also proved. The method combines a DINEOF reconstruction algorithm, a BAR reconstruction model and a NARX neural network model, aims at the problem of data loss of the chlorophyll A concentration product, integrates high-frequency information and low-frequency information of the chlorophyll A concentration product, and realizes the reconstruction of the chlorophyll A concentration product in a large-range and long-time sequence.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A time-series based chlorophyll A concentration product space-time reconstruction method is characterized by comprising the following steps:
s1, downloading a target area length time sequence chlorophyll A concentration data product, and masking a target area by using an area of interest;
s2, evaluating the coverage and precision conditions of the chlorophyll A concentration data product on a target area, and determining an initial data set of the chlorophyll A concentration;
s3, removing chlorophyll A concentration data with the coverage rate smaller than a preset threshold value;
s4, reconstructing the long-time-scale chlorophyll A concentration data by using a preset DINEOF reconstruction algorithm, and obtaining low-frequency information of a chlorophyll A concentration product;
s5, carrying out time series data prediction by using a BAR model, and extracting high-frequency information of long-time-series-scale chlorophyll A concentration data;
s6, carrying out data processing on the low-frequency information and the high-frequency information reconstructed in the steps S4 and S5 by using an NARX neural network model;
s7, calling an ntol neural network tool box of the mATLAB to generate reconstructed chlorophyll A concentration data;
s8, comparing the reconstructed chlorophyll A concentration value with the actually measured chlorophyll A concentration, and evaluating the fitting condition of the reconstructed missing chlorophyll A concentration and the actually measured value;
and S9, generating a long-time chlorophyll A concentration reconstruction product set.
2. The method for spatiotemporal reconstruction of a chlorophyll-a concentration product based on time series according to claim 1, wherein said evaluating the coverage and accuracy status of chlorophyll-a concentration data product over a target area, determining an initial data set of chlorophyll-a concentration comprises the steps of:
s21, counting the number of invalid values of the chlorophyll A concentration product, and dividing the number of the invalid values with the total number of the picture elements of the chlorophyll product to obtain a chlorophyll A concentration coverage value in each period;
and S22, converting the long time sequence data and the geographical coordinates of the actually-measured chlorophyll A concentration sampling point into pixel coordinates in a chlorophyll A concentration product, and performing linear fitting analysis on the actually-measured concentration value and the product value to obtain a correlation coefficient and a root mean square error.
3. The time-series based chlorophyll A concentration product space-time reconstruction method of claim 1, wherein said reconstructing long-time-scale chlorophyll A concentration data using a preset DINEOF reconstruction algorithm and obtaining low-frequency information of chlorophyll A concentration product comprises the steps of:
s41, constructing a time-space chlorophyll A concentration data matrix X to be reconstructed 0 m*n Calculating a matrix X of pitch values for valid data points of all non-missing data m*n And setting a cross validation set X C ；
S42, performing iterative decomposition on chlorophyll A concentration data;
s43, calculating a minimum cross validation error and an optimal modal retention number;
s44, reconstructing a missing data area and extracting chlorophyll A concentration trend low-frequency information;
where m and n are the numbers in the spatial and temporal dimensions, respectively.
4. The time-series based chlorophyll A concentration product space-time reconstruction method of claim 3, wherein said iterative decomposition of chlorophyll A concentration data comprises the steps of:
s421, matrix X m*n Performing first singular value decomposition, and setting the modal number to be 1 to obtain a matrix X;
s422, reconstructing the matrix X to obtain a matrix
And will cross-validate set X C And matrix
Carrying out comparison;
s423, the reconstructed matrix
Carrying out secondary decomposition to obtain the correction value of the missing point matrix
S424, repeating the step S42, setting the modal retention number p to be 1, and calculating the cross validation matrix X C Performing repeated iteration on the RMSE of the secondary reconstruction value and the real value;
s425, until the root mean square error is smaller than the root mean square error of the previous time 1e -4 At doubling, the iteration stops and the root mean square error converges.
5. The time-series based chlorophyll A concentration product space-time reconstruction method of claim 4, wherein said matrix X is calculated by:
X＝U m*n S m*n V n*n T
in the formula, matrixes Um x m, sm x n and Vn x n are respectively a diagonal matrix and a time modal component which are formed by a characteristic mode and a singular value of a corresponding space dimension vector after SVD decomposition, and T is a matrix transposition;
the matrix
The calculation formula of (2) is as follows:
in the formula, i is the spatial index of the matrix, j is the time index of the matrix, ut and vt are the t-th column of the spatial characteristic mode and the t-th column of the temporal characteristic mode, a t Is the t column singular value;
correction values of the defect point matrix
The calculation formula of (2) is as follows:
in the formula, X re Represented as the new matrix at the first reconstruction of the singular value decomposition.
6. The time-series based chlorophyll A concentration product spatiotemporal reconstruction method according to claim 5, wherein said calculating minimum cross validation error and optimal modal retention number comprises the steps of:
s431, setting the value of the mode retention number P to 1,2,3, … K max And recording the root mean square error R corresponding to the retention number P values of different modes P ；
S432, repeating the step S42, and performing singular value decomposition and default value reconstruction until the cross validation condition is met when the mode retention number is k;
and S433, in the results of the past iterations, a minimum RP value is always present, and the P value at the moment is taken as the optimal modal retention number k.
7. The time-series based chlorophyll A concentration product space-time reconstruction method of claim 6, wherein said reconstructing missing data area and extracting chlorophyll A concentration trend low frequency information comprises the following steps:
will cross-validate set X C Matrix values as cross-validation are restored to the original matrix X 0 m*n And reconstructing the data to obtain the chlorophyll A concentration trend low-frequency information.
8. The time-series based chlorophyll A concentration product space-time reconstruction method of claim 1, wherein said using BAR model to develop time series data prediction and extracting long time-series scale chlorophyll A concentration data high frequency information comprises the steps of:
s51, evaluating the relevance between adjacent data of time series chlorophyll A concentration products by using a BAR model;
s52, reconstructing missing pixel data by using the mean value of the forward autoregressive data and the backward autoregressive data, and determining high-frequency information of the chlorophyll A concentration;
wherein, the stable time series calculation formula of the BAR model is as follows:
in the formula, y T Is a stationary time series of the autoregressive model;
p' is the optimal order of the autoregressive model;
ε T is a mean of 0 and a variance of σ 2 White gaussian noise.
9. The time-series based chlorophyll A concentration product space-time reconstruction method of claim 1, wherein said data processing of the low-frequency information and high-frequency information reconstructed in steps S4 and S5 by using NARX neural network model comprises the steps of:
s61, using a nonlinear fitting process function in the NARX neural network model, and taking the reconstructed chlorophyll A concentration data as a target vector;
s62, reconstructing chlorophyll A concentration data by taking low-frequency information of a DINEOF reconstructed long-time-sequence chlorophyll A concentration result and BAR model reconstructed concentration high-frequency information as input quantities;
wherein, the calculation formula of the target vector is as follows:
y(t)＝f(y(t-1),y(t-2),...y(t-n y ),u(t-1),u(t-2),...,u(t-n u ))
wherein f is a nonlinear fitting process function of the NARX neural network;
y (t) is the reconstructed chlorophyll A concentration data as a target vector;
u (t) is DINEOF reconstruction result and BAR reconstruction result data.
10. The time-series based chlorophyll A concentration product space-time reconstruction method of claim 1, wherein said calling the ntol neural network toolbox of the martlab, generating the reconstructed chlorophyll A concentration data comprises the following steps:
and generating reconstructed chlorophyll A concentration data by using a hyperboloid tangent S-shaped function as a transmission function of a hidden layer and an output layer, using a gradient descent momentum as a learning function and using a conjugate gradient function as a training function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210651174.9A CN115146181A (en) | 2022-06-10 | 2022-06-10 | Time sequence-based time-space reconstruction method for chlorophyll A concentration product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210651174.9A CN115146181A (en) | 2022-06-10 | 2022-06-10 | Time sequence-based time-space reconstruction method for chlorophyll A concentration product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115146181A true CN115146181A (en) | 2022-10-04 |
Family
ID=83408622
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210651174.9A Withdrawn CN115146181A (en) | 2022-06-10 | 2022-06-10 | Time sequence-based time-space reconstruction method for chlorophyll A concentration product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115146181A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117435867A (en) * | 2023-12-21 | 2024-01-23 | 中关村睿宸卫星创新应用研究院 | Method, device, medium and equipment for determining chlorophyll concentration change time law |
-
2022
- 2022-06-10 CN CN202210651174.9A patent/CN115146181A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117435867A (en) * | 2023-12-21 | 2024-01-23 | 中关村睿宸卫星创新应用研究院 | Method, device, medium and equipment for determining chlorophyll concentration change time law |
| CN117435867B (en) * | 2023-12-21 | 2024-03-08 | 中关村睿宸卫星创新应用研究院 | Method, device, medium and equipment for determining chlorophyll concentration change time law |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Alizamir et al. | On the implementation of a novel data-intelligence model based on extreme learning machine optimized by bat algorithm for estimating daily chlorophyll-a concentration: Case studies of river and lake in USA | |
| CN108108836A (en) | A kind of ozone concentration distribution forecasting method and system based on space-time deep learning | |
| CN112699959B (en) | Multi-source multi-scale precipitation data fusion method and device based on energy functional model | |
| CN116504330B (en) | Pollutant concentration inversion method and device, electronic equipment and readable storage medium | |
| CN116879192B (en) | Water bloom prediction method, device, equipment and medium based on satellite remote sensing data | |
| Chidepudi et al. | A wavelet-assisted deep learning approach for simulating groundwater levels affected by low-frequency variability | |
| CN111798132B (en) | Cultivated land dynamic monitoring method and system based on multi-source time sequence remote sensing depth cooperation | |
| CN115146181A (en) | Time sequence-based time-space reconstruction method for chlorophyll A concentration product | |
| Wang et al. | Multivariate DINEOF reconstruction for creating long-term cloud-free chlorophyll-a data records from SeaWiFS and MODIS: A case study in Bohai and Yellow Seas, China | |
| Arıman | Determination of inactive water quality variables by MODIS data: A case study in the Kızılırmak Delta-Balik Lake, Turkey | |
| CN117974444A (en) | Sea surface height super-resolution reconstruction method based on diffusion model | |
| Lin et al. | Optimizing plankton survey strategies using observing system simulation experiments | |
| Richit et al. | Modelling forest regeneration for performance-oriented riparian buffer strips | |
| CN117633502A (en) | Multi-factor quantitative inversion method for soil moisture supply | |
| CN116702981A (en) | Ozone concentration prediction method and device, electronic equipment and medium | |
| Di Noia et al. | Global tropospheric ozone column retrievals from OMI data by means of neural networks | |
| CN114047563A (en) | All-weather assimilation method of infrared hyperspectrum | |
| Selch | Comparing salinity models in Whitewater Bay using remote sensing | |
| CN117740690B (en) | Remote sensing estimation method, system and readable storage medium for dissolved oxygen of water meter | |
| Tanaka et al. | Verification of deforestation in East Asia by spatial logit models due to population and relief energy | |
| Salazar-Neira et al. | Above-ground biomass estimation based on multi-angular l-band measurements of brightness temperatures | |
| CN115511887B (en) | Drought index prediction method and device based on artificial intelligence fusion | |
| CN115859596B (en) | Space-time simulation method for soil heavy metal accumulation process in urban-suburban gradient area | |
| CN118626863A (en) | Time constraint-considered space-time seamless high-resolution leaf area index inversion method | |
| Zhang | Reconstruction of climate fields using machine-learning methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20221004 |
|
| WW01 | Invention patent application withdrawn after publication |