CN117036889A - MLP-based remote sensing image fusion method - Google Patents
MLP-based remote sensing image fusion method Download PDFInfo
- Publication number
- CN117036889A CN117036889A CN202311055805.1A CN202311055805A CN117036889A CN 117036889 A CN117036889 A CN 117036889A CN 202311055805 A CN202311055805 A CN 202311055805A CN 117036889 A CN117036889 A CN 117036889A
- Authority
- CN
- China
- Prior art keywords
- images
- resolution
- image
- mlp
- fine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 10
- 230000004927 fusion Effects 0.000 claims abstract description 15
- 238000001228 spectrum Methods 0.000 claims abstract description 5
- 238000012952 Resampling Methods 0.000 claims abstract description 4
- 230000003044 adaptive effect Effects 0.000 claims abstract description 4
- 102220298895 rs1025502215 Human genes 0.000 claims description 2
- 102220162169 rs575633576 Human genes 0.000 claims description 2
- 102220212642 rs747431847 Human genes 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 abstract description 3
- 238000011160 research Methods 0.000 abstract description 2
- 230000008859 change Effects 0.000 description 5
- 238000000034 method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
Abstract
The invention discloses a remote sensing image fusion method based on MLP, which comprises the following steps: 1. resampling the MODIS multispectral image with the spatial resolution of 500m to 480m so as to match the resolution of the image after spatial spectrum fusion; 2. taking the coarse resolution images at three moments t1, t2 and t3 and the fine resolution images at two moments t1 and t3 as inputs, and taking the fine resolution images at the moment t2 as fused output images; 3. calculating coarse difference images C12, C23 and C13 required by the StfNet-MLP space-time fusion model; 4. predicting fine difference images F12 and F23 from C12, C23 and C13 and F1 and F3; 5. and reconstructing the fine resolution image F2_pre at the time t2 by adopting an adaptive local weighting strategy in the combination module. The invention can avoid the unilateral performance of a single remote sensing image, thereby providing an important data basis for the long time sequence research of forest monitoring.
Description
Technical Field
The invention relates to a remote sensing image fusion method, in particular to a remote sensing image fusion method based on MLP.
Background
The change of the forest resources can reflect the growth condition of forest vegetation, detect the change of forest environment and measure the ecological quality, and is an important index for reflecting the richness of the forest resources and the ecological balance condition. The accurate acquisition of the forest vegetation coverage area and the quantification of the space-time variation of the vegetation coverage have important scientific significance for revealing the functional mechanism law of the forest ecological structure.
The forest growth period is slow, the characteristic of unobvious change in a short time is provided, and the analysis of the forest coverage area change is a long-term process, so that a long-time-sequence forest coverage remote sensing image set is necessary data for constructing a monitoring model. However, the remote sensing image acquired by a single sensor is limited in the current situation that the space-time resolution is not balanced, and the problem can be solved by utilizing the fusion of the multi-mode sensor and the multi-source remote sensing image.
Disclosure of Invention
Aiming at the current situation that the remote sensing image acquired by a single sensor is limited by the fact that the space-time resolution is not balanced, the invention provides a remote sensing image fusion method based on MLP.
The invention aims at realizing the following technical scheme:
a remote sensing image fusion method based on MLP comprises the following steps:
resampling the MODIS multispectral image with the spatial resolution of 500m to 480m to match the resolution of the image after spatial spectrum fusion;
step two, taking the coarse resolution images at three moments t1, t2 and t3 and the fine resolution images at two moments t1 and t3 as inputs, and taking the fine resolution images at the moment t2 as fused output images;
step three, calculating to obtain coarse difference images C12, C23 and C13 required by the StfNet-MLP space-time fusion model according to the following formula:
C12＝C2-C1
C13＝C3-C1
C23＝C3-C2
step four, predicting fine difference images F12 and F23 from the coarse difference images C12, C23 and C13 and the adjacent fine resolution images F1 and F3:
F12＝M 0 (θ 0 ；C12,C13,F1)
F23＝M 1 (θ 1 ；C23,C13,F3)
wherein M0 and M1 represent two sub-networks of the StfNet-MLP space-time fusion model, and θ is a model parameter to be optimized;
step five, reconstructing a fine resolution image F2_pre at the time t2 by adopting an adaptive local weighting strategy in a combination module:
F2 pre ＝a*(F1-F12)+(1-a)*(F3-F23)
where α and 1- α are weighting parameters of the predicted images f2_pre of F1 and F3, respectively.
Compared with the prior art, the invention has the following advantages:
the fusion between the multispectral images can acquire data with high time resolution and high spatial resolution, effectively extract forest variation information in the remote sensing images, accurately reflect the distribution pattern of forest coverage in space-time, further reflect the real variation condition of forest resources, and avoid the one-sided performance of a single remote sensing image, thereby providing an important data basis for the long time sequence research of forest monitoring.
Drawings
Fig. 1 is a flow chart of MLP-based spatio-temporal fusion.
Detailed Description
The following description of the present invention is provided with reference to the accompanying drawings, but is not limited to the following description, and any modifications or equivalent substitutions of the present invention should be included in the scope of the present invention without departing from the spirit and scope of the present invention.
The invention provides a remote sensing image fusion method based on MLP, which is to obtain Landsat remote sensing image sequences with high space-time resolution, ensure the usability of fused data, use a vision network architecture which is used for discarding convolution and self-attention and completely uses a multi-layer perceptron (MLP), and design an improved StfNet space-time fusion method (called StfNet-MLP) on the basis. As shown in fig. 1, the method comprises the following steps:
before fusion, firstly resampling the MODIS multispectral image with 500m spatial resolution to 480m so as to match the resolution of the image after spatial spectrum fusion. The coarse resolution images (MODIS) at three times (t 1, t2, t 3) and the fine resolution images (images after spatial spectrum fusion) at two times (t 1, t 3) are taken as inputs, and the fine resolution images at the time t2 are taken as fused output images. In the constructed StfNet-MLP space-time fusion model, a deeper MLP network structure is used to replace a four-layer convolution network in the original StfNet algorithm. In fig. 1, C represents a coarse resolution image, and F represents a fine resolution image. The difference image required by the model is calculated according to the formula (1), for example, C12 is obtained by making a difference between the coarse resolution image C2 at the time t2 and the coarse resolution image C1 at the time t 1.
Meanwhile, considering the time dependency, fine difference images F12 and F23 are predicted from the corresponding coarse difference images C12, C23 and C13 and the adjacent fine resolution images F1 and F3, and their mathematical expressions are shown in the formula (2) and the formula (3).
F12＝M 0 (θ 0 ；C12,C13,F1) (2)
F23＝M 1 (θ 1 ；C23,C13,F3) (3)
Where M0 and M1 represent two sub-networks of the StfNet-MLP model, and θ is the model parameter to be optimized. And finally, reconstructing a fine resolution image F2_pre at the time t2 by adopting an adaptive local weighting strategy in the combination module, and giving a specific calculation process by a formula (4).
F2 pre ＝a*(F1-F12)+(1-a)*(F3-F23) (4)
Where α and 1- α are weighting parameters of the predicted images f2_pre of F1 and F3, respectively. In order to determine the weighting parameters at the time of reconstruction, we consider that the higher the similarity of the coarse image, the more reliable the prediction of the image. That is, if the change between the two coarse images C2 and Ck (k=1 or 3) is small, it is possible that the target image F2 is more similar to the adjacent image Fk, i.e., the result reconstructed from Fk should be more accurate.
Claims (1)
1. The MLP-based remote sensing image fusion method is characterized by comprising the following steps of:
resampling the MODIS multispectral image with the spatial resolution of 500m to 480m to match the resolution of the image after spatial spectrum fusion;
step two, taking the coarse resolution images at three moments t1, t2 and t3 and the fine resolution images at two moments t1 and t3 as inputs, and taking the fine resolution images at the moment t2 as fused output images;
step three, calculating to obtain coarse difference images C12, C23 and C13 required by the StfNet-MLP space-time fusion model according to the following formula:
C12＝C2-C1
C13＝C3-C1
C23＝C3-C2
step four, predicting fine difference images F12 and F23 from the coarse difference images C12, C23 and C13 and the adjacent fine resolution images F1 and F3:
F12＝M 0 (θ 0 ；C12,C13,F1)
F23＝M 1 (θ 1 ；C23,C13,F3)
wherein M0 and M1 represent two sub-networks of the StfNet-MLP space-time fusion model, and θ is a model parameter to be optimized;
step five, reconstructing a fine resolution image F2_pre at the time t2 by adopting an adaptive local weighting strategy in a combination module:
F2 pre ＝a*(F1-F12)+(1-a)*(F3-F23)
where α and 1- α are weighting parameters of the predicted images f2_pre of F1 and F3, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311055805.1A CN117036889A (en) | 2023-08-22 | 2023-08-22 | MLP-based remote sensing image fusion method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311055805.1A CN117036889A (en) | 2023-08-22 | 2023-08-22 | MLP-based remote sensing image fusion method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117036889A true CN117036889A (en) | 2023-11-10 |
Family
ID=88627906
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311055805.1A Pending CN117036889A (en) | 2023-08-22 | 2023-08-22 | MLP-based remote sensing image fusion method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117036889A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113012044A (en) * | 2021-02-19 | 2021-06-22 | 北京师范大学 | Remote sensing image space-time fusion method and system based on deep learning |
| CN114565843A (en) * | 2022-02-22 | 2022-05-31 | 中国电子科技集团公司第五十四研究所 | Time series remote sensing image fusion method |
| CN115601281A (en) * | 2022-11-04 | 2023-01-13 | 吉林大学（Cn） | Remote sensing image space-time fusion method and system based on deep learning and electronic equipment |
-
2023
- 2023-08-22 CN CN202311055805.1A patent/CN117036889A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113012044A (en) * | 2021-02-19 | 2021-06-22 | 北京师范大学 | Remote sensing image space-time fusion method and system based on deep learning |
| CN114565843A (en) * | 2022-02-22 | 2022-05-31 | 中国电子科技集团公司第五十四研究所 | Time series remote sensing image fusion method |
| CN115601281A (en) * | 2022-11-04 | 2023-01-13 | 吉林大学（Cn） | Remote sensing image space-time fusion method and system based on deep learning and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| GUANGSHENG CHEN 等: "StfMLP: Spatiotemporal Fusion Multilayer Perceptron for Remote-Sensing Images", IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, vol. 20, 27 December 2022 (2022-12-27), pages 2 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fu et al. | Joint camera spectral sensitivity selection and hyperspectral image recovery | |
| CN111145131A (en) | Infrared and visible light image fusion method based on multi-scale generation type countermeasure network | |
| CN111291639B (en) | Cross-source ship feature fusion learning and identifying method based on layered variation self-coding | |
| CN113298058A (en) | Water quality prediction inversion method and system based on hyperspectral image of unmanned aerial vehicle | |
| CN110880163B (en) | Low-light color imaging method based on deep learning | |
| CN110533166B (en) | Indoor positioning method based on time-space fusion characteristics | |
| KR20190036459A (en) | Hyperspectral Imaging Reconstruction Method Using Artificial Intelligence and Apparatus Therefor | |
| KR20190036442A (en) | Hyperspectral Imaging Reconstruction Method Using Artificial Intelligence and Apparatus Therefor | |
| CN114353946B (en) | Diffraction snapshot spectrum imaging method | |
| CN115018711B (en) | Image super-resolution reconstruction method for warehouse scheduling | |
| CN115731191A (en) | Narrow-band spectral imaging method based on neural network | |
| Meng et al. | Spatio–temporal–spectral collaborative learning for spatio–temporal fusion with land cover changes | |
| Liu et al. | Hyperspectral remote sensing imagery generation from RGB images based on joint discrimination | |
| Ma et al. | Multi-task interaction learning for spatiospectral image super-resolution | |
| Sun et al. | Tuning IR-cut filter for illumination-aware spectral reconstruction from RGB | |
| CN117036889A (en) | MLP-based remote sensing image fusion method | |
| CN113256733A (en) | Camera spectral sensitivity reconstruction method based on confidence voting convolutional neural network | |
| CN111640069B (en) | Compressive imaging method, system and device based on light sensing network and phase compensation | |
| CN117079105B (en) | Remote sensing image spatial spectrum fusion method and device, electronic equipment and storage medium | |
| CN111551252A (en) | Spectral response design method based on neural network | |
| CN114429424B (en) | Remote sensing image super-resolution reconstruction method suitable for uncertain degradation modes | |
| CN116433548A (en) | Hyperspectral and panchromatic image fusion method based on multistage information extraction | |
| Taphanel et al. | Speed-up chromatic sensors by optimized optical filters | |
| CN113899453A (en) | Rapid spectral imaging system and imaging method based on sub-sampling | |
| CN115146844A (en) | Multi-mode traffic short-time passenger flow collaborative prediction method based on multi-task learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |