CN115564692B - Full color-multispectral-hyperspectral integrated fusion method considering breadth difference - Google Patents
Full color-multispectral-hyperspectral integrated fusion method considering breadth difference Download PDFInfo
- Publication number
- CN115564692B CN115564692B CN202211087146.5A CN202211087146A CN115564692B CN 115564692 B CN115564692 B CN 115564692B CN 202211087146 A CN202211087146 A CN 202211087146A CN 115564692 B CN115564692 B CN 115564692B
- Authority
- CN
- China
- Prior art keywords
- image
- hyperspectral
- conv
- multispectral
- full
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 20
- 230000004927 fusion Effects 0.000 claims abstract description 42
- 238000001228 spectrum Methods 0.000 claims abstract description 33
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000005070 sampling Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000000034 method Methods 0.000 claims description 32
- 230000008569 process Effects 0.000 claims description 15
- 238000012549 training Methods 0.000 claims description 15
- 230000002776 aggregation Effects 0.000 claims description 12
- 238000004220 aggregation Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 11
- 238000012937 correction Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 230000004931 aggregating effect Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 4
- 241000282326 Felis catus Species 0.000 claims description 3
- 230000005855 radiation Effects 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 2
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 239000013589 supplement Substances 0.000 abstract description 2
- 230000003595 spectral effect Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 230000008034 disappearance Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013501 data transformation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10036—Multispectral image; Hyperspectral image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10041—Panchromatic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The application relates to a full color-multispectral-hyperspectral integrated fusion method taking into account the difference of breadth, which comprises the steps of preprocessing and accurately registering full color images, multispectral images and hyperspectral images; constructing three coding branches, and acquiring 3 groups of space-spectrum consistency characteristics with different space scales; and constructing a single-branch decoding module according to the space-spectrum consistency characteristics, carrying out the same processing after overlapping the characteristics with the minimum scale with other corresponding scale characteristics while carrying out level-by-level up sampling to obtain the fused image with the corresponding label size. The beneficial effects of the application are as follows: the application can reduce the difficulty of overlarge difference of spatial resolution in fusion of hyperspectral and full-color images, and can supplement the space-spectral information of the hyperspectral image in a limited area of breadth and simultaneously meet the integrated fusion of full-color-multispectral-hyperspectral images in a region of consistent breadth.
Description
Technical Field
The application relates to the field of remote sensing image processing, in particular to a full color-multispectral-hyperspectral integrated fusion method taking into account the difference of breadth.
Background
With the development of earth observation technology, a large number of remote sensing satellites continuously emit, and earth remote sensing has entered the development stage of multi-platform and multi-angle observation. The development of novel sensor and novel technologies such as earth observation sensor network and the like enables the remote sensing data acquisition capacity with high spatial resolution and high spectral resolution to be further improved. The hyperspectral image with fine spectrum and hundreds of wave bands has wide application prospect in the fields of target identification, resource investigation, environment monitoring, military reconnaissance and the like. However, due to the limitation of physical conditions of the sensor, including peak signal-to-noise ratio, data storage and transformation of the imaging system, a trade-off needs to be made between spatial resolution, spectral resolution and breadth, so that a broad-width hyperspectral image with high spatial resolution cannot be obtained at the same time. Accordingly, multispectral images with multiple spectral bands and single-band panchromatic images have higher spatial resolution.
The space-spectrum fusion is used as one of research hot spots in the field of remote sensing image processing, mainly solves the mutual restriction of the space and the spectrum resolution of the multi-source image, and generates a fusion image with high space resolution and high spectrum resolution at the same time, including panchromatic-multi-spectrum fusion, panchromatic (or multi-spectrum) -hyperspectral fusion and panchromatic-multispectral-hyperspectral fusion. Panchromatic-multispectral fusion aims at improving the spatial resolution of multispectral through high-resolution panchromatic images, and the method is proposed earlier, and a plurality of technologies are developed and popularized to other fusion tasks; panchromatic (or multispectral) -hyperspectral fusion aims to improve the spatial quality of spectral images by high spatial resolution images, and can generally be divided into the following categories: component substitution class, multi-resolution analysis class, variation optimization class, and deep learning-based class; the full color-multispectral-hyperspectral fusion aims to simultaneously utilize a high-resolution full color image and a medium-resolution multispectral image so as to greatly improve the spatial resolution of a low-resolution hyperspectral image and greatly improve the application value of the hyperspectral image, but the above air-spectrum fusion method is not applicable to fusion of full color-multispectral-hyperspectral images with different widths in a real scene without considering the influence of the width. In addition, compared with full color-multispectral fusion and full color (or multispectral) -hyperspectral fusion and fusion based on a step-by-step fusion method (full color-multispectral+hyperspectral image, full color+multispectral-hyperspectral image and the like), how to solve the problem that the full color-multispectral-hyperspectral image is in space, spectrum and breadth in a real scene, and under the difference of three dimensions, the realization of the collaborative high-fidelity fusion of the multisensor image is more challenging and practical.
Disclosure of Invention
The application aims to overcome the defects in the prior art and provides a full color-multispectral-hyperspectral integrated fusion method under consideration of the difference of breadth.
In a first aspect, a full color-multispectral-hyperspectral integrated fusion method is provided that accounts for breadth differences, comprising:
s1, acquiring a full-color image, a multispectral image and a hyperspectral image, and preprocessing the full-color image, the multispectral image and the hyperspectral image;
s2, accurately registering the full-color image, the multispectral image and the hyperspectral image; then, carrying out data enhancement processing on each group of images, and ensuring that the images to be fused have the same ground feature scene information and projection system;
s3, constructing three coding branches by utilizing the difference of the full-color image, the multispectral image and the hyperspectral image in the spatial scale;
s4, obtaining 3 groups of space-spectrum consistency characteristics with different space scales through multi-branch coding;
s5, constructing a single-branch decoding module according to the space-spectrum consistency characteristics, sampling the characteristics with the minimum scale layer by layer, and overlapping the characteristics with other corresponding scale characteristics, and then performing the same processing to obtain the fused image with the corresponding label size.
Preferably, the method further comprises:
s6, in the process of supervised training, optimizing by adopting an L1 norm loss function, so that the fusion image generated in the S5 is more approximate to the reference image; the reference image is an original wide-width high-resolution hyperspectral image; expressed as:
wherein Loss represents a label H calculated by using L1 norm when N input images are subjected to each training iteration ref And fusion result H result The difference between them is averaged and summed.
Preferably, in S1, the preprocessing includes: radiation calibration, FLASH atmospheric correction, orthographic correction and geometric correction; in addition, the hyperspectral image requires bad band removal and mask processing.
Preferably, S3 includes:
s301, establishing an enhanced convolution module with a residual error idea according to the precision requirement required by the integrated fusion task, wherein the enhanced convolution module is used for realizing feature extraction and is expressed as follows:
Conv_1＝Conv 3 (σ(Conv 3 (σ(Conv 3 (input)))))
Conv_2＝Conv 3 (σ(Conv 3 (σ(Conv 3 (Conv_1)))))
Conv_E＝Conv_1+Conv_2
wherein input represents input image characteristics, sigma (·) represents a Relu activation function, conv_1 and conv_2 represent results of a first layer and a second layer of the enhanced convolution module, respectively, and conv_e represents the enhanced convolution module;
s302, constructing three coding branches according to the difference of the input full-color image, multispectral image and hyperspectral image in the spatial scale, wherein each branch is designed with a coding feature layer with three scales, and the size of each layer corresponds to the spatial size of the original input full-color image, multispectral image and hyperspectral image;
s303, constructing a module for enhancing hyperspectral breadth and spatial resolution, wherein the module firstly superimposes the characteristics acquired by a full-color branch and a multispectral branch with hyperspectral characteristics in the channel dimension to obtain a rough broad hyperspectral image; secondly, inputting the rough wide-width hyperspectral image into a pixel prediction module (Pixel Correlation module, PCM) to obtain a hyperspectral image with consistent width, wherein the hyperspectral image is expressed as:
H 3 ＝PCM(σ(Conv 3 (Cat(HS，M 3 ，P 3 ))))
wherein HS represents a coarse broad hyperspectral image, sigma (·) represents a Relu activation function, cat represents a stitching operation of the input features in the channel dimension, conv 3 Indicating that the convolution kernel size is 3 x 3; m is M 1 、M 2 、M 3 Characteristic information representing a multispectral branch; p (P) 1 、P 2 、P 3 Characteristic information representing the full color branch.
Preferably, in S302, for the panchromatic branch, the features are encoded in the original size using convolution to obtain the features P of the panchromatic encoded branch 1 、P 2 、P 3 The method comprises the steps of carrying out a first treatment on the surface of the For the multispectral branch, firstly acquiring characteristic information M from multispectral images 2 Respectively up-sampling and down-sampling to obtain M 1 、M 3 The method comprises the steps of carrying out a first treatment on the surface of the The above process is expressed as:
P 1 ＝Conv_E(PAN)
P 2 ＝Down_3(Conv_E(P 1 ))
P 3 ＝Down_4(Conv_E(P 2 ))
M 2 ＝Conv_E(MS)
M 1 ＝Up_4(Conv_E(M 2 ))
M 3 ＝Down_3(Conv_E(M 2 ))
where PAN represents full color image, MS represents multispectral image, up_4 represents Up-sampling by 4 times, down_3 and down_4 represent Down-sampling by 3 times and 4 times, respectively.
Preferably, S4 includes:
s401, according to different contributions of different branch coding features to an ideal image, aggregating the ideal image into a uniform depth structure by adopting a multi-level space-spectrum aggregation module, wherein the aggregation is expressed as follows:
F i ＝Conv 3 (Cat(H i ，M i ，P i ))
wherein F is i Representing the aggregation of three coding features from different branches into a unified feature space using convolution;
s402, for input features, four parallel 3X 3 convolution layers are used to obtain x respectively 1 ，x 2 ，x 3 ，x 4 Features, then, through 1 x 1 convolution learning of correlation information among feature groups, and meanwhile, sequentially expanding receptive fields among layers by using jump connection; secondly, each branch maps the features generated by the three coding branches to an ideal space in a way of adaptively capturing the features of the different channels by a channel attention module (Channel Attention Block, CAB); then, the outputs of the four branches are overlapped, the number of channels is adjusted by a 1X 1 convolution layer, and the outputs are overlapped with the initial input characteristics to form a residual structure, so that a space-spectrum characteristic consistency characteristic block Z is obtained i The method comprises the steps of carrying out a first treatment on the surface of the Expressed by the formula:
wherein k is a branch number, x k Input features representing the kth branch, conv 1 Indicating that the convolution kernel size is 1 x 1,after representing the output of the kth branch, the outputs of the four branches are connected, then a 1 x 1 convolution is performed to adjust the channel and add it to the input feature F i In which the output characteristic Z of the embedded multi-level residual information is obtained i ，i∈{1，2，3}。
Preferably, in S5, the process of acquiring the fusion image is expressed as:
H result ＝Conv 3 (Cat(Up-4(Conv 3 (Cat(Up_3(conv 3 (z 1 ))，Conv 3 (Z 2 ))))，Conv 3 (Z 3 )))
wherein Z is 1 、Z 2 、Z 3 Three sets of code features representing S4 outputs, H result Representing the fused image.
In a second aspect, there is provided a full color-multispectral-hyperspectral integrated fusion device taking into account the difference in breadth for performing any one of the full color-multispectral-hyperspectral integrated fusion methods of the first aspect, comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring full-color images, multispectral images and hyperspectral images and preprocessing the full-color images, the multispectral images and the hyperspectral images;
the registration module is used for accurately registering the full-color image, the multispectral image and the hyperspectral image; then, carrying out data enhancement processing on each group of images, and ensuring that the images to be fused have the same ground feature scene information and projection system;
the first construction module is used for constructing three coding branches by utilizing the difference of the full-color image, the multispectral image and the hyperspectral image in the spatial scale;
the second acquisition module is used for acquiring 3 groups of space-spectrum consistency characteristics with different space scales through multi-branch coding;
and the second construction module is used for constructing a single-branch decoding module according to the space-spectrum consistency characteristics, carrying out the same processing after the characteristics with the minimum scale are sampled up level by level and overlapped with other corresponding scale characteristics, and obtaining the fused image with the corresponding label size.
In a third aspect, a full color-multispectral-hyperspectral integrated fusion framework formed by the full color-multispectral-hyperspectral integrated fusion method according to any one of the first aspects is provided, and the full color-multispectral-hyperspectral integrated fusion framework can simultaneously input a low-spatial-resolution hyperspectral image with a narrow breadth, a multispectral image with a middle spatial resolution and a low spectral resolution, and a full-color image with a highest spatial resolution, and finally output an integrated fusion image with characteristics of high spectral resolution, high spatial resolution and a large breadth.
In a fourth aspect, a computer storage medium having a computer program stored therein is provided; the computer program, when run on a computer, causes the computer to perform any of the full color-multispectral-hyperspectral integrated fusion methods of the first aspect.
In a fifth aspect, there is provided a computer program product for causing a computer to perform any of the full color-multispectral-hyperspectral integrated fusion methods of the first aspect when said computer program product is run on the computer.
The beneficial effects of the application are as follows: the application provides a method for integrating hyperspectral images and full-color images by taking the multispectral images as an intermediary to reduce the difficulty of overlarge difference of spatial resolution in the integration of the hyperspectral images and the full-color images, and in addition, the method can supplement the space-spectral information of the hyperspectral images in the limited area of the breadth and simultaneously meet the integration of the full-color-multispectral-hyperspectral images in the area of consistent breadth.
Drawings
FIG. 1 is a flow chart of a full color-multispectral-hyperspectral integrated fusion method taking into account the difference in breadth;
FIG. 2 is a schematic diagram of an image preprocessing process;
FIG. 3 is an integrated fusion flow diagram;
FIG. 4 is a schematic diagram of a multi-branch encoding structure;
FIG. 5 is a schematic diagram of an enhanced convolution module;
FIG. 6 is a detailed description of the branches of hyperspectral image enhancement coding;
FIG. 7 is a schematic diagram of a multi-level spatial-spectral aggregation module;
fig. 8 is a schematic diagram of a decoding module.
Detailed Description
The application is further described below with reference to examples. The following examples are presented only to aid in the understanding of the application. It should be noted that it will be apparent to those skilled in the art that modifications can be made to the present application without departing from the principles of the application, and such modifications and adaptations are intended to be within the scope of the application as defined in the following claims.
Example 1:
the application provides a full color-multispectral-hyperspectral integrated fusion method considering the difference of breadth for realizing the collaborative high-fidelity fusion of multisensor images. Firstly, preprocessing and data enhancement operations are carried out on full-color, multispectral and hyperspectral images, after the full-color, multispectral and hyperspectral images pass through a multi-branch coding network, coding features of the full-color and multispectral image branches are used according to scale differences of the coding branches, and non-local networks are utilized for reconstructing large-scene hyperspectral image information to obtain primary restored fusion features. And secondly, generating multi-level coding feature layers after multi-branch coding, respectively overlapping the channel dimensions by groups for effectively utilizing effective information of the feature layers, and then sending the information into a multi-level space-spectrum aggregation module to unify information of different levels into a depth space-spectrum structure according to contribution degrees. And finally, decoding links different layers of coding features on the same system by using an upsampling operation, and performing iterative optimization by using deep learning to obtain a high-resolution hyperspectral image with a large scene. The fusion image produced by the method can be used for the maximum utilization of space-spectrum-breadth information.
Specifically, as shown in fig. 1, the method includes:
s1, obtaining a full-color image, a multispectral image and a hyperspectral image, and preprocessing the full-color image, the multispectral image and the hyperspectral image.
The multi-sensor data includes full color images, multi-spectral images, and hyperspectral images, as shown in fig. 2, the present application first pre-processes the multi-sensor data. Pretreatment before fusion of full-color, multispectral and hyperspectral images mainly comprises operations such as radiation calibration, FLASH atmospheric correction, orthographic correction, geometric correction and the like; in addition, the hyperspectral image needs to be removed by one step to avoid the distortion wave band in the hyperspectral image from affecting the final fused image precision. In addition, since the hyperspectral image has narrower breadth compared with the full-color image and the multispectral image in the real situation, mask processing is applied to the hyperspectral image during registration between the images so as to facilitate the subsequent fusion step
S2, accurately registering the full-color image, the multispectral image and the hyperspectral image; and then, carrying out data enhancement processing on each group of images, and ensuring that the images to be fused have the same ground feature scene information and projection system.
In S2, according to the raw data obtained in S1, a set of raw images cannot be directly input into the network for training due to the limitation of experimental hardware. In addition, the data demand is larger during the deep learning network training, and in order to avoid the over fitting during the network training and improve the robustness of the model and the generalization capability of the model, the data enhancement processing can be adopted, namely, after a group of original images are segmented into small images, the processing such as overturning, rotating and cutting is carried out on each group of images.
S3, constructing three coding branches by utilizing the difference of the full-color image, the multispectral image and the hyperspectral image in the spatial scale as shown in fig. 3 and 4.
It should be noted that, the spatial resolution of the multispectral image is more similar to that of the panchromatic image than that of the hyperspectral image, and the multispectral image spectral information has a similar spectral range to that of the hyperspectral image and that of the panchromatic image to be fused. The S3 can reduce the difficulty of overlarge difference of spatial and spectral resolution when the hyperspectral image and the full-color image are fused, and effectively combines the advantages of the multi-branch encoding and decoding network. In fig. 4, the green, blue, orange arrows indicate the initial feature inflow direction of the input image in the full-color, hyperspectral, and multispectral coding branches, respectively, and the black arrows indicate the extracted feature orientations at the time of coding.
S3 comprises the following steps:
s301, establishing an enhanced convolution module with a residual error idea according to the precision requirement required by the integrated fusion task, wherein the enhanced convolution module is used for realizing feature extraction and is expressed as follows:
Conv_1＝Conv 3 (σ(Conv 3 (σ(Conv 3 (input)))))
Conv_2＝Conv 3 (σ(Conv 3 (σ(Conv 3 (Conv_1)))))
Conv_E＝Conv_1+Conv_2
wherein input represents input image features, σ (·) represents a Relu activation function, conv_1 and conv_2 represent first and second layer results of the enhanced convolution module, respectively, and conv_e represents the enhanced convolution module.
The enhanced convolution block with residual error concept can maintain the structural information of data and avoid gradient disappearance during feature extraction. As shown in fig. 5, the residual enhancement module consists of two residual groups, wherein each layer of convolution chunks consists of one convolution of kernel size, step size 1, and one ReLU activation function σ (·) except for the third and sixth layers.
S302, constructing three coding branches according to the difference of the input full-color image, multispectral image and hyperspectral image in spatial scale, wherein each branch is designed with coding feature layers with three scales, and the size of each layer corresponds to the spatial size of the original input full-color image, multispectral image and hyperspectral image.
In S302, for the panchromatic branch, the features are encoded according to the original size by convolution to obtain the features P of the panchromatic branch 1 、P 2 、P 3 The method comprises the steps of carrying out a first treatment on the surface of the For the multispectral branch, firstly acquiring characteristic information M from multispectral images 2 Respectively up-sampling and down-sampling to obtain M 1 、M 3 The method comprises the steps of carrying out a first treatment on the surface of the The above process is expressed as:
P 1 ＝Conv_E(PAN)
P 2 ＝Down_3(Conv_E(P 1 ))
P 3 ＝Down_4(Conv_E(P 2 ))
M 2 ＝Conv_E(MS)
M 1 ＝Up_4(Conv_E(M 2 ))
M 3 ＝Down_3(Conv_E(M 2 ))
where PAN represents full color image, MS represents multispectral image, up_4 represents Up-sampling by 4 times, down_3 and down_4 represent Down-sampling by 3 times and 4 times, respectively.
The hyperspectral image branch is used for encoding according to the full-color and multispectral image branch, and the encoding input of the hyperspectral image branch comprises encoding characteristic information of the full-color and multispectral image branch of an original hyperspectral image.
S303, aiming at a hyperspectral image branch, fully considering the difference of breadth and spatial resolution caused by the imaging characteristic of a sensor, comprehensively utilizing the characteristics of a multi-source image, respectively acquiring complementary characteristics from a full-color and multispectral coding branch, and constructing an enhancement module aiming at the hyperspectral breadth and the spatial resolution. The module firstly superimposes the characteristics acquired by the full-color branch and the multispectral branch with the hyperspectral characteristics in the channel dimension to obtain a rough broad hyperspectral image; secondly, inputting the rough broad-width hyperspectral image into a pixel prediction module PCM, as shown in FIG. 6, the module refines the prediction of the current pixel by learning the similar neighbor pixels, captures the remote correlation of the global image, thereby further refining the rough image, improving the consistency of the high-resolution feature recovery of a large scene, obtaining hyperspectral image with consistent width, comprising fine hyperspectral feature information H 3 ，H 2 ，H 1 Expressed as:
H 3 ＝PCM(σ(Conv 3 (Cat(HS，M 3 ，P 3 ))))
wherein HS represents a coarse broad hyperspectral image, sigma (·) represents a Relu activation function, cat represents a stitching operation of the input features in the channel dimension, conv 3 Indicating that the convolution kernel size is 3 x 3; m is M 1 、M 2 、M 3 Characteristic information representing a multispectral branch; p (P) 1 、P 2 、P 3 Characteristic information representing the full color branch.
S4, as shown in FIG. 7, 3 groups of space-spectrum consistency characteristics with different space scales are obtained through multi-branch coding.
S4 comprises the following steps:
s401, according to different contributions of different branch coding features to an ideal image, aggregating the ideal image into a uniform depth structure by adopting a multi-level space-spectrum aggregation module, wherein the aggregation is expressed as follows:
F i ＝Conv 3 (Cat(H i ，M i ，P i ))
wherein F is i Representing the aggregation of three coding features from different branches into a unified feature space using convolution;
s402, for input features, four parallel 3X 3 convolution layers are used to obtain x respectively 1 ，x 2 ，x 3 ，x 4 The features are then subjected to 1 multiplied by 1 convolution to learn the correlation information among the feature groups, and meanwhile, the receptive field is sequentially enlarged by utilizing jump connection among each layer, so that the learning capacity of the spatial features is improved; secondly, each branch maps the characteristics generated by three coding branches to an ideal space in a self-adaptive mode of capturing the characteristics of different channels through a channel attention module CAB; then, the outputs of the four branches are overlapped and the number of channels is adjusted by a 1X 1 convolution layer, and the outputs are overlapped with the initial input characteristics to form a residual structure to more effectively reserve space-spectrum information, and meanwhile, the information loss caused by gradient disappearance can be prevented to obtain a space-spectrum characteristic consistency characteristic block Z i The method comprises the steps of carrying out a first treatment on the surface of the Expressed by the formula:
wherein k is a branch number, x k Input features representing the kth branch, conv 1 Indicating that the convolution kernel size is 1 x 1,representing the kth branchAfter output, the outputs of the four branches are connected, then 1×1 convolved to adjust the channel, and added to the input feature F i In which the output characteristic Z of the embedded multi-level residual information is obtained i ，i∈{1，2，3}。
S5, constructing a single-branch decoding module according to the space-spectrum consistency characteristics, and in order to combine the characteristic information on different scales, as shown in fig. 8, carrying out the same processing after the characteristics with the minimum scale are sampled up level by level and overlapped with the characteristics of other corresponding scales, so as to obtain the fused image with the corresponding label size.
In S5, the process of acquiring the fusion image is expressed as:
H result ＝Conv 3 (Cat(Up_4(Conv 3 (Cat(Up_3(conv 3 (Z 1 ))，Conv 3 (Z 2 ))))，Conv 3 (Z 3 )))
wherein Z is 1 、Z 2 、Z 3 Three sets of code features representing S4 outputs, H result Representing the fused image.
In addition, the method provided by the application further comprises the following steps:
s6, in the process of supervised training, optimizing by adopting an L1 norm loss function, so that the fusion image generated in the S5 is more approximate to the reference image; the reference image is an original wide-width high-resolution hyperspectral image; expressed as:
wherein Loss represents a label H calculated by using L1 norm when N input images are subjected to each training iteration ref And fusion result H result The difference between them is averaged and summed.
Because it is difficult to obtain a hyperspectral image of a true high space-large breadth in a true imaging environment, a plurality of pairs of training data are obtained by performing simulated space and spectral degradation on a training image and applying a mask to the hyperspectral image based on the Wald protocol. In the process of supervised training, the method adopts the L1 norm loss function to optimize, and obtains a broad-width high-spatial-resolution hyperspectral image.
Example 2:
the data employed in example 2 are the existing domestic resource 02D (ZY-02D) satellite hyperspectral data and the internationally published Chikusei, pavia Center and Pavia University hyperspectral images. The application adopts computer software ENVI5.3, MATLAB2019a and Pycharm, and Pytorch framework based on NVIDIA RTX 1080Ti GPU to realize automatic operation flow. The following describes the integrated fusion step of the multi-source image in detail with reference to fig. 3.
Step one, preprocessing a data set. Firstly, aiming at the ZY-1 02D image acquired under the real condition, image registration is needed, a multisource image registration flow tool based on ENVI5.3 software is added, manual auxiliary point selection correction is added, and finally three groups of images after geographic registration are obtained. The registration accuracy is crucial to the fusion result of the images, wherein the registration accuracy is smaller than 0.6 pixel, and sub-pixel level registration is achieved. In addition, the hyperspectral image is subjected to bad wave band removal, so that the influence of a distorted wave band in the hyperspectral image on the final fused image precision is avoided. Finally, due to the lack of real images, corresponding panchromatic-multispectral-hyperspectral images cannot be obtained, and the hyperspectral images in the areas of Chikusei and Pavia are respectively passed through Wald protocols to obtain low-resolution narrow-width hyperspectral, medium-resolution multispectral, high-resolution panchromatic images and high-resolution wide-width hyperspectral images serving as references.
And secondly, performing processes such as overturning, rotating and cutting on the multi-source image data to realize a data set expansion mode so as to avoid the problems of overfitting and the like caused by less data in the deep learning training process.
And thirdly, because the space and spectrum resolution difference between the full-color and hyperspectral images is larger, taking the multispectral images at the middle level into consideration to respectively construct full-color-multispectral-hyperspectral branch coding streams according to the space scale difference, and simultaneously, aiming at hyperspectral characteristic branches, the characteristic information of the full-color and multispectral branches is interacted into the paths, and meanwhile, the non-local network structure is combined to complement the limited part of the hyperspectral images, so that the information interference caused by the imaging condition difference is effectively reduced, and the expression capability among the multisource images is enhanced.
And step four, characteristic information extracted from the multi-branch coding network, according to the difference of scale space, a multi-level space-spectrum aggregation module considering contribution of space-spectrum components in different coding branches is provided, input information is divided into four branches by utilizing four parallel convolutions, the branches comprise channel attention modules, and jump connection exists among the branches to enlarge the receptive field and improve the multi-scale space information extraction capacity. And finally, aggregating the four pieces of branch characteristic information into a unified depth structure by using a residual error model.
Step five, carrying out hierarchical interpolation up-sampling on the 3 coding features with different scales obtained in the step four from the minimum scale feature, overlapping the coding features with other coding features with corresponding scales on the channel dimension when the coding features meet the corresponding scales, and carrying out up-sampling until a fused image with the same space and spectrum scale as the label image is obtained.
Step six, according to the supervised deep network training strategy, an L1 norm is utilized to establish a loss model of the fusion image and the label image, an ADMM optimizer is adopted to optimize, and the fusion result is enabled to be continuously approximate to the label image in the training process, so that a broad-width high-spatial-resolution hyperspectral image is obtained.
Claims (6)
1. The full color-multispectral-hyperspectral integrated fusion method considering the difference of the widths is characterized by comprising the following steps:
s1, acquiring a full-color image, a multispectral image and a hyperspectral image, and preprocessing the full-color image, the multispectral image and the hyperspectral image; in S1, the preprocessing includes: radiation calibration, FLASH atmospheric correction, orthographic correction and geometric correction; in addition, the hyperspectral image also needs to be subjected to bad wave band removal and mask treatment;
s2, accurately registering the full-color image, the multispectral image and the hyperspectral image; then, carrying out data enhancement processing on each group of images, and ensuring that the images to be fused have the same ground feature scene information and projection system;
s3, constructing three coding branches by utilizing the difference of the full-color image, the multispectral image and the hyperspectral image in the spatial scale;
s3 comprises the following steps:
s301, establishing an enhanced convolution module with a residual error idea according to the precision requirement required by the integrated fusion task, wherein the enhanced convolution module is used for realizing feature extraction and is expressed as follows:
Conv_1＝Conv 3 (σ(Conv 3 (σ(Conv 3 (input)))))
Conv_2＝Conv 3 (σ(Conv 3 (σ(Conv 3 (Conv_1)))))
Conv_E＝Conv_1+Conv_2
wherein input represents input image characteristics, sigma (·) represents a Relu activation function, conv_1 and conv_2 represent results of a first layer and a second layer of the enhanced convolution module, respectively, and conv_e represents the enhanced convolution module;
s302, constructing three coding branches according to the difference of the input full-color image, multispectral image and hyperspectral image in the spatial scale, wherein each branch is designed with a coding feature layer with three scales, and the size of each layer corresponds to the spatial size of the original input full-color image, multispectral image and hyperspectral image;
in S302, for the panchromatic branch, the features are encoded according to the original size by convolution to obtain the features P of the panchromatic branch 1 、P 2 、P 3 The method comprises the steps of carrying out a first treatment on the surface of the For the multispectral branch, firstly acquiring characteristic information M from multispectral images 2 Respectively up-sampling and down-sampling to obtain M 1 、M 3 The method comprises the steps of carrying out a first treatment on the surface of the The above process is expressed as:
P 1 ＝Conv_E(PAN)
P 2 ＝Down_3(Conv_E(P 1 ))
P 3 ＝Down_4(Conv_E(P 2 ))
M 2 ＝Conv_E(MS)
M 1 ＝Up_4(Conv_E(M 2 ))
M 3 ＝Down_3(Conv_E(M 2 ))
wherein PAN represents full-color image, MS represents multispectral image, up_4 represents Up-sampling by 4 times, down_3 and down_4 represent Down-sampling by 3 times and 4 times respectively;
s303, constructing a module for enhancing hyperspectral breadth and spatial resolution, wherein the module firstly superimposes the characteristics acquired by a full-color branch and a multispectral branch with hyperspectral characteristics in the channel dimension to obtain a rough broad hyperspectral image; secondly, inputting the rough broad-width hyperspectral image into a pixel prediction module PCM to obtain a hyperspectral image with consistent width, wherein the hyperspectral image is expressed as:
H 3 ＝PCM(σ(Conv 3 (Cat(HS，M 3 ，P 3 ))))
wherein HS represents a coarse broad hyperspectral image, sigma (·) represents a Relu activation function, cat represents a stitching operation of the input features in the channel dimension, conv 3 Indicating that the convolution kernel size is 3 x 3; m is M 1 、M 2 、M 3 Characteristic information representing a multispectral branch; p (P) 1 、P 2 、P 3 Characteristic information representing a full-color branch;
s4, obtaining 3 groups of space-spectrum consistency characteristics with different space scales through multi-branch coding;
s5, constructing a single-branch decoding module according to the space-spectrum consistency characteristics, sampling the characteristics with the minimum scale layer by layer, and overlapping the characteristics with other corresponding scale characteristics, and then performing the same processing to obtain the fused image with the corresponding label size.
2. The full color-multispectral-hyperspectral integrated fusion method taking into account the difference in breadth as claimed in claim 1, further comprising:
s6, in the process of supervised training, optimizing by adopting an L1 norm loss function, so that the fusion image generated in the S5 is more approximate to the reference image; the reference image is an original wide-width high-resolution hyperspectral image; expressed as:
wherein Loss represents a label H calculated by using L1 norm when N input images are subjected to each training iteration ref And fusion result H result The difference between them is averaged and summed.
3. The full color-multispectral-hyperspectral integrated fusion method taking into account the difference in breadth as claimed in claim 2, wherein S4 comprises:
s401, according to different contributions of different branch coding features to an ideal image, aggregating the ideal image into a uniform depth structure by adopting a multi-level space-spectrum aggregation module, wherein the aggregation is expressed as follows:
F i ＝Conv 3 (Cat(H i ，M i ，P i ))
wherein F is i Representing the aggregation of three coding features from different branches into a unified feature space using convolution;
s402, for input features, four parallel 3X 3 convolution layers are used to obtain x respectively 1 ，x 2 ，x 3 ，x 4 Features, then, through 1 x 1 convolution learning of correlation information among feature groups, and meanwhile, sequentially expanding receptive fields among layers by using jump connection; secondly, each branch maps the characteristics generated by three coding branches to an ideal space in a self-adaptive mode of capturing the characteristics of different channels through a channel attention module CAB; then, the outputs of the four branches are overlapped, the number of channels is adjusted by a 1X 1 convolution layer, and the outputs are overlapped with the initial input characteristics to form a residual structure, so that a space-spectrum characteristic consistency characteristic block Z is obtained i The method comprises the steps of carrying out a first treatment on the surface of the Expressed by the formula:
wherein k is a branch number, x k Input features representing the kth branch, conv 1 Indicating that the convolution kernel size is 1 x 1,after representing the output of the kth branch, the outputs of the four branches are connected, then a 1 x 1 convolution is performed to adjust the channel and add it to the input feature F i In which the output characteristic Z of the embedded multi-level residual information is obtained i ，i∈{1，2，3}。
4. The full-color-multispectral-hyperspectral integrated fusion method considering the difference of breadth as claimed in claim 3, wherein in S5, the process of obtaining the fused image is expressed as:
H result ＝Conv 3 (Cat(Up_4(Conv 3 (Cat(Up_3(Conv 3 (Z 1 ))，Conv 3 (Z 2 ))))，Conv 3 (Z 3 )))
wherein Z is 1 、Z 2 、Z 3 Three sets of code features representing S4 outputs, H result Representing the fused image.
5. A full color-multispectral-hyperspectral integrated fusion device taking into account the difference in breadth, characterized by being used for performing the full color-multispectral-hyperspectral integrated fusion method of any one of claims 1 to 4, comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring full-color images, multispectral images and hyperspectral images and preprocessing the full-color images, the multispectral images and the hyperspectral images;
the registration module is used for accurately registering the full-color image, the multispectral image and the hyperspectral image; then, carrying out data enhancement processing on each group of images, and ensuring that the images to be fused have the same ground feature scene information and projection system;
the first construction module is used for constructing three coding branches by utilizing the difference of the full-color image, the multispectral image and the hyperspectral image in the spatial scale;
the second acquisition module is used for acquiring 3 groups of space-spectrum consistency characteristics with different space scales through multi-branch coding;
and the second construction module is used for constructing a single-branch decoding module according to the space-spectrum consistency characteristics, carrying out the same processing after the characteristics with the minimum scale are sampled up level by level and overlapped with other corresponding scale characteristics, and obtaining the fused image with the corresponding label size.
6. A computer storage medium, wherein a computer program is stored in the computer storage medium; the computer program, when run on a computer, causes the computer to perform the full color-multispectral-hyperspectral integrated fusion method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211087146.5A CN115564692B (en) | 2022-09-07 | 2022-09-07 | Full color-multispectral-hyperspectral integrated fusion method considering breadth difference |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211087146.5A CN115564692B (en) | 2022-09-07 | 2022-09-07 | Full color-multispectral-hyperspectral integrated fusion method considering breadth difference |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115564692A CN115564692A (en) | 2023-01-03 |
| CN115564692B true CN115564692B (en) | 2023-12-05 |
Family
ID=84739281
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211087146.5A Active CN115564692B (en) | 2022-09-07 | 2022-09-07 | Full color-multispectral-hyperspectral integrated fusion method considering breadth difference |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115564692B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116129278B (en) * | 2023-04-10 | 2023-06-30 | 牧马人(山东)勘察测绘集团有限公司 | Land utilization classification and identification system based on remote sensing images |
| CN117437123A (en) * | 2023-09-27 | 2024-01-23 | 宁波大学 | Spectrum and texture attention fusion depth spectrum super-resolution method |
| CN117726916B (en) * | 2024-02-18 | 2024-04-19 | 电子科技大学 | Implicit fusion method for enhancing image resolution fusion |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103325096A (en) * | 2013-06-25 | 2013-09-25 | 中国科学院遥感与数字地球研究所 | Method for reconstructing wide hyperspectral image based on fusion of multispectral/hyperspectral images |
| CN108921809A (en) * | 2018-06-11 | 2018-11-30 | 上海海洋大学 | Multispectral and panchromatic image fusion method under integral principle based on spatial frequency |
| CN112634137A (en) * | 2020-12-28 | 2021-04-09 | 西安电子科技大学 | Hyperspectral and full-color image fusion method based on AE extraction of multi-scale spatial spectrum features |
| CN113112533A (en) * | 2021-04-15 | 2021-07-13 | 宁波甬矩空间信息技术有限公司 | SAR-multispectral-hyperspectral integrated fusion method based on multiresolution analysis |
| CN113222835A (en) * | 2021-04-22 | 2021-08-06 | 海南大学 | Remote sensing full-color and multi-spectral image distributed fusion method based on residual error network |
| CN113744136A (en) * | 2021-09-30 | 2021-12-03 | 华中科技大学 | Image super-resolution reconstruction method and system based on channel constraint multi-feature fusion |
| CN113763299A (en) * | 2021-08-26 | 2021-12-07 | 中国人民解放军军事科学院国防工程研究院工程防护研究所 | Panchromatic and multispectral image fusion method and device and application thereof |
| CN113793289A (en) * | 2021-09-10 | 2021-12-14 | 海南大学 | Multi-spectral image and panchromatic image fuzzy fusion method based on CNN and NSCT |
| CN114638761A (en) * | 2022-03-24 | 2022-06-17 | 华南理工大学 | Hyperspectral image panchromatic sharpening method, device and medium |
| CN114936995A (en) * | 2022-06-15 | 2022-08-23 | 郑州轻工业大学 | Multi-scale feature enhanced remote sensing image fusion method based on large nuclear attention mechanism |
| CN115018750A (en) * | 2022-08-08 | 2022-09-06 | 湖南大学 | Medium-wave infrared hyperspectral and multispectral image fusion method, system and medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10482576B2 (en) * | 2018-03-19 | 2019-11-19 | Mitsubishi Electric Research Laboratories, Inc. | Systems and methods for multi-spectral image super-resolution |
-
2022
- 2022-09-07 CN CN202211087146.5A patent/CN115564692B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103325096A (en) * | 2013-06-25 | 2013-09-25 | 中国科学院遥感与数字地球研究所 | Method for reconstructing wide hyperspectral image based on fusion of multispectral/hyperspectral images |
| CN108921809A (en) * | 2018-06-11 | 2018-11-30 | 上海海洋大学 | Multispectral and panchromatic image fusion method under integral principle based on spatial frequency |
| CN112634137A (en) * | 2020-12-28 | 2021-04-09 | 西安电子科技大学 | Hyperspectral and full-color image fusion method based on AE extraction of multi-scale spatial spectrum features |
| CN113112533A (en) * | 2021-04-15 | 2021-07-13 | 宁波甬矩空间信息技术有限公司 | SAR-multispectral-hyperspectral integrated fusion method based on multiresolution analysis |
| CN113222835A (en) * | 2021-04-22 | 2021-08-06 | 海南大学 | Remote sensing full-color and multi-spectral image distributed fusion method based on residual error network |
| CN113763299A (en) * | 2021-08-26 | 2021-12-07 | 中国人民解放军军事科学院国防工程研究院工程防护研究所 | Panchromatic and multispectral image fusion method and device and application thereof |
| CN113793289A (en) * | 2021-09-10 | 2021-12-14 | 海南大学 | Multi-spectral image and panchromatic image fuzzy fusion method based on CNN and NSCT |
| CN113744136A (en) * | 2021-09-30 | 2021-12-03 | 华中科技大学 | Image super-resolution reconstruction method and system based on channel constraint multi-feature fusion |
| CN114638761A (en) * | 2022-03-24 | 2022-06-17 | 华南理工大学 | Hyperspectral image panchromatic sharpening method, device and medium |
| CN114936995A (en) * | 2022-06-15 | 2022-08-23 | 郑州轻工业大学 | Multi-scale feature enhanced remote sensing image fusion method based on large nuclear attention mechanism |
| CN115018750A (en) * | 2022-08-08 | 2022-09-06 | 湖南大学 | Medium-wave infrared hyperspectral and multispectral image fusion method, system and medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于多分辨率分析的GF-5和GF-1遥感影像空―谱融合;孟祥超;孙伟伟;任凯;杨刚;邵枫;符冉迪;;遥感学报(第04期);第52-60页 * |
| 基于自适应注入模型的遥感图像融合方法;杨勇;卢航远;黄淑英;涂伟;李露奕;;北京航空航天大学学报(第12期);10-22 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115564692A (en) | 2023-01-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115564692B (en) | Full color-multispectral-hyperspectral integrated fusion method considering breadth difference | |
| CN109727207B (en) | Hyperspectral image sharpening method based on spectrum prediction residual convolution neural network | |
| CN110544212B (en) | Convolutional neural network hyperspectral image sharpening method based on hierarchical feature fusion | |
| CN109003239B (en) | Multispectral image sharpening method based on transfer learning neural network | |
| AU2020281143B1 (en) | Creating super-resolution images | |
| CN112184591A (en) | Image restoration method based on deep learning image Moire elimination | |
| CN111008936B (en) | Multispectral image panchromatic sharpening method | |
| CN116309070A (en) | Super-resolution reconstruction method and device for hyperspectral remote sensing image and computer equipment | |
| CN113902620A (en) | Video super-resolution system and method based on deformable convolution network | |
| CN115760814A (en) | Remote sensing image fusion method and system based on double-coupling deep neural network | |
| CN115018750A (en) | Medium-wave infrared hyperspectral and multispectral image fusion method, system and medium | |
| CN115526779A (en) | Infrared image super-resolution reconstruction method based on dynamic attention mechanism | |
| Long et al. | Dual self-attention Swin transformer for hyperspectral image super-resolution | |
| Deng et al. | Multiple frame splicing and degradation learning for hyperspectral imagery super-resolution | |
| CN113902646A (en) | Remote sensing image pan-sharpening method based on depth layer feature weighted fusion network | |
| CN114332625A (en) | Remote sensing image colorizing and super-resolution method and system based on neural network | |
| CN116740340A (en) | Calculation spectrum imaging error correction method based on deep learning | |
| CN116309227A (en) | Remote sensing image fusion method based on residual error network and spatial attention mechanism | |
| CN115760638A (en) | End-to-end deblurring super-resolution method based on deep learning | |
| CN115082344A (en) | Dual-branch network panchromatic sharpening method based on detail injection | |
| CN114638761A (en) | Hyperspectral image panchromatic sharpening method, device and medium | |
| CN112991174A (en) | Method and system for improving resolution of single-frame infrared image | |
| CN113870126B (en) | Bayer image recovery method based on attention module | |
| WO2021073747A1 (en) | Raw to rgb image transformation | |
| Wang et al. | Underwater image quality optimization: Researches, challenges, and future trends |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |