CN116612010A - Super-resolution image reconstruction method based on spectrum related information and space high-low frequency information - Google Patents
Super-resolution image reconstruction method based on spectrum related information and space high-low frequency information Download PDFInfo
- Publication number
- CN116612010A CN116612010A CN202310717620.6A CN202310717620A CN116612010A CN 116612010 A CN116612010 A CN 116612010A CN 202310717620 A CN202310717620 A CN 202310717620A CN 116612010 A CN116612010 A CN 116612010A
- Authority
- CN
- China
- Prior art keywords
- layer
- resolution
- low frequency
- spatial
- super
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001228 spectrum Methods 0.000 title claims abstract description 102
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000004927 fusion Effects 0.000 claims abstract description 83
- 238000000605 extraction Methods 0.000 claims abstract description 30
- 238000012549 training Methods 0.000 claims abstract description 18
- 230000006870 function Effects 0.000 claims description 61
- 230000004913 activation Effects 0.000 claims description 49
- 238000005070 sampling Methods 0.000 claims description 33
- 230000003595 spectral effect Effects 0.000 claims description 20
- 238000000926 separation method Methods 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 7
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 2
- 239000011800 void material Substances 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 5
- 238000004088 simulation Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000007547 defect Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 229910003460 diamond Inorganic materials 0.000 description 2
- 239000010432 diamond Substances 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 241000512668 Eunectes Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/58—Extraction of image or video features relating to hyperspectral data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a super-resolution image reconstruction method based on spectrum related information and space high-low frequency information. The implementation steps are as follows: on the basis of respectively constructing a space spectrum combined characteristic extraction sub-network and a multi-level characteristic fusion sub-network of space high-low frequency information; constructing a super-resolution network with spectrum related information and spatial high-low frequency information feature fusion; training the super-resolution network by using the generated training set; and carrying out super-resolution reconstruction on the hyperspectral image. The invention utilizes the mixed attention of the spectrum to enable the reconstruction process to effectively combine the joint information of the spectrum dimension and the space dimension, enhances the nonlinear learning capability between the characteristics of different wave bands of the hyperspectral image, enables the high-resolution hyperspectral image obtained by reconstruction to be clearer, and improves the capability of reconstructing texture details and edge characteristics of the hyperspectral image.
Description
Technical Field
The invention belongs to the technical field of image processing, and further relates to a super-resolution image reconstruction method based on spectral correlation and spatial high-low frequency information in the technical field of image reconstruction. The method can be used for carrying out super-resolution reconstruction on the hyperspectral image so as to enlarge the target image without losing details.
Background
With the development and iteration of artificial intelligence and hardware devices, deep learning models have been largely successful in the field of computer vision, which can learn complex image features and patterns to better infer information implicit in images. For the hyperspectral image, the hyperspectral image can acquire richer spectral information, each pixel point not only contains information of three RGB wave bands, but also contains information of other wave bands, so that the characteristics of the material, the composition, the structure and the like of an object can be better reflected, and meanwhile, the correlation exists between each spectral wave band.
The patent literature of the university of Chinese civil Legend, rocket and army engineering, which applies for the university of Chinese Legend, discloses a super-pixel sparse representation method for the super-spectrum super-resolution reconstruction (application number: 2020106816377, 2020.07.15, publication number: CN 111861885A). The method comprises the following implementation steps: (1) And clustering MSI pixels by using a segmentation algorithm to generate super-pixels, wherein the super-pixels are a set of homogeneous pixels. (2) Spectral features and scale information are learned from HSI and MSI, respectively. (3) And decomposing MSI superpixels on the learned spectral characteristics by adopting joint sparse regularization to obtain coefficient information. (4) And generating a high-resolution hyperspectral image by utilizing the learned spectral characteristics and coefficient information, and completing the super-pixel sparse representation for hyperspectral super-resolution reconstruction. However, the method still has the defect that the method ignores the characteristics of different stages, and because the characteristics of different stages carry different semantic information, the characteristics are transmitted to the network terminal in a simple serial connection mode, so that the voice information of the characteristics near the network head end is lost, and the joint information among the different characteristics of each stage in the existing model cannot be fully mined.
A hyperspectral super-Resolution reconstruction method based on multiscale feature mapping is proposed by jin et al in its published paper "Multiscale Feature Mapping Network for Hyperspectral Image Super-Resolution" (Published as a conference paper at RS 2021). The method comprises the following specific implementation steps: (1) The shallow features are extracted from the input image by a convolution layer having a convolution kernel size of 3 x 3. (2) The shallow features are thus passed through four diamond-shaped modules that introduce wavelet transforms, each diamond-shaped module comprising three parts: original feature map identity mapping, feature upsampling and feature downsampling. (3) The output features of the first three diamond modules are fused with a convolution layer of convolution kernel size 1 x 1 and added to the output of the fourth diamond module. (4) The integrated features are up-sampled by sub-pixel convolution and the final result map is obtained. However, the method still has the disadvantage that the method ignores the characteristic that the response curves of different features in the spectral dimension are different. Because of the different spectral curve responses corresponding to different pixels in the hyperspectral image, the attention mechanism can be applied to emphasize spectral features. However, existing 3D convolution-based spectral attention mechanisms can only extract features of several adjacent spectra, or trade larger convolution kernel parameters for a wider spectral receptive field. Furthermore, if spatial attention and channel attention are simply extended to a three-dimensional space, information existing in a spectral dimension cannot be effectively utilized.
Disclosure of Invention
The invention aims at solving the defects of the technology, and provides a super-resolution image reconstruction method based on spectrum related information and space high-low frequency information, which is used for solving the problem that spectrum and space characteristics in a hyperspectral image cannot be effectively extracted and utilized.
In order to achieve the above purpose, the invention constructs super-resolution network of fusion of spectrum related information and spatial high-low frequency information characteristics respectively. And by utilizing the characteristic that a large amount of information exists in the spectrum band of the hyperspectral image, the characteristic extraction and the characteristic emphasis are carried out on the spectrum band. Because the invention aims at the extraction problem of the combined characteristic information of the spectrum and the space dimension, a space spectrum combined characteristic extraction sub-network is constructed, the characteristic extraction and the emphasis can be carried out on the spectrum band information of the hyperspectral image, the combination is carried out with the space and channel information, and the characteristics of the spectrum dimension, the space dimension and the channel dimension of the hyperspectral image are extracted from the angles of hierarchical fusion and cross fusion. Because the invention adopts a cascade network structure, the invention constructs a multistage feature fusion sub-network with space high-low frequency separation, the sub-network can distinguish high-low frequency information by utilizing the difference of receptive fields, and respectively carries out different feature mapping and fusion on the high-low frequency information, and further processes and fuses the high-low frequency information, so that the network can fully utilize the feature relation from different stages. The method solves the problem that the prior art can not effectively extract and utilize the spectrum and space characteristics in the hyperspectral image.
The implementation steps of the invention are as follows:
step 1, constructing a spatial spectrum joint feature extraction sub-network:
the method comprises the steps of constructing a spatial spectrum joint feature extraction sub-network, wherein the spatial spectrum joint feature extraction sub-network comprises a first spatial spectrum joint attention layer, a second spatial spectrum joint attention layer, a third spatial spectrum joint attention layer, a fourth spatial spectrum joint attention layer, a first convolution layer, a first activation layer, a second convolution layer and a second activation layer; the first to fourth spatial spectrum joint attention layers have the same structure; setting the sizes of the first convolution layer and the second convolution layer to be 1 multiplied by 1, setting the step length to be 1, setting the output filling zero to be [0, 0], and setting the cavity value to be [1, 1]; the first and second active layers are realized by using a LeakyReLu function, and the slopes of the function in the negative number part are all set to be 0.01;
step 2, constructing a multi-level feature fusion sub-network of the space high-low frequency information:
constructing a multi-level characteristic fusion sub-network of the space high-low frequency information, which is formed by sequentially connecting a space high-low frequency separation layer, a fusion layer, a first convolution layer, a second convolution layer, a first activation layer, a third convolution layer and a second activation layer in series; the fusion layer is formed by connecting a space high-frequency fusion layer and a space low-frequency fusion layer in parallel;
the parameters of the multistage feature fusion subnetwork for setting the space high-low frequency information are as follows: setting the sizes of the first convolution layer to the third convolution layer to be 1 multiplied by 1, setting the step length to be 1, setting the output filling zero to be 0,0 and setting the cavity value to be 1,1 and 1; the first activation layer is realized by adopting a sigmoid function; the second active layer is realized by adopting a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01;
step 3, constructing a super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion:
constructing a super-resolution network formed by serially connecting a shallow feature extraction layer and a mapping reconstruction layer and integrating spectrum related information and spatial high-low frequency information features; the mapping reconstruction layer is formed by connecting a space spectrum combined characteristic extraction sub-network and a multi-level characteristic fusion sub-network of space high-low frequency information in parallel;
step 4, generating a training set:
selecting at least 25 hyperspectral images, wherein the number C, the length H and the width W of spectrum bands of the resolution of each hyperspectral image are respectively more than or equal to 10, 32 and 32; carrying out data preprocessing on each hyperspectral image to obtain an image pair of the hyperspectral image, and forming a training set by all the image pairs;
step 5, training a super-resolution network:
inputting the training set into a super-resolution network with the characteristic fusion of the spectrum related information and the spatial high-low frequency information, and iteratively updating network parameters by adopting an Adam optimization algorithm until the loss function converges to obtain the super-resolution network with the characteristic fusion of the trained spectrum related information and the spatial high-low frequency information;
step 6, carrying out super-resolution reconstruction on the hyperspectral image:
and 5, preprocessing the low-spatial-resolution hyperspectral image to be reconstructed by adopting the same method as the step 5, inputting the preprocessed hyperspectral image to a trained network, reconstructing the hyperspectral image with super resolution, and outputting the reconstructed hyperspectral image with high resolution.
Compared with the prior art, the invention has the following advantages:
firstly, because the invention constructs a super-resolution network with the characteristic fusion of spectrum related information and space high-low frequency information, the invention can reconstruct hyperspectral data from the angle of space spectrum joint characteristics and the angle of multi-level characteristics, overcomes the defect that the hyperspectral data cannot be efficiently extracted by utilizing the data characteristics of spectrum dimensions in a multi-level manner in the prior art, and can reconstruct hyperspectral images in a super-resolution manner from spectrum and space dimensions more effectively.
Secondly, because the spatial spectrum combined characteristic extraction sub-network constructed by the invention utilizes spectrum high-frequency information to carry out attention calculation on spectrum dimensions, the invention can carry out characteristic extraction and emphasis on spectrum band information of hyperspectral images, and is combined with space and channel information, the characteristics of spectrum dimensions, space dimensions and channel dimensions of hyperspectral images are extracted from the angles of hierarchical fusion and cross fusion, the defect that the combined characteristic information of the space and the spectrum dimensions cannot be fully mined in the prior art is overcome, the invention can effectively combine the combined characteristic information of the spectrum and the space dimensions, and the attention mechanism is utilized to carry out emphasis and fusion on the characteristics from different dimensions, so that the space details and spectrum correlation information which can be utilized in the hyperspectral super-resolution reconstruction process are greatly increased, and the reconstruction performance of hyperspectral images is improved.
Third, because the multi-level characteristic fusion sub-network with the space high-low frequency separation constructed by the invention can distinguish the high-low frequency information by utilizing the difference of receptive fields and respectively map the different characteristics of the high-low frequency information, the network can adapt to different characteristic expression, and the expression capability of the network is enhanced. The method overcomes the defects that the characteristics of different stages are ignored or simply connected to the tail end of the network in the prior art, and the characteristic information of each stage in the existing model cannot be fully considered for further mining, so that the nonlinear target capacity and generalization of the reconstruction network are improved, the high-frequency details and texture characteristics in the hyperspectral super-resolution reconstruction result are increased, and the hyperspectral image reconstruction quality is more superior.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a schematic diagram of a spatial spectrum joint feature extraction sub-network constructed by the invention;
FIG. 3 is a schematic diagram of a spatial spectrum joint attention layer constructed in the invention;
FIG. 4 is a schematic diagram of a multi-level feature fusion sub-network of spatial high-low frequency information constructed in the present invention;
FIG. 5 is a schematic diagram of a spatial high-low frequency information separation layer constructed in accordance with the present invention;
FIG. 6 is a schematic view of a spatial high-frequency fusion layer constructed in accordance with the present invention;
FIG. 7 is a schematic diagram of a spatial low-frequency fusion layer constructed in accordance with the present invention;
fig. 8 is a schematic diagram of a super-resolution network structure of feature fusion of spectrum related information and spatial high-low frequency information constructed by the invention.
Detailed Description
The invention is further described below with reference to the drawings and examples.
The implementation steps of an embodiment of the present invention are further described with reference to fig. 1.
The spectrum related information and space high-low frequency information based network constructed by the embodiment of the invention comprises two sub-networks: and the space spectrum combined characteristic extraction sub-network and the space high-low frequency information multi-level characteristic fusion sub-network. Wherein, the spatial spectrum joint feature extraction sub-network comprises a spatial spectrum joint attention layer. The space high-low frequency information multistage feature fusion sub-network comprises a space high-low frequency information separation layer, a space high-frequency fusion layer and an inter-low frequency fusion layer.
And step 1, constructing a spatial spectrum joint feature extraction sub-network.
The structure of the spatial-spectral joint feature extraction subnetwork of the present invention is further described with reference to fig. 2.
The method comprises the steps of constructing a spatial spectrum joint feature extraction sub-network, wherein the spatial spectrum joint feature extraction sub-network comprises a first spatial spectrum joint attention layer, a second spatial spectrum joint attention layer, a third spatial spectrum joint attention layer, a fourth spatial spectrum joint attention layer, a first convolution layer, a first activation layer, a second convolution layer and a second activation layer. Wherein the first to fourth spatial spectrum joint attention layers have the same structure. Setting the sizes of the first convolution layer and the second convolution layer to be 1 multiplied by 1, setting the step length to be 1, setting the output filling zero to be [0, 0], and setting the cavity value to be [1, 1]; the first and second active layers are both implemented using the LeakyReLu function, and the slope of the function in the negative portion is set to 0.01.
The structure of the spatial joint attention layer is further described with reference to fig. 3.
Constructing an empty spectrum combined attention layer, and sequentially connecting the structures of the empty spectrum combined attention layer in series: a first convolution layer, a first activation layer, a first parallel layer, a sixth convolution layer, a sixth activation layer, a seventh convolution layer, a seventh activation layer, a second parallel layer, an eighth convolution layer, a ninth convolution layer, an eighth activation layer. The first parallel layer is formed by connecting a first branch and a second branch in parallel, the second convolution layer and the second activation layer are connected in series to form the first branch, and the fourth convolution layer and the fourth activation layer are connected in series to form the second branch. The second parallel layer is formed by connecting a third branch and a fourth branch in parallel, the third convolution layer and the third activation layer are connected in series to form a third branch, and the fifth convolution layer and the fifth activation layer are connected in series to form a fourth branch.
The parameters of each layer of the spatial spectrum combined attention layer are set as follows: the sizes of the first convolution layer, the second convolution layer, the third convolution layer and the sixth convolution layer are all set to be 3 multiplied by 1, the step length is all set to be 1, the output filling zero is all set to be [1, 0], and the cavity value is all set to be [1, 1]. The fourth, fifth and seventh convolution layers are all set to 3×1×1 in size, 1 in step length, 2,0,0 in output zero padding, 2,1 in hole value, 1×1 in size, 1 in step length, 0 in output zero padding, 1 in hole value. The first to fifth active layers are all realized by using a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01; the sixth through ninth active layers are implemented using a sigmoid function for calculating the magnitude of the attention of the input features in the spectral dimension.
And 2, constructing a multi-level feature fusion sub-network of the space high-low frequency information.
The structure of the multi-level feature fusion subnetwork of spatial high-low frequency information constructed in accordance with the present invention is further described with reference to fig. 4.
And constructing a multistage characteristic fusion sub-network of the space high-low frequency information, which is formed by sequentially connecting a space high-low frequency separation layer, a fusion layer, a first convolution layer, a second convolution layer, a first activation layer, a third convolution layer and a second activation layer in series. The fusion layer is formed by connecting a space high-frequency fusion layer and a space low-frequency fusion layer in parallel.
The parameters of the multistage feature fusion subnetwork for setting the space high-low frequency information are as follows: setting the sizes of the first convolution layer to the third convolution layer to be 1 multiplied by 1, setting the step length to be 1, setting the output filling zero to be 0,0 and setting the cavity value to be 1,1 and 1; the first activation layer is realized by adopting a sigmoid function and is used for calculating the attention of the input feature in the channel dimension; the second active layer is implemented using the LeakyReLu function, which is set to 0.01 in slope in the negative part.
The space high-low frequency separation layer is as shown in fig. 5, and the structure for building the space high-low frequency separation layer is sequentially connected in series as follows: the first combination layer is formed by connecting two branches in parallel, the second convolution layer and the second activation layer are connected in series to form a first branch, and the third convolution layer and the third activation layer are connected in series to form a second branch.
The parameters for setting the spatial high-low frequency separation layer are as follows, the sizes of the first to third convolution layers are set to 1 x 1,3 x 3, the step sizes are all set to 1, the output filling zero padding is respectively set to 0,0 and 1, [2, 2] the void values are set to [1, 1], [2, 2] respectively; the first to third active layers are all realized by using a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01; the downsampling layer adopts a tri-linear interpolation function as a downsampling function, and the sampling multiple is set to be 2 times and is used for reducing the size of the input feature map in the space dimension by two times.
The structure of the constructed spatial high frequency fusion layer is further described with reference to fig. 6.
And constructing a space high-frequency fusion layer consisting of a high-frequency parallel layer, a second convolution layer and a second activation space which are connected in series. The high-frequency parallel layer is formed by connecting a first branch and a second branch in parallel, the first convolution layer and the first activation layer are connected in series to form the first branch, and the downsampling layer, the third convolution layer, the third activation layer and the upsampling layer are connected in series to form the second branch.
The parameters for setting the spatial high-frequency fusion layer are as follows, the sizes of the first to third convolution layers are set to 3 x 3, the step length is set to be 1, the output filling zero padding is set to be 1,1 and 1, and the cavity value is set to be 1,1 and 1. The first to third active layers are each implemented using a LeakyReLu function, and the slope of the function in the negative portion is set to 0.01. The downsampling layer adopts a tri-linear interpolation function as a downsampling function, and the sampling multiple is set to be 2 times and is used for reducing the size of the input feature map in the space dimension by two times; the up-sampling layer adopts a tri-linear interpolation function as a sampling function of up-sampling, and the sampling multiple is set to be 2 times and is used for amplifying the size of the input characteristic diagram in the space dimension by two times.
The structure of the spatial low frequency fusion layer constructed in accordance with the present invention is further described with reference to fig. 7.
And constructing a space low-frequency fusion layer formed by sequentially connecting a low-frequency parallel layer, a second convolution layer and a second activation layer in series. The low-frequency parallel layer is formed by connecting a first branch and a second branch in parallel, the first convolution layer and the first activation layer are connected in series to form the first branch, and the up-sampling layer, the third convolution layer, the third activation layer and the down-sampling layer are connected in series to form the second branch.
Setting parameters of the space low-frequency fusion layer as follows, setting the sizes of the first convolution layer to the third convolution layer to be 3 multiplied by 3, setting the step length to be 1, setting the output filling zero to be [1, 1], and setting the cavity value to be [1, 1]; the first to third active layers are all realized by using a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01; the up-sampling layer adopts a tri-linear interpolation function as a sampling function of up-sampling, and the sampling multiple is set to be 2 times and is used for amplifying the size of an input feature map in the space dimension by two times; the downsampling layer adopts a tri-linear interpolation function as a downsampling function, and the sampling multiple is set to be 2 times and is used for reducing the size of the input feature map in the space dimension by two times.
And 3, constructing a super-resolution network in which the spectrum related information and the spatial high-low frequency information are fused.
A super-resolution network consisting of a shallow feature extraction layer and a mapping reconstruction layer which are connected in series is built, as shown in figure 8.
The shallow layer feature extraction layer is formed by serially connecting a convolution layer and an activation layer, the convolution kernel size of the convolution layer is set to be 3 multiplied by 3, the step length is set to be 1, and the cavity value is set to be [1, 1]; the active layer is implemented using the LeakyReLu function, which is set to 0.01 in slope in the negative part.
The mapping reconstruction layer is formed by connecting a spatial spectrum combined characteristic extraction sub-network and a multi-level characteristic fusion sub-network of the spatial high-low frequency information in parallel.
And 4, generating a training set.
In the embodiment of the invention, 25 hyperspectral images are selected from a CAVE data set, and the number C, the length H and the width W of spectrum bands of the resolution of each hyperspectral image are respectively 10, 32 and 32.
And carrying out data preprocessing on each hyperspectral image to obtain an image pair of the hyperspectral image, and forming a training set by all the image pairs.
The data preprocessing step comprises the following steps:
the first step, respectively carrying out downsampling operations of 1, 0.75 and 0.5 times on each original high-resolution hyperspectral image to obtain high-resolution hyperspectral images under three corresponding sampling times, wherein the resolution of the high-resolution hyperspectral images under each sampling time is as followsr represents a downsampling coefficient;
secondly, carrying out normalization processing on the high-resolution hyperspectral image under each sampling multiple according to the following formula:
I HR ＝I hr /65535.0
wherein I is HR Representing normalized high resolution image, I hr Representing a high resolution image at each sampling multiple.
Thirdly, downsampling is carried out on each normalized high-resolution image, and each downsampled low-resolution image and the normalized high-resolution image form an image pair.
And step 5, training a super-resolution network with the characteristic fusion of the spectrum related information and the spatial high-low frequency information.
Inputting the training set into a super-resolution network with the characteristic fusion of the spectrum related information and the spatial high-low frequency information, and iteratively updating network parameters by adopting an Adam optimization algorithm until the loss function converges to obtain the super-resolution network with the characteristic fusion of the trained spectrum related information and the spatial high-low frequency information.
The parameters of the Adam optimization algorithm are set as follows: the exponential decay rate was set to 0.9 and 0.999, respectively, eps was set to 1e-8, and the step size was set to 0.0001.
The loss function is as follows:
wherein, loss (·) represents the Loss function,representing an ith normalized high-resolution hyperspectral image input into a super-resolution network of feature fusion of spectrum related information and spatial high-low frequency information, < >>The ith high-resolution hyperspectral image output in the super-resolution network which represents the feature fusion of the spectrum related information and the spatial high-low frequency information is represented, N represents the total number of batch images of one time which are input into a training set in the network, sigma (·) represents the summation operation, and I·| represents the absolute value taking operation.
Step 6, carrying out super-resolution reconstruction on the hyperspectral image:
and 5, preprocessing the low-spatial-resolution hyperspectral image to be reconstructed by adopting the same method as the step 5, inputting the preprocessed hyperspectral image to a trained network, reconstructing the hyperspectral image with super resolution, and outputting the reconstructed hyperspectral image with high resolution.
The effects of the present invention are further described below in conjunction with simulation experiments:
1. simulation experiment conditions:
the hardware platform of the simulation experiment of the invention: the display card is NVIDIARTX2080Ti x4, the display memory is 11GB x4, the processor is Intel (R) Core (TM) CPU i9-10900X@3.70GHz, and the memory is 32GB.
The software platform of the simulation experiment of the invention: a code running environment comprising important environment libraries such as PyTorch-1.1.0+cu100, matplotlib-3.3.3 and the like is built in the Python 3.6 virtual environment of Anaconda.
2. Simulation content and result analysis:
the CAVE data set used in the simulation experiment of the invention is that 25 hyperspectral images are randomly selected from 32 hyperspectral images created by Japanese SONY company, and each hyperspectral image is cut to obtain hyperspectral images with the size of 64 multiplied by 31. And rotating, downsampling and normalizing each cut hyperspectral image to form a training set. The remaining 7 hyperspectral images in the CAVE dataset were assembled into a test set.
The simulation experiment of the invention is to construct a super-resolution network with the characteristic fusion of the spectrum related information and the spatial high-low frequency information by adopting the method, train the super-resolution network by using the CAVE data set of the training set, then input the CAVE hyperspectral image processed in the test set into the super-resolution network with the characteristic fusion of the trained spectrum related information and the spatial high-low frequency information for reconstructing the super-resolution image, and obtain 7 super-resolution images after the reconstruction of the hyperspectral image.
In order to verify the simulation experiment effect of the present invention, the up-sampling coefficient was set to 8, and the average peak signal-to-noise ratio of 7 hyperspectral super-resolution reconstructed images was calculated using the following average peak signal-to-noise ratio PSNR (Peak Signal to Noise Ratio) formula, and the result was 33.723dB.
Wherein MSE (I) HR ,I SR ) Representing the input of two hyperspectral images I to a formula HR And I SR The mean square error MSE is calculated and,and->Representing a corresponding hyperspectral image I HR And I SR Pixel value size with upper coordinates (I, j), PSNR (I HR ,I SR ) Representing the input of two hyperspectral images I to a formula HR And I SR Calculated peak signal-to-noise ratio PSNR value, log 10 (. Cndot.) represents a base 10 logarithmic operation,/->Representing summing the images by width W and length H, respectively.
The average spectral angle similarity of 7 hyperspectral super-resolution reconstructed images was calculated using the average spectral angle similarity SAM (SpectralAngle Mapper) formula described below, and the result was 5.027.
Wherein θ (z * ,z h ) Representing the pair of two sets of vectors z input to the formula * And z h Calculating cosine similarity, cos -1 (. Cndot.) represents a cosine similarity calculation operation, SAM (I HR ,I SR ) Representing the input of two hyperspectral images I to a formula HR And I SR The value of the SAM that is calculated is,representing summing each pixel of the image.
The simulation experiment shows that: according to the invention, by adding the spectrum and spatial combined characteristic extraction network to the network, the combined characteristics of the spatial domain and the spectrum domain in the hyperspectral image data can be fully excavated, and the attention mechanism of the spatial combined characteristics is adjusted by utilizing the characteristic correlation of the spectrum dimension, so that the spatial and spectral information is fused to a great extent, the capturing capacity of the spectral information in the super-resolution reconstruction process is improved, and the reconstruction effect is effectively increased. Meanwhile, the added multi-level feature fusion network of the space high-low frequency information carries out feature multiplexing and fusion on multi-level features generated in the process of feature network transmission, different features are grouped in a mode of introducing the space high-low frequency information, feature communication and fusion between groups and in the groups greatly improve the utilization rate of the network to the features, information fusion of image features at all levels is promoted, the integral efficiency of an algorithm is improved, and the method is a very practical hyperspectral image super-resolution method.
Claims (10)
1. A super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion is characterized in that a spatial spectrum combined feature extraction sub-network is constructed, and a multi-level feature fusion sub-network of spatial high-low frequency information is constructed; the reconstruction method comprises the following steps:
step 1, constructing a spatial spectrum joint feature extraction sub-network:
the method comprises the steps of constructing a spatial spectrum joint feature extraction sub-network, wherein the spatial spectrum joint feature extraction sub-network comprises a first spatial spectrum joint attention layer, a second spatial spectrum joint attention layer, a third spatial spectrum joint attention layer, a fourth spatial spectrum joint attention layer, a first convolution layer, a first activation layer, a second convolution layer and a second activation layer; the first to fourth spatial spectrum joint attention layers have the same structure; setting the sizes of the first convolution layer and the second convolution layer to be 1 multiplied by 1, setting the step length to be 1, setting the output filling zero to be [0, 0], and setting the cavity value to be [1, 1]; the first and second active layers are realized by using a LeakyReLu function, and the slopes of the function in the negative number part are all set to be 0.01;
step 2, constructing a multi-level feature fusion sub-network of the space high-low frequency information:
constructing a multi-level characteristic fusion sub-network of the space high-low frequency information, which is formed by sequentially connecting a space high-low frequency separation layer, a fusion layer, a first convolution layer, a second convolution layer, a first activation layer, a third convolution layer and a second activation layer in series; the fusion layer is formed by connecting a space high-frequency fusion layer and a space low-frequency fusion layer in parallel;
the parameters of the multistage feature fusion subnetwork for setting the space high-low frequency information are as follows: setting the sizes of the first convolution layer to the third convolution layer to be 1 multiplied by 1, setting the step length to be 1, setting the output filling zero to be 0,0 and setting the cavity value to be 1,1 and 1; the first activation layer is realized by adopting a sigmoid function; the second active layer is realized by adopting a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01;
step 3, constructing a super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion:
constructing a super-resolution network formed by serially connecting a shallow feature extraction layer and a mapping reconstruction layer and integrating spectral correlation information and spatial high-low frequency information features; the mapping reconstruction layer is formed by connecting a space spectrum combined characteristic extraction sub-network and a multi-level characteristic fusion sub-network of space high-low frequency information in parallel;
step 4, generating a training set:
selecting at least 25 hyperspectral images, wherein the number C, the length H and the width W of spectrum bands of the resolution of each hyperspectral image are respectively more than or equal to 10, 32 and 32; carrying out data preprocessing on each hyperspectral image to obtain an image pair of the hyperspectral image, and forming a training set by all the image pairs;
step 5, training a super-resolution network:
inputting the training set into a super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion, and iteratively updating network parameters by adopting an Adam optimization algorithm until a loss function converges to obtain a trained super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion;
step 6, carrying out super-resolution reconstruction on the hyperspectral image:
and 5, preprocessing the low-spatial-resolution hyperspectral image to be reconstructed by adopting the same method as the step 5, inputting the preprocessed hyperspectral image to a trained network, reconstructing the hyperspectral image with super resolution, and outputting the reconstructed hyperspectral image with high resolution.
2. The super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion according to claim 1, wherein the structure of the spatial spectrum joint attention layer in step 1 is sequentially connected in series: a first convolution layer, a first activation layer, a first parallel layer, a sixth convolution layer, a sixth activation layer, a seventh convolution layer, a seventh activation layer, a second parallel layer, an eighth convolution layer, a ninth convolution layer, an eighth activation layer; the first parallel layer is formed by connecting a first branch and a second branch in parallel, the second convolution layer and the second activation layer are connected in series to form the first branch, and the fourth convolution layer and the fourth activation layer are connected in series to form the second branch; the second parallel layer is formed by connecting a third branch and a fourth branch in parallel, the third convolution layer and the third activation layer are connected in series to form a third branch, and the fifth convolution layer and the fifth activation layer are connected in series to form a fourth branch.
3. The super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion according to claim 1, wherein the parameters of each layer of the spatial spectrum joint attention layer in step 1 are set as follows: setting the sizes of the first convolution layer, the second convolution layer, the third convolution layer and the sixth convolution layer to be 3 multiplied by 1, setting the step length to be 1, setting the output filling zero to be [1, 0], and setting the cavity value to be [1, 1]; the sizes of the fourth, fifth and seventh convolution layers are all set to 3 multiplied by 1, the step sizes are all set to 1, the output filling zero padding is all set to [2,0,0], the cavity values are all set to [2, 1], setting the sizes of the eighth convolution layer and the ninth convolution layer to be 1 multiplied by 1, setting the step length to be 1, setting the output filling zero to be 0,0 and setting the cavity value to be 1,1 and 1; the first to fifth active layers are all realized by using a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01; the sixth to ninth active layers are implemented using sigmoid functions.
4. The super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion according to claim 1, wherein in the step 2, the spatial high-low frequency separation layer is formed by sequentially connecting a first convolution layer, a first activation layer, a first combination layer and a downsampling layer in series; the first combination layer is formed by connecting two branches in parallel, the second convolution layer and the second activation layer are connected in series to form a first branch, and the third convolution layer and the third activation layer are connected in series to form a second branch; the parameters for setting the spatial high-low frequency separation layer are as follows, the sizes of the first to third convolution layers are set to 1 x 1,3 x 3, the step sizes are all set to 1, the output filling zero padding is respectively set to 0,0 and 1, [2, 2] the void values are set to [1, 1], [2, 2] respectively; the first to third active layers are all realized by using a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01; the downsampling layer adopts a tri-linear interpolation function as a downsampling function, and the sampling multiple is set to be 2.
5. The super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion according to claim 1, wherein in step 2, the spatial high-frequency fusion layer is composed of a high-frequency parallel layer, a second convolution layer and a second active space in series; the high-frequency parallel layer is formed by connecting a first branch and a second branch in parallel, the first convolution layer and the first activation layer are connected in series to form a first branch, and the downsampling layer, the third convolution layer, the third activation layer and the upsampling layer are connected in series to form a second branch; setting the sizes of the first convolution layer to the third convolution layer to be 3 multiplied by 3, setting the step length to be 1, setting the output filling zero to be [1, 1], and setting the cavity value to be [1, 1]; the first to third active layers are all realized by using a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01; the downsampling layer adopts a tri-linear interpolation function as a downsampling function, and the sampling multiple is set to be 2; the upsampling layer adopts a tri-linear interpolation function as the upsampling sampling function, and the sampling multiple is set to 2.
6. The super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion according to claim 1, wherein in step 2, the spatial low frequency fusion layer is formed by sequentially connecting a low frequency parallel layer, a second convolution layer and a second activation layer in series; the low-frequency parallel layer is formed by connecting a first branch and a second branch in parallel, the first convolution layer and the first activation layer are connected in series to form the first branch, and the up-sampling layer, the third convolution layer, the third activation layer and the down-sampling layer are connected in series to form the second branch; setting the sizes of the first convolution layer to the third convolution layer to be 3 multiplied by 3, setting the step length to be 1, setting the output filling zero to be [1, 1], and setting the cavity value to be [1, 1]; the first to third active layers are all realized by using a LeakyReLu function, and the slope of the function in the negative number part is set to be 0.01; the up-sampling layer adopts a tri-linear interpolation function as a sampling function of up-sampling, and the sampling multiple is set to be 2; the downsampling layer adopts a tri-linear interpolation function as a downsampling function, and the sampling multiple is set to be 2.
7. The super-resolution network based on spectrum related information and spatial high-low frequency information feature fusion according to claim 1, wherein the shallow feature extraction layer in step 3 is formed by serially connecting a convolution layer and an activation layer; the convolution kernel size of the convolution layer is set to be 3 multiplied by 3, the step length is set to be 1, and the hole value is set to be [1, 1]; the active layer is implemented using the LeakyReLu function, which is set to 0.01 in slope in the negative part.
8. The super-resolution network based on the feature fusion of spectrum related information and spatial high-low frequency information according to claim 1, wherein the step of preprocessing the data in step 4 is as follows:
the first step, respectively carrying out downsampling operations of 1, 0.75 and 0.5 times on each original high-resolution hyperspectral image to obtain high-resolution hyperspectral images under three corresponding sampling times, wherein the resolution of the high-resolution hyperspectral images under each sampling time is as followsr represents a downsampling coefficient of 1, 0.75 and 0.5, respectively;
secondly, carrying out normalization processing on the high-resolution hyperspectral image under each sampling multiple according to the following formula:
I HR ＝I hr /65535.0
wherein I is HR Representing normalized high resolution image, I hr Representing a high resolution image at each sampling multiple;
thirdly, downsampling is carried out on each normalized high-resolution image, and each downsampled low-resolution image and the normalized high-resolution image form an image pair.
9. The super-resolution network based on the feature fusion of spectrum related information and spatial high-low frequency information according to claim 1, wherein the parameters of Adam optimization algorithm in step 5 are set as follows: the exponential decay rate was set to 0.9 and 0.999, respectively, eps was set to 1e-8, and the step size was set to 0.0001.
10. The super-resolution network based on spectral correlation information and spatial high-low frequency information feature fusion according to claim 1, wherein the loss function in step 5 is as follows:
wherein, loss (·) represents the Loss function,representing an ith normalized high resolution hyperspectral image input into a super resolution network based on fusion of spectral related information and spatial high and low frequency information features, < >>Representing an ith high-resolution hyperspectral image output in a super-resolution network based on feature fusion of spectrum related information and spatial high-low frequency information, N represents the total number of batch images of one time input into a training set in the network, sigma (·) represents a summation operation, and I·| represents an absolute value taking operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310717620.6A CN116612010A (en) | 2023-06-16 | 2023-06-16 | Super-resolution image reconstruction method based on spectrum related information and space high-low frequency information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310717620.6A CN116612010A (en) | 2023-06-16 | 2023-06-16 | Super-resolution image reconstruction method based on spectrum related information and space high-low frequency information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116612010A true CN116612010A (en) | 2023-08-18 |
Family
ID=87685430
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310717620.6A Pending CN116612010A (en) | 2023-06-16 | 2023-06-16 | Super-resolution image reconstruction method based on spectrum related information and space high-low frequency information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116612010A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117437123A (en) * | 2023-09-27 | 2024-01-23 | 宁波大学 | Spectrum and texture attention fusion depth spectrum super-resolution method |
| CN117522687A (en) * | 2023-11-03 | 2024-02-06 | 西安电子科技大学 | Super-resolution reconstruction method of hyperspectral image based on particle dynamics |
-
2023
- 2023-06-16 CN CN202310717620.6A patent/CN116612010A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117437123A (en) * | 2023-09-27 | 2024-01-23 | 宁波大学 | Spectrum and texture attention fusion depth spectrum super-resolution method |
| CN117522687A (en) * | 2023-11-03 | 2024-02-06 | 西安电子科技大学 | Super-resolution reconstruction method of hyperspectral image based on particle dynamics |
| CN117522687B (en) * | 2023-11-03 | 2024-05-14 | 西安电子科技大学 | Super-resolution reconstruction method of hyperspectral image based on particle dynamics |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113222822B (en) | Hyperspectral image super-resolution reconstruction method based on multi-scale transformation | |
| CN113420662B (en) | Remote sensing image change detection method based on twin multi-scale difference feature fusion | |
| CN110660038B (en) | Multispectral image and full-color image fusion method based on generation countermeasure network | |
| CN116612010A (en) | Super-resolution image reconstruction method based on spectrum related information and space high-low frequency information | |
| CN111429349B (en) | Hyperspectral image super-resolution method based on spectrum constraint countermeasure network | |
| CN114254715B (en) | Super-resolution method, system and application of GF-1WFV satellite image | |
| CN110533591B (en) | Super-resolution image reconstruction method based on codec structure | |
| CN113538246B (en) | Remote sensing image super-resolution reconstruction method based on unsupervised multi-stage fusion network | |
| CN112699844A (en) | Image super-resolution method based on multi-scale residual error level dense connection network | |
| CN113096015B (en) | Image super-resolution reconstruction method based on progressive perception and ultra-lightweight network | |
| CN110838089A (en) | Fast image denoising method based on OctBlock dense block | |
| CN110751699B (en) | Color reconstruction method of optical remote sensing image based on convolutional neural network | |
| CN112883908A (en) | Space-frequency characteristic consistency-based SAR image-to-optical image mapping method | |
| CN114529482B (en) | Image compressed sensing reconstruction method based on wavelet multichannel depth network | |
| CN115909077A (en) | Hyperspectral image change detection method based on unsupervised spectrum unmixing neural network | |
| CN113191947B (en) | Image super-resolution method and system | |
| CN114331894A (en) | Face image restoration method based on potential feature reconstruction and mask perception | |
| CN110807746B (en) | Hyperspectral image sharpening method based on detail embedded injection convolutional neural network | |
| Synthiya Vinothini et al. | Attention-Based SRGAN for Super Resolution of Satellite Images | |
| CN117689579B (en) | SAR auxiliary remote sensing image thick cloud removal method with progressive double decoupling | |
| CN117474765B (en) | DEM super-resolution reconstruction system based on reference image texture transfer | |
| CN113902618B (en) | Image super-resolution algorithm based on multi-modal spatial filtering | |
| CN116152441B (en) | Multi-resolution U-net curved surface reconstruction method based on depth priori | |
| CN111161152B (en) | Image super-resolution method based on self-adaptive convolutional neural network | |
| Thool et al. | Combined wavelet and multiwavelet-based convolutional neural network (CWM-CNN) for remote sensing image super resolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |