CN112508786B - Satellite image-oriented arbitrary-scale super-resolution reconstruction method and system - Google Patents
Satellite image-oriented arbitrary-scale super-resolution reconstruction method and system Download PDFInfo
- Publication number
- CN112508786B CN112508786B CN202011404879.8A CN202011404879A CN112508786B CN 112508786 B CN112508786 B CN 112508786B CN 202011404879 A CN202011404879 A CN 202011404879A CN 112508786 B CN112508786 B CN 112508786B
- Authority
- CN
- China
- Prior art keywords
- satellite image
- resolution
- image
- satellite
- reconstruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 19
- 230000003321 amplification Effects 0.000 claims abstract description 16
- 238000013507 mapping Methods 0.000 claims abstract description 16
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 7
- 230000004927 fusion Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 238000005065 mining Methods 0.000 claims description 4
- 238000012360 testing method Methods 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000004913 activation Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000004821 distillation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a satellite image-oriented arbitrary-scale super-resolution reconstruction method and system, which improve the expression capacity of a satellite image by improving a meta-amplification module and edge enhancement based on the characteristics of the satellite image, realize reconstruction and comprise low-resolution satellite image shallow layer feature extraction, adopt a residual error dense network to extract image deep layer features and fuse the low-resolution satellite image features; a weight prediction network in the element amplification module is adopted to obtain filter weights, and a more accurate weight is obtained by optimizing a projection mapping function, so that a more accurate reconstruction result of any resolution of the satellite image is obtained and is used as an intermediate reconstructed satellite image; clear expression of satellite image texture is achieved through satellite image edge enhancement, structural information of the satellite image is fully mined, and a final high-resolution satellite image is obtained. The method can improve the expression capability of satellite images with different resolutions, and the information such as edge details and the like is clearer, so that the method can be widely applied to various fields such as earth observation and the like.
Description
Technical Field
The invention belongs to the field of image super-resolution, and particularly relates to an arbitrary-scale super-resolution reconstruction scheme for satellite images.
Background
In recent years, satellite images are widely used in many fields of earth observation, such as city planning, disaster monitoring, sea area monitoring, and moving object monitoring. The space resolution of satellite images acquired on the ground is far lower than that of natural images due to the limitation of hardware conditions such as the power consumption of satellite-borne equipment. The super-resolution reconstruction technology of the satellite images conjectures and recovers a clearer high-resolution image from one or more frames of low-resolution images by learning the mapping relation between the low-resolution images and the high-resolution images. The spatial resolution of the image is enhanced by designing a new technical scheme, and the method has important significance for strengthening the data expression capability of the satellite image and expanding the application scene of the satellite image.
The existing super-resolution methods (documents 1 and 2) of satellite images mainly adopt a deep learning-based method to model complex data, jump connection and dense connection are introduced to fully extract input information, and finer feature textures can be expressed, so that the reconstruction performance of model images is better improved. However, the super-resolution of different scale factors is considered as an independent task, and each scale factor only trains a specific model for integer scale factors (such as X2, X3 and X4), which has limitations in practical application.
In order to solve the super-resolution reconstruction of an image at an arbitrary scale, an arbitrary magnification network for super-resolution is proposed (document 3). The method trains a single model to solve the super-resolution of any scale factor including non-integer scale factors, utilizes a meta-amplification module to replace a traditional amplification module, uses the scale factors as the weights of an input dynamic prediction filter, and maps low-resolution features into high-resolution images with different sizes through the weights. However, due to the characteristics of weak texture and low resolution of the satellite image, the method has difficulty in recovering fine image content and clear edge contour information for the satellite image.
The related documents of the present invention:
[1]K.Jiang,Z.Wang,P.Yi,J.Jiang,J.Xiao,and Y.Yao,"Deep distillation recursive network for remote sensing imagery super-resolution,"Remote Sensing,vol.10,no.11,p.1700,2018.
[2]T.Lu,J.Wang,Y.Zhang,Z.Wang,and J.Jiang,"Satellite image super-resolution via multi-scale residual deep neural network,"Remote Sensing,vol.11,no.13,p.1588,2019.
[3]X.Hu,H.Mu,X.Zhang,Z.Wang,T.Tan,and J.Sun,"Meta-sr:A magnification-arbitrary network for super-resolution,"in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2019,pp.1575-1584.
disclosure of Invention
In order to solve the technical problem, the invention provides a satellite image-oriented arbitrary-scale super-resolution reconstruction technology, which is based on two characteristics of a satellite image: the method comprises the following steps of designing an improved element amplification module and an improved edge enhancement module to improve the expression capability of a satellite image by 'weak texture' and 'low resolution'.
The invention discloses a super-resolution reconstruction method of any scale for satellite image, which improves the expression ability of satellite image through improving element amplification module and edge enhancement based on the characteristics of satellite image,
firstly, shallow layer feature extraction of a low-resolution satellite image is carried out, deep layer features of the image are extracted by adopting a residual dense network, and the features of the final low-resolution satellite image are obtained through fusion;
then, amplifying the satellite image at any scale, wherein the method comprises the steps of adopting a weight prediction network in a unit amplification module to obtain filter weights, obtaining more accurate weights by optimizing a projection mapping function, and further obtaining more accurate reconstruction results of the satellite image at any resolution as intermediate reconstruction satellite images;
and finally, realizing clear expression of the texture of the satellite image by satellite image edge enhancement, fully mining the structural information of the satellite image, and obtaining the final high-resolution satellite image.
Furthermore, the optimized projection mapping function is set as follows,
defining the scale factor of the magnification as r, and describing the corresponding point (i ', j') of a certain pixel point (i, i) on the high-resolution image on the low-resolution image as
Wherein T (i, j) tableProjection mapping function [ alpha ], [ beta ] and [ alpha ], [ beta ] is]Representing a rounding function.
Furthermore, the satellite image edge enhancement is implemented as follows,
extracting edge characteristic graph from the intermediate reconstruction satellite image, and recording the input intermediate reconstruction satellite image as
The output is an edge feature map Iedge；
Reconstruction of satellite images from the middle
Extracting a structure tensor, and normalizing to obtain an image MASK (MASK);
utilizing image MASK MASK to restrain the edge characteristic graph to obtain enhanced edge information Isharp，Isharp＝Iedge·Imask；
Intermediate result to high resolution reconstruction image
And enhanced edge information IsharpAre superimposed according to
Obtaining a final super-resolution reconstruction satellite image ISR。
When extracting the edge feature map from the intermediate reconstructed satellite image, it is preferable to perform edge extraction using 11-layer convolutional layer cascade.
On the other hand, the invention also provides an arbitrary-scale super-resolution reconstruction system for the satellite images, which is used for realizing the arbitrary-scale super-resolution reconstruction method for the satellite images.
And, including the following modules,
the first module is used for extracting shallow features of the low-resolution satellite image, extracting deep features of the image by adopting a residual dense network, and fusing to obtain the final features of the low-resolution satellite image;
the second module is used for amplifying the satellite image in any scale, and comprises the steps of adopting a weight prediction network in the element amplification module to obtain filter weight, obtaining more accurate weight by optimizing a projection mapping function, and further obtaining a more accurate reconstruction result of the satellite image in any resolution as an intermediate reconstruction satellite image;
and the third module is used for realizing clear expression of the texture of the satellite image through satellite image edge enhancement, fully mining the structural information of the satellite image and obtaining the final high-resolution satellite image.
Alternatively, the system comprises a processor and a memory, wherein the memory is used for storing program instructions, and the processor is used for calling the stored instructions in the memory to execute the satellite image-oriented arbitrary scale super-resolution reconstruction method.
Alternatively, a readable storage medium is included, on which a computer program is stored, which, when executed, implements a method for satellite image oriented arbitrary scale super resolution reconstruction as described above.
By adopting the technical scheme, the method and the device can be used for generating the satellite image with any resolution, improve the data expression capability of the satellite image and improve the user viewing experience of the mobile terminal with different resolutions. Compared with the prior art, the invention has the following advantages and beneficial effects:
1) compared with the prior art, the method solves a new problem, namely the problem of reconstruction of the super-resolution of any scale of the satellite image.
2) Compared with the prior art, the invention provides a super-resolution reconstruction framework facing satellite images based on an enhancement element amplification module.
3) Compared with the prior art, the method provided by the invention has the advantages that the characteristics of the satellite image are mined, the edge enhancement algorithm is designed by utilizing the characteristics of the satellite image, the arbitrary-scale super-resolution reconstruction technology of the satellite image is realized, the expression capability of the satellite image is enhanced, and the method can be widely applied to various fields of earth observation.
Drawings
FIG. 1 is a main flow diagram of an embodiment of the present invention.
Fig. 2 is a network configuration diagram of an embodiment of the present invention.
Fig. 3 is a schematic diagram of a residual dense block according to an embodiment of the present invention.
Detailed Description
In order to facilitate the understanding and implementation of the present invention for those of ordinary skill in the art, the present invention is further described in detail with reference to the accompanying drawings and examples, it is to be understood that the embodiments described herein are merely illustrative and explanatory of the present invention and are not restrictive thereof.
The invention provides a satellite image-oriented arbitrary-scale super-resolution reconstruction technology, which is based on two characteristics of a satellite image: the method comprises the following steps of designing an improved element amplification module and an improved edge enhancement module to improve the expression capability of a satellite image by 'weak texture' and 'low resolution'. Firstly, shallow features of an image are obtained by utilizing two convolutional layers, and then features of a low-resolution image are obtained based on a residual dense network. And the arbitrary scale amplification of the satellite image is realized through the improved element amplification module to obtain an intermediate result. And finally, realizing clear expression of the texture of the satellite image through an edge enhancement module to obtain a final high-resolution satellite image.
In this example, 4 NVIDIA GTX2080Ti GPUs were used to run experiments in parallel, with training data from a common satellite image data set WHU-RS 19. WHU-RS19 is a dataset of remote sensing images collected from Google Earth, covering 19 categories, including: airports, bridges, farmlands, football fields, industrial areas, residential areas, rivers, etc., with 50 images per category. The size of the HR image is 600 × 600 pixels. The example randomly selected 900 images and used 800, 50 and 50 images as training set, validation set and test sample, respectively. Since the performance of the super-resolution reconstruction method is related to the similarity of the test images and the training images, embodiments perform additional tests on the open satellite image dataset NWPU-resic 45 to ensure the robustness and generalization of the present invention.
Referring to fig. 1, a flow of a method for reconstructing super-resolution at any scale for satellite images according to an embodiment of the present invention includes the following steps:
step 1, extracting the characteristics of the low-resolution satellite image, namely extracting the characteristics of the low-resolution satellite image by using a residual error dense network, and comprising the following substeps:
step 1.1, collecting satellite image public data sets shot by different satellites, selecting image sequences of different scenes, and respectively setting a training set, a verification set and a test set after cutting and screening;
step 1.2, for the training set image in step 1.1, firstly, double trilinear interpolation is adopted to obtain corresponding 1.1 to 4.0 low-resolution images with different scales (namely, the scale step length is 0.1), the formed high-resolution-low-resolution image pair is used as the input of a depth network, and the method is the same as most of the existing depth learning-based super-resolution reconstruction algorithms, and the embodiment also adopts two cascaded convolutional layers to extract the shallow feature F of the image1。
Step 1.3, extracting deep features of the image by using a residual dense module, fusing local features and global features of the satellite image, and obtaining final features F of the low-resolution satellite imageLR；
Referring to fig. 3, the residual dense module in this step can be implemented by a dense connected block composed of a plurality of convolutional layers with activation functions, and a feature fusion layer and a convolutional layer, for example, as in reference [3 ].
Referring to fig. 2, in the embodiment, the feature extraction module includes two cascaded convolutional layers, a plurality of cascaded residual error dense blocks, a feature fusion layer, a convolutional layer, and a final convolutional layer, which are sequentially arranged, and outputs of the plurality of residual error dense blocks are all input to the feature fusion layer for fusion.
And 2, arbitrarily amplifying the satellite image to realize the mapping from the low-resolution satellite image characteristic to the high-resolution image with an arbitrary resolution.
The meta-amplification module in the prior art comprises 3 parts: position projection, weight prediction and feature mapping. The position projection is realized by adopting a down-rounding function, and the corresponding point (i ', j') of a certain pixel point (i, j) on the high-resolution image on the low-resolution image can be described as
T (i, j) represents a projection mapping function,
representing a floor function. This operation results in a large loss of image pixel values, which are of paramount importance for satellite images due to their weak texture characteristics. In order to reduce the loss of pixel values, the invention proposes to use a rounding function instead of a rounding function, i.e. an improved projection mapping function of
[]Representing a rounding function.
Referring to fig. 2, the present step implementation includes the following sub-steps,
step 2.1, finding out the pixel point on the high-resolution image to be projected and mapped to the corresponding pixel point on the low-resolution image, defining the amplified scale factor as r, and describing the corresponding point (i ', j') of a certain pixel point (i, j) on the high-resolution image on the low-resolution image as
Wherein T (i, j) represents a projection mapping function [ alpha ], [ beta ] represents a)]Representing a rounding function.
Step 2.2, a weight prediction network is formed by two full connection layers and an activation function layer, the filter weight from the low-resolution feature to the high-resolution image is predicted, and referring to fig. 2, the full connection layers, the activation function layer and the full connection layers are sequentially arranged;
the weight prediction network can be represented as W (i, j) ═ ψ (V (i, j); θ), where W () represents the weight of the filter, ψ () represents the weight prediction operation, and V (i, j) is the weight prediction network input, defined as
r represents a scale factor and θ is a parameter of the weighting network.
Step 2.3. obtaining the low resolution satellite image from step 1.2Characteristic FLR(i, j) is multiplied by the weight obtained in step 2.2 to obtain the intermediate result of the high-resolution reconstructed image
Can be expressed as
Where Φ represents a matrix multiplication operation, the intermediate reconstructed image can thus be represented as
Step 3, satellite image edge enhancement, extracting edge characteristics of the satellite image, and recovering high-frequency information in the satellite image, wherein the preferred implementation process in the embodiment comprises the following substeps,
step 3.1, extracting the edge feature map from the intermediate reconstructed satellite image, preferably using 11 layers of convolutional layer cascade to realize edge extraction, and recording the input intermediate reconstructed satellite image as
The output is an edge feature map Iedge；
Step 3.2. reconstruction of satellite images from the middle
And extracting the structure tensor, and obtaining an image MASK after normalization. First, the structure tensor of the image is calculated
Wherein IxAnd IyRefers to the horizontal and vertical gradients, I, of the imagexyAnd IyxRespectively representing the sequential calculation of horizontal and vertical gradients and the sequential calculation of vertical and horizontal gradients for an image. Then, the trace of the matrix is adopted to judge the amount of high-frequency information in the image pixel. The image MASK is calculated as follows:
where M, n represent the height and width of the image, respectively, S (M, n) is the structure tensor, tr (S (M, n)) represents the trace of the structure tensor, McA mask representing the image. In order to adjust the dynamic range of the pixel values of the mask to 0-255 and maintain the relative relationship between the matrix elements, the embodiment normalizes the image mask to obtain Imask. Is a specific exercise
Wherein max () and min () represent the maximum value and the minimum value of the image pixel, respectively, and (x, y) represent the pixel point on the image mask.
Step 3.3. to avoid introducing noise, the embodiment utilizes image mask ImaskConstraining the edge characteristic graph to obtain enhanced edge information IsharpCan be expressed as image edge information IedgeAnd an image mask ImaskDot product of (c): i issharp＝Iedge·Imask。
Step 3.4. intermediate reconstruction of satellite images
And enhanced edge information IsharpAre superimposed according to
Obtaining a final super-resolution reconstruction satellite image ISR。
Step 4, evaluating performance of super-resolution reconstruction image of satellite image in any scale
Step 4.1, verifying and testing the performance of the super-resolution reconstruction image of the satellite image in any scale respectively by adopting the verification set and the test set obtained in the step 1.1, and adopting the peak signal-to-noise ratio (PSNR) of the image as an objective evaluation index of the image quality
And 4.2, adopting a classical double-three nonlinear interpolation algorithm Bicubic and an image super-resolution arbitrary amplification network Meta-SR as a comparison algorithm, wherein the method provided by the invention is called Ours.
In specific implementation, the automatic operation of the above processes can be realized by adopting a software mode. Experiments by adopting the process show that clear edge information of the image can be obtained by adopting the satellite image arbitrary-scale super-resolution technology after edge enhancement, and the effect is obviously superior to that of the existing image arbitrary-scale super-resolution algorithm.
Based on the results obtained by the steps 1-3, it can be seen in table 1 that the method of the present invention is superior to other comparison algorithms under different non-integer-multiple scale factors.
Table 1 results for any of the scale factors on the different methods, the test data set is WHU-RS19, the best results are shown in bold.
Therefore, according to the satellite image-oriented hyper-resolution reconstruction framework based on the improved element amplification model, the structural characteristics of the satellite image are explicitly expressed under the model, and the satellite image reconstruction with any resolution can be realized.
In specific implementation, a person skilled in the art can implement the automatic operation process by using a computer software technology, and a system device for implementing the method, such as a computer-readable storage medium storing a corresponding computer program according to the technical solution of the present invention and a computer device including a corresponding computer program for operating the computer program, should also be within the scope of the present invention.
In some possible embodiments, a satellite image-oriented arbitrary-scale super-resolution reconstruction system is provided, comprising the following modules,
the first module is used for extracting shallow features of the low-resolution satellite image, extracting deep features of the image by adopting a residual dense network, and fusing to obtain the final features of the low-resolution satellite image;
the second module is used for amplifying the satellite image in any scale, and comprises the steps of adopting a weight prediction network in the element amplification module to obtain filter weight, obtaining more accurate weight by optimizing a projection mapping function, and further obtaining a more accurate reconstruction result of the satellite image in any resolution as an intermediate reconstruction satellite image;
and the third module is used for realizing clear expression of the texture of the satellite image through satellite image edge enhancement, fully mining the structural information of the satellite image and obtaining the final high-resolution satellite image.
In some possible embodiments, a satellite image-oriented arbitrary-scale super-resolution reconstruction system is provided, which includes a processor and a memory, the memory is used for storing program instructions, and the processor is used for calling the stored instructions in the memory to execute a satellite image-oriented arbitrary-scale super-resolution reconstruction method as described above.
In some possible embodiments, a satellite image-oriented arbitrary-scale super-resolution reconstruction system is provided, which includes a readable storage medium, on which a computer program is stored, and when the computer program is executed, the satellite image-oriented arbitrary-scale super-resolution reconstruction method is implemented.
It should be understood that parts of the specification not set forth in detail are well within the prior art.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Claims (6)
1. A satellite image-oriented arbitrary scale super-resolution reconstruction method is characterized by comprising the following steps: based on the characteristics of the satellite images, the representation capability of the satellite images is improved by improving the element amplification module and the edge enhancement, the reconstruction realization process is as follows,
firstly, shallow layer feature extraction of a low-resolution satellite image is carried out, deep layer features of the image are extracted by adopting a residual dense network, and the features of the final low-resolution satellite image are obtained through fusion;
then, amplifying the satellite image at any scale, wherein the method comprises the steps of adopting a weight prediction network in a unit amplification module to obtain filter weights, obtaining more accurate weights by optimizing a projection mapping function, and further obtaining more accurate reconstruction results of the satellite image at any resolution as intermediate reconstruction satellite images;
finally, clear expression of satellite image textures is achieved through satellite image edge enhancement, structural information of the satellite images is fully mined, and a final high-resolution satellite image is obtained;
the optimized projection mapping function is set as follows,
defining the amplified scale factor as r, and describing the corresponding point (i ', j') of a certain pixel point (i, j) on the high-resolution image on the low-resolution image as
Wherein T (i, j) represents a projection mapping function [ alpha ], [ beta ] represents a)]Represents a rounding function;
the satellite image edge enhancement is implemented as follows,
extracting edge characteristic graph from the intermediate reconstruction satellite image, and recording the input intermediate reconstruction satellite image as
The output is an edge feature map Iedge；
Reconstruction of satellite images from the middle
Extracting a structure tensor, and normalizing to obtain an image MASK (MASK);
utilizing image MASK MASK to restrain the edge characteristic graph to obtain enhanced edge information Isharp，Isharp＝Iedge·Imask；
Intermediate result to high resolution reconstruction image
And enhanced edge information IsharpAre superimposed according to
Obtaining a final super-resolution reconstruction satellite image ISR。
2. The satellite image-oriented arbitrary-scale super-resolution reconstruction method according to claim 1, characterized in that: when extracting the edge feature map from the intermediate reconstructed satellite image, the edge extraction is preferably implemented by using 11-layer convolutional layer cascade.
3. An arbitrary scale super-resolution reconstruction system for satellite images is characterized in that: the method for realizing any-scale super-resolution reconstruction of the satellite-oriented image according to any one of claims 1-2.
4. The satellite image-oriented arbitrary-scale super-resolution reconstruction system according to claim 3, wherein: comprises the following modules which are used for realizing the functions of the system,
the first module is used for extracting shallow features of the low-resolution satellite image, extracting deep features of the image by adopting a residual dense network, and fusing to obtain the final features of the low-resolution satellite image;
the second module is used for amplifying the satellite image in any scale, and comprises the steps of adopting a weight prediction network in the element amplification module to obtain filter weight, obtaining more accurate weight by optimizing a projection mapping function, and further obtaining a more accurate reconstruction result of the satellite image in any resolution as an intermediate reconstruction satellite image;
and the third module is used for realizing clear expression of the texture of the satellite image through satellite image edge enhancement, fully mining the structural information of the satellite image and obtaining the final high-resolution satellite image.
5. The satellite image-oriented arbitrary-scale super-resolution reconstruction system according to claim 3, wherein: comprising a processor and a memory for storing program instructions, the processor being configured to invoke the stored instructions in the memory to perform a method of satellite image oriented arbitrary scale super resolution reconstruction as claimed in any of claims 1-2.
6. The satellite image-oriented arbitrary-scale super-resolution reconstruction system according to claim 3, wherein: comprising a readable storage medium having stored thereon a computer program which, when executed, implements a method for satellite image oriented arbitrary-scale super-resolution reconstruction as claimed in any one of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011404879.8A CN112508786B (en) | 2020-12-03 | 2020-12-03 | Satellite image-oriented arbitrary-scale super-resolution reconstruction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011404879.8A CN112508786B (en) | 2020-12-03 | 2020-12-03 | Satellite image-oriented arbitrary-scale super-resolution reconstruction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112508786A CN112508786A (en) | 2021-03-16 |
| CN112508786B true CN112508786B (en) | 2022-04-29 |
Family
ID=74968439
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011404879.8A Active CN112508786B (en) | 2020-12-03 | 2020-12-03 | Satellite image-oriented arbitrary-scale super-resolution reconstruction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112508786B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115205122B (en) * | 2022-09-06 | 2022-12-06 | 深圳大学 | Method, system, apparatus and medium for generating hyper-resolution image maintaining structure and texture |
| CN116523759B (en) * | 2023-07-04 | 2023-09-05 | 江西财经大学 | Image super-resolution reconstruction method and system based on frequency decomposition and restarting mechanism |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015180053A1 (en) * | 2014-05-28 | 2015-12-03 | 北京大学深圳研究生院 | Method and apparatus for rapidly reconstructing super-resolution image |
| CN107833182A (en) * | 2017-11-20 | 2018-03-23 | 西安建筑科技大学 | The infrared image super resolution ratio reconstruction method of feature based extraction |
| CN107833183A (en) * | 2017-11-29 | 2018-03-23 | 安徽工业大学 | A kind of satellite image based on multitask deep neural network while super-resolution and the method for coloring |
| CN108346133A (en) * | 2018-03-15 | 2018-07-31 | 武汉大学 | A kind of deep learning network training method towards video satellite super-resolution rebuilding |
| CN110866870A (en) * | 2019-10-29 | 2020-03-06 | 中山大学 | Super-resolution processing method for amplifying medical image by any multiple |
-
2020
- 2020-12-03 CN CN202011404879.8A patent/CN112508786B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015180053A1 (en) * | 2014-05-28 | 2015-12-03 | 北京大学深圳研究生院 | Method and apparatus for rapidly reconstructing super-resolution image |
| CN107833182A (en) * | 2017-11-20 | 2018-03-23 | 西安建筑科技大学 | The infrared image super resolution ratio reconstruction method of feature based extraction |
| CN107833183A (en) * | 2017-11-29 | 2018-03-23 | 安徽工业大学 | A kind of satellite image based on multitask deep neural network while super-resolution and the method for coloring |
| CN108346133A (en) * | 2018-03-15 | 2018-07-31 | 武汉大学 | A kind of deep learning network training method towards video satellite super-resolution rebuilding |
| CN110866870A (en) * | 2019-10-29 | 2020-03-06 | 中山大学 | Super-resolution processing method for amplifying medical image by any multiple |
Non-Patent Citations (2)
| Title |
|---|
| Long Term Background Reference Based Satellite Video Coding;Xu Wang,et al.;《ICASSP》;20190517;第1822-1826页 * |
| 星地协同的卫星视频高效压缩方法;肖晶等;《武汉大学学报》;20181231;第43卷(第12期);第2197-2204页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112508786A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111047516B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| Chen et al. | Real-world single image super-resolution: A brief review | |
| CN110119780B (en) | Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network | |
| Ren et al. | Single image super-resolution using local geometric duality and non-local similarity | |
| Kim et al. | Deep residual network with enhanced upscaling module for super-resolution | |
| CN109903255A (en) | A kind of high spectrum image Super-Resolution method based on 3D convolutional neural networks | |
| CN111105352A (en) | Super-resolution image reconstruction method, system, computer device and storage medium | |
| KR20140136019A (en) | Method and apparatus for performing hierarchical super-resolution of an input image | |
| Xiao et al. | A dual-UNet with multistage details injection for hyperspectral image fusion | |
| CN112508786B (en) | Satellite image-oriented arbitrary-scale super-resolution reconstruction method and system | |
| CN105488759B (en) | A kind of image super-resolution rebuilding method based on local regression model | |
| Dong et al. | Real-world remote sensing image super-resolution via a practical degradation model and a kernel-aware network | |
| CN112699844A (en) | Image super-resolution method based on multi-scale residual error level dense connection network | |
| CN113362338A (en) | Rail segmentation method, device, computer equipment and rail segmentation processing system | |
| Zou et al. | Joint wavelet sub-bands guided network for single image super-resolution | |
| CN117575915A (en) | Image super-resolution reconstruction method, terminal equipment and storage medium | |
| Deng et al. | Multiple frame splicing and degradation learning for hyperspectral imagery super-resolution | |
| Li | Image super-resolution using attention based densenet with residual deconvolution | |
| Wang et al. | Image super-resolution using multi-granularity perception and pyramid attention networks | |
| Fang et al. | Arbitrary scale super resolution network for satellite imagery | |
| Wang et al. | Asymmetric dual-direction quasi-recursive network for single hyperspectral image super-resolution | |
| CN109559278A (en) | Super resolution image reconstruction method and system based on multiple features study | |
| Liu et al. | Gradient prior dilated convolution network for remote sensing image super-resolution | |
| CN116503251A (en) | Super-resolution reconstruction method for generating countermeasure network remote sensing image by combining hybrid expert | |
| CN113205005B (en) | Low-illumination low-resolution face image reconstruction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |