CN112419155B - Super-resolution reconstruction method for fully-polarized synthetic aperture radar image - Google Patents

Super-resolution reconstruction method for fully-polarized synthetic aperture radar image Download PDF

Info

Publication number
CN112419155B
CN112419155B CN202011348480.2A CN202011348480A CN112419155B CN 112419155 B CN112419155 B CN 112419155B CN 202011348480 A CN202011348480 A CN 202011348480A CN 112419155 B CN112419155 B CN 112419155B
Authority
CN
China
Prior art keywords
resolution
training
image
aperture radar
attention
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011348480.2A
Other languages
Chinese (zh)
Other versions
CN112419155A (en
Inventor
沈焕锋
林镠鹏
李�杰
袁强强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN202011348480.2A priority Critical patent/CN112419155B/en
Publication of CN112419155A publication Critical patent/CN112419155A/en
Application granted granted Critical
Publication of CN112419155B publication Critical patent/CN112419155B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G06T2207/10044—Radar image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Abstract

The invention provides a method for reconstructing a super-resolution image of a full-polarization aperture radar based on a multi-scale attention mechanism, which constructs a residual convolution neural network embedded with modules of up-sampling, spatial attention, channel attention, multi-scale attention, adaptive loss functions and the like of a feature layer, combines a supervision type and zero-throw type training mechanism and obtains a network for training convergence. And performing super-resolution reconstruction on the low-resolution synthetic aperture radar image to be processed through the trained network to obtain a high-resolution fully-polarized synthetic aperture radar image, and effectively reconstructing the spatial information of the image while maintaining the polarization information of the image.

Description

Super-resolution reconstruction method for fully-polarized synthetic aperture radar image
Technical Field
The invention relates to the field of remote sensing image processing and computer vision, in particular to a method for reconstructing a super-resolution image of a fully-polarized synthetic aperture radar based on a multi-scale attention mechanism.
Background
The fully-polarized synthetic aperture radar can obtain various scattering information of ground objects in a single scene through different polarization modes, and therefore plays an important role in ship identification, post-earthquake evaluation, land utilization classification and other applications. However, due to the limitation of various factors such as the signal bandwidth and the antenna size of the synthetic aperture radar system, the spatial resolution of the image is inevitably reduced while multi-polarization information is obtained. Therefore, the method for reconstructing the spatial resolution by using the super-resolution reconstruction technology is an important way for improving the spatial information of the fully-polarized synthetic aperture radar.
The existing super-resolution method for the image of the fully-polarized synthetic aperture radar mainly has three types. The first is a frequency domain method based on fourier transform shift characteristics. The method can process a linear degradation model, but is difficult to better process a complete polarization synthetic aperture radar complex degradation model. The second category is spatial domain methods based on image priors. The method only uses the prior information of the image, and does not effectively utilize the external information. The third category of methods is based on deep learning. The method depends on an external database and can better fit a complex degradation process, but the network structure used by the method is relatively simple at present, and the utilization of the internal features of the image is relatively less. Therefore, it is necessary to develop a technique that can fit the complex degradation process of the fully-polarized synthetic aperture radar and efficiently reconstruct the spatial resolution thereof while maintaining the polarization information.
Disclosure of Invention
In order to solve the technical problems of the existing super-resolution reconstruction algorithm, the invention provides a super-resolution reconstruction method of a fully-polarized synthetic aperture radar image based on a multi-scale attention mechanism, so as to obtain the fully-polarized synthetic aperture radar image with high spatial resolution.
The technical scheme provided by the invention is as follows: a super-resolution reconstruction method for a fully-polarized synthetic aperture radar image based on a multi-scale attention mechanism comprises the following steps:
step 1, establishing an observation model of a full-polarization synthetic aperture radar image, wherein the model provides a degradation relation between a high-spatial-resolution full-polarization synthetic aperture radar image and a low-spatial-resolution full-polarization synthetic aperture radar image;
step 2, image preprocessing, namely constructing a training data set by utilizing the preprocessed images;
step 3, constructing a super-resolution reconstruction network of the fully-polarized synthetic aperture radar image based on a multi-scale attention mechanism;
the super-resolution reconstruction network takes a low-resolution synthetic aperture radar image as input, performs dimensionality increasing operation through convolution, and obtains a high-resolution feature layer through up-sampling of the feature layer; then, inputting the obtained high-resolution feature layer into a multi-scale attention module, extracting features of multiple scales, and performing cascade operation on the extracted feature layer; each scale attention module comprises three embedded modules, namely a space attention module, a channel attention module and an attention fusion module; finally, performing dimensionality reduction operation on the feature layer obtained by the multi-scale attention module through convolution to obtain a high-resolution synthetic aperture radar image;
step 4, training the super-resolution reconstruction network constructed in the step 3 to be convergent by using the training data set constructed in the step 2;
and 5, performing super-resolution reconstruction on the low-resolution synthetic aperture radar image by using the converged super-resolution reconstruction network trained in the step 4 to obtain a high-resolution fully-polarized synthetic aperture radar image.
Further, the observation model in step 1 is constructed as follows,
representing the high-resolution fully-polarized synthetic aperture radar image as x, and representing the degraded low-resolution fully-polarized synthetic aperture radar image as y, then the observation model of the fully-polarized synthetic aperture radar image is represented as:
y＝fds(x) (1)
wherein f isds(.) represents a down-sampling function.
Further, step 2 comprises the following two parts;
step 2.1, preprocessing the image, including radiation correction, terrain correction and multi-view processing, to obtain a corrected image; and (4) according to the observation model established in the step (1), performing down-sampling processing on the high-resolution image to obtain a low-resolution image. Obtaining a high-resolution and low-resolution fully-polarized synthetic aperture radar image pair containing the same ground object through cutting, and constructing a data set;
and 2.2, enhancing the data set constructed in the step 2.1, wherein the enhancing comprises rotating by 90 degrees, rotating by 180 degrees, rotating by 270 degrees and turning over to obtain a training data set.
Further, the super-resolution reconstruction network in the step 3 specifically includes;
step 3.1, constructing a feature layer upsampling module, wherein the module is used for performing upsampling operation on a low-resolution feature layer result obtained by convolution from the aspect of the feature layer to obtain a high-resolution feature layer, and the module is defined as:
Fhr＝fus(Flr) (2)
wherein, FhrRepresenting a high-resolution characteristic layer, FlrRepresenting low resolution feature layers, fus(.) represents an upsampling function;
step 3.2: constructing a spatial attention module, wherein the spatial attention module is used for weighting the spatial weight of the characteristic layer of the fully-polarized synthetic aperture radar and enhancing the spatial resolution of the image of the fully-polarized synthetic aperture radar, and the spatial attention module is defined as:
Figure BDA0002800581920000021
wherein the content of the first and second substances,
Figure BDA0002800581920000022
representing feature layers weighted by a spatial attention module,
Figure BDA0002800581920000023
spatial domain, C, H, representing a feature layer,W represents the number of channels, height and width of the feature layer respectively,
Figure BDA0002800581920000031
a layer of spatial attention input features is represented,
Figure BDA0002800581920000032
a spatial attention weight map is represented,
Figure BDA0002800581920000033
representing an element multiplication operation; the spatial attention weight map calculation method is as follows:
Figure BDA0002800581920000034
wherein, FspaRepresents the input feature layer, sigma (.) and delta (.) represent the Sigmoid activation function and the ReLU activation function, respectively,
Figure BDA0002800581920000035
which represents a convolution operation, the operation of the convolution,
Figure BDA0002800581920000036
and
Figure BDA0002800581920000037
two weight terms and two bias terms respectively representing spatial attention modules;
step 3.3, constructing a channel attention module, wherein the module is used for weighting the polarization channel weight of the full-polarization synthetic aperture radar feature layer and maintaining the polarization information of each channel, and the module is defined as:
Figure BDA0002800581920000038
wherein the content of the first and second substances,
Figure BDA0002800581920000039
representing feature layers weighted by the channel attention module,
Figure BDA00028005819200000310
a layer of channel attention input features is represented,
Figure BDA00028005819200000311
a channel attention weight map is shown,
Figure BDA00028005819200000312
representing an element multiplication operation; the channel attention weight map calculation method is as follows:
Figure BDA00028005819200000313
wherein, FchaRepresenting input feature layer, Pavg(.) represents an average pooling operation,
Figure BDA00028005819200000314
and
Figure BDA00028005819200000315
two weight terms and two bias terms, respectively, representing channel attention, wherein,
Figure BDA00028005819200000316
step 3.4, an attention fusion module for performing information fusion on the spatial attention weight-weighting result and the channel attention weight-weighting result obtained in the steps 3.2 and 3.3, the module being defined as:
Figure BDA00028005819200000317
wherein, FfusFor the fused results, Concat (.) represents the feature layer cascading operation, WfusAnd bfusRespectively representing a weight term and a bias term of the attention fusion module;
step 3.5, constructing a multi-scale attention module, wherein the module comprises three scales, namely an original scale, an original scale downscaling scale and an original scale upscaling scale; the original scale is used for extracting the features of the target with the conventional size, the original scale downscaling is used for extracting the features of the small target, and the original scale upscaling is used for extracting the features of the large target; the module embeds the attention mechanism described in steps 3.2-3.4, which is defined as:
Figure BDA00028005819200000318
wherein, FmsFeature layer, W, representing a multiscale attention ModulemsAnd bmsRespectively representing a weight term and a bias term of the multi-scale attention module;
the original scale feature layer calculation method comprises the following steps:
Fs0＝Ffus(F) (9)
wherein, Fs0Representing a layer of original scale features, Ffus(.) represents an embedded attention fusion module, F represents an input feature layer, and the original scale down-scale feature layer is calculated as follows:
Fs1＝fds(Ffus(fus(F))) (10)
wherein, Fs1Representing a layer of original downscaled features, fds(.) represents the down-sampling function, fus(.) represents an upsampling function, and the original scale up feature layer is calculated as follows:
Fs2＝fus(Ffus(fds(F))) (11)
wherein, Fs2Representing an original scale up feature layer;
step 3.6, constructing an adaptive loss function, wherein the adaptive loss function consists of two parts, including an L1 loss function for avoiding overfitting of network parameters caused by abnormal values and an L2 loss function for maintaining numerical relationships, and the adaptive loss function is defined as the following form:
Ltotal(Θ)＝αL1(Θ)+βL2(Θ) (12)
wherein L istotal(Θ) denotes the adaptive loss function, L1(Θ) represents the L1 loss function, L2(Θ) represents the L2 loss function, Θ being a neural network parameter, α and β being regularization parameters for adjusting the weights of the L1 and L2 loss functions; specifically, the L1 loss function is defined as:
Figure BDA0002800581920000041
wherein N represents a training image pair
Figure BDA0002800581920000042
Number of (2), xi,yiRespectively representing a high-resolution fully-polarized synthetic aperture radar image and a low-resolution fully-polarized synthetic aperture radar image, rho, of the ith pair of training imagesiRepresenting a residual error between a sampling result on the characteristic layer and the high-resolution synthetic aperture radar image, and ξ (·) representing the output of the super-resolution reconstruction network;
the L2 loss function is defined as:
Figure BDA0002800581920000043
the calculation method of the self-adaptive regularization parameter comprises the following steps:
Figure BDA0002800581920000044
further, the training mechanism adopted in step 4 includes: a supervised training mechanism and a zero throw training mechanism; training the super-resolution reconstruction network by using a supervision type training mechanism to obtain a converged pre-training network, and then further training the pre-training network by using a zero throw type training mechanism to obtain a converged reconstruction network;
step 4.1, a supervision training mechanism, wherein the supervision training mechanism is to use an external database for supervision training; carrying out supervised training on the super-resolution reconstruction network through the low-high split-joint aperture radar image pair matched with the external data set by using the training data set constructed in the step 2 to obtain a convergent pre-training network, and fully utilizing the external information of the external data set through the mechanism;
4.2, a zero throw training mechanism, namely training by using data internal information, namely training a pre-training network by using a low-resolution aperture radar image; specifically, the training mechanism performs degradation processing on a low-resolution image to be processed by using a formula (1) to obtain a down-sampling result, in the training mechanism, the down-sampling result is used as a target image of a pre-training network, the low-resolution image to be processed is used as a reference image of the pre-training network, a pairing relation between the down-sampling result of the low-resolution image and the low-resolution image is established, the pre-training network is subjected to self-supervision training, and through the mechanism, the internal information of the image to be processed is fully utilized.
The invention has the advantages that:
(1) and performing super-resolution reconstruction on the image to be processed directly through an end-to-end residual convolutional neural network to obtain the high-resolution fully-polarized synthetic aperture radar.
(2) By embedding space attention and channel attention in a multi-scale module and designing L1 and L2 adaptive loss functions, a network capable of effectively extracting multi-scale target space information and polarization information is constructed.
(3) Through two training mechanisms, external database information and image internal information can be effectively utilized.
Drawings
Fig. 1 is a flowchart of a super-resolution reconstruction method according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a network framework according to an embodiment of the present invention.
Detailed Description
The technical scheme of the invention is explained in detail in the following by combining the attached drawings and the embodiment.
Step 1: establishing an observation model of a full-polarization synthetic aperture radar image, wherein the embodiment represents a high-resolution full-polarization synthetic aperture radar image as x, and represents a degraded low-resolution full-polarization synthetic aperture radar image as y, and then the observation model of the full-polarization synthetic aperture radar image can be represented as follows:
y＝fds(x) (1)
wherein f isds(.) represents a down-sampling function.
Step 2: and (5) image preprocessing.
Step 2.1: and preprocessing the image such as radiation correction, terrain correction, multi-view processing and the like to obtain a corrected image. And (4) performing down-sampling processing on the high-resolution image according to the observation model established in the step (1) to obtain a low-resolution image. And obtaining a high-resolution and low-resolution fully-polarized synthetic aperture radar image pair containing the same ground object by cutting, wherein the overlapping area between the image pair is 20%.
Step 2.2: and (3) enhancing the data set constructed in the step (2.1), wherein the data set is effectively expanded by rotating by 90 degrees, rotating by 180 degrees, rotating by 270 degrees and turning over.
And step 3: and constructing a full-polarization synthetic aperture radar image super-resolution reconstruction network based on a multi-scale attention mechanism, wherein the network is an end-to-end residual convolution neural network. As shown in fig. 2, the residual convolutional neural network includes: and taking a low-resolution synthetic aperture radar image as input, performing dimensionality increasing operation through 1 × 1 convolution, and performing upsampling on the feature layer to obtain a high-resolution feature layer. And then inputting the obtained high-resolution feature layer into a multi-scale attention module, extracting features of three scales, and performing cascade operation on the extracted feature layer. Wherein, each scale attention module comprises three embedded modules of a space attention module, a channel attention module and an attention fusion module. And finally, performing dimensionality reduction operation on the feature layer obtained by the multi-scale attention module through 1 × 1 convolution to obtain a high-resolution synthetic aperture radar image. Six sub-modules are embedded in the residual convolutional neural network, and the residual convolutional neural network comprises: the system comprises a feature layer up-sampling module, a space attention module, a channel attention module, an attention fusion module, a multi-scale attention module and an adaptive loss function.
Step 3.1: and constructing a feature layer up-sampling module. The module is used for performing an upsampling operation on a low-resolution feature layer result obtained by convolution from a feature layer level to obtain a feature layer with a high resolution, and the module can be defined as follows:
Fhr＝fus(Flr) (2)
wherein, FhrRepresenting a high-resolution characteristic layer, FlrRepresenting low resolution feature layers, fus(.) represents an upsampling function.
Step 3.2: a spatial attention module is constructed. The module is used for weighting the spatial weight of the full-polarization synthetic aperture radar characteristic layer and enhancing the spatial resolution of the full-polarization synthetic aperture radar image. This module may be defined as:
Figure BDA0002800581920000061
wherein the content of the first and second substances,
Figure BDA0002800581920000062
representing feature layers weighted by a spatial attention module,
Figure BDA0002800581920000063
spatial domain, C, H, representing a feature layer,W represents the number of channels, height and width of the feature layer respectively,
Figure BDA0002800581920000064
a layer of spatial attention input features is represented,
Figure BDA0002800581920000065
a spatial attention weight map is represented,
Figure BDA0002800581920000066
representing an element multiplication operation. The spatial attention weight map calculation method is as follows:
Figure BDA0002800581920000067
wherein, FspaRepresents the input feature layer, sigma (.) and delta (.) represent the Sigmoid activation function and the ReLU activation function, respectively,
Figure BDA0002800581920000068
representing a convolution operation.
Figure BDA0002800581920000071
And
Figure BDA0002800581920000072
two weight terms and two bias terms representing spatial attention modules, respectively.
Step 3.3: a channel attention module is constructed. The module is used for weighting the polarization channel weight of the full-polarization synthetic aperture radar characteristic layer and keeping the polarization information of each channel. The module may be defined as:
Figure BDA0002800581920000073
wherein the content of the first and second substances,
Figure BDA0002800581920000074
representing feature layers weighted by the channel attention module,
Figure BDA0002800581920000075
a layer of channel attention input features is represented,
Figure BDA0002800581920000076
a channel attention weight map is shown,
Figure BDA0002800581920000077
representing an element multiplication operation. The channel attention weight map calculation method is as follows:
Figure BDA0002800581920000078
wherein, FchaRepresenting input feature layer, Pavg(.) represents an average pooling operation,
Figure BDA0002800581920000079
and
Figure BDA00028005819200000710
two weight terms and two bias terms, respectively, representing channel attention, wherein,
Figure BDA00028005819200000711
step 3.4: and an attention fusion module. The module is used for carrying out information fusion on the space attention weight weighting result and the channel attention weight weighting result obtained in the step 3.2 and the step 3.3. The module may be defined as:
Figure BDA00028005819200000712
wherein, FfusFor the fused results, Concat (.) represents the feature layer cascading operation, WfusAnd bfusRespectively representing the weight term and the bias term of the attention fusion module.
Step 3.5: a multi-scale attention module is constructed. The module comprises three scales which are an original scale, an original scale downscaling scale and an original scale upscaling scale respectively. The original scale is used for extracting the features of the target with the conventional size, the original scale down scale is used for extracting the features of the small target, and the original scale up scale is used for extracting the features of the large target. The module embeds the attention mechanism described in steps 3.2-3.4, which is defined as:
Figure BDA00028005819200000713
wherein, FmsFeature layer, W, representing a multiscale attention ModulemsAnd bmsRespectively representing the weight term and the bias term of the multi-scale attention module.
The original scale feature layer calculation method comprises the following steps:
Fs0＝Ffus(F) (9)
wherein, Fs0Representing a layer of original scale features, Ffus(.) represents an inline attention fusion module and F represents the feature layer of the input.
The method for calculating the original scale down-scale feature layer comprises the following steps:
Fs1＝fds(Ffus(fus(F))) (10)
wherein, Fs1Representing a layer of original downscaled features, fds(.) represents the down-sampling function, fus(.) represents an upsampling function.
The original scale upscaling feature layer calculation method comprises the following steps:
Fs2＝fus(Ffus(fds(F))) (11)
wherein, Fs2Representation sourceAnd (5) starting scale up of the feature layer.
Step 3.6: an adaptive loss function is constructed. The adaptive loss function consists of two parts, including an L1 loss function for avoiding overfitting of network parameters due to outliers, and an L2 loss function for maintaining numerical relationships, which can be defined as the following form:
Ltotal(Θ)＝αL1(Θ)+βL2(Θ) (12)
wherein L istotal(Θ) denotes the adaptive loss function, L1(Θ) represents the L1 loss function, L2(Θ) represents the L2 loss function, with Θ being the neural network parameter and α and β being the regularization parameters used to adjust the weights of the L1 and L2 loss functions. Specifically, the L1 loss function may be defined as:
Figure BDA0002800581920000081
wherein N represents a training image pair
Figure BDA0002800581920000082
Number of (2), xi,yiRespectively representing a high-resolution fully-polarized synthetic aperture radar image and a low-resolution fully-polarized synthetic aperture radar image, rho, of the ith pair of training imagesiAnd the residual error between the sampling result on the characteristic layer and the high-resolution synthetic aperture radar image is represented, and xi (.) represents the output of the super-resolution reconstruction network.
The L2 loss function may be defined as:
Figure BDA0002800581920000083
the calculation method of the self-adaptive regularization parameter comprises the following steps:
Figure BDA0002800581920000084
and 4, step 4: and (3) training the super-resolution reconstruction network constructed in the step (3) by using the training data set constructed in the step (2). The training mechanism adopted by the invention comprises: a supervised training mechanism and a zero throw training mechanism. And training the super-resolution reconstruction network by using a supervision type training mechanism to obtain a convergent pre-training network. And then, further training the pre-training network by using a zero-throw type training mechanism to obtain a converged super-resolution reconstruction network.
Step 4.1: a supervised training mechanism. The mechanism is to use an external database for supervision training and is characterized in that the training data set constructed in the step 2 is used, and supervised training is carried out on the super-resolution reconstruction network through the low-high split-combined aperture radar image pair matched with the external data set, so as to obtain a convergent pre-training network. By this mechanism, the external information of the external data set is fully utilized.
Step 4.2: a zero throw training mechanism. The mechanism is to use data internal information to train, and is characterized in that the low-resolution and integration aperture radar image is used for training a pre-training network. Specifically, the training mechanism performs a degradation process on the low-resolution image to be processed by using a formula (1) to obtain a down-sampling result. In the training mechanism, a downsampling result is used as a target image of a pre-training network, a low-resolution image to be processed is used as a reference image of the pre-training network, a pairing relation between a downsampling result of a low-resolution image and the low-resolution image is established, and self-supervision training is carried out on the pre-training network. By the mechanism, the internal information of the image to be processed is fully utilized.
And 5: and (5) super-resolution reconstruction of the image to be processed. And (4) performing super-resolution reconstruction on the low-resolution synthetic aperture radar image to be processed by utilizing the converged super-resolution convolutional neural network trained in the step (4) to obtain a high-resolution fully-polarized synthetic aperture radar image.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.

Claims (7)

1. A super-resolution reconstruction method for a fully-polarized synthetic aperture radar image is characterized by comprising the following steps:
step 1, establishing an observation model of a full-polarization synthetic aperture radar image, wherein the model provides a degradation relation between a high-spatial-resolution full-polarization synthetic aperture radar image and a low-spatial-resolution full-polarization synthetic aperture radar image;
step 2, image preprocessing, namely constructing a training data set by utilizing the preprocessed images;
step 3, constructing a super-resolution reconstruction network of the fully-polarized synthetic aperture radar image based on a multi-scale attention mechanism;
the super-resolution reconstruction network takes a low-resolution synthetic aperture radar image as input, performs dimensionality increasing operation through convolution, and obtains a high-resolution feature layer through up-sampling of the feature layer; then, inputting the obtained high-resolution feature layer into a multi-scale attention module, extracting features of multiple scales, and performing cascade operation on the extracted feature layer; each scale attention module comprises three embedded modules, namely a space attention module, a channel attention module and an attention fusion module; finally, performing dimensionality reduction operation on the feature layer obtained by the multi-scale attention module through convolution to obtain a high-resolution synthetic aperture radar image;
step 4, training the super-resolution reconstruction network constructed in the step 3 to be convergent by using the training data set constructed in the step 2;
and 5, performing super-resolution reconstruction on the low-resolution synthetic aperture radar image by using the converged super-resolution reconstruction network trained in the step 4 to obtain a high-resolution fully-polarized synthetic aperture radar image.
2. The method of claim 1, wherein: the observation model in step 1 is constructed as follows,
representing the high-resolution fully-polarized synthetic aperture radar image as x, and representing the degraded low-resolution fully-polarized synthetic aperture radar image as y, then the observation model of the fully-polarized synthetic aperture radar image is represented as:
y＝fds(x) (1)
wherein f isds(.) represents a down-sampling function.
3. The method of claim 1, wherein: step 2 comprises the following two parts;
step 2.1, preprocessing the image, including radiation correction, terrain correction and multi-view processing, to obtain a corrected image; according to the observation model established in the step 1, performing down-sampling processing on the high-resolution image to obtain a low-resolution image, obtaining a high-resolution and low-resolution fully-polarized synthetic aperture radar image pair containing the same ground object by cutting, and constructing a data set;
and 2.2, performing data enhancement on the data set constructed in the step 2.1 to obtain a training data set.
4. The method of claim 1, wherein: the super-resolution reconstruction network in the step 3 specifically comprises the following steps;
step 3.1, constructing a feature layer upsampling module, wherein the module is used for performing upsampling operation on a low-resolution feature layer result obtained by convolution from the aspect of the feature layer to obtain a high-resolution feature layer, and the module is defined as:
Fhr＝fus(Flr) (2)
wherein, FhrRepresenting a high-resolution characteristic layer, FlrRepresenting low resolution feature layers, fus(.) represents an upsampling function;
step 3.2: constructing a spatial attention module, wherein the spatial attention module is used for weighting the spatial weight of the characteristic layer of the fully-polarized synthetic aperture radar and enhancing the spatial resolution of the image of the fully-polarized synthetic aperture radar; the module is defined as:
Figure FDA0002800581910000021
wherein the content of the first and second substances,
Figure FDA0002800581910000022
representing feature layers weighted by a spatial attention module,
Figure FDA0002800581910000023
spatial domain, C, H, representing a feature layer,W represents the number of channels, height and width of the feature layer respectively,
Figure FDA0002800581910000024
a layer of spatial attention input features is represented,
Figure FDA0002800581910000025
a spatial attention weight map is represented,
Figure FDA0002800581910000026
representing an element multiplication operation; the spatial attention weight map calculation method is as follows:
Figure FDA0002800581910000027
wherein, FspaRepresents the input feature layer, sigma (.) and delta (.) represent the Sigmoid activation function and the ReLU activation function, respectively,
Figure FDA0002800581910000028
which represents a convolution operation, the operation of the convolution,
Figure FDA0002800581910000029
and
Figure FDA00028005819100000210
two weight terms and two bias terms respectively representing spatial attention modules;
step 3.3, constructing a channel attention module, wherein the module is used for weighting the polarization channel weight of the full-polarization synthetic aperture radar characteristic layer and maintaining the polarization information of each channel; the module is defined as:
Figure FDA00028005819100000211
wherein the content of the first and second substances,
Figure FDA00028005819100000212
representing feature layers weighted by the channel attention module,
Figure FDA00028005819100000213
a layer of channel attention input features is represented,
Figure FDA00028005819100000214
a channel attention weight map is shown,
Figure FDA00028005819100000215
representing an element multiplication operation; the channel attention weight map calculation method is as follows:
Figure FDA00028005819100000216
wherein, FchaRepresenting input feature layer, Pavg(.) represents an average pooling operation,
Figure FDA00028005819100000217
and
Figure FDA00028005819100000218
two weight terms and two bias terms, respectively, representing channel attention, wherein,
Figure FDA00028005819100000219
step 3.4, an attention fusion module for performing information fusion on the spatial attention weight-weighting result and the channel attention weight-weighting result obtained in the steps 3.2 and 3.3, the module being defined as:
Figure FDA00028005819100000220
wherein, FfusFor the fused results, Concat (.) represents the feature layer cascading operation, WfusAnd bfusRespectively representing a weight term and a bias term of the attention fusion module;
step 3.5, constructing a multi-scale attention module, wherein the module comprises three scales, namely an original scale, an original scale downscaling scale and an original scale upscaling scale; the original scale is used for extracting the features of the target with the conventional size, the original scale downscaling is used for extracting the features of the small target, and the original scale upscaling is used for extracting the features of the large target; the module embeds the attention mechanism described in steps 3.2-3.4, which is defined as:
Figure FDA0002800581910000031
wherein, FmsFeature layer, W, representing a multiscale attention ModulemsAnd bmsRespectively representing a weight term and a bias term of the multi-scale attention module;
the original scale feature layer calculation method comprises the following steps:
Fs0＝Ffus(F) (9)
wherein, Fs0Representing a layer of original scale features, Ffus(.) represents an embedded attention fusion module, F represents an input feature layer, and the original scale down-scale feature layer is calculated as follows:
Fs1＝fds(Ffus(fus(F))) (10)
wherein, Fs1Representing original downscalingCharacteristic layer, fds(.) represents the down-sampling function, fus(.) represents an upsampling function, and the original scale up feature layer is calculated as follows:
Fs2＝fus(Ffus(fds(F))) (11)
wherein, Fs2Representing an original scale up feature layer;
step 3.6, constructing an adaptive loss function, wherein the adaptive loss function consists of two parts, including an L1 loss function for avoiding overfitting of network parameters caused by abnormal values and an L2 loss function for maintaining numerical relationships, and the adaptive loss function is defined as the following form:
Ltotal(Θ)＝αL1(Θ)+βL2(Θ) (12)
wherein L istotal(Θ) denotes the adaptive loss function, L1(Θ) represents the L1 loss function, L2(Θ) represents the L2 loss function, Θ being a neural network parameter, α and β being regularization parameters for adjusting the weights of the L1 and L2 loss functions; specifically, the L1 loss function is defined as:
Figure FDA0002800581910000032
wherein N represents a training image pair
Figure FDA0002800581910000033
Number of (2), xi,yiRespectively representing a high-resolution fully-polarized synthetic aperture radar image and a low-resolution fully-polarized synthetic aperture radar image, rho, of the ith pair of training imagesiRepresenting a residual error between a sampling result on the characteristic layer and the high-resolution synthetic aperture radar image, and ξ (·) representing the output of the super-resolution reconstruction network;
the L2 loss function is defined as:
Figure FDA0002800581910000041
the calculation method of the self-adaptive regularization parameter comprises the following steps:
Figure FDA0002800581910000042
5. the method of claim 2, wherein: the training mechanism adopted in step 4 comprises: a supervised training mechanism and a zero throw training mechanism; training the super-resolution reconstruction network by using a supervision type training mechanism to obtain a converged pre-training network, and then further training the pre-training network by using a zero throw type training mechanism to obtain a converged reconstruction network;
step 4.1, a supervision training mechanism, wherein the supervision training mechanism is to use an external database for supervision training; carrying out supervised training on the super-resolution reconstruction network through the low-high split-joint aperture radar image pair matched with the external data set by using the training data set constructed in the step 2 to obtain a convergent pre-training network, and fully utilizing the external information of the external data set through the mechanism;
4.2, a zero throw training mechanism, namely training by using data internal information, namely training a pre-training network by using a low-resolution aperture radar image; specifically, the training mechanism performs degradation processing on a low-resolution image to be processed by using a formula (1) to obtain a down-sampling result, in the training mechanism, the down-sampling result is used as a target image of a pre-training network, the low-resolution image to be processed is used as a reference image of the pre-training network, a pairing relation between the down-sampling result of the low-resolution image and the low-resolution image is established, the pre-training network is subjected to self-supervision training, and through the mechanism, the internal information of the image to be processed is fully utilized.
6. The method of claim 2, wherein: the overlap area between image pairs in step 2.1 is 20%.
7. The method of claim 2, wherein: the data enhancement in step 2.2 includes rotation by 90 °, rotation by 180 °, rotation by 270 °, and flipping.
CN202011348480.2A 2020-11-26 2020-11-26 Super-resolution reconstruction method for fully-polarized synthetic aperture radar image Active CN112419155B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011348480.2A CN112419155B (en) 2020-11-26 2020-11-26 Super-resolution reconstruction method for fully-polarized synthetic aperture radar image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011348480.2A CN112419155B (en) 2020-11-26 2020-11-26 Super-resolution reconstruction method for fully-polarized synthetic aperture radar image

Publications (2)

Publication Number Publication Date
CN112419155A CN112419155A (en) 2021-02-26
CN112419155B true CN112419155B (en) 2022-04-15

Family

ID=74842530

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011348480.2A Active CN112419155B (en) 2020-11-26 2020-11-26 Super-resolution reconstruction method for fully-polarized synthetic aperture radar image

Country Status (1)

Country Link
CN (1) CN112419155B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112801928B (en) * 2021-03-16 2022-11-29 昆明理工大学 Attention mechanism-based millimeter wave radar and visual sensor fusion method
CN113052848B (en) * 2021-04-15 2023-02-17 山东大学 Chicken image segmentation method and system based on multi-scale attention network
CN113658047A (en) * 2021-08-18 2021-11-16 北京石油化工学院 Crystal image super-resolution reconstruction method
CN113793267B (en) * 2021-09-18 2023-08-25 中国石油大学(华东) Self-supervision single remote sensing image super-resolution method based on cross-dimension attention mechanism
CN114254715B (en) * 2022-03-02 2022-06-03 自然资源部第一海洋研究所 Super-resolution method, system and application of GF-1WFV satellite image
CN114972041B (en) * 2022-07-28 2022-10-21 中国人民解放军国防科技大学 Polarization radar image super-resolution reconstruction method and device based on residual error network
CN116128727B (en) * 2023-02-02 2023-06-20 中国人民解放军国防科技大学 Super-resolution method, system, equipment and medium for polarized radar image

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109584161A (en) * 2018-11-29 2019-04-05 四川大学 The Remote sensed image super-resolution reconstruction method of convolutional neural networks based on channel attention
CN109903228A (en) * 2019-02-28 2019-06-18 合肥工业大学 A kind of image super-resolution rebuilding method based on convolutional neural networks
CN110992270A (en) * 2019-12-19 2020-04-10 西南石油大学 Multi-scale residual attention network image super-resolution reconstruction method based on attention
AU2020100200A4 (en) * 2020-02-08 2020-06-11 Huang, Shuying DR Content-guide Residual Network for Image Super-Resolution

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109584161A (en) * 2018-11-29 2019-04-05 四川大学 The Remote sensed image super-resolution reconstruction method of convolutional neural networks based on channel attention
CN109903228A (en) * 2019-02-28 2019-06-18 合肥工业大学 A kind of image super-resolution rebuilding method based on convolutional neural networks
CN110992270A (en) * 2019-12-19 2020-04-10 西南石油大学 Multi-scale residual attention network image super-resolution reconstruction method based on attention
AU2020100200A4 (en) * 2020-02-08 2020-06-11 Huang, Shuying DR Content-guide Residual Network for Image Super-Resolution

Also Published As

Publication number Publication date
CN112419155A (en) 2021-02-26

Similar Documents

Publication Publication Date Title
CN112419155B (en) Super-resolution reconstruction method for fully-polarized synthetic aperture radar image
Song et al. Spatiotemporal satellite image fusion using deep convolutional neural networks
Zou et al. Ship detection in spaceborne optical image with SVD networks
WO2020056791A1 (en) Method and apparatus for super-resolution reconstruction of multi-scale dilated convolution neural network
CN112507997B (en) Face super-resolution system based on multi-scale convolution and receptive field feature fusion
CN109727207B (en) Hyperspectral image sharpening method based on spectrum prediction residual convolution neural network
CN107358260B (en) Multispectral image classification method based on surface wave CNN
CN112083422B (en) Single-navigation InSAR system end-to-end classification method based on multistage deep learning network
WO2023000159A1 (en) Semi-supervised classification method, apparatus and device for high-resolution remote sensing image, and medium
CN108734171A (en) A kind of SAR remote sensing image ocean floating raft recognition methods of depth collaboration sparse coding network
CN114283120B (en) Domain-adaptive-based end-to-end multisource heterogeneous remote sensing image change detection method
CN111161271A (en) Ultrasonic image segmentation method
Natteshan et al. Effective SAR image segmentation and classification of crop areas using MRG and CDNN techniques
CN113793289A (en) Multi-spectral image and panchromatic image fuzzy fusion method based on CNN and NSCT
CN113239736A (en) Land cover classification annotation graph obtaining method, storage medium and system based on multi-source remote sensing data
CN115527056A (en) Hyperspectral image classification method based on dual-hybrid convolution generation countermeasure network
Xiao et al. A review of remote sensing image spatiotemporal fusion: Challenges, applications and recent trends
CN115457509A (en) Traffic sign image segmentation algorithm based on improved space-time image convolution
CN115331109A (en) Remote sensing image target detection method based on rotation equal-variation convolution channel attention enhancement and multi-scale feature fusion
CN112950780A (en) Intelligent network map generation method and system based on remote sensing image
CN116563682A (en) Attention scheme and strip convolution semantic line detection method based on depth Hough network
CN114092803A (en) Cloud detection method and device based on remote sensing image, electronic device and medium
Zhao et al. Multiple Attention Network for Spartina Alterniflora Segmentation Using Multi-temporal Remote Sensing Images
CN113887472A (en) Remote sensing image cloud detection method based on cascade color and texture feature attention
Jiang et al. Semantic segmentation network combined with edge detection for building extraction in remote sensing images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant