CN115170406A - High-precision fusion method for optical image and SAR (synthetic aperture radar) intensity image - Google Patents

Abstract

The invention provides a high-precision fusion method of an optical image and an SAR intensity image, which comprises the steps of establishing an image coordinate system, determining the position of each pixel of two data sources, calculating the deviation between the SAR image and an optical image of a basic map, achieving sub-pixel level registration precision by using a frequency spectrum diversity method, reducing fusion blur, enabling the SAR and optical registration precision to be less than 0.1 pixel, then carrying out HSI transformation on two RGB images to obtain respective brightness components, then carrying out wavelet transformation on the brightness components, selecting a certain fusion rule to obtain new brightness components, and then carrying out inverse transformation to obtain fused RGB images. The method overcomes the difficulties that the cloud shielding is difficult to obtain by singly using the optical image, makes up the defect that the SAR image lacks the spectral information of the ground object, improves the monitoring accuracy of the safety environment of the power transmission and transformation equipment, and provides first-hand image data for timely discovering the power transmission line and the surrounding disasters.

Description

A high-precision fusion method of optical image and SAR intensity image

technical field

The invention relates to the technical field of high-precision fusion of heterogeneous satellite remote sensing images, in particular to a high-precision fusion method of optical images and SAR intensity images.

Background technique

High spatial resolution optical satellite remote sensing images can monitor the operation environment of power transmission and transformation, and obtain information on vegetation coverage, topographical features, new houses, and large-scale construction by third parties within a certain range of power transmission and transformation. The spectral characteristics of optical images can realize the monitoring of various surface changes in the power transmission and transformation operating environment, but due to the high spatial resolution optical images cannot penetrate the cloud layer, the revisit period is long, and the acquisition is difficult, often for several months. To obtain a high-quality image of a scene, if a medium-resolution optical image is used, it is difficult to realize the safety monitoring of power transmission and transformation equipment due to the low spatial resolution. SAR (Synthetic Aperture Radar, Synthetic Aperture Radar) has all-day, all-weather and is not affected by clouds and rain. Using high spatial resolution SAR images can obtain dynamic information of the surface in time, but SAR images do not contain spectral information of ground objects, and need to combine optical Images can better identify surface objects. In view of this situation, the technology of obtaining power transmission and transformation equipment by using high spatial resolution SAR and medium spatial resolution optical image fusion came into being.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method capable of high-precision fusion of optical images and SAR intensity images, which not only has the characteristics of SAR images to monitor surface changes in time, but also retains the advantages of optical images to identify surface objects.

In order to achieve the above object, the present invention is achieved through the following technical solutions.

A high-precision fusion method of optical image and SAR intensity image, comprising the steps of:

Obtain the high spatial resolution X-band SAR image S1 of the study area and the same wide medium spatial resolution optical image L1 taken on the same day in the same area;

Perform data preprocessing on images S1 and L1 to obtain preprocessed images S2 and L2 respectively;

Perform sub-pixel registration on images S2 and L2, use L2 as the extrusion base map, and use the spectral classification method to register S2 to L2 to reduce fusion blur;

The registered high-resolution SAR image S2 re-resolution optical image is used for fusion processing, the two data sources are normalized into grayscale images, and then the two RGB images are transformed by HSI to obtain their respective luminance components. The luminance component is subjected to wavelet transform, the fusion rule is selected and a new luminance component is obtained, and finally the fused RGB image is inversely transformed.

Further, the preprocessing of S1 includes: orbit correction, image cropping, thermal noise removal, and terrain correction using DEM data.

Further, the preprocessing of L1 includes: radiometric calibration, atmospheric correction, orthorectification, image fusion, image enhancement and cropping of the study area.

Further, the sub-pixel level registration of the images S2 and L2 includes the following steps:

Carry out pixel resampling, use the pixel value of adjacent 4 points to utilize bilinear interpolation to sample the L2 image to be consistent with the S2 pixel spacing, and obtain the resampling image L3;

Register S2 with L3, use a multi-constrained GAN network to learn the mapping relationship between visible light and remote sensing SAR images, use the trained model to expand the number and diversity of training samples, and use neural networks to extract features for image patch matching prediction, After rough registration, an offset parameter file is generated, and the offset is estimated. Finally, the trained model is used to remove the offset to obtain sub-pixel registration, and new SAR images and optical images S3 and L4 are exported respectively.

Further, the image fusion adopts the texture weighting enhancement method to make the texture features of the SAR image clearer. The specific steps include:

After the noise reduction is completed, image reconstruction is carried out, the texture of the area with strong backscattering is strengthened, and its weight ratio is increased, and the texture of the area with weak backscattering is weakened, so as to highlight the texture of the ground object, and the SAR with enhanced texture information is marked as S4. Its formula is as follows:

为自定义权重系数，是一个常数，S3_w为后向散射强的区域影像，S3_v为后向散射弱的区域影像。in,

is a custom weight coefficient, which is a constant, S3 _w is the image of the area with strong backscatter, and S3 _v is the image of the area with weak backscatter.

The advantages of the present invention are:

Based on heterogeneous satellite remote sensing images, the present invention extracts feature information from two images to perform image fusion, overcomes difficulties such as difficulty in obtaining cloud occlusion by using optical images alone, makes up for the shortcoming of SAR images lacking spectral information of ground objects, and improves the safety of power transmission and transformation equipment. The accuracy of environmental monitoring provides first-hand image data for timely detection of power transmission lines and surrounding third-party external breakages, large-scale construction, and landslides.

The spectral classification method is used to register SAR and optical images to reduce fusion blur caused by low registration accuracy.

The texture weighted fusion algorithm is used to enhance the texture information of the SAR image, so that the fused image contains both strong texture and strong spectrum, which can effectively identify the environmental changes around the transmission line.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

A high-precision fusion method of optical image and SAR intensity image, comprising the following steps:

S1. Obtain the high spatial resolution X-band SAR image S1 in the study area and the medium spatial resolution optical image L1 of the same width taken on the same day in the same area, and perform data preprocessing on the images S1 and L1:

The preprocessing of the image S1 mainly includes orbit correction, image cropping, thermal noise removal, and terrain correction using DEM data to prepare for registration with the medium-resolution optical image, that is, the L1 image. Thermal noise is the noise inherent in the SAR satellite system. It is due to the energy generated during the operation of the satellite itself. It is necessary to repeat the image for many times to remove the influence of thermal noise. In side-view imaging, terrain fluctuations will distort SAR images, and lead to perspective shrinkage, overlapping, shadows, and other phenomena, so terrain correction is required. Image S2 is obtained after processing.

The preprocessing of the medium-resolution optical L1 image mainly includes radiometric calibration, atmospheric correction, orthorectification, image fusion, image enhancement and cropping of the study area. Radiometric calibration converts the gray value (DN value) of the image into radiance value or apparent reflectance in image processing to eliminate the error of the sensor itself and determine the accurate radiation value at the entrance of the sensor. In particular, the fusion here is only the fusion of resolution and spectral information. After fusion, the study area is cropped. The cropping boundary and size are based on S1, and finally the medium-resolution optical satellite image L2 is obtained.

S2. Perform sub-pixel-level registration on S2 and L2, use L2 as the base map, and use the spectral classification method to register S2 to L2. The registration accuracy reaches 0.1 pixel, which is ready for fusion and reduces fusion blur.

To achieve sub-pixel registration accuracy, general geo-referencing is far from enough, and special processing is required.

First, perform pixel resampling. Because the resolution of the L2 image is low, it is necessary to sample the L2 image to the same pixel spacing as the S2 pixel, and use the bilinear interpolation method to resample, using the pixel values of the adjacent 4 points, according to its The distance from the interpolation point is given different weights to perform linear interpolation. The principle is to use the pixel values of the four critical points of the upper left, upper right, lower left and lower right corresponding to the target point to obtain the pixel value of the target point. After the resampling is completed, the image L3 is obtained, and the specific calculation formula is as follows:

a=(1-t)(1-u), b=(1-t)×u, c=t×u, d=t(1-u)

Among them, a, b, c, d are the four coordinate coefficients, respectively, u, t are the ratio of the critical point and the target point on the two-dimensional coordinate axis.

Use a multi-constrained GAN network to learn the mapping relationship between visible light and remote sensing SAR images, and then use the trained model to expand the number and diversity of training samples, and then use the neural network to extract features for image patch matching prediction. After rough registration Generate an offset parameter file, estimate the offset, and finally use the trained model to remove the offset to obtain sub-pixel registration. After the registration is completed, a new SAR image and optical image need to be exported, which are defined as S3 and L4, respectively.

S3. Use the registered high-resolution SAR image S2 and the medium-resolution optical image to perform fusion processing:

S31. Texture weighting enhancement, so that the texture of the SAR image is highlighted. The principle is to perform secondary noise reduction. After the noise reduction is completed, image reconstruction is performed to strengthen the texture of the area with strong backscattering and increase its weight ratio. The texture of the area with weak scattering is weakened, thereby highlighting the texture of the ground object, and the SAR with enhanced texture information is marked as S4. Its formula is as follows:

S32. Use the ground object texture information in S4 and the spectral information of L4 to fuse, first perform HSI transformation on the optical image, transform the optical image from RGB to HSI space, and separate the intensity component (texture information) I and spectral component H of the image. and S; then use the separated intensity component I to perform histogram matching on the radar image, so that the histogram distribution trend of SAR and optical image is consistent, so as to effectively maintain the spectral information; the wavelet transform algorithm is applied to the optical image intensity component and histogram On the SAR image after image matching, the intensity component and the high-frequency component and low-frequency component of the SAR image are obtained respectively, and the low-frequency and high-frequency components of the two are fused respectively; the wavelet inverse transform is applied to the low-frequency fusion result and the high-frequency fusion result. Then, a new intensity component I' is obtained, and the HSI inverse transform is performed between the new intensity component I' and the hue component H and saturation component S separated by the HSI transformation of the optical image, and the optical and radar fusion result of the RGB space is obtained, Get the fused RGB image.

Finally, it should be noted that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, the The technical solutions described in the foregoing embodiments may be modified, or some technical features thereof may be equivalently replaced. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (5)

1. A high-precision fusion method of an optical image and an SAR intensity image is characterized by comprising the following steps:

acquiring an SAR image S1 of a high spatial resolution X wave band in a research area and a medium spatial resolution optical image L1 of the same width shot in the same day in the same area;

performing data preprocessing on the images S1 and L1 to respectively obtain preprocessed images S2 and L2;

performing sub-pixel level registration on the images S2 and L2, using the L2 as an extrusion base map, and registering the S2 to the L2 by using a frequency spectrum classification method to reduce fusion blurring;

and performing fusion processing by using the registered high-resolution SAR image S2 to obtain optical images with resolution, normalizing the two data sources into a gray-scale image, performing HSI (high-resolution image integration) conversion on the two RGB images to obtain respective brightness components, performing wavelet conversion on the brightness components, selecting a fusion rule, obtaining new brightness components, and finally performing inverse conversion on the new brightness components to obtain the fused RGB image.

2. The method for high-precision fusion of an optical image and an SAR intensity image according to claim 1, wherein the preprocessing of S1 comprises: track correction, image cropping, thermal noise removal and terrain correction by utilizing DEM data.

3. The method for high-precision fusion of an optical image and an SAR intensity image according to claim 1, wherein the preprocessing of L1 comprises: radiometric calibration, atmospheric correction, orthorectification, image fusion, image enhancement, and region of interest cropping.

4. The method for high-precision fusion of an optical image and an SAR intensity image according to claim 1, wherein the sub-pixel level registration of the images S2 and L2 comprises the steps of:

pixel resampling is carried out, pixel values of 4 adjacent points are used for sampling the L2 image to be consistent with the S2 pixel interval by a bilinear interpolation method, and a resampled image L3 is obtained;

registering the S2 and the L3, learning the mapping relation between the visible light and the remote sensing SAR images by using a multi-constrained GAN network, expanding the number and diversity of training samples by using a trained model, extracting features by using a neural network to perform image block matching prediction, generating an offset parameter file after coarse registration, performing offset estimation, finally removing the offset by using the trained model to obtain sub-pixel level registration, and respectively deriving new SAR images and optical images S3 and L4.

5. The high-precision fusion method of the optical image and the SAR intensity image according to claim 1, wherein the image fusion adopts a texture weighting enhancement method to make the texture feature of the SAR image clearer, specifically comprising the steps of:

after noise reduction is finished, image reconstruction is carried out, the texture of the region with strong backscattering is enhanced, the weight ratio of the region is enhanced, the texture of the region with weak backscattering is weakened, therefore, the texture of the ground object is highlighted, the SAR image with enhanced texture information is marked as S4, and the formula is as follows:

wherein,

in order to self-define the weight coefficient,

is an image of the region where the back-scattering is strong,

the image of the region with weak backscattering.