CN111563866B - Multisource remote sensing image fusion method - Google Patents

Abstract

The invention relates to the technical field of image processing, in particular to a multi-source remote sensing image fusion method, which comprises the following steps: firstly, utilizing IHS conversion to obtain a brightness I component, a chromaticity H component and a saturation S component of the multi-spectrum image after up-sampling; filtering the component I by adopting guide filtering, and constructing self-adaptive fractional order differentiation for enhancing edge detail information in the full-color image; respectively carrying out wavelet transformation on the filtered multispectral image I component and the enhanced panchromatic image to obtain a high-frequency component and a low-frequency component of the multispectral image I component and the enhanced panchromatic image, wherein the high-frequency component adopts an absolute value maximization principle, and the low-frequency component adopts a weighted average principle; and taking the result obtained by wavelet inverse transformation as a new I component, and finally obtaining a fusion image by utilizing IHS inverse transformation. The method combines the guided filtering and the fractional differential to fuse the full-color image and the multispectral image, effectively inhibits the spectrum distortion phenomenon of the fusion result, and reduces the loss of space detail information.

Description

A Multi-source Remote Sensing Image Fusion Method

technical field

The invention relates to the technical field of image processing, in particular to a multi-source remote sensing image fusion method.

Background technique

Multi-source remote sensing image fusion refers to the image processing technology that superimposes two or more remote sensing images of the same scene from different sensors to obtain a comprehensive image with more accurate and complete information. Image fusion is not only an important part of remote sensing data processing, but also has a wide range of applications in environmental detection, urban planning, military reconnaissance and other fields. In recent years, with the continuous development of signal processing technology, researchers have conducted a lot of research on image fusion methods.

At present, multi-source remote sensing image fusion is mainly divided into three levels: pixel level fusion, feature level fusion and decision level fusion. Compared with feature level fusion and decision level fusion, pixel level fusion has better performance in terms of accuracy and timeliness. Multi-source remote sensing image fusion methods for pixel level mainly include three categories: image fusion based on component replacement, image fusion based on multi-resolution analysis and image fusion based on pattern. The image fusion method based on component substitution has the characteristics of low complexity and easy implementation, and the spatial details of the fusion result are well preserved, but the processing involves spatial transformation operations, so spectral distortion often occurs; images based on multi-resolution analysis The fusion method is not prone to spectral distortion, but the ringing phenomenon often occurs during the fusion process, which leads to the loss of spatial features in the fusion result; the model-based image fusion method is not prone to spectral distortion and loss of spatial features, but the methods involved in the fusion process High complexity. At this stage, although image fusion methods continue to emerge, they still face many problems and difficulties, which are mainly reflected in the following aspects: 1) The spatial information of the fused image is easily lost; 2) The spectrum of the fused image is prone to distortion.

Therefore, solving the problems of spatial information loss and spectral distortion in the process of multi-source remote sensing image fusion has become one of the important tasks for those skilled in the art.

Contents of the invention

The technical problem to be solved by the present invention is to provide a multi-source remote sensing image fusion method to effectively suppress the spectral distortion phenomenon of fusion results and reduce the loss of spatial detail information.

The technical solution adopted by the present invention for realizing the above object comprises the following steps:

Step 1: Obtain the multispectral image and the panchromatic image of the same ground object, and use the bicubic interpolation method to upsample the multispectral image so that the size of the multispectral image is consistent with that of the panchromatic image;

Step 2: Use IHS transformation to transform the color space of the multispectral image, extract the brightness I component of the multispectral image for further processing, and retain the chroma H component and saturation S component of the multispectral image for subsequent IHS inverse transformation ;

Step 3: Construct an adaptive fractional differential to enhance the edge details of the panchromatic image and preserve the contour information of the object, and at the same time, use guided filtering to filter the brightness I component of the multispectral image;

Step 4: Perform wavelet transformation on the panchromatic image after fractional differential processing and the brightness I component of the multispectral image after guided filtering processing to obtain the transformed high and low frequency components, where the high frequency component adopts the principle of taking the largest absolute value , the low-frequency component adopts the weighted average principle;

Step 5: Obtain the result image after wavelet inverse transformation through wavelet reconstruction;

Step 6: Use the result obtained by inverse wavelet transform as the I _new component, and perform IHS inverse transform with the H component and S component to obtain a fused image.

The present invention has the following advantages and beneficial effects:

1. Compared with the direct processing of the luminance component of the upsampled multispectral image in the prior art, the problem of block effect is easy to occur; the present invention introduces guided filtering with structure transfer characteristics and edge-preserving smoothing characteristics, and realizes the Suppression of blocking artifacts during image fusion. At the same time, the spatial texture information and local detail information of the I component of the multispectral image are also enhanced, thereby helping to improve the visual effect of the fused image.

2. Compared with the direct histogram matching of the luminance component and the full-color image in the prior art, the gray level is reduced and some details are lost; the present invention introduces fractional differentiation into image fusion, especially in combination with image Statistical features improve the order of fractional differentiation, avoid artificially setting a fixed order, and realize adaptive fractional differentiation. It can not only effectively preserve the flat part of the image as the base layer, but also enhance the edge details of the image.

Description of drawings

Fig. 1 is the image fusion flowchart of the method of the present invention.

Fig. 2 is a test image used in the experiment of the method of the present invention, and a comparison diagram of fusion effects of different methods on the test image.

Among them, Fig. 2(a) is a panchromatic image, Fig. 2(b) is a multispectral image, Fig. 2(c) is a fused image obtained by the IHS transform method, Fig. 2(d) is a fused image obtained by the Brovey method, Fig. 2(e) is the fused image obtained by the PCA method, Fig. 2(f) is the fused image obtained by the DWT method, Fig. 2(g) is the fused image obtained by the ATWT-M3 method, and Fig. 2(h) is the fused image obtained by the ATWT method Fusion image, Fig. 2(i) is the fusion image obtained by AWLP method, Fig. 2(j) is the fusion image obtained by GS method, Fig. 2(k) is the fusion image obtained by HPF method, Fig. 2(l) is the fusion image obtained by MTF- The fused image obtained by the GLP method, Fig. 2(m) is the fused image obtained by the method of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

As shown in Figure 1, a multi-source remote sensing image fusion method includes the following steps:

Step 1: Obtain the multispectral image and the panchromatic image of the same ground object, and use the bicubic interpolation method to upsample the multispectral image so that the size of the multispectral image is consistent with that of the panchromatic image.

Step 2: Use IHS transformation to transform the color space of the multispectral image, extract the brightness I component of the multispectral image for further processing, and retain the chroma H component and saturation S component of the multispectral image for subsequent IHS inverse transformation , the calculation formulas of I component, H component and S component are respectively:

H＝tan ^-1 [s ₁ /s ₂ ] (2)

Among them, R, G, and B are the red, green, and blue bands of the multispectral image, respectively.

Step 3: Construct an adaptive fractional differential to enhance the edge details of the panchromatic image and preserve the outline information of the object. The specific method is:

Assuming that the size of the panchromatic image f(i,j) is M×N, then the spatial frequency calculation formula is:

Among them, RF and CF represent the row frequency and column frequency of the panchromatic image f(i,j) respectively. The larger the value of the spatial frequency of the image, the richer the spatial information of the image and the stronger the layering of the image. In addition, the calculation formula of the average gradient of the panchromatic image f(i,j) is:

The larger the value of the average gradient, the more prominent the details such as the edge and texture of the image, and the higher the definition. Next, the spatial frequency and average gradient of the panchromatic image are normalized using the arccotangent nonlinear normalization function, namely:

Considering that the spatial frequency of the image and the numerical value of the average gradient are equally important to the differential order, the following average weighting process is performed on the two:

Since the Tanh function is a monotonically increasing function in the range of real numbers and presents a nonlinear growth trend, this characteristic conforms to the law of the variation of the differential order with image statistics, so the Tanh function is used to construct the order function as follows.

When the fractional differential order v∈[0.5,0.7], the texture details of the image can be highlighted and the image contour information can be fully preserved. Therefore, the present invention carries out a correction process on f(Y), wherein β and α are respectively set to 0.5 and 0.7, and then the calculation function v of the adaptive fractional differential order is obtained.

At the same time, the guided filtering is used to filter the brightness I component of the multispectral image, and the specific method is as follows:

First, set the radius of the guided filter r=7, and the regularization parameter ε=10 ⁻⁶ . The values of the linear coefficients a _(k,l) and b _(k,l) for calculating the guided filtering are:

和μ_(k,l)分别是局部窗ω_(k,l)所含像素的方差和均值，

代表局部窗ω_(k,l)所含多光谱图像I分量的像素均值，ε是正则化参数。考虑到在引导滤波的处理过程中，一个像素点(i,j)可能同时被多个局部窗ω_(k,l)滑过。因而，需对线性系数a_(k,l)和b_(k,l)进行均值处理，即：In the formula, |ω| represents the number of pixels in a rectangular local window ω (k,l) centered on pixel _(k,l) and radius r,

and μ _(k,l) are the variance and mean of the pixels contained in the local window ω _(k,l) , respectively,

Represents the pixel mean of the multispectral image I component contained in the local window ω _(k,l) , and ε is a regularization parameter. Considering that during the process of guided filtering, a pixel point (i, j) may be slid by multiple local windows ω _{(k, l)} at the same time. Therefore, the linear coefficients a _(k,l) and b _(k,l) need to be averaged, namely:

和

代入引导滤波的线性定义模型，从而得到滤波后的输出图像。Will

and

Substitute into the linear definition model of guided filtering to get the filtered output image.

The result after guided filtering is used as the base layer of the image, and the detail layer of the image is obtained by subtracting the I component of the multispectral image from the base image, and then the gray scale variation range of the detail layer is linearly transformed, and finally added to the base image to obtain the texture Structurally enhanced images.

Step 4: Perform wavelet transformation on the panchromatic image after fractional differential processing and the luminance I component of the multispectral image after guided filtering processing to obtain transformed high and low frequency components. Among them, the high-frequency component adopts the principle of taking the largest absolute value, and the low-frequency component adopts the weighted average principle.

Step 5: Obtain the result image after wavelet inverse transformation through wavelet reconstruction.

Step 6: Use the result obtained by wavelet inverse transform as the I _new component, and perform IHS inverse transform with the H component and S component to obtain the fused image. The calculation formula is:

Among them, R _new , G _new , and B _new are the red, green, and blue bands of the fused image respectively.

The present invention selects registered multispectral images and panchromatic images as test images, and conducts comparative studies with IHS, Brovey, PCA, DWT, ATWT-M3, ATWT, AWLP, GS, HPF and MTF-GLP methods.

The experimental results are as follows:

Experiment 1, the fusion results of different methods on the test image are shown in Figure 2. After comparative analysis, it is found that the fusion result obtained by the method of the present invention not only preserves the spectral features of the image, but also enhances the detailed texture of the ground objects in the image, so that the image has higher definition and better visual effect.

Experiment 2, in order to improve the accuracy of the quality evaluation of fusion results, the method of the present invention adopts several common evaluation indicators, including: average gradient (AG), mean value (ME), standard deviation (SD), information entropy (IE) , Mutual Information (MI) and Spatial Frequency (SF). The larger the value of the above evaluation index, the richer the spatial information of the image and the stronger the sense of hierarchy, as shown in Table 1. According to Table 1, it can be seen that compared with other methods, the fusion results obtained by the method of the present invention have achieved different degrees of improvement in various quality evaluation indicators, and have certain comprehensive advantages. The result shows that the method of the present invention has richer details of the fused image and better visual effect.

Table 1 Statistical table of quality evaluation indicators of image fusion results

To sum up, the multi-source remote sensing image fusion method disclosed in the present invention can effectively reduce the spectral distortion phenomenon and the loss of spatial detail information of fusion results, and has high effectiveness and feasibility.

The preferred implementation mode of this patent has been described in detail above, but the scope of protection of this patent is not limited to the above-mentioned implementation mode. Within the knowledge of those of ordinary skill in the art, it can also be done without departing from the purpose of this patent. Other changes should be included in the protection scope of the present invention.

Claims (2)

1. A multi-source remote sensing image fusion method is characterized in that, comprising the following steps: Step 1: Obtain the multispectral image and the panchromatic image of the same ground object, and use the bicubic interpolation method to upsample the multispectral image so that the size of the multispectral image is consistent with that of the panchromatic image; Step 2: Use IHS transformation to transform the color space of the multispectral image, extract the brightness I component of the multispectral image for further processing, and retain the chroma H component and saturation S component of the multispectral image for subsequent IHS inverse transformation ; Step 3: Construct an adaptive fractional differential to enhance the edge details of the panchromatic image and preserve the contour information of the object, and at the same time, use guided filtering to filter the brightness I component of the multispectral image; Step 4: Perform wavelet transformation on the panchromatic image after fractional differential processing and the brightness I component of the multispectral image after guided filtering processing to obtain the transformed high and low frequency components, where the high frequency component adopts the principle of taking the largest absolute value , the low-frequency component adopts the weighted average principle; Step 5: Obtain the result image after wavelet inverse transformation through wavelet reconstruction; Step 6: take the result obtained by the inverse wavelet transform as the I _new component, and perform IHS inverse transform with the H component and the S component to obtain the fused image; The brightness I component, chroma H component, and saturation S component obtained in the step 2 are respectively:

H＝tan ^-1 [s ₁ /s ₂ ]

Among them, R, G, and B are the red, green, and blue bands of the multispectral image, respectively; The adaptive fractional order differential method adopted in the step 3, the specific method is: First, calculate the spatial frequency and average gradient of the panchromatic image f(i,j), where the spatial frequency is obtained using the following formula:

In the formula, the size of the panchromatic image f(i,j) is M×N, and RF and CF represent the row frequency and column frequency of the panchromatic image f(i,j) respectively; in addition, the panchromatic image f(i,j ) The formula for calculating the average gradient is:

Next, the spatial frequency and average gradient of the panchromatic image are normalized using the arccotangent nonlinear normalization function, namely:

Continue to weight the two equally as follows:

Finally, the Tanh function is used to construct the differential order v as follows:

Among them, β and α take 0.5 and 0.7 respectively; In the step 3, the specific method of filtering the I component of the multispectral image by using guided filtering is as follows: First, set the radius of guided filtering r=7, regularization parameter ε=10 ^-6 , and calculate the linear coefficients a _(k,l) and b _(k,l) of guided filtering as follows:

In the formula, |ω| represents the number of pixels in a rectangular local window ω (k,l) centered on pixel _(k,l) and radius r,

and μ _(k,l) are the variance and mean of the pixels contained in the local window ω _(k,l) , respectively,

Represents the pixel mean value of the multispectral image I component contained in the local window ω _(k,l) , ε is a regularization parameter; Then perform mean value processing on the linear coefficients a _(k,l) and b _(k,l) , namely:

Bundle

and

Substitute into the linear definition model of guided filtering to get the filtered output image:

The result after guided filtering is used as the base layer of the image, and the detail layer of the image is obtained by subtracting the I component of the multispectral image from the base image, and then the gray scale variation range of the detail layer is linearly transformed, and finally added to the base image to obtain the texture Structurally enhanced images. 2. a kind of multi-source remote sensing image fusion method according to claim 1, is characterized in that, described step 6 carries out IHS inverse transformation with the I _new component that obtains and H component, S component, and the computing formula of final fusion image is :

Among them, R _new , G _new , and B _new are the red, green, and blue bands of the fused image respectively.

Images