CN1588448A - Image optimum fusing method based on fuzzy integral - Google Patents

Abstract

An image optimization fusion method based on fuzzy integrals. In IHS space, the low-frequency baseband coefficients obtained by multi-layer wavelet decomposition of the intensity component of the multispectral image, and the low-frequency baseband coefficients obtained by the corresponding multi-layer wavelet decomposition of the high-resolution image, Using fuzzy integral to synthesize spectral information and spatial resolution, two single-factor indicators, iteratively optimizes pixel-level fusion, and at the same time performs high-frequency detail feature fusion on the high-frequency sub-band coefficients after wavelet decomposition, and then fuses the processed The obtained high-frequency sub-band coefficients and low-frequency coefficients are subjected to corresponding wavelet inverse transformation to obtain a new intensity component I', and then the optimized fusion image is obtained after IHS inverse transformation. The invention combines the characteristics of the IHS fusion method and the wavelet fusion method, so that the fused image not only achieves the highest spatial resolution, but also reduces the color distortion to the greatest extent, and effectively improves the spectral information index of the fused image.

Description

Image Optimal Fusion Method Based on Fuzzy Integral

technical field

The present invention relates to an image optimization fusion method based on fuzzy integral, combined with the time-frequency characteristics of wavelet multi-resolution decomposition and IHS (Intensity-Chroma Hue-Saturation) transformation fusion method, using fuzzy integral to synthesize spectral information and The two single-factor indicators of spatial resolution are used to optimize the fusion of remote sensing images and effectively improve the spectral information indicators of fused images. They can be widely used in various military or civilian remote sensing information processing systems and digital city spatial information systems. .

Background technique

Effectively fusing high-resolution panchromatic remote sensing images and low-resolution multispectral remote sensing images, and balancing the two characteristic indicators of spatial detail information and spectral information in the fusion results, is one of the research hotspots in multi-source remote sensing image fusion technology.

The IHS fusion method first proposed by Haydn et al. is one of the classic practical algorithms. This method transforms the multispectral image from RGB (Red-Green-Green-Blue) space to IHS space through IHS transformation, and at the same time linearly stretches the high-resolution panchromatic image, so that the mean and variance of the stretched image coincides with the intensity component I ₀ in the IHS space. Then, the stretched high-resolution image is used as a new intensity component, and together with the H and S components, it is transformed into the original RGB space according to the IHS inverse transformation formula. In this way, the fused image not only has a higher spatial resolution, but also maintains the same hue and saturation of the original low-resolution multispectral image. However, this classic IHS fusion method has certain defects. Because the data of different bands have different spectral characteristic curves, the IHS fusion method distorts the original spectral characteristics and produces different degrees of spectral degradation, which is not conducive to image The correct identification and classification, especially for the image fusion of multi-sensor remote sensing images of different time phases, the IHS fusion method cannot make the color tone of the fusion image consistent with the color tone of the original multispectral image, which is caused by the transformation of spectral information. Images cannot be used for object recognition and inversion. Te-Ming et al. conducted a mathematical proof in the IHS space, discussed the defects of the IHS fusion method, and concluded that: although the high-resolution panchromatic image I _new used to replace the intensity component I ₀ is performed before the replacement The matching of the statistical characteristics of the image is achieved, but the matching error δ=I _new -I leads to color distortion.

When the fuzzy measure is used to represent the emphasis on the evaluation index, the introduction of fuzzy integral can effectively synthesize the two single factor indexes of spectral information index and spatial resolution. On the basis of IHS fusion method and wavelet transform fusion, fuzzy integral can be convenient Fast image optimization fusion. At present, there is no report on the method of using fuzzy integral for image optimal fusion.

Contents of the invention

The purpose of the present invention is to address the shortcomings of the above-mentioned IHS transformation and fusion technology, to provide an optimal fusion method for remote sensing images, which introduces fuzzy integrals as two single-factor indicators of comprehensive spectral information index and spatial resolution, which can improve the spatial resolution of the fused image. It can reduce the color distortion and effectively improve the spectral information index of the fusion image.

In order to achieve such a goal, the present invention combines the low-frequency baseband coefficients obtained by multi-layer wavelet decomposition of the intensity component of the multispectral image with the low-frequency baseband coefficients obtained by the corresponding multi-layer wavelet decomposition of the high-resolution image in the IHS space to obtain spatial details. The fuzzy optimization fusion of the two characteristic indicators of information and spectral information, the high-frequency detail feature fusion is performed on the high-frequency sub-band coefficients after wavelet decomposition, and then the corresponding wavelet inverse transform is performed on the wavelet coefficients to obtain new intensity components, and then The fused image is obtained after IHS inverse transformation.

Because the wavelet transform has good frequency division characteristics in the transform domain, and the statistical characteristics of the wavelet coefficients reflect the significant features such as edges, lines and regions of the remote sensing image, the present invention introduces the multi-resolution analysis (Multi-resolution Analysis) method of the wavelet transform into high In the fusion of high-resolution panchromatic remote sensing images and low-resolution multispectral remote sensing images.

Method of the present invention comprises following specific steps:

1. Perform IHS transformation on the multispectral image B to be fused to obtain the chroma H, saturation S, and intensity component I in the IHS color space, and then perform wavelet decomposition on the I component to obtain low-frequency baseband coefficients and high-frequency subband coefficients .

2. Perform linear stretching and histogram matching on the high-resolution image A to be fused, and then perform wavelet decomposition to obtain low-frequency baseband coefficients and high-frequency sub-band coefficients. The number of decomposition layers is the same as that of the multispectral image I component.

3. Determine a 3×3 spatial domain window, and obtain the mean value μ(2 ^j ) and variance D(2 ^j ) of the high-frequency sub-band coefficients of the I component of image B and the high-frequency sub-band coefficients of image A, respectively.

4. On the corresponding resolution layer, the high-frequency sub-band coefficients are fused according to formula (1) for high-frequency detail features:

In formula (1), 2 ^j is the number of wavelet decomposition layers, W ^k (2 ^j , x, y) is the fusion result of high-frequency sub-band coefficients obtained at 2 ^j resolution; W _A ^K (2j, x, y) and W _B ^K (2 ^j , x, y) are the high-frequency sub-band coefficients corresponding to the I component in image A and image B respectively, and D _A ^K and D _B ^K are respectively centered on (x, y) The variance of the 3×3 spatial domain window.

5. For the I component of image B and the low-frequency baseband coefficient of image A, the pixel-level fusion is optimized according to formula (2), and the k _opt of the weight coefficient is optimized according to the spectral information evaluation index and the spatial resolution evaluation index. Using fuzzy integral to synthesize spectral information index and two single factor indexes of spatial resolution:

A(2 ^j , x, y)=k ₁ A _A (2 ^j , x, y)+k ₂ A _B (2 ^j , x, y) (2)

In formula (2), A _A (2 ^j , x, y), A _B (2 ^j , x, y) are the low-frequency baseband data of 2 ^j resolution corresponding to the I component in image A and image B respectively, k ₁ , k ₂ is the weight coefficient that needs to be optimized. According to normalization requirements, k ₁ and k ₂ satisfy k ₁ +k ₂ =1, that is, the determination of the optimization coefficient can be attributed to k _opt =k ₁ =1-k ₂ satisfying the objective function. The weight coefficient in the optimization fusion iteration process satisfies 0≤k _opt ≤1;

According to the formula S=sup{min[e(u ₁ ), g(E ₁ )], min[e(u ₂ ), g(E ₂ )]} to calculate the fuzzy integral value E _i determined by the optimization evaluation index, find S _Kopt = max(E _i ), and the iterative value k _opt corresponding to S _Kopt is the optimal weight coefficient. In the formula, g(E ₁ ) and g(E ₂ ) are two items of spectral information index and spatial resolution The importance of indicators, e(u ₁ ) is the spectral information index, e(u ₂ ) is the spatial resolution index, according to the size of e(u ₁ ), e(u ₂ ), u ₁ and u ₂ are the spectral information and spatial resolution from small to large sorting positions.

6. Carry out corresponding wavelet inverse transform to the low-frequency baseband coefficients of the obtained pixel-level fused wavelet coefficients and the high-frequency wavelet coefficients to obtain a new intensity component I';

7. Perform IHS inverse transformation on I', H, and S to obtain the fused image C.

When the present invention introduces fuzzy integrals to perform pixel-level optimization and fusion of the two single-factor indexes of spectral information index and spatial resolution:

Definition: Let X be the domain of discourse, e is a measurable function from X to [0, 1], A∈P(X), then the fuzzy integral S of e with respect to the fuzzy measure g on the set A is defined as (3) :

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; A</span>}}{<span\; class="notranslate">\; \&Integral;</span>}\mathrm{<span\; class="notranslate">\; e</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0,1</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; sup</span>}\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; \&cap;</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

$<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0,1</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; \&cap;</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Wherein, E _α ={x|e(x)≥α}, P(x) is a power set of X. g(·) is the fuzzy measure. When fuzzy integral is used for optimal fusion, fuzzy measure can represent the degree of importance.

The key to optimal fusion using fuzzy integral is the definition of fuzzy measure g(x), which can be measured by gλ. In the case that the domain of discourse X={x ₁ , x ₂ , x ₃ ,...x _n } (factor set) is limited, when λ=0, as long as the single point set (single factor set) { _xi } gλ fuzzy measure gλ( _xi ) of gλ, then the measure of any AX can be obtained. For multi-spectral and high-resolution image fusion, domain X={x ₁ , x ₂ }, there are two evaluation factors, x ₁ =spectral information, x ₂ =spatial resolution. The degree of importance is gλ(x ₁ ), gλ(x ₂ ), simply expressed as g ₁ , g ₂ , then g({x ₁ })=g ₁ , g({x ₂ })=g ₂ , g({ x ₁ , x ₂ })=g({x ₁ })+g({x ₂ })=1. e(x) represents the spectral information evaluation index and the resolution evaluation index. The corresponding evaluation index of domain X is e(x ₁ )=E _SP , e(x ₂ )=E _HF is simply expressed as e ₁ , e ₂ . According to the definition of fuzzy integral, formula (4) can be obtained:

$\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{<span\; class="notranslate">\; \&Integral;</span>}\mathrm{<span\; class="notranslate">\; e</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0,1</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; sup</span>}\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&cap;</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

$<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0,1</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

According to the size of e ₁ and e ₂ , sort x ₁ and x ₂ and record them as u ₁ and u ₂ in ascending order. At this time, there are two situations for the value of α: when α=e(u ₁ ), E _α =E ₁ ={u ₁ , u ₂ }, then g(E ₁ )=1; when α=e(u ₂ ), E _ε =E ₂ ={u ₂ }, then g(E ₂ )=g({u ₂ }).

According to the definition of fuzzy integral, it can be expressed as formula (5):

S=sup{min[e(u ₁ ), g(E ₁ )], min[e(u ₂ ), g(E ₂ )]} (5)

When e ₁ >e ₂ , the sorting position of x ₁ and x ₂ from small to large is u ₁ = x ₂ , u ₂ = x ₁ , correspondingly e(u ₁ )=e ₂ , g(E ₁ )=1 , e(u ₂ )=e ₁ , g(E ₂ )=g({u ₂ })=g({x ₁ })=g ₁ , then the fuzzy integral S=max(e ₁ ·g ₁ , e ₂ ).

When e ₁ < e ₂ , the sorting position of x ₁ and x ₂ from small to large is u ₁ = x ₁ , u ₂ = x ₂ , correspondingly e(u ₁ )=e ₁ , g(E ₁ )=1 , e(u ₂ )=e ₂ , g(E ₂ )=g({u ₂ })=g({x ₂ })=g ₂ , then the fuzzy integral S=max(e ₁ , e ₂ ·g ₂ ).

When e ₁ =e ₂ , S can take any one of the above two values.

In the process of optimizing the fusion iteratively obtaining the weight coefficient k _opt , the fuzzy integral value of the two single-factor indexes of spectral information index and spatial resolution is obtained by using formula (5), and then S _Kopt =max(E _i ) , the iteration value k _opt corresponding to S _Kopt is the optimal weight coefficient, so that the optimal weight k _opt can be determined conveniently and quickly.

The present invention combines the characteristics of the IHS fusion method and the wavelet fusion method, and performs pixel-level optimal fusion and high-frequency detail feature fusion of the high-frequency sub-band coefficients on the weight coefficients of the wavelet baseband coefficients respectively, and its beneficial effects are embodied as follows: The fused image not only achieves the highest spatial resolution, but also minimizes color distortion. The two feature indexes of spatial detail information and spectral information in the fusion result are balanced, and the spectral information index of the fusion image is effectively improved. At the same time, the fuzzy integral method is introduced to effectively integrate the two single-factor indicators of spectral information and spatial resolution, and it is convenient and quick to determine the optimal weight value in the process of pixel-level optimal fusion, and the result is more in line with human subjective perception of fused images. feel.

Description of drawings

Fig. 1 is the flow chart of the present invention - image optimization and fusion method based on fuzzy integral.

Fig. 2 is the comparison between the remote sensing image fusion results of the present invention and the IHS and WT (wavelet) methods. Among them, Figure 2(a) is the multispectral remote sensing image (256×256); Figure 2(b) is the high spatial resolution panchromatic remote sensing image (256×256); Figure 2(c) is the fusion result of the IHS method, Fig. 2(d) is the fusion result of the WT method, and Fig. 2(e) is the optimized fusion result of the present invention.

Fig. 3 is the performance evaluation index curve of the present invention.

Detailed ways

In order to better understand the technical solution of the present invention, further description will be made below in conjunction with the accompanying drawings.

The detailed process of the method of the present invention is shown in Figure 1. The present invention selects a multi-spectral remote sensing image B (256×256) as shown in Figure 2(a), and a high-resolution panchromatic remote sensing image A (256×256) as shown in Figure 2(b). After A and B are strictly registered, the implementation Follow the steps below:

1. Perform IHS transformation on the multispectral image B to obtain the chroma H, saturation S and intensity component I in the IHS color space respectively, and then perform three-level wavelet decomposition on the I component to obtain the low-frequency baseband coefficient and high-frequency sub-band coefficient .

2. Perform linear stretching and histogram matching on the high-resolution image A, and then perform wavelet decomposition. The number of decomposition layers is also 3 layers, and the low-frequency baseband coefficients and high-frequency sub-band coefficients are obtained.

3. Determine a 3×3 spatial domain window, and obtain the mean value μ(2 ^j ) and variance D(2 ^j ) of the high-frequency subband coefficients of the I component in image B and the high-frequency wavelet coefficients of image A respectively.

4. Determine the high-frequency sub-band coefficients of the corresponding resolution layer according to the following formula, and perform high-frequency detail feature fusion to obtain the high-frequency detail feature fusion result of the corresponding resolution layer.

5. The method of introducing fuzzy integral to synthesize the two single factor indicators of spectral information index and spatial resolution is as follows: according to the following formula

S=sup{min[e(u ₁ ), g(E ₁ )], min[e(u ₂ ), g(E ₂ )]}

Calculate the fuzzy integral value E _i determined by the optimization evaluation index, obtain S _Kopt =max(E _i ), and the iteration value k _opt corresponding to S _Kopt is the optimal weight coefficient.

Among them, the fusion image optimization evaluation index is defined as follows:

(i) Spectral information evaluation index

The evaluation index of spectral information is defined by the degree of correlation between fusion image and multispectral image.

Let f be the fused image and f ₀ be the multispectral image. Define spectral information evaluation E _SP index as (6) formula.

${\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; SP</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; Corr</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; npix</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}{\sqrt{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; npix</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; npix</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, npix is the number of pixels in the image, f and f ₀ represent the average gray value of the image, and the degree of correlation Corr(f, f ₀ ) reflects the degree of similarity between images f and f ₀ .

(ii) Spatial resolution evaluation index

The spatial resolution index is defined by the degree of correlation between the high-frequency component of the grayscale corresponding to the fused image and the high-frequency component of the high-resolution image. Let f _H be a high-resolution panchromatic image. First convert the fused image into a grayscale image, and then perform wavelet decomposition to obtain four components (f ^a , f ^h , f ^v , f ^d ) of the fused image, which respectively represent the low frequency component of the fused image and the high frequency in the horizontal direction component, the high-frequency component in the vertical direction, and the high-frequency component in the diagonal direction. Similarly, four components (f _H ^a , f _H ^h , f _H ^v , f _H ^d ) of the high-resolution image wavelet decomposition can be obtained. Define the spatial resolution evaluation index as shown in formula (7):

${\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; HF</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; Corr</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; Corr</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; Corr</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

In order to verify the effectiveness of the optimal weight calculated based on the fuzzy integral, pixel-level fusion can be performed according to formula (8), and the weight coefficient k _opt is optimized according to the objective function shown in formula (8):

The weight coefficients k ₁ and k ₂ of the baseband data fusion in the fusion criterion are corrected according to the optimization evaluation index of the fusion image. With the increase of the fusion weight k ₁ of low-frequency baseband coefficients of high-resolution images, the spatial resolution evaluation index E _HF increases, and the spectral information evaluation index E _SP decreases. Therefore, k _opt in formula (6) is the weight coefficient that maximizes the objective function, that is, the weight coefficient that maximizes E _SP and E _HF at the same time. The value range of k _opt is [0, 1].

According to the optimization objective function of the weight coefficient k _opt , in the value interval [0, 1], with the increase of the weight coefficient fusion weight k ₁ , the spatial resolution evaluation index E _HF and spectral information can be obtained according to formula (9) The curve of the evaluation index E _SP , the intersection point of the curve is the k _opt of the optimal weight coefficient, as shown in Figure 3 .

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; MinE</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}$ $<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

In the formula (9), E(i) is the evaluation index, and i is the optimization times of weight coefficient _k1 . Figure 3 shows the curves of E _SP and E _HF obtained at [0, 1] with an optimal step size of 0.001. It can be seen that both E _SP and E _HF after normalization are nonlinear. The weight coefficient k _opt satisfying the optimization objective function is the intersection point of E _SP and E _HF k _opt =0.424 (convergence accuracy δ≤0.001).

As shown in Figure 3, it is obvious that fuzzy integral is introduced to comprehensively evaluate the two single-factor indicators of spectral information index and spatial resolution, and the iterative value k _opt corresponding to S _Kopt (S _Kopt = max(E _i )) is calculated and obtained That is, the intersection point of the curves of E _HF and E _SP calculated according to formula (9), which is the k _opt of the optimal weight coefficient.

6. Perform corresponding wavelet inverse transform on the low-frequency baseband coefficients of the obtained pixel-level optimally fused wavelet coefficients and the high-frequency wavelet coefficients of the feature-level fusion to obtain a new intensity component I'.

7. Perform IHS inverse transformation on I', H, and S to obtain the fused image C.

Fusion result comparison: Fig. 2 (c) is the fusion result (256 * 256) of IHS method, Fig. 2 (d) is the fusion result (256 * 256) of WT method, Fig. 2 (e) optimization fusion result of the present invention ( 256×256).

In the process of optimization, according to the two feature evaluation indexes defined in formulas (4)~(5), they are substituted into formula (5), and the value of fuzzy integral S is calculated according to every 100 steps. The results are shown in Table 1 (note that degrees g ₁ =0.7, g ₂ =0.3). At the same time, the fuzzy integral value S is drawn as shown in Figure 3. Obviously, the fuzzy integral value S presents a non-linear change process with a peak point, which is the intersection point of the index curve obtained after normalizing the two characteristic indexes - the optimal weight coefficient k _opt , which makes the fusion The final image not only achieves the highest spatial resolution, but also minimizes color distortion.

Table 1 Comparison of evaluation indicators of fusion results Find Uber step length Spectral Information E _SP Spatial resolution E _HF Fuzzy integral S K ₁ 0.99323 0.8869 0.70000 K ₂ 0.99159 0.90191 0.69077 K ₃ 0.98812 0.91298 0.67116 K ₄ 0.98237 0.92122 0.6998 K ₅ 0.97384 0.92718 0.82135 K ₆ 0.96197 0.93127 0.74776 K ₇ 0.94615 0.93389 0.62011 K ₈ 0.92579 0.93535 0.45575 K ₉ 0.90032 0.93594 0.30000 K ₁₀ 0.86932 0.93589 0.29965

The changes in the evaluation values of the two factors, spectral information and spatial resolution information shown in Table 1, are in line with human subjective evaluation feelings. In other words, Figure 2(e) obtained by using the image fusion method based on fuzzy integral conforms to people's subjective evaluation habits and subjective feelings.

Table 2 Comparison of evaluation indicators for the results of different fusion methods band correlation coefficient mean gradient IHS R 0.5624 10.6146 G 0.4644 11.3663 B 0.5060 9.5875 WT R 0.6457 11.0157 G 0.5512 13.5479 B 0.5714 10.5447 Method in this paper R 0.8488 10.5142 G 0.7691 11.1475 B 0.8247 9.2163

Table 2 shows the quantitative evaluation results of the fused images obtained by different fusion methods using the two indicators of correlation coefficient and average gradient. As can be seen from Table 2, the fuzzy integral-based image optimization fusion method of the present invention can greatly improve the spectral information retention performance of the fused image, but it is slightly worse than the wavelet transform (WT) method in improving the spatial resolution. , and the spatial resolution index of the fused image obtained by the IHS method is basically the same. Therefore, the method of the present invention minimizes color distortion while achieving the highest spatial resolution.

Claims (1)

1, a kind of image optimization fusion method based on fuzzy integral, it is characterized in that comprising following concrete steps: (1) Perform IHS transformation on the multispectral image B to be fused to obtain the chroma H, saturation S and intensity component I in the IHS color space, and then perform wavelet decomposition on the I component to obtain the low-frequency baseband coefficients and high-frequency subbands coefficient; (2) Perform linear stretching and histogram matching on the high-resolution image A to be fused, and then perform wavelet decomposition to obtain low-frequency baseband coefficients and high-frequency sub-band coefficients. The number of decomposition layers is the same as that of the multispectral image I component. ; (3) Determine a 3×3 spatial domain window, and obtain the mean value μ(2 ^j ) and variance D(2 ^j ) of the high-frequency sub-band coefficients of the I component of image B and the high-frequency sub-band coefficients of image A respectively; (4) The high-frequency sub-band coefficients corresponding to the resolution layer follow the following formula

Perform high-frequency detail feature fusion, where 2 ^j is the number of wavelet decomposition layers, W ^k (2 ^j , x, y) is the high-frequency sub-band coefficient fusion result obtained at 2 ^j resolution, W _A ^k (2 ^j , x, y) and W _B ^k (2 ^j , x, y) are the corresponding high-frequency sub-band coefficients of the I component in image A and image B respectively, and D _A ^k and D _B ^k are respectively (x, y) is the variance of the 3×3 airspace window of the central pixel; (5) For the I component of image B and the low-frequency baseband coefficient of image A, according to the following formula A(2 ^j , x, y)=k ₁ A _A (2 ^j , x, y)+k ₂ A _B (2 ^j , x, y) for optimized pixel-level fusion, where A _A (2 ^j , x, y), A _B (2 ^j , x, y) are the low-frequency baseband data of 2 ^j resolution corresponding to the I component in image A and image B respectively, k ₁ , k ₂ are the weight coefficients that need to be optimized, k ₁ +k ₂ =1; Introduce fuzzy integral to synthesize the two single factor indicators of spectral information index and spatial resolution, and optimize the weight coefficient k _opt =k ₁ =1-k ₂ in the process of fusion iteration, satisfying 0≤k _opt ≤1; according to the formula S=sup{min[e(u ₁ ), g(E ₁ )], min[e(u ₂ ), g(E ₂ )]} to calculate the fuzzy integral value determined by the optimal evaluation index E _i , get S _Kopt = max(E _i ), and the iterative value k _opt corresponding to S _Kopt is the optimal weight coefficient, where g(E ₁ ) and g(E ₂ ) are the spectral information index and spatial The importance of the two indicators of resolution, e(u ₁ ) is the spectral information indicator, e(u ₂ ) is the spatial resolution indicator, according to the size of e(u ₁ ), e(u ₂ ), u ₁ and u ₂ is the sorting position of spectral information and spatial resolution from small to large; (6) Carry out corresponding wavelet inverse transform to the low-frequency baseband coefficients of the wavelet coefficients of the pixel-level optimization fusion obtained, and each high-frequency wavelet coefficient of the feature-level fusion carried out, to obtain a new intensity component I '; (7) Perform IHS inverse transformation on I', H, and S to obtain the fused image C.

Images