CN104616261A - Method for fusing Shearlet domain multi-spectral and full-color images based on spectral characteristics - Google Patents

Abstract

The invention relates to a method for fusing Shearlet domain multi-spectral and full-color images based on spectral characteristics. The method comprises the steps of (1) performing IHS transformation for a multi-spectral image; (2) performing Shearlet transformation and decomposition for the BIMFs and the full-color image to obtain a low-frequency sub-band and a plurality of high-frequency sub-band; (3) building a fusion rule and selecting fusion coefficients, namely, decomposing the low-frequency coefficients through laplacian pyramid, selecting the low-frequency fusion coefficients by the comparing and bigger selection method, and fusing the corresponding high-frequency coefficients according to the fusion rule based on PCNN (Pulse Coupled Neural Network); (4) performing IHS reverse transformation to obtain the final fused image. With the adoption of the fusion algorithm, the spectrum information of the images is protected while the space details expression capacity is improved, and thus the fusion effect is improved.

Description

Multispectral and the panchromatic image fusion method in a kind of Shearlet territory based on spectral characteristic

Technical field

The present invention relates to a kind of method of technical field of image processing, be specifically related to the multispectral and panchromatic image fusion method in a kind of Shearlet territory based on spectral characteristic.

Background technology

Remote sensing image cannot possess high spatial resolution and high spectral resolution simultaneously, needs to utilize spectral resolution to improve spatial resolution.Remote sensing image fusion by certain regular calculation process, obtains the multi-source data of space or time upper redundancy than any single data more accurately, abundanter information, generates the composograph that a width has new space, wave spectrum, temporal characteristics.

Full-colour image has high spatial resolution, the minutia of expressive atural object, but spectral resolution is not enough; Multispectral image spectral information enriches, and be beneficial to identification ground object target profile, but spatial resolution is low.Multispectral fusion with full-colour image both can utilize the high resolving power of full-colour image to improve the resolution of multispectral image, can make full use of again the distinctive meticulous expression to target in multispectral image, obtain abundanter target information.

Improvement component replacement method and multiresolution analysis are the two kinds of methods solving spectrum problem of dtmf distortion DTMF, by improving conventional IHS transformation for mula, substituted for matrix multiplication operation with addition of matrices computing, range of fusion is expanded to the fusion more than three wave bands, to reduce spectrum degreeof tortuosity; Part research, by adding regulating parameter, in order to control the injection degree of spatial detail, makes fusion results between standard I HS conversion and BROVERY convert.

In order to obtain better syncretizing effect, increasing research gets up to use by component method of substitution with based on multiresolution analysis methods combining; The fusion method of IHS conversion combined with wavelet transformed is representative wherein, first to the luminance component pre-service of Pan image, MS image, then wavelet decomposition is carried out, the high frequency coefficient of luminance component is replaced with the high frequency coefficient of full-colour image, retain the low frequency component of MS image, fusion treatment obtains new luminance component, and the spectral information carried due to it is protected, and decreases spectrum distortion to a certain extent; Similar utilizes IHS conversion in conjunction with the fusion method of contourlet transformation, first the luminance component of MS image and PAN figure is done Histogram Matching, uses multiresolution analysis that the detailed information of PAN image is injected into luminance component, obtain high-resolution principal component; Only carry out between luminance component and PAN image owing to merging, part decreases the spectrum problem of dtmf distortion DTMF of fused images; Although said method decreases the spectrum distortion of fused images, but because it all comprises down-sampling and up-sampling operation, produce Pseudo-Gibbs artifacts, have lost the partial spatial information of Pan image, the spatial resolution of fusion evaluation is often not as good as traditional IHS fusion method, fusion results is responsive to the selection and comparison of wavelet basis function Sum decomposition level simultaneously, and Selecting parameter is comparatively complicated.

In order to overcome above-mentioned shortcoming, BEMD can be utilized to replace above-mentioned multiresolution analysis method, reason is that BEMD has than the more intuitive character representation mode of wavelet analysis and frequency characteristic more flexibly, image is decomposed and does not need in conversion, select basis function but can obtain the multi-resolution representation of similar wavelet transformation, and do not introduce redundant information in goal decomposition.Decompose each intersubband of BIMFs obtained and there is correlativity, be similar to the correlativity between wavelet field different scale, simultaneously because decomposable process is without down-sampling, avoid the Pseudo-Gibbs artifacts that can cause in wavelet transformation.BEMD has the ability of good Image Edge-Detection and texture recognition, is beneficial to the extraction of image detail protection and image texture.The each subband of BIMFs contains the major side profile of original image, i.e. spatial structural form, and surplus then contains the tendency information of image, corresponding to the low frequency component of remote sensing images, carries its spectral information.So utilize BEMD to realize multi-resolution analysis, spectral information can be protected better.

Summary of the invention

Technical matters to be solved by this invention is: provide a kind of Shearlet territory based on spectral characteristic multispectral and panchromatic image fusion method, the visual characteristic of human eye under color background has been taken into account while improving spatial resolution, retain the spectral information in the former luminance component of multispectral image as far as possible, decrease spectrum distortion.

Technical scheme of the present invention is: the multispectral and panchromatic image fusion method in a kind of Shearlet territory based on spectral characteristic, comprises the following steps:

(1) IHS conversion is carried out to multispectral image, obtain luminance component P _i, chromatic component P _s, saturation degree component P _h, by the luminance component P of multispectral image _iobtain accumulateing modular function BIMF in n-1 two dimension that frequency arranges from high to low by two-dimensional empirical mode decomposition BEMD _t, t=1,2 ... n-1, n>3 and surplus res;

(2) modular function BIMF is accumulate in calculating n-1 two dimension _tenergy and sort, first three component sum B of statistics energy ordering _mS, by B _mSwith full-colour image I _pANdecompose respectively by Shearlet conversion, Decomposition order is three layers, obtains a respective low frequency sub-band: with three layers of high-frequency sub-band: m=1,2,3, accumulate modular function BIMF in two dimension _tthe computing formula of energy is such as formula shown in (1):

$E\left({\mathrm{BIMF}}_t\right)=\left(\underset{x=1}{\overset{i}{\mathrm{\Σ}}}\underset{y=1}{\overset{j}{\mathrm{\Σ}}}D{{(x,y)}^2}_t\right)---\left(1\right)$

Wherein, D (x, y) _tfor BIMF _tcoefficient, i, j represent BIMF respectively _tline number and columns;

(3) set up fusion rule and select fusion coefficients: utilize Laplacian pyramid low frequency coefficient, obtain corresponding coefficient of dissociation L _{mS, ij}and L _{pAN, ij}, the large method of relatively getting shown in through type (2) selects low frequency fusion coefficients L _{f, ij}:

$L_{F,\mathrm{ij}}=\left\{\begin{array}{c}{L}_{\mathrm{MS},\mathrm{ij}},\mathrm{if}(L_{\mathrm{MS},\mathrm{ij}}\&GreaterEqual;L_{\mathrm{PAN},\mathrm{ij}})\\ L_{\mathrm{PAN},\mathrm{ij}},\mathrm{if}(L_{\mathrm{MS},\mathrm{ij}}<L_{\mathrm{PAN},\mathrm{ij}})\end{array}\right.---\left(2\right)$

Corresponding high frequency coefficient is merged by the fusion rule formulated based on binary channels Pulse Coupled Neural Network, specific as follows:

1. will input two input ends of binary channels Pulse Coupled Neural Network respectively, utilize the external drive of CSF factor CSF as binary channels Pulse Coupled Neural Network of formula (3) design factor;

CSF(f)＝2.6×(0.0192+0.114f)exp[-(0.114f) ^1.1] (3)

Wherein, it is spatial frequency; f _x, f _ythe spatial frequency of level and vertical direction respectively,

$f_x=\sqrt{\frac{1}{\mathrm{MN}}\underset{i=0}{\overset{M-1}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N-1}{\mathrm{\&Sigma;}}}{[f(i,j)-f(i,j-1)]}^2}---\left(4\right)$

$f_y=\sqrt{\frac{1}{\mathrm{MN}}\underset{i=1}{\overset{M-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{N-1}{\mathrm{\&Sigma;}}}{[f(i,j)-f(i-1,j)]}^2}---\left(5\right)$

Wherein, M is source images line number, and N is source images columns;

2. compare according to formula (6) fusion coefficients obtaining all directions subband:

$H_{F,\mathrm{ij}}^{m,k}=\left\{\begin{array}{cc}{B}_{\mathrm{MS},\mathrm{ij}}^{h,m},\mathrm{if}& \mathrm{CSF}\left(B_{\mathrm{MS},\mathrm{ij}}^{h,m}\right)>\mathrm{CSF}\left(I_{\mathrm{PAN},\mathrm{ij}}^{h,m}\right)\\ I_{\mathrm{PAN},\mathrm{ij}}^{h,m},\mathrm{if}& \mathrm{CSF}\left(B_{\mathrm{MS},\mathrm{ij}}^{h,m}\right)<\mathrm{CSF}\left(I_{\mathrm{PAN},\mathrm{ij}}^{h,m}\right)\end{array}\right.---\left(6\right)$

Wherein, m and k refers to that m layer decomposes respectively, a kth directional subband, it is high-frequency sub-band at the coefficient that (i, j) puts;

3. Shearlet inverse transformation is utilized to reconstruct L _{f, ij}with obtain fusion coefficients F _i, utilize formula (7) to obtain new luminance component P _i':

P _I'＝F _I+res (7)

Wherein, res is the surplus that step (1) obtains;

(4) by new luminance component P _i', chromatic component P _s, saturation degree component P _hcarry out IHS inverse transformation, obtain final fused images.

The present invention's advantage is compared with prior art:

BEMD and human eye contrast sensory characteristic apply in remote sensing image fusion by the present invention, first source images realizes color space conversion through IHS conversion, I component is obtained comprising the intrinsic mode function of high fdrequency component and comprising the surplus of low frequency component by two-dimensional empirical mode decomposition, then Pan image and IMFs are decomposed by Shearlet conversion the coefficient of dissociation obtained under different scale multiple directions, fusion rule considers human eye characteristic, utilize based on color contrast sensitivity excitation binary channels PCNN and laplacian pyramid select fusion coefficients, fused images is obtained again by the reconstruct of IHS inverse transformation, new algorithm make use of the advantage that Shearlet conversion can realize the expression of image optimum.

For improving the spectrum distortion that traditional fusion method causes, the present invention utilizes color contrast sensitivity characteristic, first I component is obtained by IHS conversion, realize multispectral image spectral information (the Res component in I component) in conjunction with BEMD to be again separated with spatial information (the BIMFs component in I component), spatial information in Pan image is injected into fused images by the mode merged with IMFs, in order to meet human eye characteristic, utilize Shearlet to convert and respectively Pan image and IMFs are decomposed into the coefficient under different scale multiple directions, select to merge high frequency coefficient by the binary channels PCNN encouraged based on color contrast susceptibility, realize the extraction of color background hypograph feature, blending algorithm protects the spectral information of image as far as possible while strengthening spatial detail expressive ability, improve syncretizing effect.

Accompanying drawing explanation

Fig. 1 is that schematic flow sheet is merged in the present invention.

Fig. 2 is various blending algorithm effectiveness comparison schematic diagram.

Embodiment

Below in conjunction with accompanying drawing, the present invention is described in further detail.

As shown in Figure 1, a kind of Shearlet territory based on spectral characteristic is multispectral comprises the following steps with panchromatic image fusion method:

(1) IHS conversion is carried out to multispectral image, obtain luminance component P _i, chromatic component P _s, saturation degree component P _h, by the luminance component P of multispectral image _iobtain accumulateing modular function BIMF in a frequency n-1 from high to low two dimension by two-dimensional empirical mode decomposition BEMD _t, t=1,2 ... n-1, n>3 and surplus res;

(2) modular function BIMF is accumulate in calculating n-1 two dimension _tenergy to go forward side by side line ordering, first three component sum B of statistics energy ordering _mS, by B _mSwith full-colour image I _pANcarry out Shearlet respectively and convert three layers of decomposition, obtain a respective low frequency sub-band: with three layers of high-frequency sub-band: m=1,2,3, wherein 1-3 layer comprises 10 respectively, 10,18 directional subbands; Modular function BIMF is accumulate in two dimension _tthe computing formula of energy is such as formula shown in (1):

$E\left({\mathrm{BIMF}}_t\right)=\left(\underset{x=1}{\overset{i}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{j}{\mathrm{\&Sigma;}}}D{{(x,y)}^2}_t\right)---\left(1\right)$

Wherein, D (x, y) _tfor BIMF _tcoefficient, i, j represent BIMF respectively _tline number and columns;

(3) set up fusion rule and select fusion coefficients: utilize Laplacian pyramid low frequency coefficient, obtain corresponding coefficient of dissociation L _{mS, ij}and L _{pAN, ij}, the large method of relatively getting shown in through type (2) selects low frequency fusion coefficients L _{f, ij}:

$L_{F,\mathrm{ij}}=\left\{\begin{array}{c}{L}_{\mathrm{MS},\mathrm{ij}},\mathrm{if}(L_{\mathrm{MS},\mathrm{ij}}\&GreaterEqual;L_{\mathrm{PAN},\mathrm{ij}})\\ L_{\mathrm{PAN},\mathrm{ij}},\mathrm{if}(L_{\mathrm{MS},\mathrm{ij}}<L_{\mathrm{PAN},\mathrm{ij}})\end{array}\right.---\left(2\right)$

Corresponding high frequency coefficient is merged by the fusion rule formulated based on binary channels Pulse Coupled Neural Network PCNN, specific as follows:

1. will be input to two input ends of binary channels Pulse Coupled Neural Network PCNN respectively, utilize the external drive of CSF factor CSF as binary channels Pulse Coupled Neural Network PCNN of formula (3) design factor;

CSF(f)＝2.6×(0.0192+0.114f)exp[-(0.114f) ^1.1] (3)

Wherein, it is spatial frequency; f _x, f _ythe spatial frequency of level and vertical direction respectively,

$f_x=\sqrt{\frac{1}{\mathrm{MN}}\underset{i=0}{\overset{M-1}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N-1}{\mathrm{\&Sigma;}}}{[f(i,j)-f(i,j-1)]}^2}---\left(4\right)$

$f_y=\sqrt{\frac{1}{\mathrm{MN}}\underset{i=1}{\overset{M-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{N-1}{\mathrm{\&Sigma;}}}{[f(i,j)-f(i-1,j)]}^2}---\left(5\right)$

Wherein, M is source images line number, and N is source images columns;

2. compare according to formula (6) fusion coefficients obtaining all directions subband:

$H_{F,\mathrm{ij}}^{m,k}=\left\{\begin{array}{cc}{B}_{\mathrm{MS},\mathrm{ij}}^{h,m},\mathrm{if}& \mathrm{CSF}\left(B_{\mathrm{MS},\mathrm{ij}}^{h,m}\right)>\mathrm{CSF}\left(I_{\mathrm{PAN},\mathrm{ij}}^{h,m}\right)\\ I_{\mathrm{PAN},\mathrm{ij}}^{h,m},\mathrm{if}& \mathrm{CSF}\left(B_{\mathrm{MS},\mathrm{ij}}^{h,m}\right)<\mathrm{CSF}\left(I_{\mathrm{PAN},\mathrm{ij}}^{h,m}\right)\end{array}\right.---\left(6\right)$

Wherein, m and k refers to that m layer decomposes respectively, a kth directional subband; it is high-frequency sub-band at the coefficient that (i, j) puts, be high-frequency sub-band, obtained by step (2).

3. Shearlet inverse transformation is utilized to reconstruct L _{f, ij}with obtain fusion coefficients F _i, utilize formula (7) to obtain new luminance component P _i':

P _I'＝F _I+res (7)

Wherein, res is the surplus obtained in step (1);

(4) by new luminance component P _i', chromatic component P _s, saturation degree component P _hcarry out IHS inverse transformation, the image after finally being merged.

As shown in Figure 2, the effect that compares with other several fusion method of the inventive method.Figure (a) is Pan image; figure (b) is MS image; figure (c)-(g) is respectively the effect utilizing and carry out image co-registration based on GIHS, wavelet transform (DWT), principal component analysis (PCA) (PCA), Brovey and the inventive method; by relatively seeing; utilize the inventive method to carry out image co-registration and not only effectively enrich the background information of image; and the details protected to greatest extent in image, meet human-eye visual characteristic.

In objective evaluation Indexes Comparison table shown in table 1, by standard deviation (Std), information entropy (En), spatial frequency (SF), average gradient (Gradient), related coefficient (CC), spectrum torsion resistance (WD) weighs the fused image quality that different fusion method obtains, standard deviation (Std), information entropy (En), spatial frequency (SF), average gradient (Gradient), related coefficient (CC) is larger, illustrate that in fused images, characteristic information is abundanter, more close with source images, syncretizing effect is better, spectrum torsion resistance (WD) weighs spectrum distortion level, be worth the distortion of larger explanation spectrum more serious.Can be seen by data in table 1, the first five index of this method is compared with additive method and is all had clear improvement, the fused images that display this method generates has larger partial gradient, grey level distribution is disperseed more, image texture is abundanter, and details is given prominence to, and spectrum torsion resistance (WD) is that all methods are minimum, illustrate that the loss of the inventive method spectral information is less, syncretizing effect is better.

Table 1

Claims (1)

1. an and panchromatic image fusion method multispectral based on the Shearlet territory of spectral characteristic, is characterized in that, comprise the following steps:

(1) IHS conversion is carried out to multispectral image, obtain luminance component P _i, chromatic component P _s, saturation degree component P _h, by the luminance component P of multispectral image _iobtain accumulateing modular function BIMF in n-1 two dimension that frequency arranges from high to low by two-dimensional empirical mode decomposition BEMD _t, t=1,2 ... n-1, n>3 and surplus res;

(2) modular function BIMF is accumulate in calculating n-1 two dimension _tenergy and sort, first three component sum B of statistics energy ordering _mS, by B _mSwith full-colour image I _pANdecompose respectively by Shearlet conversion, Decomposition order is three layers, obtains a respective low frequency sub-band: with three layers of high-frequency sub-band: m=1,2,3, accumulate modular function BIMF in two dimension _tthe computing formula of energy is such as formula shown in (1):

$E\left({\mathrm{BIMF}}_t\right)=\left(\underset{x=1}{\overset{i}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{j}{\mathrm{\&Sigma;}}}D{{(x,y)}^2}_t\right)---\left(1\right)$

Wherein, D (x, y) _tfor BIMF _tcoefficient, i, j represent BIMF respectively _tline number and columns;

(3) set up fusion rule and select fusion coefficients: utilize Laplacian pyramid low frequency coefficient, obtain corresponding coefficient of dissociation L _{mS, ij}and L _{pAN, ij}, the large method of relatively getting shown in through type (2) selects low frequency fusion coefficients L _{f, ij}:

$L_{F,\mathrm{ij}}=\left\{\begin{array}{c}{L}_{\mathrm{MS},\mathrm{ij}}\mathrm{if}(L_{\mathrm{MS},\mathrm{ij}}\&GreaterEqual;L_{\mathrm{PAN},\mathrm{ij}})\\ L_{\mathrm{PAN},\mathrm{ij}},\mathrm{if}(L_{\mathrm{MS},\mathrm{ij}}<L_{\mathrm{PAN},\mathrm{ij}})\end{array}\right.---\left(2\right)$

Corresponding high frequency coefficient is merged by the fusion rule formulated based on binary channels Pulse Coupled Neural Network, specific as follows:

1. will input two input ends of binary channels Pulse Coupled Neural Network respectively, utilize the external drive of CSF factor CSF as binary channels Pulse Coupled Neural Network of formula (3) design factor;

CSF(f)＝2.6×(0.0192+0.114f)exp[-(0.114f) ^1.1] (3)

Wherein, it is spatial frequency; f _x, f _ythe spatial frequency of level and vertical direction respectively,

$f_x=\sqrt{\frac{1}{\mathrm{MN}}\underset{i=0}{\overset{M-1}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N-1}{\mathrm{\&Sigma;}}}{[f(i,j)-f(i,j-1)]}^2}---\left(4\right)$

$f_y=\sqrt{\frac{1}{\mathrm{MN}}\underset{i=1}{\overset{M-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{N-1}{\mathrm{\&Sigma;}}}{[f(i,j)-f(i-1,j)]}^2}---\left(5\right)$

Wherein, M is source images line number, and N is source images columns;

2. compare according to formula (6) fusion coefficients obtaining all directions subband:

$H_{F,\mathrm{ij}}^{m,k}=\left\{\begin{array}{cc}{B}_{\mathrm{MS},\mathrm{ij}}^{h,m},\mathrm{if}& \mathrm{CSF}\left(B_{\mathrm{MS},\mathrm{ij}}^{h,m}\right)>\mathrm{CSF}\left(I_{\mathrm{PAN},\mathrm{ij}}^{h,m}\right)\\ I_{\mathrm{PAN},\mathrm{ij}}^{h,m},\mathrm{if}& \mathrm{CSF}\left(B_{\mathrm{MS},\mathrm{ij}}^{h,m}\right)<\mathrm{CSF}\left(I_{\mathrm{PAN},\mathrm{ij}}^{h,m}\right)\end{array}\right.---\left(6\right)$

Wherein, m and k refers to that m layer decomposes respectively, a kth directional subband, it is high-frequency sub-band at the coefficient that (i, j) puts;

3. Shearlet inverse transformation is utilized to reconstruct L _{f, ij}with obtain fusion coefficients F _i, utilize formula (7) to obtain new luminance component P _i':

P _I'＝F _I+res (7)

Wherein, res is the surplus that step (1) obtains;

(4) by new luminance component P _i', chromatic component P _s, saturation degree component P _hcarry out IHS inverse transformation, obtain final fused images.