CN103208102A - Remote sensing image fusion method based on sparse representation - Google Patents

Abstract

The invention discloses a remote sensing image fusion method based on sparse representation. The method comprises the following steps of: firstly, establishing a linear regression model between a multispectral image and a brightness component thereof; secondly, performing sparse representation on a panchromatic image and the multispectral image by using high and low resolution dictionaries respectively, and acquiring sparse representation coefficients of the brightness component of the multispectral image according to the linear regression model; thirdly, extracting detail components according to the sparse representation coefficients of the panchromatic image and the brightness component, and implanting the detail components to the sparse representation coefficients of each band of the multispectral image under a general component replacement fusion framework; and finally, performing image restoration to obtain a multispectral image with high spatial resolution. According to the method, the sparse representation technology is introduced into the field of remote sensing image fusion, so that the defect that high spatial resolution and spectral information cannot be simultaneously preserved in the prior art is overcome; and the fusion result of the method is superior to that of the conventional remote sensing image fusion method on the aspects of spectral preservation and spatial resolution improvement.

Description

A kind of remote sensing image fusion method based on rarefaction representation

Technical field

The present invention relates to image processing techniques, specifically is a kind of remote sensing image fusing method based on rarefaction representation.

Background technology

Along with the continuous development of remote sensing technology, the remote sensing image data of the different spatial resolutions that is obtained by various satellite sensors, temporal resolution, spectral resolution provides abundant valuable resource for human earth observation.Yet for the remote sensing images of optical system, be conflicting between its spatial resolution and the spectral resolution, under the situation of certain signal to noise ratio (S/N ratio), the raising of spectral resolution is cost to sacrifice spatial resolution.Full-colour image spatial resolution height has abundant spatial detail information, the minutia of expression atural object that can be detailed, but spectral information is less; The multispectral image spectral information is abundant, be conducive to atural object identification, but imaging resolution is low.Increasing remote sensing application need be got up the advantages of the two, generates the multispectral image with higher spatial resolution, to satisfy the needs to the more profound application of image.The effective way that remote sensing image fusion addresses this problem just.

Present blending algorithm can roughly be divided into two classes: the blending algorithm of replacing based on the color space component and based on the blending algorithm of ARSIS model.The fusion of replacing based on the color space component generally is to merge in the pixel grey scale space of image, as IHS(Intensity-Hue-Saturation), PCA(Principal Component Analysis) etc., such algorithm has improved the spatial resolution of fused images effectively and has but introduced serious spectrum distortion simultaneously.Merging based on the ARSIS model is by inferring that the radio-frequency component that the MS image lacks improves its spatial resolution, as HPF(High-pass Filtering), WTF(Wavelet Transform Fusion) etc., this type of algorithm has solved component and has replaced the serious problem of blending algorithm spectrum distortion, but occurs excessively phenomenon such as injection or counteracting of details in the multispectral image after merging easily.

Summary of the invention

The objective of the invention is to overcome above-mentioned prior art shortcoming, propose a kind of remote sensing image fusing method based on rarefaction representation, to improve the spatial resolution of fused image, reduce spectrum distortion and color distortion simultaneously.

For achieving the above object, the present invention replaces at general component merges the remote sensing image fusion that realizes under the framework based on rarefaction representation, and its technical scheme is the linear regression model (LRM) of at first setting up between multispectral image and its luminance component; Secondly utilize the high-resolution and low-resolution dictionary of training respectively full-colour image and multispectral image to be carried out rarefaction representation, and obtain multispectral image luminance component rarefaction representation coefficient according to linear regression model (LRM); Rarefaction representation coefficient according to full-colour image and luminance component extracts the details composition then, and is injected in the rarefaction representation coefficient of each wave band of multispectral image under general component replacement (GCOS) fusion framework; Carry out the multispectral image that image restoration obtains high spatial resolution at last.

The present invention includes following steps:

1, utilize imaging device to obtain low resolution multispectral image and high resolving power full-colour image respectively

2, make up multispectral image and luminance component linear regression model (LRM)

Utilize least square method to determine each band image of multispectral image and the linear relationship between the multispectral image luminance component.

3, the high low resolution dictionary of training is right

Select the abundant high resolving power natural image of one group of detailed information, and the analog image degenerative process obtains corresponding low-resolution image.Utilization based on the characteristics of image dictionary to (J.Yang, J.Wright, T.Huang et al..Image Super-Resolution via Sparse Representation ［ J ］ .IEEE Trans.Image Processing, 2010,19:2861～2873) method is carried out dictionary study to high-definition picture and corresponding low-resolution image, obtains high low resolution and crosses complete dictionary.

4, utilize orthogonal matching pursuit algorithm (Y.C.Pati, R.Rezaiifar, P.S.Krishnaprasad.Orthogonal matching pursuit:recursive function approximation with applications to wavelet decomposition ［ C ］ .Proceedings of the27th Annual Asilomar Conference on Signals, Systems, and Computers, 1993,1:40～44) find the solution the sparse coefficient of each wave band image of multispectral image under the low resolution dictionary respectively, and the sparse coefficient of full-colour image under the high resolving power dictionary.

5, utilize multispectral image that step 2 obtains and the sparse coefficient of the low resolution multispectral image of luminance component linear regression model (LRM) and step 4 acquisition to find the solution the sparse coefficient of low resolution multispectral image luminance component;

6, utilizing the sparse coefficient of the high resolving power full-colour image that the sparse coefficient of the luminance component that step 5 obtains and the maximum rule of absolute value obtain step 4 to carry out part replaces;

7, the sparse coefficient of full-colour image after the part of utilizing step 6 to obtain is replaced deducts the sparse coefficient of the luminance component of step 5 acquisition, obtains the sparse coefficient of detailed information;

8, utilize the sparse coefficient of each wave band of low resolution multispectral image of step 4 acquisition to add the sparse coefficient of detailed information that step 7 obtains, obtain the sparse coefficient of high-resolution multi-spectral image;

9, reconstruct high-resolution multi-spectral image:

High resolving power is crossed complete dictionary and the sparse multiplication of high resolving power, obtain the multispectral image of column vectorization.The multispectral image of column vectorization is converted into image block, returns to the position of original image, the high-resolution multi-spectral image after obtaining merging.

The invention has the beneficial effects as follows:

1, the present invention introduces the remote sensing image fusion field with rarefaction representation, overcome the defective that can't keep higher spatial resolution and spectral information in the prior art simultaneously, make the present invention to keep spectral information and spatial detail information preferably simultaneously, be conducive to the later stage to treatment of picture and identification.

2, the present invention introduces high low resolution dictionary full-colour image and multispectral image is carried out rarefaction representation, overcome the defective that fused images in the prior art is subject to full-colour image resolution, make fused images resolution that the present invention obtains than the fusion of prior art before full-colour image resolution obviously improve.

3, the present invention introduces maximum fusion rule and the rarefaction representation result of full-colour image is carried out part replaces, and has overcome the defective of spectrum distortion in the existing integration technology, makes the present invention reduce greatly than the spectrum torsion resistance of prior art.

Description of drawings

Fig. 1 is the process flow diagram that the present invention is based on the remote sensing image fusing method of rarefaction representation.

Fig. 2 is the analogous diagram that the present invention is based on the remote sensing image fusing method of rarefaction representation.

Embodiment

Elaborate to of the present invention below in conjunction with drawings and Examples: present embodiment has provided detailed embodiment and process being the example of implementing under the prerequisite with the technical solution of the present invention, but protection scope of the present invention should not be limited to following embodiment.

1, utilize imaging device to obtain low resolution multispectral image and high resolving power full-colour image respectively;

Utilize multispectral imaging equipment and full-colour image imaging device to obtain low resolution multispectral image and high resolving power respectively

Full-colour image, and read in low resolution multispectral image and high resolving power full-colour image.

Low resolution multispectral image size in the embodiment of the invention is 128 * 128 * 4, and resolution is 16m; High resolving power full-colour image size is 512 * 512 * 4, and resolution is 4m.

2, make up multispectral image and luminance component linear regression model (LRM)

The high resolving power full-colour image is resampled downwards, obtain the spatial resolution full-colour image identical with multispectral image and be expressed as Pan ^l, find the solution formula (1) by minimum two-value method, obtain weights coefficient g _bWith biasing constant bias, wherein, MS _bThe b band image of representing original multispectral image, min are represented to minimize, || || ₂For finding the solution two norms; Then, through type (2) utilizes gained linear relationship simulation low resolution multispectral image luminance component I.

$\{g_b,\mathrm{bias}\}=\underset{g_b,\mathrm{bias}}{\min }{\left|\right|{\mathrm{Pan}}^l-\underset{b=1}{\overset{B}{\mathrm{\Σ}}}{g}_b{\mathrm{MS}}_b+\mathrm{bias}\left|\right|}_2---\left(1\right)$

$I=\underset{b=1}{\overset{B}{\mathrm{\Σ}}}{g}_b{\mathrm{MS}}_b+\mathrm{bias}---\left(2\right)$

The weights coefficient magnitude of red, green, blue, each spectral coverage of near infrared is respectively 0.067,0.189,0.228,0.319 in the embodiment of the invention, and the biasing constant is 0.098.

3, the high low resolution dictionary of training is right

At first select the abundant high resolving power natural image of one group of detailed information, and the analog image degenerative process obtains corresponding low-resolution image.Simultaneously, in order to reduce the inconvenience that high low resolution sample dimension difference causes in the dictionary training process, low-resolution image made progress to resample makes it identical with the high-definition picture size.Mode random extraction size with piece on last single order at low-resolution image, the second derivative image is the low resolution sample block of 8 * 8 sizes, vectorization side by side, corresponding position after its average of high resolution graphics image subtraction is extracted size and is similarly the vectorization arranged side by side of 8 * 8 high resolving power sample block, in embodiments of the present invention, randomly draw 1024 pairs of high low resolution sample block altogether.

The high-resolution and low-resolution sample of supposing said extracted is expressed as X respectively _hAnd X _l, Dui Ying high-resolution and low-resolution dictionary D then _hAnd D _lCan obtain by following target equation:

$\{D_h,D_l,\mathrm{\α}\}=\arg \underset{D_h,D_l,\mathrm{\α}}{\min }\{{\left|\right|X_h-D_h\mathrm{\α}\left|\right|}_2^2+{\left|\right|X_l-D_l\mathrm{\α}\left|\right|}_2^2+\mathrm{\λ}{\left|\right|\mathrm{\α}\left|\right|}_1\}---\left(3\right)$

The final goal function can be expressed as follows:

$\{D,\mathrm{\α}\}=\arg \underset{D,\mathrm{\α}}{\min }\{{\left|\right|X-\mathrm{D\α}\left|\right|}_2^2+\mathrm{\λ}{\left|\right|\mathrm{\α}\left|\right|}_1\}---\left(4\right)$

4, wherein, X=[X _h; X _l], D=[D _h; D _l].High low resolution dictionary D in the embodiment of the invention _hAnd D _lSize be 64 * 1024.Min{} represents to minimize,

For finding the solution two norms of Y-D α, α is sparse coefficient, and λ is the weight factor of balance fidelity and degree of rarefication.Utilize the orthogonal matching pursuit algorithm to find the solution the sparse coefficient of each wave band image of multispectral image under the low resolution dictionary respectively, and the sparse coefficient of full-colour image under the high resolving power dictionary.

The concrete steps of described orthogonal matching pursuit algorithm are as follows:

The low resolution multispectral image that step 1 is obtained is resampled to the identical size of high resolving power full-colour image that obtains with step 1, is expressed as MS ^lWith size be 8 * 8 window from left to right, travel through each wave band and the full-colour image of multispectral image from top to bottom, and each image block is converted to length is 64 column vector, be expressed as

Wherein N is the number of image block in the single image,

Expression multispectral image MS ^lB wave band.

For

Utilize dictionary D respectively _l, D _hReach the OMP algorithm and carry out finding the solution of rarefaction representation coefficient, can obtain corresponding multispectral each wave band and full-colour image rarefaction representation coefficient

α _Pan:

${\mathrm{\α}}_{{\mathrm{MS}}_b^l}=\arg \underset{{\mathrm{\α}}_{{\mathrm{MS}}_b^l}}{\min }{\left|\right|{\mathrm{\α}}_{{\mathrm{MS}}_b^l}\left|\right|}_{0}s.t.{\left|\right|x_i^{{\mathrm{MS}}_b^l}-D_l{\mathrm{\α}}_{{\mathrm{MS}}_b^l}\left|\right|}_2^2\≤\mathrm{\ϵ}---\left(5\right)$

${\mathrm{\α}}_{\mathrm{Pan}}=\arg \underset{{\mathrm{\α}}_{\mathrm{Pan}}}{\min }{\left|\right|{\mathrm{\α}}_{\mathrm{Pan}}\left|\right|}_{0}s.t.{\left|\right|x_i^{\mathrm{Pan}}-D_h{\mathrm{\α}}_{\mathrm{Pan}}\left|\right|}_2^2\≤\mathrm{\ϵ}---\left(6\right)$

5, utilize multispectral image and luminance component linear regression model (LRM) to obtain the sparse coefficient of multispectral image luminance component.

Because atom has corresponding relation in rarefaction representation coefficient and the dictionary, the size of rarefaction representation coefficient value has been reacted the degree of corresponding atom conspicuousness, therefore, according to formula (7), can obtain the rarefaction representation coefficient of low resolution luminance picture

${\mathrm{\α}}_{I_0}=\underset{b=1}{\overset{B}{\mathrm{\Σ}}}{g}_b{\mathrm{\α}}_{{\mathrm{MS}}_b^l}+\mathrm{bias}---\left(7\right)$

Wherein, g _bBe respectively weights coefficient and the biasing constant of in step 2, trying to achieve with bias, Be the sparse coefficient of multispectral each wave band that obtains in the step 4.

6, utilize the sparse coefficient of luminance component and the maximum rule of absolute value that the sparse coefficient of full-colour image is carried out the part replacement.

In order to reduce spectrum distortion in the details composition that extracts full-colour image, the present invention has at first carried out the replacement of part composition to the rarefaction representation coefficient of full-colour image.The present invention adopts the maximum fusion rule of absolute value that the rarefaction representation coefficient of full-colour image is carried out the replacement of part composition, obtains the rarefaction representation coefficient of high resolving power I component:

${\mathrm{\α}}_I\left(i\right)=\left\{\begin{array}{cc}{\mathrm{\α}}_{\mathrm{Pan}}\left(i\right)& \left|{\mathrm{\α}}_{\mathrm{Pan}}\left(i\right)\right|>\left|{\mathrm{\α}}_{I_0}\left(i\right)\right|\\ {\mathrm{\α}}_{I_0}\left(i\right)& \mathrm{other}\end{array}\right.---\left(8\right)$

Wherein, α _IThe rarefaction representation coefficient of expression high resolving power I component correspondence, i represents i element in the rarefaction representation coefficient.

7, the sparse coefficient of full-colour image after utilization part is replaced deducts the sparse coefficient of luminance component, obtains the sparse coefficient of detailed information, namely

8, the sparse coefficient of each wave band of multispectral image adds the sparse coefficient of detailed information, obtains the sparse coefficient of each wave band of high-resolution multi-spectral image.The rarefaction representation coefficient of high resolving power MS image correspondence then

${\mathrm{\α}}_{{\mathrm{MS}}_b}^h={\mathrm{\α}}_{{\mathrm{MS}}_b^l}+({\mathrm{\α}}_I-{\mathrm{\α}}_{I_0})---\left(9\right)$

9, reconstruct high-resolution multi-spectral image

High resolving power is crossed complete dictionary and the sparse multiplication of high resolving power, obtain the multispectral image of column vectorization, the multispectral image of column vectorization is converted into 8 * 8 image block, return to the position of original image, lap is got average, obtains high-resolution multispectral image.

High-resolution multi-spectral image size after merging in the example of the present invention is 512 * 512 * 4, and resolution is 4m.

Effect of the present invention can be described further by following emulation experiment.

The experiment simulation environment is Matlab7.10.0(R2010a) Service Pack3, AMD Athlon (tm) 64X2Dual Core Processor5000+2.60GHz, Windows7, Fig. 2 (a) and Fig. 2 (b) are one group and treat fused images, Fig. 2 (c) is a width of cloth reference picture, this three width of cloth image is the IKONOS satellite image, Fig. 2 (d)～(g) is respectively the fusion results of traditional algorithm GIHS, GSA, WTF, AWLP, and Fig. 2 (h) treats that to Fig. 2 (a) and Fig. 2 (b) fused images merges the high-resolution multi-spectral image that the back obtains for adopting the present invention.

As can be seen from the figure, the fusion results of algorithms of different is all comparatively approaching with reference picture aspect whole colourity and saturation degree, GIHS, GSA method overall brightness are dark partially, WTF, AWLP method have kept spectral information preferably, but detailed information is fuzzy, and fusion results of the present invention is visually clear bright, rich color, it is also fine that details keeps, and all is better than the fusion results of traditional algorithm on spatial resolution, spectral information and visual effect.

In order to prove effect of the present invention, calculate the objective evaluation index of GIHS, GSA, WTF, AWLP and fusion results of the present invention respectively, obtain related coefficient (CC), root-mean-square error (RMSE), spectrum radian (SAM), global dimension composition error (ERGAS) and general image quality evaluation index (UIQI) be as following table relatively:

As seen from the above table, related coefficient of the present invention and general image quality evaluation index are all greater than the evaluation of estimate of prior art, related coefficient is more near 1, and expression fused images and reference picture are more approaching, and fusion results is more good, the general image quality evaluation index is used for the reserving degree of objective evaluation spectral information, range of results is in [0,1], and the picture quality index is the bigger the better, more similar to reference picture, the more big expression syncretizing effect of the value of related coefficient and UIQI is more good; Root-mean-square error RMSE, spectrum radian SAM and relative dimension composition error ERGAS are all less than the evaluation of estimate of prior art, root-mean-square error is represented the size of fused images and reference picture error, root-mean-square error is more little, the effect of image co-registration is more good, the spectrum radian is represented the tortuous degree of spectrum, more near 0, syncretizing effect is more good, the more little fused images of global dimension composition error and reference picture are more approaching relatively, the effect that merges is just more good, ideal situation is 0, root-mean-square error RMSE, spectrum radian SAM, the more little expression syncretizing effect of global dimension composition error ERGAS is more good relatively.This shows that evaluation result of the present invention all is better than the evaluation result of prior art, the present invention has objective evaluation effect preferably than prior art.

Claims (6)

1. the remote sensing image fusion method based on rarefaction representation is characterized in that, comprises the following steps:

1. utilize imaging device to obtain low resolution multispectral image and high resolving power full-colour image respectively;

2. make up multispectral image and luminance component linear regression model (LRM):

Utilize least square method to determine each band image of multispectral image and the linear relationship between the multispectral image luminance component.

3. train high low resolution dictionary right

Select the abundant high resolving power natural image of one group of detailed information, and the analog image degenerative process obtains corresponding low-resolution image, utilization is carried out dictionary study based on the right method of characteristics of image dictionary to high-definition picture and corresponding low-resolution image, obtains high low resolution and crosses complete dictionary;

4. utilize the orthogonal matching pursuit algorithm sparse coefficient of each wave band image of low resolution multispectral image under the low resolution dictionary that 1. obtain of solution procedure respectively, and the sparse coefficient of high resolving power full-colour image under the high resolving power dictionary;

5. utilize the sparse coefficient of the low resolution multispectral image that 4. multispectral image that 2. step obtain and luminance component linear regression model (LRM) and step obtain to find the solution the sparse coefficient of low resolution multispectral image luminance component;

6. utilizing the sparse coefficient of the high resolving power full-colour image that 4. the sparse coefficient of the luminance component that 5. step obtain and the maximum rule of absolute value obtain step to carry out part replaces;

7. the sparse coefficient of full-colour image after utilizing part that 6. step obtain to replace deducts the sparse coefficient of the luminance component that 5. step obtain, and obtains the sparse coefficient of detailed information;

8. utilize the sparse coefficient of each wave band of low resolution multispectral image that 4. step obtain to add the sparse coefficient of detailed information that 7. step obtains, obtain the sparse coefficient of high-resolution multi-spectral image;

9. reconstruct high-resolution multi-spectral image:

High resolving power is crossed the sparse multiplication of the high-resolution multi-spectral image that 8. complete dictionary and step obtain, obtain the multispectral image of column vectorization.

2. the remote sensing image fusion method based on rarefaction representation according to claim 1 is characterized in that: the step 2. concrete steps of described structure multispectral image and luminance component linear regression model (LRM) is as follows:

The high resolving power full-colour image that 1. step is obtained resamples downwards, obtains the spatial resolution full-colour image identical with the multispectral image of low resolution, finds the solution following formula by minimum two-value method, obtains weights coefficient g _bWith biasing constant bias:

Wherein, Pan ^lBe the spatial resolution full-colour image identical with the multispectral image of low resolution, MS _bThe expression step is the b band image of the middle low resolution multispectral image that obtains 1..

3. the remote sensing image fusion method based on rarefaction representation according to claim 1 is characterized in that: 3. the right concrete steps of the high low resolution dictionary of described training are as follows for step:

Select the abundant high resolving power natural image of one group of detailed information, and the analog image degenerative process obtains corresponding low resolution natural image; Simultaneously, the low resolution natural image is made progress to resample make it identical with high resolving power natural image size; Mode random extraction size with piece on last single order at low-resolution image, the second derivative image is the low resolution sample block of N * N size, vectorization side by side, corresponding position after its average of high resolution graphics image subtraction is extracted size and is similarly N * N high resolving power sample block vectorization arranged side by side, and the high-resolution and low-resolution sample of extraction is expressed as X respectively _hAnd X _l, Dui Ying high-resolution and low-resolution dictionary D then _hAnd D _lCan obtain by following target equation:

Wherein, X=[X _h; X _l], D=[D _h; D _l], min{} represents to minimize,

For finding the solution two norms of Y-D α, α is sparse coefficient, and λ is the weight factor of balance fidelity and degree of rarefication.

4. the remote sensing image fusion method based on rarefaction representation according to claim 1, it is characterized in that: the step 4. described orthogonal matching pursuit algorithm that utilizes is found the solution the sparse coefficient of each wave band image of multispectral image under the low resolution dictionary respectively, and the concrete steps of the sparse coefficient of full-colour image under the high resolving power dictionary are as follows:

Multispectral image is resampled to identical size with full-colour image, is expressed as MS ^l, with size be

Window from left to right, travel through each wave band and the full-colour image of multispectral image from top to bottom, and each image block be converted to the column vector that length is n, be expressed as

Wherein N is the number of image block in the single image,

Expression multispectral image MS ^lB wave band;

For

Utilize dictionary D respectively _l, D _hReach the OMP algorithm and carry out finding the solution of rarefaction representation coefficient, can obtain corresponding multispectral each wave band and full-colour image rarefaction representation coefficient α _Pan, calculating formula is as follows:

。

5. the remote sensing image fusion method based on rarefaction representation according to claim 1 is characterized in that: the step 5. described concrete steps of the sparse coefficient that multispectral image and luminance component linear regression model (LRM) obtain the multispectral image luminance component of utilizing is as follows:

Obtain the rarefaction representation coefficient of low resolution multispectral image luminance component

Formula is as follows:

Wherein, g _bBe respectively weights coefficient and the biasing constant of trying to achieve in 2. in step with bias,

4. the sparse coefficient of multispectral each wave band that obtains in for step.

6. the remote sensing image fusion method based on rarefaction representation according to claim 1 is characterized in that: the step 6. described concrete steps of the sparse coefficient that multispectral image and luminance component linear regression model (LRM) obtain the multispectral image luminance component of utilizing is as follows:

Adopt the maximum fusion rule of absolute value that the rarefaction representation coefficient of full-colour image is carried out the replacement of part composition, obtain the rarefaction representation coefficient of high resolving power I component, formula is as follows:

Wherein, α _IThe rarefaction representation coefficient of expression high resolving power I component correspondence, i represents i element in the rarefaction representation coefficient,

The rarefaction representation coefficient of low resolution multispectral image luminance component.

Images