CN103208102B - A kind of remote sensing image fusion method based on rarefaction representation - Google Patents

Abstract

Based on a remote sensing image fusion method for rarefaction representation, the linear regression model (LRM) between model multispectral image and its luminance component; Secondly utilize the high-resolution and low-resolution dictionary of training respectively full-colour image and multispectral image to be carried out to rarefaction representation, and obtain multispectral image luminance component rarefaction representation coefficient according to linear regression model (LRM); Then extract details composition according to the rarefaction representation coefficient of full-colour image and luminance component, and replace under fusion framework and be injected in the rarefaction representation coefficient of the each wave band of multispectral image at general component; Finally carry out image restoration and obtain the multispectral image of high spatial resolution. Rarefaction representation technology is incorporated into remote sensing image fusion field by the present invention, overcome the defect that prior art cannot keep higher spatial resolution and spectral information simultaneously, fusion results of the present invention spectrum keep and spatial resolution raising aspect be better than traditional remote sensing image fusion method.

Description

A kind of remote sensing image fusion method based on rarefaction representation

Technical field

The present invention relates to image processing techniques, specifically a kind of remote sensing image fusing method based on rarefaction representation.

Background technology

Along with the development of remote sensing technology, the different spatial resolutions that obtained by various satellite sensors, temporal resolution,The remote sensing image data of spectral resolution enriches valuable resource for mankind's earth observation provides. But for optical systemThe remote sensing images of system, are conflicting between its spatial resolution and spectral resolution, the in the situation that of certain signal to noise ratio,The raising of spectral resolution is to sacrifice spatial resolution as cost. Full-colour image spatial resolution is high, has abundantSpatial detail information, the minutia of expression atural object that can be detailed, but spectral information is less; Multispectral image spectrum letterBreath is abundant, be conducive to atural object identification, but imaging resolution is low. Increasing remote sensing application need to be by the advantage knot of the twoAltogether, generate the multispectral image with higher spatial resolution, to meet the needs to the more profound application of image.The effective way that remote sensing image fusion addresses this problem just.

Current blending algorithm can roughly be divided into two classes: based on color space component replace blending algorithm and based on ARSISThe blending algorithm of model. The fusion of replacing based on color space component is generally to melt on the pixel grey scale space of imageClose, as IHS(Intensity-Hue-Saturation), PCA(PrincipalComponentAnalysis) etc., suchAlgorithm has effectively improved the spatial resolution of fused images and has but introduced serious spectrum distortion simultaneously. Based on ARSISIt is by inferring that the radio-frequency component that MS image lacks improves its spatial resolution, as HPF(High-pass that model mergesFiltering), WTF(WaveletTransformFusion) etc., this type of algorithm has solved component and has replaced blending algorithm lightThe problem that spectrum distortion is serious, but in multispectral image after merging, easily occur that details is excessively injected or the phenomenon such as counteracting.

Summary of the invention

The object of the invention is to overcome above-mentioned prior art shortcoming, propose a kind of Remote Sensing Image Fusion based on rarefaction representationMethod to improve the spatial resolution of fused image, reduces spectrum distortion and color distortion simultaneously.

For achieving the above object, the present invention replaces and merges the remote sensing images of realization based on rarefaction representation under framework at general componentMerge, its technical scheme is the linear regression model (LRM) between model multispectral image and its luminance component; Secondly utilizeThe high-resolution and low-resolution dictionary of training carries out rarefaction representation to full-colour image and multispectral image respectively, and according to linear regressionModel obtains multispectral image luminance component rarefaction representation coefficient; Then according to the rarefaction representation of full-colour image and luminance componentCoefficient extracts details composition, and replaces (GCOS) at general component and merge and under framework, be injected into the each wave band of multispectral imageIn rarefaction representation coefficient; Finally carry out image restoration and obtain the multispectral image of high spatial resolution.

The present invention includes following steps:

1, utilize imaging device to obtain respectively low resolution multispectral image and High-resolution Panchromatic Images

2, build multispectral image and luminance component linear regression model (LRM)

Utilize least square method to determine the line between each band image and the multispectral image luminance component of multispectral imageSexual intercourse.

3, train high low resolution dictionary pair

Select one group of high-resolution natural image that detailed information is abundant, and analog image degenerative process obtains corresponding lowImage in different resolution. Utilize based on characteristics of image dictionary (J.Yang, J.Wright, T.Huangetal..ImageSuper-ResolutionviaSparseRepresentation［J］．IEEETrans．ImageProcessing，2010,19:2861～2873) method high-definition picture and corresponding low-resolution image are carried out to dictionary learning,Obtain high low resolution and cross complete dictionary.

4, utilize orthogonal matching pursuit algorithm (Y.C.Pati, R.Rezaiifar, P.S.Krishnaprasad.Orthogonalmatchingpursuit:recursivefunctionapproximationwithapplicationstowaveletdecomposition［C］．Proceedingsofthe27thAnnualAsilomarConferenceonSignals, Systems, andComputers, 1993,1:40～44) solve respectively the each wave band of multispectral imageThe sparse coefficient of image under low resolution dictionary, and the sparse system of full-colour image under high-resolution dictionaryNumber.

5, utilize multispectral image that step 2 obtains and luminance component linear regression model (LRM) and step 4 to obtain low pointDistinguish that the sparse coefficient of rate multispectral image solves the sparse coefficient of low resolution multispectral image luminance component;

6, utilize the sparse coefficient of the luminance component that step 5 obtains and the height that the maximum rule of absolute value obtains step 4The sparse coefficient of resolution ratio full-colour image carries out part replacement;

7, the sparse coefficient of full-colour image after utilizing part that step 6 obtains to replace deducts the luminance component that step 5 obtainsSparse coefficient, obtain the sparse coefficient of detailed information;

8, utilize the sparse coefficient of the each wave band of low resolution multispectral image that step 4 obtains to add that step 7 obtains thinThe sparse coefficient of joint information, obtains the sparse coefficient of high-resolution multi-spectral image;

9, reconstruct high-resolution multi-spectral image:

High-resolution is crossed to complete dictionary and the sparse multiplication of high-resolution, obtain the multispectral image of column vector.The multispectral image of column vector is converted into image block, returns to the position of original image, obtain the high score after mergingDistinguish rate multispectral image.

The invention has the beneficial effects as follows:

1, rarefaction representation is introduced remote sensing image fusion field by the present invention, overcome in prior art and cannot keep higher simultaneouslyThe defect of spatial resolution and spectral information, makes the present invention can keep preferably spectral information and spatial detail letter simultaneouslyBreath, is conducive to processing and the identification of later stage to image.

2, the present invention introduces high low resolution dictionary full-colour image and multispectral image is carried out to rarefaction representation, has overcome existingIn technology, fused images is limited to the defect of full-colour image resolution ratio, and the fused images resolution ratio ratio that the present invention is obtained is existingThere is the front full-colour image resolution ratio of fusion of technology obviously to improve.

3, the present invention introduces maximum fusion rule the rarefaction representation result of full-colour image is carried out to part replacement, has overcome existingThe defect of spectrum distortion in integration technology, makes the present invention greatly reduce than the spectrum torsion resistance of prior art.

Brief description of the drawings

Fig. 1 is the flow chart that the present invention is based on the remote sensing image fusing method of rarefaction representation.

Fig. 2 is the analogous diagram that the present invention is based on the remote sensing image fusing method of rarefaction representation.

Detailed description of the invention

Elaborate to of the present invention below in conjunction with drawings and Examples: the present embodiment is taking technical solution of the present invention as frontPut the example of implementing, provided detailed embodiment and process, but under protection scope of the present invention should not be limited toThe embodiment stating.

1, utilize imaging device to obtain respectively low resolution multispectral image and High-resolution Panchromatic Images;

Utilize multispectral imaging equipment and full-colour image imaging device to obtain respectively low resolution multispectral image and high-resolutionFull-colour image, and read in low resolution multispectral image and High-resolution Panchromatic Images.

Low resolution multispectral image size in the embodiment of the present invention is 128 × 128 × 4, and resolution ratio is 16m; High-resolutionRate full-colour image size is 512 × 512 × 4, and resolution ratio is 4m.

2, build multispectral image and luminance component linear regression model (LRM)

High-resolution Panchromatic Images is carried out to downward resampling, obtain the full-colour picture that spatial resolution is identical with multispectral imagePicture is expressed as Pan^l, solve formula (1) by minimum two-value method, obtain weights coefficient g_bWith biasing constant bias, wherein,MS_bRepresent the b band image of original multispectral image, min represents to minimize, || ||₂For solving two norms; Then,Through type (2) utilizes gained linear relationship simulation low resolution multispectral image luminance component I.

$\{g_b,\mathrm{bias}\}=\underset{g_b,\mathrm{bias}}{\min }{\left|\right|{\mathrm{pan}}^l-\underset{b=1}{\overset{B}{\mathrm{\Σ}}}{g}_b{\mathrm{MS}}_b+\mathrm{bias}\left|\right|}_2---\left(1\right)$

$I=\underset{b=1}{\overset{B}{\mathrm{\Σ}}}{g}_b{\mathrm{MS}}_b+\mathrm{bias}---\left(2\right)$

In the embodiment of the present invention, the weights coefficient magnitude of red, green, blue, each spectral coverage of near-infrared is respectively 0.067,0.189,0.228,0.319, biasing constant is 0.098.

3, train high low resolution dictionary pair

First select one group of high-resolution natural image that detailed information is abundant, and analog image degenerative process obtains corresponding lowImage in different resolution. Meanwhile, the inconvenience causing in dictionary training process in order to reduce high low resolution sample dimension difference,Low-resolution image is carried out to upwards resampling makes it identical with high-definition picture size. Finally at low-resolution imageOn single order, second dervative image, extract at random the low resolution sample block of size as 8 × 8 sizes in the mode of piece, side by side toQuantize, the corresponding position after its average of high resolution graphics image subtraction is extracted size and is similarly 8 × 8 high-resolution sample blockVectorization side by side, in embodiments of the present invention, randomly draws 1024 pairs of high low resolution sample block altogether.

The high-resolution and low-resolution sample of supposing said extracted is expressed as X_hAnd X_l, corresponding high-resolution and low-resolution wordAllusion quotation D_hAnd D_lCan obtain by following target equation:

$\left\{\begin{array}{c}{D}_h,D_l,\mathrm{\α}\end{array}\right\}=\arg \underset{D_h,D_l,\mathrm{\α}}{\min }\left\{\begin{array}{c}{\left|\right|X_h-D_h\mathrm{\α}\left|\right|}_2^2\end{array}+{\left|\right|X_l{-D}_l\mathrm{\α}\left|\right|}_2^2+\mathrm{\λ}{\left|\right|\mathrm{\α}\left|\right|}_1\right\}---\left(3\right)$

Final goal function can be expressed as follows:

$\{D,\mathrm{\α}\}=\arg \underset{D,\mathrm{\α}}{\min }\left\{\begin{array}{c}{\left|\right|X-\mathrm{D\α}\left|\right|}_2^2+\mathrm{\λ}{\left|\right|\mathrm{\α}\left|\right|}_1\end{array}\right\}---\left(4\right)$

4, wherein, X=[X_h;X_l]，D=[D_h;D_l]. High low resolution dictionary D in the embodiment of the present invention_hWithD_lSize be 64 × 1024. Min{} represents to minimize,For solving two norms of Y-D α, α is sparseCoefficient, λ is the weight factor of balance fidelity and degree of rarefication. Utilize orthogonal matching pursuit algorithm to solve respectively multispectral figureThe sparse coefficient of the each wave band image of picture under low resolution dictionary, and the sparse system of full-colour image under high-resolution dictionaryNumber.

The concrete steps of described orthogonal matching pursuit algorithm are as follows:

It is complete that the low resolution multispectral image that step 1 is obtained is resampled to the high-resolution obtaining with step 1Color image formed objects, is expressed as MS^l. The window that is 8 × 8 by size from left to right, travels through multispectral figure from top to bottomEach wave band and the full-colour image of picture, and each image block is converted to length is 64 column vector, is expressed as Wherein N is the number of image block in single image,Represent multispectral image MS^lB wave band.

For Utilize respectively dictionary D_l、D_hAnd OMP algorithm carries out solving of rarefaction representation coefficient,Can obtain corresponding multispectral each wave band and full-colour image rarefaction representation coefficientα_Pan：

${\mathrm{\α}}_{{\mathrm{MS}}_b^l}=\arg \underset{{\mathrm{\α}}_{{\mathrm{Ms}}_b^l}}{\min }{\left|\right|{\mathrm{\α}}_{{\mathrm{MS}}_b^l}\left|\right|}_{0}s\·t{\left|\right|x_i^{{\mathrm{MS}}_b^l}-D_l{\mathrm{\α}}_{{\mathrm{MS}}_b^l}\left|\right|}_2^2\≤\mathrm{\ϵ}---\left(5\right)$

${\mathrm{\α}}_{\mathrm{Pan}}=\arg \underset{{\mathrm{\α}}_{\mathrm{Pan}}}{\min }{\left|\right|{\mathrm{\α}}_{\mathrm{Pan}}\left|\right|}_{0}s.t.{\left|\right|x_i^{\mathrm{Pan}}-D_h{\mathrm{\α}}_{\mathrm{Pan}}\left|\right|}_2^2\≤\mathrm{\ϵ}---\left(6\right)$

5, utilize multispectral image and luminance component linear regression model (LRM) to obtain the sparse system of multispectral image luminance componentNumber.

Because rarefaction representation coefficient and dictionary Atom have corresponding relation, the size of rarefaction representation coefficient value has been reacted correspondenceThe degree of atom conspicuousness, therefore, according to formula (7), can obtain the rarefaction representation coefficient of low resolution luminance picture：

${\mathrm{\α}}_{I_0}=\underset{b=1}{\overset{B}{\mathrm{\Σ}}}{g}_b{\mathrm{\α}}_{{\mathrm{MS}}_b^l}+\mathrm{bias}---\left(7\right)$

Wherein, g_bBe respectively weights coefficient and the biasing constant of in step 2, trying to achieve with bias,In step 4The sparse coefficient of multispectral each wave band obtaining.

6, utilizing the maximum rule of the sparse coefficient of luminance component and absolute value to carry out part to the sparse coefficient of full-colour image replacesChange.

In order to reduce spectrum distortion in extracting the details composition of full-colour image, first rare to full-colour image of the present inventionDredge and represent that coefficient has carried out part composition and replaced. The present invention adopts the sparse table of the maximum fusion rule of absolute value to full-colour imageShow that coefficient carries out the replacement of part composition, obtain the rarefaction representation coefficient of high-resolution I component:

${\mathrm{\α}}_I\left(i\right)=\left\{\begin{array}{cc}{\mathrm{\α}}_{\mathrm{Pan}}\left(i\right)& \left|{\mathrm{\α}}_{\mathrm{Pan}}\left(i\right)\right|>\left|{\mathrm{\α}}_{I_0}\left(i\right)\right|\\ {\mathrm{\α}}_{I_0}\left(i\right)& \mathrm{other}\end{array}\right.---\left(8\right)$

Wherein, α_IRepresent rarefaction representation coefficient corresponding to high-resolution I component, i represents the i in rarefaction representation coefficientIndividual element.

7, the sparse coefficient of full-colour image after utilization part is replaced deducts the sparse coefficient of luminance component, obtains detailed informationSparse coefficient,

8, the sparse coefficient of the each wave band of multispectral image adds the sparse coefficient of detailed information, obtains high-resolution multi-spectral figureThe sparse coefficient of the each wave band of picture. The rarefaction representation coefficient that high-resolution MS image is corresponding

${\mathrm{\α}}_{{\mathrm{MS}}_b}^h={\mathrm{\α}}_{{\mathrm{MS}}_b^l}+({\mathrm{\α}}_I-{\mathrm{\α}}_{I_0})---\left(9\right)$

9, reconstruct high-resolution multi-spectral image

High-resolution is crossed to complete dictionary and the sparse multiplication of high-resolution, obtain the multispectral image of column vector, willThe multispectral image of column vector is converted into 8 × 8 image block, returns to the position of original image, and lap is got average,Obtain high-resolution multispectral image.

High-resolution multi-spectral image size after merging in example of the present invention is 512 × 512 × 4, and resolution ratio is 4m.

Effect of the present invention can be described further by following emulation experiment.

Experiment simulation environment is Matlab7.10.0(R2010a) ServicePack3, AMDAthlon (tm) 64X2DualCoreProcessor5000+2.60GHz, Windows7, Fig. 2 (a) and Fig. 2 (b) be one group and treat fused images,Fig. 2 (c) is a width reference picture, and this three width image is IKONOS satellite image, Fig. 2 (d)～(g) respectivelyFor the fusion results of traditional algorithm GIHS, GSA, WTF, AWLP, Fig. 2 (h) is for adopting the present invention to Fig. 2 (a)And the high-resolution multi-spectral image that obtains after fused images merges of Fig. 2 (b).

As can be seen from the figure, the fusion results of algorithms of different aspect overall colourity and saturation degree with reference picture all comparativelyApproach, GIHS, GSA method overall brightness are partially dark, and WTF, AWLP method have kept spectral information preferably, butDetailed information is fuzzy, and fusion results of the present invention is visually clear bright, rich color, and it is also fine that details keeps,In spatial resolution, spectral information and visual effect, be all better than the fusion results of traditional algorithm.

In order to prove effect of the present invention, calculate respectively GIHS, GSA, WTF, AWLP and fusion of the present inventionThe objective evaluation index of result, obtain coefficient correlation (CC), root-mean-square error (RMSE), spectrum radian (SAM),Relatively global dimension composition error (ERGAS) and general image quality evaluation index (UIQI) are as following table:

As seen from the above table, coefficient correlation of the present invention and general image quality evaluation index are all greater than the evaluation of estimate of prior art,Coefficient correlation more approaches 1, represents that fused images and reference picture are more approaching, and fusion results is better, and general image quality is commentedValency index is used for the reserving degree of objective evaluation spectral information, and range of results is in [0,1], and picture quality index is the bigger the better,More similar to reference picture, the larger expression syncretizing effect of value of coefficient correlation and UIQI is better; Root-mean-square error RMSE,Spectrum radian SAM and relative dimension composition error ERGAS are all less than the evaluation of estimate of prior art, root-mean-square error tableShow fused images and reference picture mistake extent, root-mean-square error is less, and the effect of image co-registration is better, spectrum radianThe tortuous degree that represents spectrum, more approaches 0, and syncretizing effect is better, relatively the less fused images of global dimension composition errorMore approaching with reference picture, the effect of fusion is just better, and ideal situation is 0, root-mean-square error RMSE, spectrum radianSAM, relatively the less expression syncretizing effect of global dimension composition error ERGAS is better. This shows of the present inventionEvaluation result is all better than the evaluation result of prior art, and the present invention has good objective evaluation effect than prior art.

Claims (2)

1. the remote sensing image fusion method based on rarefaction representation, is characterized in that, comprises the following steps:

1. utilize imaging device to obtain respectively low resolution multispectral image and High-resolution Panchromatic Images;

2. build multispectral image and luminance component linear regression model (LRM):

Utilize least square method to determine the line between each band image and the multispectral image luminance component of multispectral imageSexual intercourse;

3. train high low resolution dictionary pair

Select one group of high-resolution natural image that detailed information is abundant, and analog image degenerative process obtains corresponding lowImage in different resolution, utilizes based on the right method of characteristics of image dictionary high-definition picture and corresponding low-resolution imageCarry out dictionary learning, obtain high low resolution and cross complete dictionary;

4. utilize the orthogonal matching pursuit algorithm each wave band image of low resolution multispectral image that 1. solution procedure obtains respectivelySparse coefficient under low resolution dictionary, and the sparse coefficient of High-resolution Panchromatic Images under high-resolution dictionary;

5. utilize low that 4. multispectral image that 2. step obtain and luminance component linear regression model (LRM) and step obtainThe sparse coefficient of resolution multi-spectral image solves the sparse coefficient of low resolution multispectral image luminance component;

6. utilize the sparse coefficient of the luminance component that 5. step obtain and the maximum rule of absolute value 4. to obtain stepThe sparse coefficient of High-resolution Panchromatic Images carries out part replacement;

7. the sparse coefficient of full-colour image after utilizing part that 6. step obtain to replace deducts the brightness that 5. step obtain and dividesThe sparse coefficient of amount, the sparse coefficient of acquisition detailed information;

8. utilize the sparse coefficient of the each wave band of low resolution multispectral image that 4. step obtain to add what 7. step obtainedThe sparse coefficient of detailed information, obtains the sparse coefficient of high-resolution multi-spectral image;

9. reconstruct high-resolution multi-spectral image:

High-resolution is crossed to the sparse multiplication of the high-resolution multi-spectral image that 8. complete dictionary and step obtain,To the multispectral image of column vector.

2. the remote sensing image fusion method based on rarefaction representation according to claim 1, is characterized in that: step 3.The right concrete steps of the high low resolution dictionary of described training are as follows:

Select one group of high-resolution natural image that detailed information is abundant, and analog image degenerative process obtains corresponding lowResolution ratio natural image; Meanwhile, low resolution natural image is carried out to upwards resampling and make itself and high-resolution naturallyImage size is identical; On the last single order at low-resolution image, second dervative image, extract at random in the mode of pieceSize is the low resolution sample block of N × N size, and vectorization side by side, after its average of high resolution graphics image subtractionCorresponding position extract size and be similarly the vectorization arranged side by side of N × N high-resolution sample block, high and low point of extractionThe rate sample of distinguishing is expressed as X_hAnd X_l, corresponding high-resolution and low-resolution dictionary D_hAnd D_lCan pass through following orderMark equation obtains:

$\{D,\mathrm{\α}\}=\arg \underset{D,\mathrm{\α}}{min}\left\{\right||X-D\mathrm{\α}|{|}_2^2+\mathrm{\λ}\left|\right|\mathrm{\α}\left|{|}_1\right\}---\left(2\right)$

Wherein, X=[X_h；X_l]，D＝[D_h；D_l], min{} represents to minimize,For solving X-D αTwo norms, α is sparse coefficient, λ is the weight factor of balance fidelity and degree of rarefication.