CN106023122A - Image fusion method based on multi-channel decomposition - Google Patents

Abstract

The invention provides an image fusion method based on multi-channel decomposition. The method comprises the steps that (1) multi-channel decomposition feature extraction is carried out on a source image, namely multi-channel decomposition is carried out on the source image to acquire the structural component and the texture component of the image; (2) a fusion rule based on sparse representation is set; (3) coefficient fusion is carried out, namely the sparse representation coefficient of an image block corresponding to each image is fused according to a certain fusion rule to acquire a sparse coefficient after structural image and texture fusion; and (4) image reconstruction is carried out. The step (2) comprises the steps that block vectorization is carried out on the multi-channel-decomposed image; sparse representation is carried out on a number of vectorized blocks; an over-complete dictionary is learned; and the sparse representation coefficient of each column vector in the dictionary is calculated. The step (4) comprises the steps that since reconstruction is the inverse process of sparse decomposition, the over-complete dictionary and the fusion sparse coefficient are combined for reconstruction; and structural and texture reconstruction is carried out to acquire a final fusion image.

Description

Image interfusion method based on Multichannel Decomposition

Technical field

The invention belongs to image fusion technology field, particularly a kind of based on multichannel image interfusion method.

Background technology

Nowadays, pattern recognition becomes more and more important at daily life with the application in industrial circle.The little image of arriving Feature extraction, the big information identification to scene of a crime.Thus, it is found that a kind of efficient image interfusion method becomes increasingly to weigh Want.The process of image co-registration is the process to source images feature extraction.Texture image and structural images are the important of source images Feature and the embodiment of different aspect, therefore take into full account that the texture of image and structural information can improve the quality of fusion image. We mainly make use of TV-L1 algorithm and the K-SVD algorithm of rarefaction representation of exploded view picture.

The structure that TV-L1 model obtains under different scale is different, thus can be well by different at metric space Structural area separates.Along with parameter lambda value tapers into, little structure is smoothed successively, and smooth effect is very clean, The key of this effect multi-resolution decomposition just.Meanwhile, details in the structure remained is it is clear that this point is for image Decomposition is critically important.So using TV-L1 model to carry out resolving into texture and structure components by image.

K-SVD algorithm is to be proposed by Michal Aharon, Michael Elad of the Institute of Technology of Israel et al. for 2006 Come, be the most classical a kind of dictionary training algorithm, and reached good training effect.Its objective is to solve lower column matrix The solution of equation:

Y=DX (1)

Wherein D is intended to the dictionary of training, and X is intended to training, the sparse coefficient matrix of Y correspondence dictionary.Dimension when matrix Time the highest, even if using computer software also to be difficult to solution matrix equation, and this algorithm solves what higher dimensional matrix solved just Problem.

Summary of the invention

It is an object of the invention to provide a kind of based on multichannel image fusion method, it is possible to realize multi-source image is carried out Efficient image co-registration.

The technical solution realizing the object of the invention is: a kind of based on multichannel image interfusion method, including following Step:

Step 1), the feature extraction of source images Multichannel Decomposition: source images is carried out Multichannel Decomposition, obtain the knot of image Structure component (structure) and texture component (texture)；

Step 2), fusion rule based on rarefaction representation is arranged: image is after Multichannel Decomposition, during image co-registration Introduce sparse representation theory.According to the super complete dictionary of structure, the image to be fused of input is carried out piecemeal vectorization arranged side by side, so Rear calculating each column vector rarefaction representation coefficient under dictionary；

Step 3), coefficient merges: the rarefaction representation coefficient of image block corresponding for each image according to certain fusion Rule merges, it is thus achieved that the sparse coefficient of image to be reconstructed.

Step 4), image reconstruction: reconstruct is the inverse process of Its Sparse Decomposition, enters with merging sparse coefficient in conjunction with super complete dictionary Line reconstruction, obtains structural images and the texture image of synthesis, the structure obtained and texture image is summed into fusion image.

Described step 1) particularly as follows: utilize TV-L1 model to carry out manifold source images A and B of two width M × N to be fused Road decomposes.It is decomposed into 2 major parts: comprise the structure components (structure image) of large scale change and to comprise image little The details coefficients (texture image) of yardstick texture.First, in catabolic process, original image f is newly defined as newly by we Image u, v ∈ R^M×N, calculate its graded.

${(\▿u)}_{i,j}\left(\begin{array}{c}{\left(\▿u\right)}_{i,j}^1\\ {(\▿u)}_{i,j}^2\end{array}\right)---\left(2\right)$

Wherein

$\begin{array}{c}{(\&dtri;u)}_{i,j}^1=u_{i+1,j}-u_{i,j},i<M,{(\&dtri;u)}_{M,j}^1=0\\ {(\&dtri;u)}_{i,j}^2=u_{i,j+1}-u_{i,j},j<N,{(\&dtri;u)}_{i,N}^2=0\end{array}---\left(3\right)$

Therefore, TV-L1 model can be drawn by following formula

$\underset{u}{min}\left|\right|\&dtri;u|{|}_1+\mathrm{\&lambda;}||u-f|{|}_1---\left(4\right)$

According to above-mentioned computational methods, we will be as follows based on TV-L1 model:

min∫_D|▽u|dx+λ∫_D|u-f|dx (5)

Wherein, u is the structure components of image, and v is the texture component of image.Formula (5) is also a convex optimization problem.First half TV (the u)=∫ of part_D| u | dx can regard image sparse signal | | u | | as₂L₁Norm, the fit term of latter half, Also it is l₁Norm.Then, (5) formula can be by minimum l₁Norm method solves.l₁Norm has focused energy and substantially locates to feature, The feature of the most floating tiny characteristics, TV-L1 model is based on l₁Norm, there is l₁The advantage of norm, thus TV-L1 model Preferably performance is had than TV-L2 model at metric space.At metric space, l₁The smooth effect of norm be " as far as possible " do Only, l₂Norm is not very clean to the smooth effect of small-scale structure, remains to find out it under the yardstick of only one of which structure The blurred contour of his structure.The structure that TV-L2 model obtains under different scale is different, thus can be well at metric space Different structural areas is separated.Along with parameter lambda value tapers into, little structure is smoothed successively, and smooth effect Fruit is very clean, the key of this effect multi-resolution decomposition just.Meanwhile, details in the structure remained it is clear that this Point is critically important for picture breakdown.

Through step 1) obtain 4 width images.Structural images S of image A_AWith texture image T_AAnd the structural images of image B S_BWith texture image T_B。

Described step 2) particularly as follows: image is after Multichannel Decomposition, during image co-registration, introduce sparse representation theory. According to the super complete dictionary of structure, the image to be fused of input is carried out piecemeal vectorization arranged side by side, then calculates each column vector and exist Rarefaction representation coefficient under dictionary.The process of the method mainly has following two parts:

(1) image column vectorization.Assume that two width sizes are the image of M × N, the sliding window utilizing a size to be 4 × 4 Carry out these images respectively overlapping being sampled as the image block that size is identical, and press row expansion 16 × 1 dimensional vector.

(2) rarefaction representation.The rarefaction representation coefficient of the column vector that each image block is corresponding obtains by solving following formula.

${\hat{s}}_i^j=\arg \min \left|\right|s_i^j|{|}_0---\left(6\right)$

$\begin{array}{cc}s.t.& {\mathrm{Ds}}_i^j=P_i^j;j=1,2,...,J\end{array}---\left(7\right)$

Here consider noise jamming, utilize the image sparse improved to represent model, meanwhile, use SOMP algorithm super complete Solve sparse coefficient on standby dictionary D, thus obtain the sparse coefficient of each image block.

We are by T_A,T_B,S_A,S_BVectorization obtains T respectively_A1,T_B1,S_A1,S_B1, make T=[T_A1,T_B1], S=[S_A1,S_B1], K-SVD algorithm carries out study respectively obtain D1 T, S are brought into respectively, [XT_A1,XT_B1] and D2, [XS_A1,XS_B1]

Described step 3) particularly as follows: the rarefaction representation coefficient of image block corresponding for each image according to certain fusion Rule merges, it is thus achieved that the sparse coefficient of image to be reconstructed.The normal form the greater i.e. taking two corresponding sparse coefficient is made For last sparse coefficient.

Assuming that the sparse coefficient that two signal training are obtained is respectively x1 and x2, size is a*b, and fusion rule is:

I.e. take the greater sparse coefficient (A) as the new image merged of two sparse coefficient column vector one normal forms.

If by sparse coefficient [XTA1, XTB1] and [XS_A1,XS_B1] merge the sparse coefficient that obtains and be respectively A1, A2.

Described step 4) particularly as follows: be the inverse process of Its Sparse Decomposition according to reconstruct, sparse with fusion in conjunction with super complete dictionary Coefficient is reconstructed, and according to the principle of dictionary learning, the dictionary D1, D2 study obtained is respectively at sparse A1 and the A2 phase merged Take advantage of.That is:

Y1=D1 × A1, (8)

Y2=D2 × A2； (9)

In (8) formula, Y1 is not the structural images merged, but the vectorization of structural images represents, so using step Rapid 2) inverse process of vectorization procedure obtains structural images structure merged, and in like manner has (9) formula to obtain the texture merged Image texture.

So, the final image Image merged i.e. is represented by:

Image=structure+texture. (10)

Accompanying drawing explanation

Fig. 1 is present invention flow chart based on multichannel Image Fusion.

Fig. 2 (a) is the left focusedimage of the inventive method.

Fig. 2 (b) is the right focusedimage of the inventive method.

Fig. 2 (c) is the structural images of left focusedimage.

Fig. 2 (d) is the texture image of left focusedimage.

Fig. 2 (e) is the structural images of right focusedimage.

Fig. 2 (f) is the texture image of right focusedimage.

Fig. 2 (g) is the fusion structure image that sliding window step-length is 8 of 8 × 8.

Fig. 2 (h) is the fusion texture image that sliding window step-length is 8 of 8 × 8.

Fig. 2 (i) be the sliding window step-length of 8 × 8 be the final fusion image of 8.

Fig. 2 (j) is the fusion structure image that sliding window step-length is 4 of 8 × 8.

Fig. 2 (k) is the fusion texture image that sliding window step-length is 4 of 8 × 8.

Fig. 2 (l) be the sliding window step-length of 8 × 8 be the final fusion image of 4.

Fig. 2 (m) is the fusion structure image that sliding window step-length is 2 of 8 × 8.

Fig. 2 (n) is the fusion texture image that sliding window step-length is 2 of 8 × 8.

Fig. 2 (o) be the sliding window step-length of 8 × 8 be the final fusion image of 2.

Detailed description of the invention

Below in conjunction with the accompanying drawings the detailed description of the invention of the present invention is described in further detail.As shown in figure (1), including Following steps:

Step 1 prepares image M × N (M=480, N=640 in the present embodiment) source images A and B to be fused, utilizes TV- Picture breakdown is become structure components and texture component by L1 model.TV-L1 model is as follows:

min∫_D|▽u|dx+λ∫_D|u-f|dx

Wherein, u is the structure components of image, and v is the texture component of image.Formula (5) is also a convex optimization problem.First half TV (the u)=∫ of part_D| u | dx can regard image sparse signal | | u | | as₂L₁Norm, the fit term of latter half, Also it is l₁Norm.

Step 2 fusion rule based on rarefaction representation is arranged: image, after Multichannel Decomposition, draws during image co-registration Enter sparse representation theory.According to the super complete dictionary of structure, the image to be fused of input is carried out piecemeal vectorization arranged side by side, then Calculate each column vector rarefaction representation coefficient under dictionary；The process of the method mainly has following two parts:

(1) image column vectorization.Two width sizes are the image of 480 × 640, the sliding window utilizing a size to be 8 × 8 Carry out these images respectively overlapping being sampled as the image block that size is identical, and press row expansion 64 × 1 dimensional vector.Concrete grammar For:

Xn (::, n)=X ((i*step)+1:(i*step+8), (j*step)+1:(j*step+8))；

Wherein, Xn (::, n) represent n-component column vector, X represents original image, and i, j represent line number and columns, step respectively Representing step-length, step-length is set as 8 in this example, the different specifications such as 4,2.Constantly fine along with step-length, the fusion knot obtained Fruit is also become better and better, and the time used is more and more longer.

(2) rarefaction representation.The rarefaction representation coefficient of the column vector that each image block is corresponding obtains by solving following formula.

${\hat{s}}_i^j=\arg \min \left|\right|s_i^j|{|}_0$

$\begin{array}{cc}s.t.& {\mathrm{Ds}}_i^j=P_i^j;j=1,2,...,J\end{array}$

Here consider noise jamming, utilize the image sparse improved to represent model, meanwhile, use SOMP algorithm super complete Solve sparse coefficient on standby dictionary D, thus obtain the sparse coefficient of each image block.

Step 3) coefficient fusion:

Assuming that the sparse coefficient that two signal training are obtained is respectively x1 and x2, size is a*b, and fusion rule is:

I.e. take the greater sparse coefficient (A) as the new image merged of two sparse coefficient column vector one normal forms.

Step 4 image reconstruction: according to the principle of dictionary learning, the dictionary D1, D2 study obtained is dilute respectively at merge Dredge A1 with A2 to be multiplied.That is:

Y1=D1 × A1, (8)

Y2=D2 × A2； (9)

In (8) formula, Y1 is not the structural images merged, but the vectorization of structural images represents, so using step Rapid 2) inverse process of vectorization procedure obtains structural images structure merged, and in like manner has (9) formula to obtain the texture merged Image texture.

So, the final image Image merged i.e. is represented by:

Image=structure+texture.

Below in conjunction with example, the present invention is further detailed explanation.

Focusedimage as left in Fig. 2 (a) the inventive method, shown in the right focusedimage of Fig. 2 (b) the inventive method, first distinguishes Read right focusedimage A and left focusedimage B, it is carried out TV-L1 decomposition, obtain structural images S1 corresponding to A with B (structure image), texture image T1 (texture image) and S2, T2, i.e. Fig. 2 (c), Fig. 2 (d), Fig. 2 (e) and figure 2(f).S1, T1, S2 and T2 are carried out respectively vectorization and obtains S11, T11, S22 and T22；Again by [S11, S22] and [T11, T22] carry out K-SVD dictionary training and obtain D1, [X11, X12] and D2, [X21, X22].Then carry out step 3) coefficient merge, [X11, X12] is fused to X1, [X21, X22] is fused to X2.Finally D1 × the X1 obtained and D2 × X2 is carried out contravariant vector Change to such as Fig. 2 (g), Fig. 2 (h) (step-length is 8), Fig. 2 (j), Fig. 2 (k) (step-length is 4) and Fig. 2 (m), Fig. 2 (n), (step-length is 2).Finally its structural images and texture image are separately summed design sketch Fig. 2 (i) that set obtains merging, Fig. 2 (l), Fig. 2 (o).It can be seen that from Fig. 2 (i) to Fig. 2 (l) again to Fig. 2 (o), the effect of fusion is become better and better, merge increasingly finer, merge Required time is more and more longer.

Claims (5)

1. an image interfusion method based on Multichannel Decomposition, it is characterised in that comprise the following steps:

Step 1), the feature extraction of source images Multichannel Decomposition: source images is carried out Multichannel Decomposition, obtain the structure of original image Component (structure image) and texture component (texture image)；

Step 2), fusion rule based on rarefaction representation is arranged: image carries out piecemeal vectorization after Multichannel Decomposition, to vector Multiple piecemeals after change carry out rarefaction representation, learn super complete dictionary, then calculate each column vector rarefaction representation under dictionary Coefficient；

Step 3), coefficient merges: the rarefaction representation coefficient of image block corresponding for each image according to certain fusion rule Merge, it is thus achieved that the sparse coefficient of image to be reconstructed；

Step 4), image reconstruction: reconstruct is the inverse process of Its Sparse Decomposition, carry out weight in conjunction with super complete dictionary with merging sparse coefficient Structure, obtains structural images and the texture image of synthesis, the structure obtained and texture image is summed into fusion image.

Image interfusion method based on Multichannel Decomposition the most according to claim 1, it is characterised in that described step 1) bag Include:

Two width source images A and B to be fused are carried out Multichannel Decomposition respectively, is decomposed into 2 major parts: comprise large scale change Structure components (structure image) and comprise the details coefficients (texture image) of image little yardstick texture.

Based on TV-L1 model, source images being carried out Multichannel Decomposition, TV-L1 model is as follows:

$\min {\&Integral;}_D|\&dtri;u|\mathrm{dx}+\mathrm{\&lambda;}{\&Integral;}_D|u-f|\mathrm{dx}$

Wherein, u is the structure components of image, and v is the texture component of image.

Image interfusion method based on Multichannel Decomposition the most according to claim 1, it is characterised in that described step 2) bag Include: image, after Multichannel Decomposition, carries out rarefaction representation respectively.According to the super complete dictionary of structure, by the figure to be fused of input As carrying out piecemeal vectorization arranged side by side, then calculate each column vector rarefaction representation coefficient under dictionary.The process of the method is main There are following two parts:

(1) image column vectorization.Assuming that two width sizes are the image of m × n, the sliding window utilizing a size to be 4 × 4 is respectively Carry out these images overlapping and be sampled as the image block that size is identical, and press row expansion 16 × 1 dimensional vector.

(2) dictionary training.For the matrix signal of two vectorizations of input, carried out K-SVD dictionary learning training.

Y=Dx

Wherein, Y represents the matrix signal through vectorization, and D represents the dictionary that study obtains, and x represents the sparse system that study obtains Number.

(3) rarefaction representation.The rarefaction representation coefficient of the column vector that each image block is corresponding obtains by solving following formula.

${\hat{s}}_i^j=\arg \min \left|\right|s_i^j|{|}_0$

$\begin{array}{cc}s.t.& {\mathrm{Ds}}_i^j=P_i^j;j=1,2,...,J\end{array}$

Here consider noise jamming, utilize the image sparse improved to represent model, meanwhile, use SOMP algorithm at super complete word Solve sparse coefficient in allusion quotation D, thus obtain the sparse coefficient of each image block.

Image interfusion method based on Multichannel Decomposition the most according to claim 1, it is characterised in that described step 3) bag Include following steps:

Assuming that the sparse coefficient that two signal training are obtained is respectively x1 and x2, size is a*b, and fusion rule is:

If matrix x=[x1, x2],

I.e. take the greater sparse coefficient (A) as the new image merged of two sparse coefficient column vector one normal forms.

Image interfusion method based on Multichannel Decomposition the most according to claim 1, it is characterised in that described step 4) bag Include following steps:

It is the inverse process of Its Sparse Decomposition according to reconstruct, the dictionary D that will obtain₁With the coefficient x merged₁It is multiplied and obtains Y₁, then by this Matrix signal is reduced to image, i.e. obtains structural images structure merged, obtains the texture maps merged by same method As texture, if original image is I, then have:

I=structure+texture.