CN101968882A - Multi-source image fusion method - Google Patents

Abstract

The invention discloses a multi-source image fusion method. The method comprises the following steps of: in an early feature extraction stage, combining wavelet kernel-based support transform with anti-aliasing contourlet, wherein on one hand, direction information which cannot be extracted by the support transform is increased, and on the other hand, an aliasing phenomenon generated in contourlet transform is eliminated; and in an image fusion stage, performing fusion judgment on low frequency signals of each fused image by establishing a high frequency judgment device by adopting a pulse coupled neural network, and performing the fusion judgment on high frequency signals of each fused image by adopting an absolute value maximum selection rule. Therefore, effective fusion of high and low frequency signals is realized.

Description

A kind of multisource image anastomosing method

Technical field

The present invention relates to the image co-registration field, be specifically related to a kind of multisource image anastomosing method.

Background technology

The processing of the image co-registration of prior art is generally Pixel-level, feature level, decision level.Pixel-level image merges and mainly carries out at original data, its purpose mainly is figure image intensifying, image segmentation and image classification, thereby for manually interpretation image or further feature level fusion provide better input information, pixel-level image merges, depend on the sensitivity of sensor sensing device, need adopt the sensor of high resolution capacity to remote image; Feature level image co-registration is meant characteristic information extraction from each sensor image, and the process of carrying out analysis-by-synthesis and processing, the characteristic information that extracts is the abundant expression amount or the sufficient statistic of Pixel Information, typical characteristic information has edge, shape, profile, angle, texture, similar luminance area etc., Feature Fusion is not the whole of image information, often when expressing feature, ignored image overall; The decision level image co-registration is to carry out the process of reasoning from logic or statistical reasoning from the information of multiple image, the foundation of a decision function often needs a large amount of samples to carry out long decision function checking computing, and image is adopted the fusion of decision level need expend the plenty of time and takies a large amount of internal memories.

Summary of the invention

In view of this, in order to address the above problem, the invention discloses a kind of multisource image anastomosing method, utilization is based on the support conversion and the combination of anti-aliasing profile ripple of small echo nuclear, realize multiple dimensioned directive feature extraction in the image characteristics extraction stage, reduced image reconstruction " cut " phenomenon simultaneously; Utilize maximum value and Pulse Coupled Neural Network combination, obtain high-precision fusion high and low frequency information, effectively extracted the notable feature of image, improved the visual effect of fused image integral body simultaneously.

The object of the present invention is achieved like this: a kind of multisource image anastomosing method may further comprise the steps:

1) adopts the small echo kernel function to treat fused images and set up multiple dimensioned support wave filter;

2) use multiple dimensioned support wave filter to carry out the support conversion, produce high and low frequency information respectively respectively treating fused images;

3) use anti-aliasing profile wave convert method that high and low frequency information is handled, obtain the anti-aliasing information of high and low frequency;

4) to the anti-aliasing information of the low frequency of respectively treating fused images, adopt pulse coupled neural metanetwork fusion rule, choose and treat that respectively the anti-aliasing information of the low frequency that can be triggered in the fused images is as the low-frequency information after merging;

5) use the absolute value maximum to choose the fusion rule method to the anti-aliasing information of the high frequency of respectively treating fused images, the anti-aliasing information of high frequency of choosing the absolute value maximum of respectively treating each pixel correspondence of fused images is as the high-frequency information after merging;

6) the high and low frequency information after the fusion that is selected is carried out inverse transformation of anti-aliasing profile ripple and support inverse transformation respectively, generate fused images.

Further, in the step 1), utilize the small echo kernel function to make up multiple dimensioned support wave filter prototype Q1, multiple dimensioned support wave filter prototype Q1 in length and breadth to utilizing the zero filling of porous wavelet filter principle, realize the structure of multi-scale transform wave filter, obtain multiple dimensioned support wave filter;

Wherein, multiple dimensioned support wave filter prototype is:

$Q_1=A(Y-b\stackrel{\→}{1})$

b＝B ^TY，

A＝Ω ^-1，

Ω=K+I/ γ, γ are normalization factor, and I is a unit matrix;

Element satisfies k among the K _{I, j}=K (x _i, x _jφ (the x of)=＜ _i) ^Tφ (x _j) i, j ∈ [1,2n+1], and φ (x) is the small echo kernel function;

Element satisfies Y among the Y _Ij=i * j, i ∈ [1,2n+1], j=1;

N is a natural number, is used for determining the scale size of multiple dimensioned support wave filter prototype Q1;

Further, step 2) specifically may further comprise the steps:

21) the multiple dimensioned support wave filter with each yardstick carries out the support conversion, the high-frequency information after obtaining decomposing with treating fused images:

DM _h＝Q _h*M _h；

Wherein, M ₁Be original image to be merged, when h 〉=2, M _hBe h layer low-frequency information;

DM _hBe h floor height information frequently, Q _hIt is the multiple dimensioned support wave filter of h yardstick;

DM _hBe Q _hAnd M _hConvolution results;

H ∈ [1, n], h is a decomposed class, n 〉=1;

22) use M _H+1=M _h-DM _hObtain h+1 layer low-frequency information M _H+1

Further, step 4) comprises the steps:

41) the anti-aliasing information of low frequency of utilizing one group of sample image is set up the Pulse Coupled Neural Network of selecting the anti-aliasing information of low frequency as input, comprises receiving element, modulating unit and pulse generation unit; The pulse generation unit excites excited principle to generate dynamic threshold according to decay;

42) the receiving element receiving step 3) the anti-aliasing information of the low frequency of respectively treating fused images that obtains is as presenting input, and modulating unit is modulated by the neighborhood link input of the anti-aliasing information of low frequency and is presented input, obtains neuron output; When neuron output value surpasses dynamic threshold, choose the corresponding anti-aliasing information of the low frequency for the treatment of fused images of this neuron output and be the low-frequency information after merging;

Further, in the step 6), the low-frequency information after the fusion that high-frequency information after the fusion that step 5) is obtained and step 4) obtain is carried out coefficient adjustment, and successively by anti-aliasing profile reconstructed wave and support reconstruct, reconstruct generates fused images.

The invention has the beneficial effects as follows: utilize support conversion and the combination of anti-aliasing profile ripple, realized multiple dimensioned directive feature extraction, reduced image reconstruction " cut " phenomenon simultaneously in the image characteristics extraction stage based on small echo nuclear; According to the high-frequency information token image detailed information of decomposing the back image, the absolute value of high-frequency information is big more, and it is strong that this place's brightness changes Shaoxing opera, and this place comprises the principle of the important information of image, use the absolute value maximum to choose algorithm to high-frequency information, make image detail information fully keep.According to people's vision physiological characteristics, the coupled neural network application to in the low-frequency information processing, has effectively been extracted the notable feature of image, improved the visual effect of fused image integral body simultaneously.

Description of drawings

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is described in further detail below in conjunction with accompanying drawing:

Fig. 1 shows the image co-registration main-process stream;

Fig. 2 shows the change of scale process of multiple dimensioned support wave filter;

Fig. 3 shows pulse coupled neural metanetwork structure.

Embodiment

Below will describe in detail the preferred embodiments of the present invention.

Whole step:

As shown in Figure 1, a kind of multisource image anastomosing method, step is:

In the image characteristics extraction stage:

1) employing small echo kernel function is treated fused images A and is treated that fused images B sets up multiple dimensioned support wave filter;

2) use multiple dimensioned support wave filter to treat fused images and carry out support conversion (that is: SVT conversion), form high and low frequency information;

3) use anti-aliasing profile wave convert method (that is: NACT conversion) that high and low frequency information is handled, obtain the anti-aliasing information of high and low frequency;

In the image co-registration stage:

4) to the anti-aliasing information of the low frequency of respectively treating fused images, adopt pulse coupled neural metanetwork (PCNN) fusion rule, choose and treat that respectively the anti-aliasing information of the low frequency that can be triggered in the fused images is as the low-frequency information after merging;

5) from respectively treat the anti-aliasing information of high frequency the fused images, choose the absolute value anti-aliasing information of high frequency as the high-frequency information after merging;

6) the high and low frequency information after the fusion that is selected is carried out inverse transformation of anti-aliasing profile ripple and support inverse transformation respectively, generate fused images.

One, image characteristics extraction

1. small echo kernel function

Support vector machine (Support vector machine, the general-purpose machinery learning method that SVM) to be Vapnik set up on nineteen ninety-five Statistical Learning Theory basis.People such as Zhang proposed small echo support vector machine theory in 2004, with the small echo nuclear subsitution common gaussian kernel, polynomial kernel etc., based on the small echo nuclear of the female little wave structure of Morlet, the small echo kernel function is defined as follows:

$K(s,s^{\′})=\underset{i=1}{\overset{L}{\mathrm{\Π}}}\cos (1.75\×\frac{(s_i-s_i^{\′})}{\mathrm{\λ}})\exp (-\frac{{\left|\right|(s_i-s_i^{\′})\left|\right|}^2}{2{\mathrm{\λ}}^2})---\left(1\right)$

Wherein, λ is a contraction-expansion factor, and it is fixed to come according to circumstances in actual applications; I ∈ [1, L], L is the space dimensionality of data s herein.

This small echo nuclear has nearly orthogonal and the characteristics that are suitable for the signal partial analysis, has improved the classification performance of support vector machine.Suykens in 1999 etc. propose a kind of novel support vector machine (LS-SVM) on Vapnik statistical learning basis, LS-SVM has better pace of learning than SVM, and calculates easy.Use small echo kernel function can be to approach any nonlinear function than higher precision as basis function in LS-SVM, compare with gaussian kernel, small echo is checked non-stationary signal and is had the meticulous step by step premium properties that approaches with feature extraction, the multi-scale filtering device of its generation can extract the notable feature of image preferably, is beneficial to image co-registration.

2. multiple dimensioned support wave filter

In the standard support vector machine, the most basic estimation problem can be optimized by QP and seeks the set of one group of weight coefficient and solve, and can be expressed as:

y＝f(x，w)＝w ^Tφ(x)+b (2)

Wherein: x ∈ R ^d, y ∈ R, R ^dThe expression input space, d represents dimension, φ (x) is Nonlinear Mapping function: R ^d→ R ^q, the dimension in q representation feature space, w are feature space R ^qAn element.Then f (x, w) its risk function R (w) can be written as:

$R\left(w\right)=\frac{C}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}L[y,f(x,w)]+\frac{{\left|\right|w\left|\right|}^2}{2}---\left(3\right)$

Wherein: C＞0 is a constant, and L[y, and f (x, w)] be loss function

$L[y,f(x,w)]=\left\{\begin{array}{cc}0,& \mathrm{if\; f}(x,w)-y<\mathrm{\&epsiv;}\\ |f(x,w)-y|-\mathrm{\&epsiv;},& \mathrm{otherwise}\end{array}\right.---\left(4\right)$

Parameter ε is majorized function f (x, w) deviation that obtains at realistic objective y.

In LS-SVM, risk function R (w) abbreviation becomes the minimization form:

$R\left(w\right)=\frac{{\left|\right|w\left|\right|}^2}{2}+\frac{\mathrm{\&gamma;}}{2}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{e}_i^2---\left(5\right)$

Wherein: γ is a constant, is normalization factor.This moment, constraint definition of equal value was:

y _i＝w ^Tφ(x _i)+b+e _i，i＝1，....N；(6)

Wherein: x _iBe sample, b ∈ R is the projection deviator, and N is the number of x sample;

The definable Lagrangian function is:

$L=\frac{{\left|\right|w\left|\right|}^2}{2}+\frac{\mathrm{\&gamma;}}{2}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{e}_i^2-\underset{i=0}{\overset{N}{\mathrm{\&Sigma;}}}{\mathrm{\&alpha;}}_i\{w^T\mathrm{\&phi;}\left(x_i\right)+b-y_i+e_i\},---\left(7\right)$

Cancellation w and e, so optimal conditions can be written as equality constraint:

$\left[\begin{array}{cc}0& {\stackrel{\&RightArrow;}{1}}^T\\ \stackrel{\&RightArrow;}{1}& \mathrm{\&Omega;}\end{array}\right]\left[\begin{array}{c}b\\ \mathrm{\&alpha;}\end{array}\right]=\left[\begin{array}{c}0\\ Y\end{array}\right]---\left(8\right)$

Wherein:

Ω=K+I/ γ, γ are normalization factor, and I is a unit matrix; The element of K satisfies k _{I, j}=K (x _i, x _jφ (the x of)=＜ _i) ^Tφ (x _j) i, j ∈ [1,2n+1], and φ (x) is the small echo kernel function, the expression inner product;

In the building process of multi-scale filtering device,

Element satisfies Y among the Y _Ij=i * j, i ∈ [1,2n+1], j=1; Therefore Y is a constant vector space.

Under the LS-SVM framework, when making up the multi-scale filtering device, estimation function f can be expressed as the linear combination of support vector, as shown in the formula:

$f\left(x\right)=\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{\mathrm{\&alpha;}}_{i}K(x,x_i)+b---\left(9\right)$

Wherein: K (x, x _i)=φ (x) ^Tφ (x _i), i=1 ... N.K is a kernel function, α _iBe support vector x _iSupport.

Through type (8) b, α can be written as formula (10) (11):

$b=\frac{{\stackrel{\&RightArrow;}{1}}^T{\mathrm{\&Omega;}}^{-1}Y}{{\stackrel{\&RightArrow;}{1}}^T{\mathrm{\&Omega;}}^{-1}{\stackrel{\&RightArrow;}{1}}^T}---\left(10\right)$

$\mathrm{\&alpha;}={\mathrm{\&Omega;}}^{-1}(Y-b\stackrel{\&RightArrow;}{1})---\left(11\right)$

In LS-SVM, can select to examine K, normalization factor γ and vector element Y=[y in advance ₁..., y _N] ^T, same method, constant matrices A and B definition also can calculate:

$A={\mathrm{\&Omega;}}^{-1},B^T=\frac{{\stackrel{\&RightArrow;}{1}}^T{\mathrm{\&Omega;}}^{-1}}{{\stackrel{\&RightArrow;}{1}}^T{\mathrm{\&Omega;}}^{-1}\stackrel{\&RightArrow;}{1}}$

Must see that (10) (11) can be written as equation (12) (13) respectively again:

b＝B ^TY (12)

$\mathrm{\&alpha;}=A(Y-b\stackrel{\&RightArrow;}{1})Y=\mathrm{QY}---\left(13\right)$

Q can pass through equation (14)

$Q=A(Y-b\stackrel{\&RightArrow;}{1})---\left(14\right)$

Be defined as the vector space size of M * N, M=2m+1 wherein, N=2n+1 generally sets m=n by rewriting matrix Q, and corresponding weighting coefficient nuclear has become new support multi-filter, and matrix Q depends on input vector, kernel function K and parameter γ.

With Q in length and breadth to utilizing the zero filling of porous wavelet filter principle, realize the structure of multi-scale transform wave filter, obtain the multiple dimensioned support wave filter of each yardstick.Show the dimensional variation form of Q, wherein Q as Fig. 2 ₁Be n=1, the coefficient of the multi-scale wavelet nuclear support vector wave filter of γ=1 o'clock.

3. multiple dimensioned support wave filter is treated fused images and is carried out the support conversion

The multiple dimensioned support wave filter of each yardstick is carried out the support conversion with treating fused images, and the high-frequency information after obtaining decomposing is as follows:

DM _h＝Q _h*M _h；

Wherein, M ₁Be original image to be merged, when h 〉=2, M _hBe h layer low-frequency information;

DM _hBe h floor height information frequently, Q _hIt is the multiple dimensioned support wave filter of h yardstick;

DM _hBe Q _hAnd M _hConvolution results;

H ∈ [1, n], h is a decomposed class, n 〉=1;

Use M _H+1=M _h-DM _hObtain h+1 layer low-frequency information M _H+1

4. anti-aliasing profile ripple (NACT) is to the high and low frequency information processing

Use a nearly step of anti-aliasing filter method to carry out operation splitting to the high frequency for the treatment of fused images, low-frequency information after decomposing through the support conversion, obtain the anti-aliasing information of high and low frequency.

Two, image co-registration

1. generate and merge the back low-frequency information

According to people's vision physiological characteristics, (Pulse-coupled neural network PCNN) applies in the image co-registration, improves the performance of image co-registration with the coupled neural network.PCNN is based on the visual theory of cat and a kind of neural network model of making up, mainly constitute: accept unit, modulating unit and pulse generation unit by three functional units, after the neuron igniting, can inspire adjacent neuron igniting, this characteristic of PCNN is that image applications is laid a good foundation.Each neuronic activity can be described by following formula:

$\left\{\begin{array}{c}{F}_l^k(i,j)\left(n\right)=s{p}_l^k(i,j)\\ L_l^k(i,j)\left(n\right)=e^{-{\mathrm{\&alpha;}}_L}{L}_l^k(i,j)(n-1)+V_L{\mathrm{\&Sigma;}}_{\mathrm{pq}}{W}_l^k(\mathrm{ij},\mathrm{pq})Y_l^k(\mathrm{ij},\mathrm{pq})(n-1)\\ U_l^k(i,j)\left(n\right)=F_i^k(i,j)\left(n\right)*(1+\mathrm{\&beta;}{L}_l^k(i,j)\left(n\right))\\ {\mathrm{\&theta;}}_l^k(i,j)\left(n\right)=e^{-{\mathrm{\&alpha;}}_{\mathrm{\&theta;}}}{\mathrm{\&theta;}}_l^k(i,j)(n-1)+V_{\mathrm{\&theta;}}{Y}_l^k(i,j)(n-1)\\ Y_l^k(i,j)\left(n\right)=\left\{\begin{array}{c}1,\mathrm{if}{U}_l^k(i,j)\left(n\right)>{\mathrm{\&theta;}}_l^k(i,j)\left(n\right)\\ \text{0,otherwise}\end{array}\right.\end{array}\right.$

Wherein,

With

Be respectively neuronic input, internal activity item and output; In the following formula after each parameter n represent number of iterations;

Represent the anti-aliasing information of low frequency of one group of sample image, import as the PCNN network;

Be neuronic link field, V _LExpression

Intrinsic electromotive force, W is the link weight coefficient matrix between the neuron, q and p are the subscript of link field, α _ITime constant for link field; β is a strength of joint coefficient between the cynapse;

Be the dynamic threshold threshold value, n represents number of iterations, if

Greater than threshold value

Then neuron will produce a pulse

Be also referred to as once igniting; In the training process of the PCNN network that is applied to the anti-aliasing frequency information of low frequency, repeatedly trigger to determine triggering times, experience repeatedly triggers the PCNN network that trains better fusion performance the anti-aliasing information of low frequency is selected.

In the PCNN network training process, the pulse generation unit excites excited principle to generate dynamic threshold according to decay in the iterative process

In circulation for the first time,

Initial value is made as 0.Dynamic threshold occurs in iterative process

The time, dynamic threshold increases at once, decays gradually by index again then, excites excitement once more up to neuron, and experience repeatedly triggers training and determines final dynamic gate limit value θ _{I, j}

The PCNN network application that trains in the anti-aliasing Information Selection of low frequency as shown in Figure 3, receiving element receive frequency territory coordinate (i, the anti-aliasing information F of the low frequency of j) locating _{I, j}As presenting input, modulating unit is modulated by the neighborhood link input of the anti-aliasing information of low frequency and is presented input, obtains neuron output Ui, j;

As neuron output Ui, j surpasses the definite dynamic threshold θ of PCNN that trains _{I, j}, choose the corresponding anti-aliasing information of the low frequency for the treatment of fused images of this neuron output and be the low-frequency information after merging.

2. generate and merge the back high-frequency information

The anti-aliasing information of the high frequency of respectively treating fused images that obtains after anti-aliasing profile wavelength-division is separated, the anti-aliasing information of high frequency of absolute value maximum of choosing each image slices vegetarian refreshments correspondence is as the high-frequency information after merging.

3. reconstruct generates fused images

High and low frequency information after the fusion that is selected is carried out inverse transformation of anti-aliasing profile ripple and support inverse transformation respectively, generate fused images.

The above only preferably is not limited to the present invention for of the present invention, and obviously, those skilled in the art can carry out various changes and modification and not break away from the spirit and scope of the present invention the present invention.Like this, if of the present invention these are revised and modification belongs within the scope of claim of the present invention and equivalent technologies thereof, then the present invention also is intended to comprise these changes and modification interior.

Claims (5)

1. multisource image anastomosing method is characterized in that: may further comprise the steps:

1) adopts the small echo kernel function to treat fused images and set up multiple dimensioned support wave filter;

2) use multiple dimensioned support wave filter to carry out the support conversion, produce high and low frequency information respectively respectively treating fused images;

3) use anti-aliasing profile wave convert method that high and low frequency information is handled, obtain the anti-aliasing information of high and low frequency;

4) to the anti-aliasing information of the low frequency of respectively treating fused images, adopt pulse coupled neural metanetwork fusion rule, choose and treat that respectively the anti-aliasing information of the low frequency that can be triggered in the fused images is as the low-frequency information after merging;

5) use the absolute value maximum to choose the fusion rule method to the anti-aliasing information of the high frequency of respectively treating fused images, the anti-aliasing information of high frequency of choosing the absolute value maximum of respectively treating each pixel correspondence of fused images is as the high-frequency information after merging;

6) the high and low frequency information after the fusion that is selected is carried out inverse transformation of anti-aliasing profile ripple and support inverse transformation respectively, generate fused images.

2. a kind of multisource image anastomosing method as claimed in claim 1 is characterized in that:

In the step 1), utilize the small echo kernel function to make up multiple dimensioned support wave filter prototype Q1, multiple dimensioned support wave filter prototype Q1 in length and breadth to utilizing the zero filling of porous wavelet filter principle, realize the structure of multi-scale transform wave filter, obtain multiple dimensioned support wave filter;

Wherein, multiple dimensioned support wave filter prototype is:

$Q_1=A(Y-b\stackrel{\&RightArrow;}{1})$

b＝B ^TY，

A＝Ω ^-1，

Ω=K+I/ γ, γ are normalization factor, and I is a unit matrix;

Element satisfies k among the K _{I, j}=K (x _i, x _jφ (the x of)=＜ _i) ^Tφ (x _j) i, j ∈ [1,2n+1], and φ (x) is the small echo kernel function;

Element satisfies y among the Y _Ij=i * j, i ∈ [1,2n+1], j=1;

N is a natural number, is used for determining the scale size of multiple dimensioned support wave filter prototype Q1.

3. a kind of multisource image anastomosing method as claimed in claim 2 is characterized in that:

Step 2) specifically may further comprise the steps:

21) the multiple dimensioned support wave filter with each yardstick carries out the support conversion, the high-frequency information after obtaining decomposing with treating fused images:

DM _h＝Q _h*M _h；

Wherein, M ₁Be original image to be merged, when h 〉=2, M _hBe h layer low-frequency information;

DM _hBe h floor height information frequently, Q _hIt is the multiple dimensioned support wave filter of h yardstick;

DM _hBe Q _hAnd M _hConvolution results;

H ∈ [1, n], h is a decomposed class, n 〉=1;

22) use M _H+1=M _h-DM _hObtain h+1 layer low-frequency information M _H+1

4. a kind of multisource image anastomosing method as claimed in claim 3 is characterized in that:

Step 4) comprises the steps:

41) the anti-aliasing information of low frequency of utilizing one group of sample image is selected the Pulse Coupled Neural Network of the anti-aliasing information of low frequency as input, comprises receiving element, modulating unit and pulse generation unit; The pulse generation unit excites excited principle to generate dynamic threshold according to decay;

42) the receiving element receiving step 3) the anti-aliasing information of the low frequency of respectively treating fused images that obtains is as presenting input, and modulating unit is modulated by the neighborhood link input of the anti-aliasing information of low frequency and is presented input, obtains neuron output; When neuron output value surpasses dynamic threshold, choose the corresponding anti-aliasing information of the low frequency for the treatment of fused images of this neuron output and be the low-frequency information after merging.

5. as each described a kind of multisource image anastomosing method in the claim 1 to 4, it is characterized in that: in the step 6), low-frequency information after the fusion that high-frequency information after the fusion that step 5) is obtained and step 4) obtain is carried out coefficient adjustment, by anti-aliasing profile reconstructed wave and support reconstruct, reconstruct generates fused images successively.

Images