CN1794300A - Multisensor image fusion method based on optimized small wave filter set - Google Patents

Abstract

This invention relates to a merge method for multiple sensor images based on an optimum small wave filter set, which applies small waves to carry out multi-dimension dissolution to the being merged images to get a series of HF components and a lowest frequency component, then applies a character pick up method based on the visual property to the HF part for merging and applies the weighted average method to merge the low frequency part to get the merged image by small wave inverse transformation to the merged results, finally applies the smallest mean-root-square error to evaluate the merged images and carries out optimized search to small coefficient taking the pass-band energy and the non-pass band energy of the small filter coefficient so as to get the optimum design of the small filter set.

Description

Multi-sensor image fusion method based on the optimal wavelet bank of filters

Technical field

The present invention relates to a kind of multi-sensor image fusion method based on the optimal wavelet bank of filters, be the image interfusion method of a multiple dimensioned and statistical method combination in the information fusion field, in systems such as optical imagery, targeted surveillance, safety inspection, all can be widely used.

Background technology

Image fusion technology is the fusion of visual information in the multi-sensor information fusion, it utilizes the different imaging mode of various imaging sensors, for different images provides complementary information, increase amount of image information, reduce the raw image data amount, raising is to the adaptability of environment, and is more reliable to obtain, useful information is for observing or further handling more accurately.Image fusion technology is an emerging technology that combines sensor, signal Processing, Flame Image Process and artificial intelligence etc., become a kind of very important and useful graphical analysis and computer vision technique in recent years, had a wide range of applications in fields such as automatic target identification, computer vision, remote sensing, robot, Medical Image Processing and Military Application.

At present representative method has the method, wavelet transform etc. of turriform conversion based on multiple dimensioned method.The process that multi-scale image merges be at first with images after registration through multiple dimensioned decomposition, decomposition method comprises methods such as Laplce, gradient pyramid and wavelet decomposition; Every layer of feature of seeing image at this yardstick or wave band as of decomposing the back image, the energy norm that reacts according to these features is weighted average or selection, to reach the purpose of fusion.Because wavelet transform when extracting the image low-frequency information, has obtained the detail of the high frequency of three directions again, in theory, compares with traditional fusion method based on the tower conversion, has better decomposition effect.Therefore, the critical problem that decomposes based on the multiresolution of wavelet transformation is the design problem of wavelet filter group.At present the filter set designing method that exists all concentrates on and how to design on the bi-orthogonal filter group with accurate reconstruct, and good accurate reconstruction property can guarantee that bank of filters do not bring any error to signal.Yet, also there is not at present a kind of filter design method that is directed to the image co-registration problem, in the application of image co-registration, the information of fused images does not need all information in the original image are all comprised to come in, and the intervention of little reconstructed error is little to merging Effect on Performance.The present invention introduces a kind of optimal wavelet filter set designing method that is applicable to image co-registration, and applies it to multiple focussing image, in the fusion of multi-sensor image.

Summary of the invention

The objective of the invention is at the deficiency that has now based on the multi-resolution image integration technology existence of small echo, a kind of method of devise optimum wavelet filter group is provided, thereby find a kind of wavelet transformation mode that is more suitable in image co-registration, to improve the picture quality after merging, reach desirable practical function.

For realizing such purpose, technical scheme of the present invention is: a kind of multi-sensor image fusion method based on the optimal wavelet bank of filters, it is characterized in that, and comprise following concrete steps:

1) original image that adopts 9/7 traditional wavelet filter group to treat fusion carries out the multiresolution expansion: two wave filters original image signal being imported the wavelet basis function structure decompose, and obtain a plurality of high fdrequency components and a low frequency component;

2) HFS that adopts the edge feature amplitude to be connected the probability characteristics fused images with the edge merges; Adopt the low frequency component of weighted average method fused images;

3) obtain the fusion results of image HFS and low frequency part by above step, can obtain fusion results by inverse wavelet transform;

4) adopt the method for lowest mean square root error that fused image is estimated, the evaluation result of whole blending algorithm and performance of filter parameter are that passband energy and stopband energy are as the objective function of optimizing, parameters optimization is a wavelet coefficient, carry out the design of wavelet filter group, so that root-mean-square error reaches minimum, obtain the wavelet filter group coefficient that fused images adopts;

5) adopt the adaptive modeling method for annealing to optimize wavelet filter group coefficient: optimization method adopts method search wavelet filter group coefficient in optimizing the space of adaptive modeling annealing, search in a very little scope optimizing space constraint, finally obtain optimum wavelet filter group coefficient;

6) method of the symmetrical improvement rate of employing is determined the bank of filters of this global optimum;

7) utilize the bank of filters of this optimized design to merge multi-resolution image, multi-sensor image at last, obtain fusion results.

Image interfusion method of the present invention has following beneficial effect:

The optimal wavelet territory multiresolution analysis method of image has the ability of extracting the validity feature of image, because the design of wavelet filter group is to be objective function with the image co-registration, make designed bank of filters can extract the validity feature of image, and with the different frequency range of the present image of these mark sheets, be on the multiresolution level, choosing of these features helped improving the image co-registration result.Carry out image interfusion method at HFS, make and in fusion process, can extract information as far as possible, and avoid because the erroneous decision that produces under the noise situation with edge feature based on visual signature.Adopt the adaptive modeling method for annealing can make and optimize the result, avoid falling into local minizing point for global optimum's point to the devise optimum wave filter.Based on above 3 reasons, the fusion performance that can fully improve image based on the multi-sensor image fusion method of optimal wavelet bank of filters of the present invention, picture quality after the fusion is improved significantly, show significant and practical value for the subsequent treatment and the image of various application systems.

Description of drawings

Fig. 1 is the multi-sensor image fusion method schematic flow sheet based on the optimal wavelet bank of filters of the present invention;

Fig. 2 treats fused images and desirable fusion results synoptic diagram for the inventive method embodiment's;

Fig. 3 is that different fusion methods are to the infrared and contrast synoptic diagram visible images fusion results.

Embodiment

Below in conjunction with drawings and Examples technical scheme of the present invention is further described.

A kind of multi-sensor image fusion method based on the optimal wavelet bank of filters is undertaken by flow process shown in Figure 1.By shown in Figure 1, among Fig. 1, treat that fused images is A and B.At first respectively image A, B are carried out wavelet transformation, picture signal is showed with multiple dimensioned form.The multiple dimensioned expression of signal has two parts, and a part is the HFS of reflected signal sudden change, the detail section of signal just, and another part is the low frequency part of reflected signal general picture.Employing is merged based on the system of selection of visual characteristic to HFS, adopts the weighted mean algorithm to merge to low frequency part.At last the high and low frequency part that obtains just can be obtained fused images through inverse wavelet transform.The evaluation result that fused image obtains merging by lowest mean square root error assessment with its input as optimizing process, is optimized wavelet coefficient and chooses, and the result that is optimized feeds back to the wavelet transformation process, optimizes multiresolution and decomposes and restructuring procedure.So repeatedly, obtain optimal wavelet bank of filters coefficient.Adopt the method for symmetrical improvement rate to determine the bank of filters of this global optimum then, utilize the bank of filters of this optimized design to merge multi-resolution image, multi-sensor image at last, can obtain comparatively desirable fusion results.

Concrete implementation step is:

1, the original image that adopts 9/7 traditional wavelet filter group to treat fusion carries out the multiresolution expansion:

Image is carried out wavelet transform the time, need the output of high and low frequency wave filter that will be by wavelet transformation to carry out down-sampled.Concrete conversion process such as formula (1) and the formula (2) of iterating:

$w_{i+1}\left(n\right)=\underset{k}{\mathrm{\Σ}}g\left(k\right)\·s_i\left(\frac{n-k}{2}\right)----\left(1\right)$

$s_{i+1}\left(n\right)=\underset{k}{\mathrm{\Σ}}h\left(k\right)\·s_i\left(\frac{n-k}{2}\right)----\left(2\right)$

G (k) is the decomposition high frequency filter of small echo, and h (k) is the decomposition low frequency filter of small echo.

Obtain the high fdrequency component w of small echo by formula (1) and formula (2) ₀, w ₁..., w _NWith lowest frequency component s _NN is the number of plies of wavelet transformation.

2, adopt based on the feature extracting method of visual characteristic HFS and merge image:

At first define a scanning window that size is suitable,, comprise its 8 adjacent pixels at a location of pixels exactly and handle together as 3*3.

Visual characteristic has mainly been considered two parts:

PI _X(m，n)＝C(m，n)·I(m，n) (3)

PI _X(m n) is meant visual characteristic, and subscript X represents to treat fused images, and (m n) is the amplitude that signal changes to C, the absolute value of the HFS of picture signal just, the amplitude that the present invention utilizes the absolute value of the HFS of small echo to change as signal; I (m n) is the topology that picture signal changes, and calculates by following formula:

sign＝sign(C _X(m，n)) (4)

I(m，n)＝p _X(m，n)·(1-p _X(m，n)) (5)

If the HFS of small echo is that then sign is 1 more than or equal to zero; If less than zero, then sign is 0.p _X(m, n) be symbol sign value with the center identical around the probable value of pixel number.

Convergence strategy is as follows:

Low frequency component to image directly is weighted average method, and its weights are respectively 1/2.

3, obtain the fusion results of image HFS and low frequency part by above step, can obtain final fusion results by inverse wavelet transform.

$s_i\left(n\right)=\underset{k}{\mathrm{\Σ}}\tilde{h}(n-k)\·s_{i+1}(2^i\·n)+\underset{k}{\mathrm{\Σ}}\tilde{g}\left(n-k\right)\·w_{i+1}(2^i\·n).----\left(7\right)$

What formula (7) was represented is the inverse transformation process (signal reconstruction) of small echo. Be the reconstructed high frequency wave filter of small echo, It is the reconstruct low frequency filter of small echo.Obtain fused images.

4, adopt the method for lowest mean square root error (RMSE) that fused image is estimated.The evaluation result of whole blending algorithm and performance of filter parameter (passband energy and stopband energy) are as the objective function of optimizing, and parameters optimization is a wavelet coefficient, thereby carries out the design of wavelet filter group.

First objective function root-mean-square error as definition (8):

$\mathrm{RMSE}=\sqrt{\frac{1}{M\·N}\underset{i=1}{\overset{M}{\mathrm{\Σ}}}\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{(R(i,j)-F(i,j))}^2},----\left(8\right)$

M wherein, N is the line number and the columns of image; (i is that the algorithm fusion results is at (i, the grey scale pixel value of j) locating j) to F; (i is that the canonical reference fused images is at (i, the grey scale pixel value of j) locating j) to F.The P value is more little, and fusion results is excellent more; Otherwise the P value is big more, and fusion results is poor more.

An odd length and symmetrical low-pass filter h ₀(n), n=1,2 ..., N ₀Fourier transform H ₀(z) be:

$F_0\left(e^{\mathrm{j\ω}}\right)=H_0\left(\mathrm{\ω}\right)e^{-j\frac{N_0-1}{2}\mathrm{\ω}},----\left(9\right)$

Wherein

$H_0\left(\mathrm{\ω}\right)=h_0((N_0+1)/2)+\underset{n=1}{\overset{\frac{N_0-1}{2}}{\mathrm{\Σ}}}2h_0\left(n\right)\cos (\frac{N_0+1}{2}-n)\mathrm{\ω}.----\left(10\right)$

Cutoff frequency is ω _sStopband ENERGY E s (h ₀) be:

$E_s\left(h_0\right)={\∫}_{{\mathrm{\ω}}_s}^{\mathrm{\π}}{H}_0^2\left(\mathrm{\ω}\right)\mathrm{d\ω}$

$=h_0^2((N_0+1)/2)(\mathrm{\π}-{\mathrm{\ω}}_s)-4h_0((N_0+1)/2)\underset{n=1}{\overset{(N_0-1)/2}{\mathrm{\Σ}}}{h}_0\left(n\right)\frac{\sin (\frac{N_0+1}{2}-n){\mathrm{\ω}}_s}{\frac{N_0+1}{2}-n}$

$+2\underset{n=1}{\overset{(N_0-1)/2}{\mathrm{\Σ}}}{h}_0{\left(n\right)}^2[(\mathrm{\π}-{\mathrm{\ω}}_s)-\frac{\sin (N_0+1-2n){\mathrm{\ω}}_s}{N_0+1-2n}]$

$-2\underset{n=1}{\overset{(N_0-1)/2}{\mathrm{\Σ}}}\underset{m=1,m\≠n}{\overset{(N_0-1)/2}{\mathrm{\Σ}}}{h}_0\left(n\right)h_0\left(m\right)[\frac{\sin (n-m){\mathrm{\ω}}_s}{n--m}+\frac{\sin (N_0+1-n-m){\mathrm{\ω}}_s}{N_0+1-n-m}]$ (11)

If cutoff frequency is ω _pPassband energy E _p(h ₀) be:

$E_p\left(h_0\right)={\∫}_0^{{\mathrm{\ω}}_p}{(H_0\left(\mathrm{\ω}\right)-1)}^2\mathrm{d\ω}$

$={(h_0((N_0+1)/2)-1)}^2{\mathrm{\ω}}_p+4(h_0((N_0+1)/2)-1)\underset{n=1}{\overset{(N_0-1)/2}{\mathrm{\Σ}}}{h}_0\left(n\right)\frac{\sin (\frac{N_0+1}{2}-n){\mathrm{\ω}}_p}{\frac{N_0+1}{2}-n}$

$+2\underset{n=1}{\overset{(N_0-1)/2}{\mathrm{\Σ}}}{h}_0{\left(n\right)}^2[{\mathrm{\ω}}_p-\frac{\sin (N_0+1-2n){\mathrm{\ω}}_p}{N_0+1-2n}]$

$+2\underset{n=1}{\overset{(N_0-1)/2}{\mathrm{\Σ}}}\underset{m=1,m\≠n}{\overset{(N_0-1)/2}{\mathrm{\Σ}}}{h}_0\left(n\right)h_0\left(m\right)[\frac{\sin (n-m){\mathrm{\ω}}_p}{n-m}+\frac{\sin (N_0+1-n-m){\mathrm{\ω}}_p}{N_0+1-n-m}]$ (12)

Second objective function with stopband and passband energy and represent:

$\underset{h_0,h_1}{\min }E(h_0,h_1),----\left(13\right)$

E (h wherein ₀, h ₁)=E _s(h ₀)+E _s(h ₁)+E _p(h ₀)+E _p(h ₁) be the gross energy of all filter transmission bands and stopband, the objective function of devise optimum wavelet filter just can be written as following form like this:

$\underset{h_0,h_1}{\min }-w\left(\mathrm{RMSE}(A,R,F)\right)+(1-w)E(h_0,h_1),----\left(14\right)$

0≤w≤1st wherein, weights, A represent it is Image Fusion, and R is the canonical reference image, and F is a fused image.Notice that when w=1 objective function becomes root-mean-square error (Root Mean SquareError is abbreviated as RMSE).

5, adopt the adaptive modeling method for annealing to obtain optimum wavelet filter group coefficient.

From selected traditional wavelet coefficient is initial solution, in a series of Markov chains that when successively decreasing, produce by means of controlled variable t, utilize a new explanation generation device and acceptance criterion, repeat to comprise the test of " produce new explanation---calculating target function is poor---judging whether to accept new explanation---accepts (or giving up) new explanation " these four tasks, constantly to the current iteration of separating, thereby reach the implementation that makes the objective function optimum.At above-mentioned image co-registration process, the committed step in the simulated annealing process is described as follows:

A) new explanation generation device.Picked at random i and j between 1～m, in current the separating if i and j the identical individual state of separating of i that then changes of the image co-registration index that bank of filters obtained; If difference then exchanges its state, that is:

If Xi=1-Xi is Xi=Xj

If Xi=1-Xi and Xj=1-Xj are Xi ≠ Xj

B) about the adjustment of initial point.Because the restriction strictness of bound in the model constrained condition, if L＜Lb (merge index and do not reach minimum index request) then chooses the pairing bank of filters coefficient of other indexs successively, change the optimization state of its correspondence, make it become current optimum solution, repeat this process up to eligible.

C) acceptance criterion.The Metropolis acceptance criterion of taking to expand judges whether to accept new explanation, if new explanation is feasible and be better than current separate then and accept; Otherwise accept new explanation by exp (Δ W/t) or 0 probability, that is:

D) stopping criterion.When being decremented to setting value E, controlled variable t stops algorithm.

6, the method for the symmetrical improvement rate of employing is determined the bank of filters of this global optimum.

The symmetry improvement rate is a kind of nonlinear method, is for fear of the unusual value when parameters optimization designs.For two hypothesis A and B, its performance number is respectively (A ₁..., A _m) and (B ₁..., B _m).Symmetry improvement rate S is:

$S_i=\left\{\begin{array}{ccc}{A}_i/B_i-1& \mathrm{if}& A_i\≥B_i\\ 1-A_i/B_i& \mathrm{if}& A_i\≤B_i\end{array}\right.,----\left(15\right)$

$\stackrel{\‾}{S}=\frac{1}{m}\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{S}_i.----\left(16\right)$

In our method, (A ₁..., A _m) be that certain bank of filters design result is applied to the resulting fusion results of m group image, symmetrical improvement rate is used for the total evaluation bank of filters in the resulting performance of different training image groups.Here, we adopt the bank of filters of maximum symmetrical improvement rate value as final optimization result.

7, utilize the bank of filters of this optimized design to merge multi-resolution image, multi-sensor image at last, can obtain comparatively desirable fusion results.

Fig. 2, Fig. 3 are seen in enforcement of the present invention, and the image by Fig. 2 is optimized wavelet coefficient respectively, and the wavelet filter group after will optimizing is used for the infrared image of Fig. 3 (a) expression and the visible images of Fig. 3 (b) expression are merged.Fig. 2 is used to optimize the wavelet filter group, and uppermost four lines image is four parts of poly Jiao " clock " image, and bottom line is " cameraman " image, and the image of rightmost column is the canonical reference image of preceding two row; Fig. 3 (c) is the result who merges based on the laplacian pyramid multiresolution method; Fig. 3 (d) is the fusion results of traditional wavelet multiresolution rate image interfusion method; Fig. 3 (e) is the multi-resolution image fusion results based on discrete wavelet; Fig. 3 (f) is the fusion results of the multiresolution method of optimal wavelet bank of filters mentioned among employing the present invention.

Table 1 is the fusion results index evaluation index of visible light/infrared image.As can be seen from the table, when the method that adopts the present invention to propose, merge performance and surpassed traditional wavelet method, the laplacian pyramid method, even surpassed the method for traditional wavelet frame.

Table 1

Image interfusion method	EMI	PMI
			Based on the LP method based on the DWT method based on the DWF method based on the ODWT method	0.4344 0.4189 0.4179 0.4723	0.6456 0.6411 0.6427 0.6574

Claims (1)

1, based on the multi-sensor image fusion method of optimal wavelet bank of filters, it is characterized in that comprising following concrete steps:

1) original image that adopts 9/7 traditional wavelet filter group to treat fusion carries out the multiresolution expansion: two wave filters original image signal being imported the wavelet basis function structure decompose, and obtain a plurality of high fdrequency components and a low frequency component;

2) HFS that adopts the edge feature amplitude to be connected the probability characteristics fused images with the edge merges; Adopt the low frequency component of weighted average method fused images;

3) obtain the fusion results of image HFS and low frequency part by above step, can obtain fusion results by inverse wavelet transform;

4) adopt the method for lowest mean square root error that fused image is estimated, the evaluation result of whole blending algorithm and performance of filter parameter are that passband energy and stopband energy are as the objective function of optimizing, parameters optimization is a wavelet coefficient, carry out the design of wavelet filter group, so that root-mean-square error reaches minimum, obtain the wavelet filter group coefficient that fused images adopts;

5) adopt the adaptive modeling method for annealing to optimize wavelet filter group coefficient: optimization method adopts method search wavelet filter group coefficient in optimizing the space of adaptive modeling annealing, search in a very little scope optimizing space constraint, finally obtain optimum wavelet filter group coefficient;

6) method of the symmetrical improvement rate of employing is determined the bank of filters of this global optimum;

7) utilize the bank of filters of this optimized design to merge multi-resolution image, multi-sensor image at last, obtain fusion results.