CN103632354A - Multi focus image fusion method based on NSCT scale product - Google Patents

Abstract

The invention discloses a multi focus image fusion method based on NSCT scale product. The method comprises the steps that non-sampling contourlet transform (NSCT) is used to decompose a multi focus image; laplace energy and the fusion rule of a low frequency sub-band are used; scale product, local laplace energy and the fusion rule of a high frequency direction sub-band are used; and NSCT inverse operation is used to acquire a fusion image. According to an experiment results, the algorithm can fully extract source image information and can inject the information into the fusion image; the influence of noise can be effectively restrained; and the fusion effect is better than the fusion effects of other methods.

Description

Based on the long-pending multi-focus image fusing method of NSCT yardstick

Technical field

The invention belongs to image processing field, relate in particular to a kind of based on the long-pending multi-focus image fusing method of NSCT yardstick.

Background technology

Image co-registration is one of the study hotspot on present image processing circle, and it is widely used in the fields such as remote sensing, machine vision, medical science, military affairs, the administration of justice and manufacturing industry.When imageing sensors such as adopting CCD or CMOS obtains image, due to the camera lens depth of field, the scenery being positioned on focussing plane can obtain projection clearly on image, and on image, is subject in various degree fuzzy at other locational scenery.The image that one width focuses on is everywhere the precondition of many subsequent treatment, the main method addressing this problem is exactly multi-focus image fusion technology, adopt different focal lengths that shooting a series of images is set, then these images are carried out to fusion treatment, obtain width fused images clearly everywhere.

At present, conventional multi-focus image fusing method is mainly divided into spatial domain and the large class methods of transform domain two.Spatial domain fusion method is mainly divided into take 3 kinds of amalgamation modes that pixel, image block, region be unit.Take pixel while carrying out multi-focus image fusion as basis, conventionally need to judge whether each pixel focuses on, its shortcoming is that calculated amount and error are all larger.Multi-focus image fusing method counting yield based on image block is higher, but how to choose suitable tile size, needs further to be studied.Multi-focus image fusing method based on region is owing to first must carrying out image dividing processing, thereby increased calculated amount, and syncretizing effect depends on the quality that image is cut apart to a great extent.Image co-registration based on transform domain comprises pyramid transform, wavelet transformation and multi-scale geometric analysis method.The Contourlet being proposed by Minh N.Do and Vetterli converts, and Laplacian pyramid is combined with anisotropic filter group, has good many resolutions and directivity, is a kind of good image representation method.But, image is being carried out in the process of Contourlet conversion, need to carry out down-sampled operation to image, thereby make Contourlet conversion not possess translation invariance, in processing, image can produce Gibbs phenomenon.In order to address this problem, Arthur L has proposed the Contourlet conversion (NSCT) of non-sampling, and this conversion has translation invariance and redundant information, therefore can effectively extract the directional information for the treatment of in fused images, obtains good syncretizing effect.

Known by studying a large amount of documents, about the algorithm of multi-focus image fusion, do not consider that noise is on the impact of image and human eye vision effect, this effect by the image after causing merging and the perception of human eye institute is inconsistent.

Summary of the invention

The object of this invention is to provide a kind of Multi-focus image fusion that can suppress noise and meet human eye vision sense organ, specifically provide a kind of based on the long-pending Multi-focus image fusion of NSCT yardstick.

Suppose active image A and B, the concrete steps of the Multi-focus image fusion based on contrast are as follows:

Step 1: image pre-service

Owing to being subject to the interference of the factors such as noise, need to carry out pre-service to multiple focussing image A and B, the present invention adopts mean filter to carry out filtering processing to image A and B, the image A after being processed ' and B '.

Step 2: picture breakdown

The present invention adopt NSCT by image A ' and B ' decompose, obtain respectively the low frequency sub-band coefficient of focusedimage A '

with high frequency direction sub-band coefficients

the low frequency sub-band coefficient of focusedimage B '

with high frequency direction sub-band coefficients wherein l represents the scale parameter decomposing, and k represents Directional Decomposition progression.

Step 3: fusion rule

A large amount of lists of references show: the fusion rule of multi-focus image fusion directly has influence on the effect of image co-registration.Consider to the object of the invention is to design a Multi-focus image fusion that suppresses noise and meet human eye vision, the present invention adopts NSCT yardstick long-pending with Laplce's energy with (SML) as fusion rule, is specifically expressed as follows:

1) convergence strategy of low frequency sub-band

The low frequency sub-band that image obtains after NSCT decomposes is the approximate description of source images, has comprised the most of ability characteristics in image.The present invention adopts Laplce's energy and (SML) edge feature of reflection image, the to a certain extent appropriately focus characteristics of token image and sharpness.

Because contrast is considered in a certain region, suppose that window size is m ₁* n ₁, the low frequency coefficient of source images A ' and B '

with

corresponding local Laplce's energy and

with

computing formula as formula (1) and formula (2):

${\mathrm{NSML}}_l^{A^{\′}}(i,j)=\underset{k_1=-(m_1-1)/2}{\overset{(m_1-1)/2}{\mathrm{\Σ}}}\underset{k_2=-(n_1-1)/2}{\overset{(n_1-1)/2}{\mathrm{\Σ}}}{W}_l(k_1,k_2)\left(\right|2L_l^{A^{\′}}(i+k_1,j+k_2)-L_l^{A^{\′}}(i+k_1-1,j+k_2)-L_l^{A^{\′}}(i+k_1+1,j+k_2)|---\left(1\right)$

$+|2L_l^{A^{\′}}(i+k_1,j+k_2)-L_l^{A^{\′}}(i+k_1,j+k_2-1)-L_l^{A^{\′}}(i+k_1,j+k_2+1)|{)}^2$

${\mathrm{NSML}}_l^{B^{\′}}(i,j)=\underset{k_1=-(m_1-1)/2}{\overset{(m_1-1)/2}{\mathrm{\Σ}}}\underset{k_2=-(n_1-1)/2}{\overset{(n_1-1)/2}{\mathrm{\Σ}}}{W}_l(k_1,k_2)\left(\right|2L_l^{B^{\′}}(i+k_1,j+k_2)-L_l^{B^{\′}}(i+k_1-1,j+k_2)-L_l^{B^{\′}}(i+k_1+1,j+k_2)|---\left(2\right)$

$+|2L_l^{B^{\′}}(i+k_1,j+k_2)-L_l^{B^{\′}}(i+k_1,j+k_2-1)-L_l^{B^{\′}}(i+k_1,j+k_2+1)|{)}^2$

Obviously, local Laplce's energy and larger, shows that this region contains abundant image information.According to Laplce's energy of source images and size, the fusion rule of low frequency coefficient is as formula (3):

$L_l^F(i,j)=\left\{\begin{array}{cc}{L}_l^{A^{\′}}(i,j),& \mathrm{if}:{\mathrm{NSML}}_l^{A^{\′}}(i,j)>{\mathrm{NSML}}_l^{B^{\′}}(i,j)\\ L_l^{B^{\′}}(i,j),& \mathrm{if}:{\mathrm{NSML}}_l^{A^{\′}}(i,j)\≤{\mathrm{NSML}}_l^{B^{\′}}(i,j)\end{array}\right.---\left(3\right)$

2) convergence strategy of high frequency direction subband

The high-frequency sub-band of image after NSCT decomposes, reflection be the edge details information of image.If source images is mixed with noise, image is together with after multiple dimensioned decomposition, noise is aliasing in the edge details information of image.If fusion rule is directly acted on to wavelet coefficient, will cause falsely dropping of fusion coefficients.Because it is wrong that the existence of noise may cause the metric calculating according to fusion rule.The present invention, by the multiply each other important structure information of the NSCT multi-scale product strengthening image that forms of the high frequency direction sub-band coefficients of adjacent two NSCT, weakens noise, and then by Laplace operator with as the criterion of pixel resolution.Concrete process is as follows:

First, the high frequency direction subband to source images A ' and B '

with carry out the long-pending computing of yardstick, see formula (4) and (5):

$F_{l,k}^{A^{\′}}(i,j)=H_{l,k}^{A^{\′}}(i,j)\underset{K=0}{\overset{k_{l-1}-1}{\mathrm{\Π}}}{H}_{l-1,K}^{A^{\′}}(i,j),---\left(4\right)$

$F_{l,l}^{B^{\′}}(i,j)=H_{l,k}^{B^{\′}}(i,j)\underset{K=0}{\overset{k_{l-1}-1}{\mathrm{\Π}}}{H}_{l-1,K}^{B^{\′}}(i,j)---\left(5\right)$

Wherein, k _lrepresent yardstick l Directional Decomposition value of series.

Secondly, at window size, be m ₁* n ₁under, calculate source images yardstick long-pending

with

local Laplce's energy and

with

computing formula is formula (6) and formula (7):

${\mathrm{LML}}_{l,k}^{A^{\′}}(i,j)=\frac{\underset{k_1=-(m_1-1)/2}{\overset{(m_1-1)/2}{\mathrm{\Σ}}}\underset{k_2=-(n_1-1)/2}{\overset{(n_1-1)/2}{\mathrm{\Σ}}}{W}_h(k_1,k_2){\mathrm{PML}}_{l,k}^{A^{\′}}(i+k_1,j+k_2)}{(2m_1+1)({2n}_1+1)},---\left(6\right)$

${\mathrm{LML}}_l^{B^{\′}}(i,j)=\frac{\underset{k_1=-(m_1-1)/2}{\overset{(m_1-1)/2}{\mathrm{\Σ}}}\underset{k_2=-(n_1-1)/2}{\overset{(n_1-1)/2}{\mathrm{\Σ}}}{W}_h(k_1,k_2){\mathrm{PML}}_{l,k}^{B^{\′}}(i+k_1,j+k_2)}{(2m_1+1)(2n_1+1)}---\left(7\right)$

Wherein,

${\mathrm{PML}}_{l,k}^{A^{\′}}(i+k_1,j+k_2)=\left(\right|2F_{l,k}^{A^{\′}}(i+k_1,j+k_2)-F_{l,k}^{A^{\′}}(i+k_1-1,j+k_2)-F_{l,k}^{A^{\′}}(i+k_1+1,j+k_2)|+$

$|2F_{l,k}^{A^{\′}}(i+k_1,j+k_2)-F_{l,k}^{A^{\′}}(i+k_1,j+k_2-1)-F_{l,k}^{A^{\′}}(i+k_{1}j+j_2+1)|)$

${\mathrm{PML}}_{l,k}^{B^{\′}}(i+k_1,j+k_2)=\left(\right|2F_{l,k}^{B^{\′}}(i+k_1,j+k_2)-F_{l,k}^{B^{\′}}(i+k_1-1,j+k_2)-F_{l,k}^{B^{\′}}(i+k_1+1,j+k_2)|+$

$|2F_{l,k}^{B^{\′}}(i+k_1,j+k_2)-F_{l,k}^{B^{\′}}(i+k_1,j+k_2-1)-F_{l,k}^{B^{\′}}(i+k_1,j{+k}_2+1)|)$

The fusion rule of high frequency direction subband is shown in formula (8):

$H_{l,k}^F(i,j)=\left\{\begin{array}{cc}{H}_{l,k}^{A^{\&prime;}}(i,j),& \mathrm{if}:{\mathrm{LML}}_{l,k}^{A^{\&prime;}}(i,j)\&GreaterEqual;{\mathrm{LML}}_{l,k}^{B^{\&prime;}}(i,j)\\ H_{l,k}^{B^{\&prime;}}(i,j),& \mathrm{if}:{\mathrm{LML}}_{l,k}^{A^{\&prime;}}(i,j)<{\mathrm{LML}}_{l,k}^{B^{\&prime;}}(i,j)\end{array}\right.---\left(8\right)$

Step 4: Image Reconstruction

Thereby each sub-band coefficients of fused images is carried out to NSCT inverse transformation and obtain final fused images.

First the present invention adopts NSCT to decompose source images, and according to the feature of low frequency sub-band and high frequency direction subband, adopts respectively different fusion rules, finally by NSCT inverse transformation, obtains multi-focus image fusion image.Experimental result shows, this algorithm is injected in fused images without abundant extraction source image information, and can effectively suppress the impact of noise, and syncretizing effect is more excellent compared with additive method.

Accompanying drawing explanation

Fig. 1 is process flow diagram of the present invention;

Fig. 2 is the multi-Focus Image Fusion Effect figure under different fusion methods are disturbed about noiseless, wherein:

The left focusedimage of Fig. 2 (a);

The right focusedimage of Fig. 2 (b);

The design sketch of Fig. 2 (c) based on DWT blending algorithm;

The design sketch of Fig. 2 (d) based on LSWT blending algorithm;

The design sketch of Fig. 2 (e) based on NSCT blending algorithm;

Fig. 2 (f) adopts syncretizing effect of the present invention.

Fig. 3 be different fusion methods about there being the multi-Focus Image Fusion Effect figure under noise, wherein:.

The left focusedimage of Fig. 3 (a);

The right focusedimage of Fig. 3 (b);

The design sketch of Fig. 3 (c) based on DWT blending algorithm;

The design sketch of Fig. 3 (d) based on LSWT blending algorithm;

The design sketch of Fig. 3 (e) based on NSCT blending algorithm;

Fig. 3 (f) adopts syncretizing effect of the present invention.

Embodiment

With reference to Fig. 1, detailed process of the present invention comprises:

Step 1: image filtering is processed

Two source images A and B are carried out to mean filter processing, obtain filtered source images A ' and B ';

Step 2: picture breakdown

Application NSCT decomposes source images A ' and B ', obtains respectively low frequency sub-band and the high frequency direction subband of source images A ', the low frequency sub-band of source images B ' and high frequency direction subband;

Step 3: fusion rule

1) adopt formula (3) as the fusion rule of low frequency sub-band;

2) first high-frequency sub-band is carried out to the long-pending computing of yardstick, at the fusion rule that adopts formula (8) as high frequency direction subband.

Step 4: Image Reconstruction

Thereby each sub-band coefficients of fused images is carried out to NSCT inverse transformation and obtain final fused images.

In order to verify the performance of algorithm of the present invention, nothing is made an uproar and carried out respectively fusion experiment with the multiple focussing image of making an uproar.In experiment, except visual effect, we also adopt mutual information (MI), Q ^aB/Fas objective evaluation index.Adopt these two indexs to be because the object of image co-registration is fuse information, and they not necessarily require to know desirable fused images.Wherein MI is used for measuring source images has how many information transfer in fusion results, Q ^{aB F}to utilize Sobel rim detection to weigh how many edge details information to transfer to fused images from source images.The value of the two is larger, illustrates that the effect merging is better.

This experiment adopts respectively Image Fusion, the Image Fusion based on LSWT conversion based on DWT conversion and based on NSCT Image Fusion, muting multiple focussing image is merged, and fusion results as shown in Figure 2 and Table 1.

The comparison of the different fusion method performance evaluation of table 1

Table 1 has provided considerable evaluation index MI and Q ^{aB F}numerical value.Further having confirmed validity and the superiority of algorithm herein, is consistent with visually drawn conclusion.

This experiment adopts respectively Image Fusion, the Image Fusion based on LSWT conversion based on DWT conversion and based on NSCT Image Fusion, the multiple focussing image of Noise is merged.Wherein, multiple focussing image carries white Gaussian noise (variance is 0.01), and fusion results as described in Figure 3.Adopt improved Y-PSNR (VPSNR) as objective evaluation index, wherein the computing formula of VPSNR is suc as formula (13) institute formula:

$\mathrm{VPSNR}=10(\log \frac{255}{{\mathrm{\&sigma;}}_{n,f}^2}-\log \frac{255}{{\mathrm{\&sigma;}}_n^2})---\left(13\right)$

Wherein, be respectively the variance of fused images and noise source image.Obviously, in fused images, institute's Noise is less, and VPSNR value is larger.When VPSNR is tending towards 0, show that the noise contents of fused images and the noise contents of source images are close; VPSNR value is less than 0, shows that the noise contents of fused images is large compared with source images.

The fusion evaluation index value of these four kinds of blending algorithms is shown in Table 2.

The comparison of the different fusion method performance evaluation of table 2 Noise multiple focussing image

The evaluation index value of the effect by Fig. 2, Fig. 3 and table 1, table 2 can be found out, by the fused images that the method that the present invention proposes obtains, comprise maximum image informations, not only there is good visual effect, also comprise abundant information, and obtained good syncretizing effect.

Claims (1)

1. based on the long-pending multi-focus image fusing method of NSCT yardstick, comprise following process:

Step 1: image pre-service

Adopt mean filter to carry out filtering processing to image A and B, the image A after being processed ' and B '.

Step 2: picture breakdown

Adopt NSCT by image A ' and B ' decompose, obtain respectively the low frequency sub-band coefficient of focusedimage A '

with high frequency direction sub-band coefficients

the low frequency sub-band coefficient of focusedimage B '

with high frequency direction sub-band coefficients

wherein l represents the scale parameter decomposing, and k represents Directional Decomposition progression.

Step 3: fusion rule

1) fusion rule of low frequency sub-band

(1) application of formula (1) and formula (2) calculate the low frequency coefficient of source images A ' and B '

with

's

with

${\mathrm{NSML}}_l^{A^{\&prime;}}(i,j)=\underset{k_1=-(m_1-1)/2}{\overset{(m_1-1)/2}{\mathrm{\&Sigma;}}}\underset{k_2=-(n_1-1)/2}{\overset{(n_1-1)/2}{\mathrm{\&Sigma;}}}{W}_l(k_1,k_2)\left(\right|2L_l^{A^{\&prime;}}(i+k_1,j+k_2)-L_l^{A^{\&prime;}}(i+k_1-1,j+k_2)-L_l^{A^{\&prime;}}(i+k_1+1,j+k_2)|---\left(1\right)$

$+|2L_l^{A^{\&prime;}}(i+k_1,j+k_2)-L_l^{A^{\&prime;}}(i+k_1,j+k_2-1)-L_l^{A^{\&prime;}}(i+k_1,j+k_2+1)|{)}^2$

${\mathrm{NSML}}_l^{B^{\&prime;}}(i,j)=\underset{k_1=-(m_1-1)/2}{\overset{(m_1-1)/2}{\mathrm{\&Sigma;}}}\underset{k_2=-(n_1-1)/2}{\overset{(n_1-1)/2}{\mathrm{\&Sigma;}}}{W}_l(k_1,k_2)\left(\right|2L_l^{B^{\&prime;}}(i+k_1,j+k_2)-L_l^{B^{\&prime;}}(i+k_1-1,j+k_2)-L_l^{B^{\&prime;}}(i+k_1+1,j+k_2)|---\left(2\right)$

$+|2L_l^{B^{\&prime;}}(i+k_1,j+k_2)-L_l^{B^{\&prime;}}(i+k_1,j+k_2-1)-L_l^{B^{\&prime;}}(i+k_1,j+k_2+1)|{)}^2$

(2) fusion rule of low frequency sub-band is formula (3)

$L_l^F(i,j)=\left\{\begin{array}{cc}{L}_l^{A^{\&prime;}}(i,j),& \mathrm{if}:{\mathrm{NSML}}_l^{A^{\&prime;}}(i,j)>{\mathrm{NSML}}_l^{B^{\&prime;}}(i,j)\\ L_l^{B^{\&prime;}}(i,j),& \mathrm{if}:{\mathrm{NSML}}_l^{A^{\&prime;}}(i,j)\&le;{\mathrm{NSML}}_l^{B^{\&prime;}}(i,j)\end{array}\right.---\left(3\right)$

2) fusion rule of high frequency direction subband

(1) the high frequency direction subband to source images A ' and B '

with

carry out the long-pending computing of yardstick, see formula (4) and (5):

$F_{l,k}^{A^{\&prime;}}(i,j)=H_{l,k}^{A^{\&prime;}}(i,j)\underset{K=0}{\overset{k_{l-1}-1}{\mathrm{\&Pi;}}}{H}_{l-1,K}^{A^{\&prime;}}(i,j),---\left(4\right)$

$F_{l,l}^{B^{\&prime;}}(i,j)=H_{l,k}^{B^{\&prime;}}(i,j)\underset{K=0}{\overset{k_{l-1}-1}{\mathrm{\&Pi;}}}{H}_{l-1,K}^{B^{\&prime;}}(i,j)---\left(5\right)$

(2) application of formula (6) and (7) calculating source images yardstick are long-pending

with

's

with

${\mathrm{LML}}_{l,k}^{A^{\&prime;}}(i,j)=\frac{\underset{k_1=-(m_1-1)/2}{\overset{(m_1-1)/2}{\mathrm{\&Sigma;}}}\underset{k_2=-(n_1-1)/2}{\overset{(n_1-1)/2}{\mathrm{\&Sigma;}}}{W}_h(k_1,k_2){\mathrm{PML}}_{l,k}^{A^{\&prime;}}(i+k_1,j+k_2)}{(2m_1+1)({2n}_1+1)},---\left(6\right)$

${\mathrm{LML}}_l^{B^{\&prime;}}(i,j)=\frac{\underset{k_1=-(m_1-1)/2}{\overset{(m_1-1)/2}{\mathrm{\&Sigma;}}}\underset{k_2=-(n_1-1)/2}{\overset{(n_1-1)/2}{\mathrm{\&Sigma;}}}{W}_h(k_1,k_2){\mathrm{PML}}_{l,k}^{B^{\&prime;}}(i+k_1,j+k_2)}{(2m_1+1)(2n_1+1)}---\left(7\right)$

Wherein,

${\mathrm{PML}}_{l,k}^{A^{\&prime;}}(i+k_1,j+k_2)=\left(\right|2F_{l,k}^{A^{\&prime;}}(i+k_1,j+k_2)-F_{l,k}^{A^{\&prime;}}(i+k_1-1,j+k_2)-F_{l,k}^{A^{\&prime;}}(i+k_1+1,j+k_2)|+$

$|2F_{l,k}^{A^{\&prime;}}(i+k_1,j+k_2)-F_{l,k}^{A^{\&prime;}}(i+k_1,j+k_2-1)-F_{l,k}^{A^{\&prime;}}(i+k_{1}j+j_2+1)|)$

${\mathrm{PML}}_{l,k}^{B^{\&prime;}}(i+k_1,j+k_2)=\left(\right|2F_{l,k}^{B^{\&prime;}}(i+k_1,j+k_2)-F_{l,k}^{B^{\&prime;}}(i+k_1-1,j+k_2)-F_{l,k}^{B^{\&prime;}}(i+k_1+1,j+k_2)|+$

$|2F_{l,k}^{B^{\&prime;}}(i+k_1,j+k_2)-F_{l,k}^{B^{\&prime;}}(i+k_1,j+k_2-1)-F_{l,k}^{B^{\&prime;}}(i+k_1,j{+k}_2+1)|)$

The fusion rule of high frequency direction subband is shown in formula (8):

$H_{l,k}^F(i,j)=\left\{\begin{array}{cc}{H}_{l,k}^{A^{\&prime;}}(i,j),& \mathrm{if}:{\mathrm{LML}}_{l,k}^{A^{\&prime;}}(i,j)\&GreaterEqual;{\mathrm{LML}}_{l,k}^{B^{\&prime;}}(i,j)\\ H_{l,k}^{B^{\&prime;}}(i,j),& \mathrm{if}:{\mathrm{LML}}_{l,k}^{A^{\&prime;}}(i,j)<{\mathrm{LML}}_{l,k}^{B^{\&prime;}}(i,j)\end{array}\right.---\left(8\right)$

Step 4: Image Reconstruction

Thereby each sub-band coefficients of fused images is carried out to NSCT inverse transformation and obtain final fused images.

Images