CN103578092A - Multi-focus image fusion method - Google Patents

Abstract

The invention discloses a multi-focus image fusion method. According to the multi-focus image fusion method, a source image is fused through a fusion algorithm based on NMF, and then a temporary fused image is obtained; the difference between the temporary fused image and the source image is obtained, and a difference image of the temporary fused image and the source image is obtained; gradient energy of a neighboring window of each pixel of the difference image is calculated, a decision matrix is built according to the gradient energy of the neighboring window of the pixels of the difference image, corresponding pixels in the source image are fused according to a certain fusion rule, and a fused image is obtained. According to the multi-focus image fusion method, secondary fusion is carried out on the source image, the temporary fused image is built by extracting global features of the source image, then the difference between the temporary fused image and the source image is obtained, focus regional characteristics of the source images are detected and judged accurately through the gradient energy of the difference image, and then the quality of the fused image is improved.

Description

A kind of multi-focus image fusing method

Technical field

The invention belongs to technical field of image processing, what be specifically related to is a kind of multi-focus image fusing method.

Background technology

Multi-focus image fusion be exactly to through obtain under the identical image-forming condition of registration about several focusedimages in a certain scene, adopt certain blending algorithm to extract clear area separately, and these regions are merged and generate in these scenes of a width all images clearly of all objects.In fields such as traffic, medical treatment, safety, logistics, be widely used.Can effectively improve utilization factor and the reliability of system to object table detection and identify of sensor image information.

Pixel-level image fusion directly adopts suitable blending algorithm to carry out fusion treatment on original image pixels gray space, and fundamental purpose is to cut apart with Images Classification and process and provide support for follow-up figure image intensifying, image.Pixel-level image fusion algorithm is compared with decision level image co-registration with feature level image co-registration, and accuracy is high, and information loss is minimum, the detailed information that can provide more feature levels and decision level image co-registration not to have.

Along with the development of computing machine and imaging technique, formed gradually in recent years following several comparatively conventional Pixel-level Multi-focus image fusion:

(1) multi-focus image fusing method based on wavelet transformation (Discrete Wavelet Transform, DWT).Its main process is that source images is carried out to wavelet decomposition, then adopts suitable fusion rule, and high and low frequency coefficient is merged, and the wavelet coefficient after merging is carried out to wavelet inverse transformation and obtain fused images.The method has good time-frequency local characteristics, has obtained good effect, but DWT can not make full use of the geometric properties that view data itself has, can not be optimum or the presentation video of " sparse ".

(2) multi-focus image fusing method of the profile wave convert based on non-lower sampling (Nonsubsampled Contourlet Transform, NSCT).Its main process is that source images is carried out to NSCT decomposition, then adopts suitable fusion rule, and high and low frequency coefficient is merged, and the wavelet coefficient after merging is carried out to NSCT inverse transformation and obtain fused images.The method can obtain good syncretizing effect, but travelling speed is slower, and coefficient of dissociation need to take a large amount of storage spaces.

(3) multi-focus image fusing method based on principal component analysis (PCA) (Principal Component Analysis, PCA).Its main process is according to row major or row, preferentially to convert source images to column vector, and calculate covariance, according to covariance matrix, ask for proper vector, determine first principal component characteristic of correspondence vector and determine accordingly the weight that each source images merges, according to weight, be weighted fusion.The method is simple, and travelling speed is fast, but easily reduces fused images contrast, affects fused image quality.

(4) multi-focus image fusing method based on spatial frequency (Spatial Frequency, SF).Its main process is source images to be carried out to piece cut apart, and then calculates each piece SF, and the SF of contrast source images corresponding blocks merges the large correspondence image piece of SF value to obtain fused images.The method is simply easy to implement, but a minute block size is difficult to self-adaptation to be determined, piecemeal is too large, easily the pixel outside focus is all comprised to come in, reduce fusion mass, fused images contrast is declined, easily produce blocking effect, piecemeal is too little characterizes limited ability to region readability, is prone to the wrong choice of piece, and to noise-sensitive.

(5) multi-focus image fusing method based on Pulse Coupled Neural Network (Pulse Coupled Neural Network, PCNN).Its main process is the outside input stimulus using single grey scale pixel value as PCNN, according to the igniting figure of PCNN, calculates spark rate corresponding to input pixel, and the pixel with larger spark rate is merged, and obtains fused images.The method can realize information transmission and information coupling automatically, and its result can better retain the feature of figure itself.But the method parameter is more, model is complicated, and running and comparing is consuming time, in addition, human eye vision is more responsive and also insensitive to the brightness of single pixel to the variation of image border, and the gray-scale value of the single pixel fused images effect that neuronic outside input produces as PCNN is unsatisfactory.

Above-mentioned Lung biopsy is the multi-focus image fusing method of comparatively commonly using, but in these methods, wavelet transformation (DWT) can not make full use of the geometric properties that view data itself has, can not be optimum or the presentation video of " sparse ", easily cause fused images to occur skew and information dropout phenomenon.Profile wave convert based on non-lower sampling (NSCT) method is because decomposable process is complicated, and travelling speed is slower, and coefficient of dissociation need to take a large amount of storage spaces in addition.Principal component analysis (PCA) (PCA) method easily reduces fused images contrast, affects fused image quality.Pulse Coupled Neural Network (PCNN) method parameter is more, and model is complicated, and running and comparing is consuming time.Commonly use for these five kinds and all exist different shortcomings, between speed and fusion mass, be difficult to be in harmonious proportion, limited application and the popularization of these methods.

Summary of the invention

Technical matters to be solved by this invention be in multi-focus image fusion field blending algorithm image characteristics extraction and minutia are represented clear not, can not adaptively selected block size and blocking effect of occurring etc., the problem that syncretizing effect is not satisfactory.For this reason, the invention provides a kind of multi-focus image fusing method, the multiple focussing image I of the method after to registration _aand I _bmerge I _aand I _bbe gray level image, and

be that size is the space of M * N, M and N are positive integer, and this fusion method comprises the following steps:

(1) build multiple focussing image I _aand I _bobserving matrix V;

(2) with Algorithms of Non-Negative Matrix Factorization, observing matrix V is decomposed, obtain basis matrix W;

(3) basis matrix W being converted to size is the matrix of M * N, and the image that this matrix is corresponding is interim fused images I ₀;

(4) respectively by interim fused images I ₀with source images I _aand I _bdiffer from, obtain differential image D _awith differential image D _b, wherein: D _a=I ₀-I _a, D _b=I ₀-I _b;

(5) calculated difference image D _awith differential image D _bgradient energy in each neighborhood of pixels, Size of Neighborhood is 5 * 5 or 7 * 7;

(6) construction feature matrix H,

(formula 1)

In (formula 1):

EODG _a(i, j) is differential image D _agradient energy in pixel (i, j) neighborhood;

EODG _b(i, j) is differential image D _bgradient energy in pixel (i, j) neighborhood;

i＝1,2,3,…,M；j＝1,2,3,…,N；

H (i, j) is that matrix H i is capable, the element of j row;

(7) build fused images F,

gray level image after being merged:

$F(i,j)=\left\{\begin{array}{cc}{I}_A(i,j)& H(i,j)=1\\ I_B(i,j)& H(i,j)=0\end{array}\right.$ (formula 2)

In (formula 2):

The gray-scale value that F (i, j) locates for the gray level image F pixel (i, j) after merging;

I _a(i, j) is gray level image I before merging _athe gray-scale value located of pixel (i, j);

I _b(i, j) is gray level image I before merging _bthe gray-scale value located of pixel (i, j).

The eigenmatrix building in step (6) is corroded to expansive working and process, and utilize the eigenmatrix after processing to build fused images.

Compared with prior art, the invention has the beneficial effects as follows:

(1) first the present invention carries out Non-negative Matrix Factorization to source images, carry out image co-registration for the first time, then the fusion results obtaining is poor with source images respectively, by the energy gradient in each neighborhood of pixels relatively, the focus characteristics of each sub-block in source images is judged, built decision matrix and merge for the second time.Source images is carried out to secondary fusion, improved the accuracy rate that each sub-block focus characteristics of source images is judged, be conducive to the extraction of clear area target, can effectively suppress blocking effect, simultaneously better to image detail character representation effect.

(2) in the present invention, image co-registration framework is flexible, can be used for the image co-registration task of other types.

To sum up, algorithm frame of the present invention is flexible, the focus characteristics of source images sub-block is judged to have higher accuracy rate, and effectively suppress blocking effect, can extract comparatively accurately clear area target detail, and it is clear that image detail is expressed, and improves fused image quality.

Accompanying drawing explanation

Below in conjunction with accompanying drawing and embodiment, the present invention is further explained to explanation.

Fig. 1 is the source images to be merged of embodiment 1;

Fig. 2 is the Non-negative Matrix Factorization (NMF of Laplce (LAP), wavelet transformation (DWT), the profile wave convert based on non-lower sampling (NSCT), spatial frequency (SF), Non-negative Matrix Factorization (NMF), local Non-negative Matrix Factorization (LNMF), sparse Non-negative Matrix Factorization (SNMF), degree of rarefication constraint _sC), the syncretizing effect figure of nine kinds of image interfusion methods of the present invention (Propose) to multiple focussing image ' clock ' Fig. 1, Proposed represents method of the present invention;

Fig. 3 is the Non-negative Matrix Factorization (NMF of Laplce (LAP), wavelet transformation (DWT), the profile wave convert based on non-lower sampling (NSCT), spatial frequency (SF), Non-negative Matrix Factorization (NMF), local Non-negative Matrix Factorization (LNMF), sparse Non-negative Matrix Factorization (SNMF), degree of rarefication constraint _sC), nine kinds of image interfusion methods of the present invention (Propose) are to the difference comparison diagram between each fused images of multiple focussing image ' clock ' and source images Fig. 1 (b);

Fig. 4 is the fused images for the treatment of of embodiment 2;

Fig. 5 is the Non-negative Matrix Factorization (NMF of Laplce (LAP), wavelet transformation (DWT), the profile wave convert based on non-lower sampling (NSCT), spatial frequency (SF), Non-negative Matrix Factorization (NMF), local Non-negative Matrix Factorization (LNMF), sparse Non-negative Matrix Factorization (SNMF), degree of rarefication constraint _sC), nine kinds of image interfusion methods of the present invention (Propose) are to multiple focussing image ' book ' Fig. 3 (a) and fused images (b);

Fig. 6 is the Non-negative Matrix Factorization (NMF of Laplce (LAP), wavelet transformation (DWT), the profile wave convert based on non-lower sampling (NSCT), spatial frequency (SF), Non-negative Matrix Factorization (NMF), local Non-negative Matrix Factorization (LNMF), sparse Non-negative Matrix Factorization (SNMF), degree of rarefication constraint _sC), nine kinds of image interfusion methods of the present invention (Propose) are to the difference comparison diagram between each fused images of multiple focussing image ' book ' and source images Fig. 3 (b).

Embodiment

In order to overcome tile size in multi-focus image fusion field, can not self-adaptation divide, details is unintelligible, part loss in detail, the not satisfactory problem of degradation shortcomings and syncretizing effect under contrast, the invention provides a kind of multi-focus image fusing method based on Non-negative Matrix Factorization, the concrete operations flow-interpret of the method is as follows:

Method of the present invention is to I _a,

two width multiple focussing images merge, and two width multiple focussing image sizes are M * N: utilize vectorial conversion operations by image I _a,

be converted to column vector, the merging of gained column vector being carried out to matrix forms observing matrix V:

$V=\left[\begin{array}{cc}{V}_A& V_B\end{array}\right]=\left[\begin{array}{cc}{v}_{1A}& v_{1B}\\ v_{2A}& v_{2B}\\ .& .\\ .& .\\ .& .\\ v_{\mathrm{qA}}& v_{\mathrm{qB}}\\ .& .\\ .& .\\ .& .\\ v_{\mathrm{MNA}}& v_{\mathrm{MNB}}\end{array}\right]$

Wherein: V _afor source images I _aimage array be converted to the matrix after column vector, V _bfor source images I _bimage array be converted to the matrix after column vector, v _qArepresent source images I _aimage array be converted to the element after column vector, v _qBrepresent source images I _bimage array be converted to the element after column vector, q=1,2,3 ..., MN.

The present invention carries out Non-negative Matrix Factorization (NMF) to observing matrix V, obtains basis matrix W, and observing matrix V can be by this basis matrix W linear expression, and the base vector number of basis matrix W is 1, and this base vector can intactly represent whole features of observing matrix V.

The present invention utilizes matrix conversion operation to convert basis matrix W to interim fused images I ₀∈ R ^{m * N}, now, interim fused images I ₀∈ R ^{m * N}homology image I _a,

corresponding, and in the same size.

Differential image D of the present invention _a, D _b∈ R ^{m * N}through interim fused images I ₀∈ R ^{m * N}difference homology image I _a,

work difference obtains, differential image D _a=I ₀-I _a, differential image D _b=I ₀-I _b, the differential image D obtaining _aand D _bbe the images that two width sizes are identical, size is M * N, and during gradient energy in calculating two differential images in respective pixel neighborhood, the position coordinates of these two pixels in differential image is separately identical.

Gradient energy of the present invention (EODG) computing method are shown below:

$\mathrm{EODG}(\mathrm{\α},\mathrm{\β})=\underset{k=-(K-1)/2}{\overset{(K-1)/2}{\mathrm{\Σ}}}\underset{l=-(L-1)/2}{\overset{(L-1)/2}{\mathrm{\Σ}}}(f_{\mathrm{\α}+k}^2+f_{\mathrm{\β}+1}^2)$

f _α+k＝[f ₀（α+k+1,β）-f（α+k+1,β）]-[f ₀（α+k,β）-f（α+k,β）]

f _β+l＝[f ₀(α,β+l+1)-f(α,β+l+1)]-[f ₀(α,β+l)-f(α,β+l)]

Wherein:

K * L is the size of pixel (α, β) neighborhood, and value is 5 * 5 or 7 * 7;

-(K-1)/2≤k≤(K-1)/2, and k round numbers;

-(L-1)/2≤l≤-(L-1)/2, and l round numbers;

F (α, β) is the gray-scale value of pixel (α, β) in source images;

F ₀(α, β) is the gray-scale value of pixel (α, β) in interim fused images.

Decision matrix H of the present invention,

in, " 1 " represents source images I _apixel be in focal zone, " 0 " represents source images I _bpixel be in focal zone.

Owing to relying on separately gradient energy as the evaluation criterion of image definition, may not extract all clear sub-blocks completely, the interregional burr that exists in decision matrix, blocks and narrow adhesion, need to carry out morphologic corrosion expansive working to decision matrix.

Below the embodiment that inventor provides, so that technical scheme of the present invention is further explained to explanation.

Embodiment 1:

Follow technical scheme of the present invention, to Fig. 1 (a) and (b), two width source images carry out fusion treatment to this embodiment, and result is as shown in the Propose in Fig. 2.Utilize the Non-negative Matrix Factorization (NMF of Laplce (LAP), wavelet transformation (DWT), the profile wave convert based on non-lower sampling (NSCT), spatial frequency (SF), Non-negative Matrix Factorization (NMF), local Non-negative Matrix Factorization (LNMF), sparse Non-negative Matrix Factorization (SNMF), degree of rarefication constraint simultaneously _sC) eight kinds of image interfusion methods to Fig. 1 (a) with (b) shown in two width source images carry out fusion treatment, result as shown in Figure 2, is carried out quality assessment to the fused images of different fusion methods, processes and calculates result shown in table 1.

The evaluation of table 1 multiple focussing image ' clock ' fused image quality.

Method	MI	Q AB/F	Q 0	Q W	Q E	Running?Time(s)
							LAP	6.9328	0.6872	0.7641	0.9154	0.5268	0.8490
DWT	5.8622	0.6317	0.6964	0.8802	0.492	0.4699
							NSCT	6.5346	0.6603	0.7566	0.8850	0.4944	205.3386
SF	7.6723	0.6768	0.7952	0.8957	0.4848	0.8953
							NMF	6.8719	0.5875	0.7770	0.8594	0.3796	1.2612
LNMF	6.9078	0.6601	0.7706	0.8566	0.3809	11.5779
							SNMF	6.8703	0.5801	0.7754	0.8561	0.3811	13.1989
NMFsc	6.8677	0.5699	0.7674	0.8487	0.3761	15.8415
							Propose	8.4516	0.7351	0.7509	0.8689	0.5439	10.7145

Embodiment 2:

Follow technical scheme of the present invention, to Fig. 4 (a) and (b), two width source images carry out fusion treatment to this embodiment, and result is as shown in the Propose in Fig. 5.

Utilize the Non-negative Matrix Factorization (NMF of Laplce (LAP), wavelet transformation (DWT), the profile wave convert based on non-lower sampling (NSCT), spatial frequency (SF), Non-negative Matrix Factorization (NMF), local Non-negative Matrix Factorization (LNMF), sparse Non-negative Matrix Factorization (SNMF), degree of rarefication constraint simultaneously _sC) eight kinds of image interfusion methods to two width source images (a) shown in Fig. 4 with (b) carry out fusion treatment, result as shown in Figure 5, is carried out quality assessment to the fused images of different fusion methods in Fig. 5, processes and calculates result shown in table 2.

The evaluation of table 2 multiple focussing image ' book ' fused image quality.

In table 1 and table 2: Method represents method; Fusion method comprises eight kinds respectively: the Non-negative Matrix Factorization (NMF of Laplce (LAP), wavelet transformation (DWT), the profile wave convert based on non-lower sampling (NSCT), spatial frequency (SF), Non-negative Matrix Factorization (NMF), local Non-negative Matrix Factorization (LNMF), sparse Non-negative Matrix Factorization (SNMF), degree of rarefication constraint _sC); Running Time represents working time, and unit is second.MI represents mutual information, is the fused image quality objective evaluation index based on mutual information.Q ^aB/Fthe marginal information total amount that representative is shifted from source images; Q ₀represent the degreeof tortuosity of fused images; Q _wrepresent that fused images shifts the degree of remarkable information from source images; Q _erepresent that fused images shifts the degree of visual information and marginal information from source images, the desired value of Q index is larger, represents that the quality of fused images is better.

From Fig. 3 and Fig. 5, can find out, all there is drift in various degree and fuzzy in other method, and method of the present invention is obviously better than the syncretizing effect of other fusion methods to the fused images of multiple focussing image Fig. 1 ' clock ' and multiple focussing image Fig. 2 ' book '.

Difference design sketch Fig. 3 from fused images Fig. 2 and Fig. 1 (b) source images, and difference design sketch Fig. 6 of fused images Fig. 5 and Fig. 4 (b) source images can find out, this method is obviously better than additive method to the extractability of source images focus area object edge and texture, can be good at the target information of focus area in source images to transfer in fused images and go.Can effectively catch the target detail information of focal zone, improve image co-registration quality.

More than exemplifying is only to illustrate of the present invention, does not form the restriction to protection scope of the present invention, within the every and same or analogous design of the present invention all belongs to protection scope of the present invention.

Claims (2)

1. a multi-focus image fusing method, the multiple focussing image I of the method after to registration _aand I _bmerge I _aand I _bbe gray level image, and I _a,

be that size is the space of M * N, M and N are positive integer, it is characterized in that, this fusion method comprises the following steps:

(1) build multiple focussing image I _aand I _bobserving matrix V;

(2) with Algorithms of Non-Negative Matrix Factorization, observing matrix V is decomposed, obtain basis matrix W;

(3) basis matrix W being converted to size is the matrix of M * N, and the image that this matrix is corresponding is interim fused images I ₀;

(4) respectively by interim fused images I ₀with source images I _aand I _bdiffer from, obtain differential image D _awith differential image D _b, wherein: D _a=I ₀-I _a, D _b=I ₀-I _b;

(5) calculated difference image D _awith differential image D _bgradient energy in each neighborhood of pixels, Size of Neighborhood is 5 * 5 or 7 * 7;

(6) construction feature matrix H,

(formula 1)

In (formula 1):

EODG _a(i, j) is differential image D _agradient energy in pixel (i, j) neighborhood;

EODG _b(i, j) is differential image D _bgradient energy in pixel (i, j) neighborhood;

i＝1,2,3,…,M；j＝1,2,3,…,N；

H (i, j) is that matrix H i is capable, the element of j row;

(7) build fused images F,

gray level image after being merged

$F(i,j)=\left\{\begin{array}{cc}{I}_A(i,j)& H(i,j)=1\\ I_B(i,j)& H(i,j)=0\end{array}\right.$ (formula 2)

In (formula 2):

The gray-scale value that F (i, j) locates for the gray level image F pixel (i, j) after merging;

I _a(i, j) is gray level image I before merging _athe gray-scale value located of pixel (i, j);

I _b(i, j) is gray level image I before merging _bthe gray-scale value located of pixel (i, j).

2. multi-focus image fusing method as claimed in claim 1, is characterized in that, the eigenmatrix building is corroded to expansive working and process, and utilize the eigenmatrix after processing to build fused images in step (6).

Images