CN1822046A - Infrared and visible light image fusion method based on regional property fuzzy - Google Patents

Abstract

This invention relates to a method for interfusing fuzzy images based on region characters including the following steps: 1, applying small wave frames to carry out multi-dimension resolutions to being interfused images to get a series of high frequency components and a lowest component, 2, carrying out K average clustering to the low frequency part of the infrared images and dividing it to three kinds, 3, expressing the three kinds to an important target region, a second important region and a background region to make a decision to the interfusion based on the fuzzy region character and measurement target of the HF part of a sensor multi-image, 4, sending the final multi-resolution images into a filter made up of a same small wave basic function to be filtered and summing up the filtered image signals, lowering the transformation layer number of the small wave frame by one, carrying out a process of the fall-sample to the filter then to transform the next layer inversely and repeating the process till finishing the inverse transformation to the entire wave frames to get a final fused image.

Description

Infrared and visible light image fusion method based on regional property fuzzy

Technical field

The present invention relates to image processing techniques, particularly relate to a kind of multi-sensor image fusion method based on regional property fuzzy, be the image interfusion method of multiple dimensioned, fuzzy theory and image region segmentation combination in the information fusion field, in systems such as optical imagery, targeted surveillance, safety inspection, all can be widely used.

Background technology

In recent years, the multi-sensor image integration technology has obtained in fields such as machine vision, remote sensing, medical science, military affairs paying close attention to widely.Multi-sensor image merges the view data about same target or scene that is meant a plurality of sensor acquisition, carries out suitable overall treatment according to certain algorithm, and it is new to produce a width of cloth, satisfies the new images of certain demand.Because not only have redundancy but also have complementarity between the image that multisensor obtains, image co-registration can obtain the image that a width of cloth is more suitable for human eye and machine recognition by these images to Same Scene of suitable fusion.

At present representative image interfusion method method based on the turriform conversion is arranged, based on the method for wavelet transform etc.The process that multi-scale image merges be at first with images after registration through multiple dimensioned decomposition, decomposition method comprises methods such as Laplce, gradient pyramid and wavelet decomposition; Every layer of feature of seeing image at this yardstick or wave band as of decomposing the back image, the energy norm that reacts according to these features is weighted average or selection, to reach the purpose of fusion.Because each pixel is to belong to some zones in the image, in order to guarantee the consistance of pixel, be necessary to develop the multi-scale image fusion method of a cover, more typically have based on the region Fusion of small echo with based on the region Fusion of Laplace transform based on the zone.In these two kinds of methods, the conspicuousness in zone is estimated the extraction carried out on each layer on multiple dimensioned, just ask for each zone multiple dimensioned coefficient absolute value and estimate as conspicuousness, fusion process is to estimate according to this, select the corresponding multiple dimensioned coefficient in that bigger zone, fusion results obtains by multiple dimensioned inverse transformation.Although the method difference of the method for their multi-scale transform and image segmentation to choosing of fusion criterion, and merges in the design estimate similarly substantially, fusion structure is similar.So caused problem of their common existence: regional conforming problem.Because estimating on each frequency range of multiple dimensioned expansion back, the conspicuousness in zone asks for, caused the selection possibility on each frequency range inconsistent, promptly, not all frequency ranges of selecting same width of cloth image respective regions, thereby regional inconsistency occurred, the contrast of target area and regional consistance all descend to some extent.

Summary of the invention

At the defective that exists in the above-mentioned prior art, technical matters to be solved by this invention provide a kind of can be in fuzzy space processing region merge decision problem, can solve the regional consistency problem in the multiresolution transform domain, with the picture quality after improve merging, reach the infrared and visible light image fusion method based on regional property fuzzy of desirable practical function.

In order to solve the problems of the technologies described above, a kind of infrared and visible light image fusion method based on regional property fuzzy provided by the present invention is characterized in that, comprises following concrete steps:

1) adopt wavelet frame will treat that fused images carries out multiple dimensioned decomposition: two wave filters original image signal being imported the wavelet basis function structure decompose, obtain a high fdrequency component and a low frequency component, then these two wave filters are carried out rising sampling processing, low frequency component after decomposing is sent into the two-layer decomposition of carrying out wavelet frame in the wave filter after rising sampling processing as input signal, obtain another group of high fdrequency component and low frequency component, wave filter carries out rising sampling processing again simultaneously, with the low frequency component that obtains after the two-layer decomposition once more as the input signal of wave filter, can further obtain the higher decomposition number of plies, finally obtain a series of high fdrequency components and lowest frequency component of wavelet frame;

2) the infrared image low frequency part is carried out the K mean cluster and be divided into three classes: the K mean algorithm is divided into three classes with the infrared image low frequency part, the all corresponding Euclidean distance of this three class with cluster centre, determine that according to Euclidean distance two norms of distance are objective function, utilize simple optimizing algorithm to make this objective function minimum, thereby draw the result that infrared Image Segmentation becomes three classes;

3) low frequency component of infrared sensor image is partly carried out after cluster segmentation becomes three classes, be expressed as and be important goal zone, inferior important area and background area; Measurement index according to fuzzy region feature and multi-sensor image HFS obtains merging decision-making;

4) the wavelet frame inverse transformation gets fusion results to the end: the multi-resolution image that will obtain is at last sent into one and is sued for peace by filtering in the wave filter of wavelet basis function structure and to filtered picture signal equally, one deck is fallen in the wavelet frame transform number of plies, to construct wave filter carries out once remaking down one deck inverse transformation after the down-sampled processing, by that analogy, thus the inverse transformation of finishing whole wavelet frame obtains final fused images.

Utilize the infrared and visible light image fusion method based on regional property fuzzy provided by the invention, owing to consider the consistency problem that merge in the zone, differentiate the attribute of All Ranges clear in advance, adopt different strategies for different zones, make information not lose the consistance characteristic with important area feature; On multiresolution method, adopted multiresolution decomposition method based on wavelet frame, this method has been avoided the translation variation characteristic of wavelet transformation, and being directed to some image registration accuracy is not that extra high image has stronger robustness; At the Region Segmentation of infrared image low-frequency range, adopted the K mean algorithm, it is simple that this algorithm has an algorithm, the characteristics that precision is high.Based on above 3 reasons, the fusion performance that can fully improve image based on the image interfusion method of regional property fuzzy of the present invention, picture quality after the fusion is improved significantly, show significant and practical value for the subsequent treatment and the image of various application systems.

Image interfusion method based on fuzzy region feature of the present invention is when guaranteeing that there is good consensus information important area and background area, and inferior important area has significant high-frequency characteristic.Thereby the contradiction of having avoided All Ranges in the simple pursuit image to keep consistency feature and having produced.And apply it in the fusion of infrared and visible images.

Description of drawings

Fig. 1 is the schematic flow sheet of the image interfusion method based on regional property fuzzy of the present invention;

Fig. 2 is the synoptic diagram of subordinate function that the present invention adopts;

Fig. 3 is that different fusion methods are to the infrared and contrast synoptic diagram visible images fusion results.

Embodiment

Below in conjunction with description of drawings embodiments of the invention are described in further detail, but present embodiment is not limited to the present invention, every employing similarity method of the present invention and similar variation thereof all should be listed protection scope of the present invention in.

The flow process of the embodiment of the invention provided a kind of image interfusion method based on regional property fuzzy in Fig. 1, treats that fused images is infrared sensor image A and ccd sensor image B referring to shown in Figure 1.At first respectively infrared sensor image A, ccd sensor image B are carried out wavelet frame (multiresolution) conversion respectively; Respectively picture signal is showed with multiple dimensioned form through coefficient of dissociation; The multiple dimensioned expression of signal has two parts, and a part is the high fdrequency component part of reflected signal sudden change, the just detail section of signal; Another part is the low frequency component part of reflected signal general picture; Adopt the clustering method that the K average is cut apart to cut apart to the infrared image low frequency component, then the low frequency component of cluster result and ccd sensor image B image is handled through regional fuzzy membership function and determined to merge decision-making, the high fdrequency component of infrared sensor image A, ccd sensor image B is handled the common decision-making of determining to merge through measuring index respectively; The high fdrequency component of infrared sensor image A, ccd sensor image B is carried out fusion treatment through merging the definite fusion computing of decision-making respectively then, form jointly with low frequency component simultaneously and merge the back coefficient, just the multi-resolution representation of fusion results image carries out multiresolution wavelet framework inverse transformation at last and just can obtain fused images.

Concrete implementation step is:

1) original image that adopts 9/7 traditional wavelet filter group to treat fusion carries out the multiresolution expansion:

The discrete wavelet framework is the correction to wavelet transform, image is carried out wavelet transform the time, generally need the output of high and low frequency wave filter that will be by wavelet transformation to carry out down-sampled.The difference of wavelet frame just is this part not to be done down-sampled processing, but the filter coefficient of small echo is revised.Concrete conversion process such as formula (1) and the formula (2) of iterating:

$w_{i+1}\left(n\right)=\underset{k}{\mathrm{\Σ}}g(2^i\·k)\·s_i(n-k)---\left(1\right)$

$s_{i+1}\left(n\right)=\underset{k}{\mathrm{\Σ}}h(2^{\′}\·k)\·s_i(n-k)---\left(2\right)$

G (k) is the decomposition high frequency filter of wavelet frame, and h (k) is the decomposition low frequency filter of wavelet frame.

Obtain the high fdrequency component w of wavelet frame by formula (1) and formula (2) ₀, w ₁..., w _NWith lowest frequency component s _NN is the number of plies of wavelet frame transform.

2) the infrared image low frequency part is carried out the K mean cluster and is divided into three classes:

In application and research process to image, people are often interested in some part in the image, and zone specific, that have peculiar property in the general correspondence image of these parts needs these regional separation and Extraction are come out thus.Image segmentation just is meant image is divided into the zone of each tool characteristic and extracts the technology of interesting target.Image segmentation is a substance of image understanding and analysis.Zone in the image is meant that interconnect, as to have consistent " meaningful " attribute pixel set.The attribute of so-called " meaningful " depends on the concrete condition of image to be analyzed, as the statistical property of the neighborhood of the color of image, gray scale, pixel or texture features etc." consistance " requires each zone to have identical or close characteristic attribute.Image partition method generally comprises: thresholding method, cluster segmentation method, statistics split plot design and region growing method and act of union separately.K average partitioning algorithm belongs to a kind of in the cluster segmentation method.

The K mean algorithm is n vector x _jBe divided into c class G _i, and ask the cluster centre of every class, make the objective function of non-similarity (or distance) index reach minimum.When selecting the 1st class G _iMiddle vector x _kWith corresponding cluster centre c _iBetween tolerance when being Euclidean distance, objective function can be defined as:

$J=\underset{i=1}{\overset{c}{\mathrm{\Σ}}}{J}_i=\underset{i=1}{\overset{c}{\mathrm{\Σ}}}\left(\underset{k,x_k\∈G_i}{\mathrm{\Σ}}{\left|\right|x_k-c_i\left|\right|}^2\right),$

Here $J_i=\underset{k,x_k\∈G_i}{\mathrm{\Σ}}{\left|\right|x_k-c_i\left|\right|}^2$ Be class G _iThe internal object function.J _iValue depend on G _iPhysical dimension and c _iThe position.Obviously, the value of j is more little, shows that the cluster effect is good more.

K mean algorithm basic thought:

1. at first picked at random c vector as the center of every class;

The two dimension of 2. establishing U and be a c * n is subordinate to matrix.If the 1st vector x _jBelong to class I, then the element u among the U _IjBe 1; Otherwise this element gets 0.That is:

3. according to u _IjThe value of calculating target function formula (4) is if the difference that it is lower than a given minimum threshold or double value is less than a parameter threshold then out of service.

According to u _IjUpgrade each cluster centre: $c_i=\frac{1}{\left|G_i\right|}\underset{k,x_k\∈G_i}{\mathrm{\Σ}}{x}_k,$ Here $\left|G_i\right|=\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{u}_{\mathrm{ij}},$ Representation class G _iThe number of interior element.Come back to step then 2.

3) low frequency component of infrared sensor image is partly carried out after cluster segmentation becomes three classes, is expressed as and is important goal zone, inferior important area and background area, carry out Fuzzy processing:

When a plurality of sensors carry out imaging to a certain scene, the real scene zone of each sensor image roughly can be divided into three kinds, these three zones can be divided by certain sensor to target susceptibility (as infrared imaging sensor), are respectively abundant inferior important area of the important area, edge or the texture information that comprise target and the background area that comprises background information.By the notion in the above-mentioned fuzzy theory, the complete or collected works in these three kinds of zones have defined a regional fuzzy set A on the real scene U.Owing to carry out the susceptibility of the sensor of area dividing to target, we are referred to as sensor of interest by system, and other imaging sensors are called context sensor.

At first to determine the fusion rule of each element when merging among the regional fuzzy set A.If the zone is divided into important area, illustrate that this regional importance of sensor of interest image is better than the respective regions of context sensor image, the method for fusion is with all multiresolution coefficients in this part zone of sensor of interest image respective regions part as fused image.If inferior important area, illustrate that then multi-sensor image is suitable in the conspicuousness feature of this zone performance, as long as will have the information of edge feature remains as fusion results, there is no need to keep all information of this zone, then in to this zone fusion process, can select that relatively large multiresolution coefficient of conspicuousness as fusion results, final fusion results is degenerated the conforming destruction in this zone.If the zone belongs to the background area, illustrate that the context sensor image is better than the sensor of interest image in this regional importance, similar to the importance zone, the method for fusion is with all multiresolution coefficients in this part zone of context sensor image respective regions part as fused image.

If the element of regional fuzzy set A is respectively A ₁, A ₂, A ₃, represent target importance zone, inferior important area and background area respectively, the provincial characteristics attribute is u (u ∈ U), then μ _A(u) be called the degree of membership of u to regional fuzzy set A, therefore when area attribute was determined, regional convergence strategy should be:

Because the importance of image-region is relative, that is to say, can not judge that this zone is important or unessential according to certain feature of image.The importance in zone is a fuzzy notion, therefore is necessary the importance attribute of image is carried out obfuscation, and fusion process is carried out at fuzzy space.

The present invention adopts normal distribution subordinate function relatively more commonly used, and as shown in Figure 2, this function is defined as:

${\mathrm{\μ}}_{A_j}\left(u\right)=\exp \left[\frac{-{(\mathrm{ME}\left(u\right)-E\left(A_j\right))}^2}{\frac{L_{\max }-L_{\min }}{2}}\right]---\left(6\right)$

Wherein, μ _Aj(u) expression u zone belongs to A _jMembership function; _LmaxAnd L _MinBe respectively the desirable cluster centre of important area and background area of image, E (A ₁)=L _Min, E (A ₃)=L _MaxE (A _j) representation attribute is A _jDesirable cluster centre is established L _Max, L _MinBe maximum, the minimal gray grade of sensor of interest image, $E\left(A_2\right)=\frac{L_{\max }+L_{\min }}{2};$ ME (u) is the actual cluster centre in u zone.The a certain sensor image that is obtained is obtained the fuzzy region feature that a series of fuzzy region degrees of membership are called this image.

A among the figure ₁, A ₂, A ₃Three elements of difference presentation video fuzzy region collection, their respectively corresponding three kinds of desirable fusion results.As ME (u)=A ₁The time presentation video zone u be background, the respective regions that can directly utilize the context sensor image is made as F as fusion results ₁As ME (u)=A ₂The time represent that regional u is time important area, we adopt the image interfusion method based on pixel ^[1], get fusion results F ₂As ME (u)=A ₃The time represent that regional u is an important area, the respective regions that directly adopts the sensor of interest image is as fusion results F ₃

At last, according to each regional characteristic, define the some μ on the degree of membership space of all images pixel _Aj(u) (j=1,2,3).According to these degree of membership points, determine the fusion results of image.

$F=\frac{\underset{i=1}{\overset{3}{\mathrm{\Σ}}}{\mathrm{\μ}}_{A_i}\left(u\right)F_i}{\underset{i=1}{\overset{3}{\mathrm{\Σ}}}{\mathrm{\μ}}_{A_i}\left(u\right)},---\left(7\right)$

F promptly is the multi-resolution representation of fusion results in the formula.It is carried out corresponding multiresolution inverse transformation, can obtain final fusion results.

4) the wavelet frame inverse transformation gets fusion results to the end: the multi-resolution image that will obtain is at last sent into one and is sued for peace by filtering in the wave filter of wavelet basis function structure and to filtered picture signal equally, one deck is fallen in the wavelet frame transform number of plies, to construct wave filter carries out once remaking down one deck inverse transformation after the down-sampled processing, by that analogy, thus the inverse transformation of finishing whole wavelet frame obtains final fused images.

$s_i\left(n\right)=\underset{k}{\mathrm{\Σ}}\tilde{h}(2^i\·n-k)\·s_{i+1}\left(n\right)+\underset{k}{\mathrm{\Σ}}\tilde{g}(2^i\·n-k)\·w_{i+1}\left(n\right)---\left(8\right)$

What formula (8) was represented is the inverse transformation process (signal reconstruction) of wavelet frame. Be the reconstructed high frequency wave filter of wavelet frame,

It is the reconstruct low frequency filter of wavelet frame.Finally obtain fused images, as Fig. 3 e.

Fig. 2, Fig. 3 are seen in enforcement of the present invention, and the fuzzy membership function by Fig. 2 carries out Fuzzy Processing to the low-frequency range of infrared image respectively, and use it for and merge the visible images that infrared image that Fig. 3 a represents and Fig. 3 b represent and merge.What Fig. 3 c represented is the fusion results that adopts wavelet transformation and traditional area blending algorithm method; The fusion results that Fig. 3 d adopts laplacian pyramid conversion and traditional area to merge; Fig. 3 e is the image co-registration result who the present invention is based on based on fuzzy region feature.

Table 1 is the fusion results index evaluation index of visible light/infrared image.As can be seen from the table, when the method that adopts the present invention to propose, merge performance and surpassed traditional wavelet method, the laplacian pyramid method, even surpassed the method for traditional wavelet frame.

Table 1

Image interfusion method	EMI	PMI
			Adopt the method for Zhang.Z to adopt the method for Piella.G based on the FRF method	0.3610 0.4760 0.4462	1.3694 1.4172 1.5037

Claims (1)

1, a kind of image interfusion method based on regional property fuzzy is characterized in that, comprises following concrete steps:

1) adopt wavelet frame will treat that fused images carries out multiple dimensioned decomposition: two wave filters original image signal being imported the wavelet basis function structure decompose, obtain a high fdrequency component and a low frequency component, then these two wave filters are carried out rising sampling processing, low frequency component after decomposing is sent into the two-layer decomposition of carrying out wavelet frame in the wave filter after rising sampling processing as input signal, obtain another group of high fdrequency component and low frequency component, wave filter carries out rising sampling processing again simultaneously, with the low frequency component that obtains after the two-layer decomposition once more as the input signal of wave filter, can further obtain the higher decomposition number of plies, finally obtain a series of high fdrequency components and lowest frequency component of wavelet frame;

2) the infrared image low frequency part is carried out the K mean cluster and be divided into three classes: the K mean algorithm is divided into three classes with the infrared image low frequency part, the all corresponding Euclidean distance of this three class with cluster centre, determine that according to Euclidean distance two norms of distance are objective function, utilize simple optimizing algorithm to make this objective function minimum, thereby draw the result that infrared Image Segmentation becomes three classes;

3) low frequency component of infrared sensor image is partly carried out after cluster segmentation becomes three classes, be expressed as and be important goal zone, inferior important area and background area; Measurement index according to fuzzy region feature and multi-sensor image HFS obtains merging decision-making;

4) the wavelet frame inverse transformation gets fusion results to the end: the multi-resolution image that will obtain is at last sent into one and is sued for peace by filtering in the wave filter of wavelet basis function structure and to filtered picture signal equally, one deck is fallen in the wavelet frame transform number of plies, to construct wave filter carries out once remaking down one deck inverse transformation after the down-sampled processing, by that analogy, thus the inverse transformation of finishing whole wavelet frame obtains final fused images.

Images