CN106600572A - Adaptive low-illumination visible image and infrared image fusion method - Google Patents

Abstract

The invention relates to an adaptive low-illumination visible image and infrared image fusion method, and belongs to the field of digital image processing. The method comprises the steps: image preprocessing, NSCT transformation, frequency domain coefficient fusion and an NSCT inversion. The method mainly achieves the multi-scale decomposition of a bright image and an infrared visible image, extracts high-frequency components and low-frequency components, and selects different fusion rules to carry out the frequency domain fusion according to the characteristics of the high-frequency components and low-frequency components. The NSCT inversion part is mainly used for carrying out the multi-scale inversion of the obtained fusion high-frequency and low-frequency components, obtaining the gray scale image after fusion, then carrying out the weighting of the gray scale image after fusion and an original color visible image, and obtaining a color fusion image. The method can effectively keep more detail information of the original image, can improve the contrast and resolution of the fusion image, and can be widely used in the fields of intelligent traffic, video monitoring, medical diagnosis, target detection and national defense security.

Description

A kind of adaptive low-light (level) visible images and infrared image fusion method

Technical field

The invention belongs to digital image processing field.

Background technology

With the fast development of sensor technology, the research of image fusion technology is increasingly becoming the focus neck of current research Domain.Different types of sensor, the usual difference of image information of capturing scenes is larger.In addition, single-sensor is obtained Information content is limited, it is difficult to meet the application demand of people.Image fusion technology, is to obtain two or more sensors Image information merge, produce the new image of a width.New images after fusion have abundant, contrast height of detailed information etc. Advantage, is widely used in the every field such as medical diagnosis, target recognition and tracking, video monitoring, intelligent transportation, national defense safety, It is worth with higher social application.Registration of the determination of fusion criterion, the raising of fused image quality and original image etc. is asked Topic, remains image co-registration art difficult point.

Image fusion technology be broadly divided into based on the fusion method of pixel point analysis, feature based calculate fusion method and Fusion method based on Analysis of Policy Making.Wherein, the fusion method based on pixel point analysis, directly clicks through to original image respective pixel The process of row analysis and synthesis, obtains fusion pixel values, is that image co-registration field is most widely used, simplest method.Based on biography The image interfusion method of system wavelet transformation, because direction Sexual behavior mode is poor, is easily introduced blocking effect, causes fused images contrast It is low, it is unfavorable for the observation and differentiation of human eye.Based on the image interfusion method of traditional contourlet transformation, due to fusion criterion choosing It is improper to take, and causes the problems such as fused images luminance dynamic range is little, detailed information is lacked, and has a strong impact on the function of vision system.

In recent years, the research attention rate more and more higher of image fusion technology.Low-light (level) visible images, often brightness is low, It is unintelligible, it is not easy to the observation and identification of target.At present, the image interfusion method based on pixel point analysis, has not yet been formed unification Mathematical Modeling, and fused image quality need further raising., there is contrast low, bright in traditional images fusion method The problems such as degree dynamic range is little, reservation detailed information is few, and most methods are just for the fusion of gray level image, it is difficult to extensively It is applied to every field.

The content of the invention

The present invention proposes a kind of adaptive low-light (level) visible images and infrared image fusion method, to solve fusion figure Image contrast is low, the problem that luminance dynamic range is little.

The present invention is adopted the technical scheme that, comprised the following steps：

(1), the collection of original low-light (level) visible images and infrared image：

Under low light conditions, original color visible ray is gathered respectively by the infrared camera and Visible Light Camera on head Image and infrared image, its resolution ratio is 640*480；

(2), by the original infrared image for obtaining and color visible image, scale invariant feature Point matching is carried out：

According to original infrared image and the scale invariant feature of color visible image, image registration is carried out, it is ensured that scene In any same position picture collection image in be also at same position；

(3), for the color visible image after registration, using HIS conversion its luminance picture is extracted；According to infrared sensing Device characteristic, strengthens the contrast of infrared image after registration：

After registration, color visible image is 3-D view, and infrared image is two dimensional image, image space dimension Difference, therefore, converted using color space HIS, extract the luminance picture I of color visible image_visible, its computing formula is such as (1) shown in：

Wherein, I_r、I_g、I_bIt is expressed as the three-channel pixel values of RGB of low-light (level) visible images；

Infrared image is carried out into pixel brightness value to negate, is conducive to strengthening the contrast of infrared image；

I_IR=L-I_ir (2)

Wherein, I_IRFor the infrared image after going instead, I_irFor the infrared image after registration, L is the gray level of infrared image, When the pixel of infrared image is 8bit, L=2^8=256；

(4), converted using NSCT, by luminance picture I_visibleWith infrared image I_IRMulti-resolution decomposition is carried out, is respectively obtained Correspondence low frequency component and high fdrequency component：

Including two parts：Not sub sampled pyramid filter and not sub sampled anisotropic filter group, not sub sampled golden word Tower wave filter group is used to realize multi-resolution decomposition process that not sub sampled anisotropic filter group to be used to realize the decomposition of frequency domain direction；

In not sub sampled pyramid filter group, the not sub sampled pyramid filter of kth level can be obtained by following formula：

Not sub sampled pyramid filter also needs to meet Bezout identities：

H₀(z)G₀(z)+H₁(z)G₁(z)=1 (4)

Wherein H₀(z)、G₀Z () is the low pass resolution filter and composite filter of not sub sampled pyramid filter；H₁ (z)、G₁Z () is the high pass resolution filter and composite filter of not sub sampled pyramid filter：

Not sub sampled anisotropic filter group, is made up of single fan-filter, and not sub sampled anisotropic filter is entered Row up-sampling operation, can effectively eliminate aliasing；

According to not sub sampled pyramid and not sub sampled anisotropic filter group, many chis are carried out to luminance picture and infrared image Degree conversion, extracts corresponding high fdrequency componentAnd low frequency componentWherein, J >=d >=1, J represents image Total Decomposition order, total Decomposition order is bigger, then the time of algorithm operation is longer, is the multiple dimensioned and multi-direction of guarantee picture breakdown Characteristic, makes J=2, k=[2,16]；

(5) using different fusion criterions, to high fdrequency componentAnd low frequency componentMelted respectively Close, the low frequency component and high fdrequency component after being merged；

Low frequency component reflects the profile information of original image, and it is the grain details information of image that high fdrequency component is represented, According to the characteristics of high fdrequency component and low frequency component, different criterions are respectively adopted and are merged, the high fdrequency component after being merged And low frequency component；

1) low frequency component fusion

For the corresponding low frequency component that step (4) is obtained, low frequency point after being merged using adaptive threshold fusion criterion Amount：

Wherein,For the low frequency component after fusion, I_THFor luminance threshold, w_thFor weight coefficient,It is respectively bright The corresponding low frequency component of degree image and infrared image；

For luminance picture and the low frequency component of infrared image, using the fusion criterion of adaptive threshold, merged, when When the low frequency component of luminance picture is more than threshold value, select luminance picture low frequency component to be the low frequency component after fusion, work as luminance graph When the low frequency component of picture is less than or equal to threshold value, arithmetic average is carried out to the low frequency component of luminance picture and infrared image, calculating can Low frequency component to after fusion；

2) high fdrequency component fusion

For the corresponding high fdrequency component that step (4) is obtained, using Pulse Coupled Neural Network ignition times big fusion is selected Criterion, high fdrequency component after being merged, Pulse Coupled Neural Network is, by a feedback-type network, to be connected by several neurons Composition, each neuron is mainly made up of receiving portion, modulating part and pulse generator, and high fdrequency component coefficient is trigger pulse The feed back input of coupled neural network, as shown in formula (7)：

Wherein,It is the output of feedback fraction,It is input signal, (i, j) represents location of pixels, and k represents d floor heights The direction number of frequency component, n ' is current iteration number of times；

The coupling part of Pulse Coupled Neural Network, can be obtained by formula (8).

WhereinIt is the output of coupling part, m and n is the scope for connecting neuron, V_LFor normalization coefficient, W_ij,mn For the weight coefficient of other neurons of connection；

Pulse Coupled Neural Network internal arithmetic process, is calculated by formula (9) and formula (10)；

Wherein,It is internal state, β, a_θAnd V_θFor fixed coefficient,It is threshold value；

Every time iterative process is shown below：

Wherein, X represents original luminance picture or infrared image；N is total iterations；Represent total igniting time Number；

By comparing ignition times, the big corresponding high fdrequency component of ignition times is selected as high fdrequency component after fusion：

For luminance picture and the high fdrequency component of infrared image, adopt and taken greatly based on Pulse Coupled Neural Network ignition times Criterion merged, using formula (7), will correspondence high fdrequency component trigger pulse coupled neural network, and according to formula (9)- (12) corresponding ignition times are calculated, by formula (13) by comparing ignition times, select big ignition times corresponding High fdrequency component is used as the high fdrequency component after fusion；

(6), the low frequency component and high fdrequency component after fusion is carried out into NSCT inverse transformations, the gray level image after just being merged I_Gray-F；

For original image is low-light (level) gray scale visible images and infrared image, by multiple dimensioned inverse transformation, obtain Grayscale fusion image, maintains low-light (level) region Infrared Image Information, while the texture information of low-light (level) gray level image is remained, Improve primary visible light gray level image contrast and definition, be easy to human eye more accurately judge and recognize the people in scene or its His object；

(7) color visible image after the fusion for, being obtained by step (6) after gray level image and registration carries out weights and asks With obtain final color fusion image I_F；

Wherein, C represents R, G, B color channel；Represent the primary visible light pixel value of C Color Channels；Represent that C leads to The final fused images pixel value in road, w is weight coefficient, general w=0.5.

Under low light conditions, the average brightness value of color visible image is relatively low, is to ensure that color fusion image retains more The color information of many visible images, can by the grayscale fusion image obtained in step 6 and original color by formula (14) See that light image carries out weights summation, obtain final color fusion image.

The present invention has following beneficial effects：

(1) the present invention be directed to resolution ratio is 640*480 low-light (level)s visible images and infrared image carries out frequency domain fusion, The method for improving new images contrast and definition.

(2) image that visible light sensor is generally obtained has abundant colour information and detailed information, but In low light conditions or other bad weathers (such as:Haze, dust and sand weather) under, many scene informations can be lost, it is impossible to round-the-clock Work.Infrared sensor, by target heat radiation, can capture hiding thermal target, by scene brightness and boisterous Affect less, but the contrast of general infrared image is low, and colourless multimedia message, single-sensor, it is difficult to fully meet reality Engineer applied.The present invention is converted using NSCT, carries out multi-resolution decomposition to original image first, extracts corresponding high fdrequency component And low frequency component.Then, low frequency component is merged by the fusion criterion using adaptive threshold.Using based on pulse coupling Close neutral net ignition times absolute value and take big fusion criterion, high fdrequency component is merged.Again by NSCT inverse transformations and Weights are sued for peace, and obtain new fused images, effectively remove blocking artifact, and the detailed information of prominent scene objects retains more Detailed information and color information, improve fused images contrast and definition, advantageously in the observation and differentiation of human eye, Low-light (level) and inclement weather conditions are can be applicable to, with extensive social application value.

(3) present invention is applicable not only to the fusion of gray level image, applies also for the fusion of coloured image, and original image point Resolution is more than 640*480.

(4) present invention has wide at aspects such as video monitoring, intelligent transportation, medical diagnosis, machine vision and national defense safeties General using value.

Description of the drawings

Fig. 1 is algorithm flow chart in application examples of the present invention；

Fig. 2 is the head schematic diagram of image capturing system in application examples of the present invention；

Fig. 3 (a) is the original low-luminance color visible images of application examples Scene 1 of the present invention；

Fig. 3 (b) is the original infrared image of application examples Scene of the present invention 1；

Fig. 4 (a) is luminance picture of the application examples Scene 1 of the present invention after feature registration；

Fig. 4 (b) is infrared image of the application examples Scene 1 of the present invention after feature registration；

Fig. 5 is the color fusion image after the fusion of application examples Scene 1；

Fig. 6 (a) is the original low-luminance color visible images of application examples Scene 2 of the present invention；

Fig. 6 (b) is the original low-light (level) infrared image of application examples Scene 2 of the present invention；

Fig. 7 (a) is luminance picture of the application examples Scene 2 of the present invention after feature registration；

Fig. 7 (b) is infrared image of the application examples Scene 2 of the present invention after feature registration；

Fig. 8 is the color fusion image after application examples Scene 2 of the present invention merges.

Specific embodiment

Comprise the following steps：

(1), the collection of original low-light (level) visible images and infrared image：

As shown in Fig. 2 for the clouds terrace system of IMAQ of the present invention, under low light conditions, by infrared on head Camera and Visible Light Camera gather respectively original color visible images and infrared image, and its resolution ratio is 640*480；

(2), by the original infrared image for obtaining and color visible image, scale invariant feature Point matching is carried out：

Because camera position is different, lens focus are different and other external environments impacts, same position in scene As being in diverse location in the color visible image and infrared image of collection in step (1), therefore, according to original infrared figure The scale invariant feature of picture and color visible image, carries out image registration, it is ensured that the picture of any same position in scene exists Same position is also in the image of collection；

(3), for the color visible image after registration, using HIS conversion its luminance picture is extracted；According to infrared sensing Device characteristic, strengthens the contrast of infrared image after registration：

After registration, color visible image is 3-D view, and infrared image is two dimensional image, image space dimension Difference, can cause fusion error, therefore, converted using color space HIS, extract the luminance picture of color visible image I_visible, its computing formula is as shown in (1)：

Wherein, I_r、I_g、I_bIt is expressed as the three-channel pixel values of RGB of low-light (level) visible images；

Infrared sensor is imaged by the infra-red radiation of object in scene, and often object (such as people) is in the picture Brightness value is higher, but under the conditions of night low-illumination, the object that eye-observation is arrived is often low-light level.Therefore, will be infrared Image carries out pixel brightness value and negates, and is conducive to strengthening the contrast of infrared image.

I_IR=L-I_ir (2)

Wherein, I_IRFor the infrared image after going instead, I_irFor the infrared image after registration, L is the gray level of infrared image, When the pixel of infrared image is 8bit, L=2^8=256；

(4), converted using NSCT, by luminance picture I_visibleWith infrared image I_IRMulti-resolution decomposition is carried out, is respectively obtained Correspondence low frequency component and high fdrequency component：

The characteristics of NSCT conversion not only has contourlet transformation multiresolution, localization, multidirectional, also with flat Motion immovability, can eliminate Gibbs phenomenons, including two parts：Not sub sampled pyramid filter and not sub sampled direction are filtered Ripple device group, not sub sampled pyramid filter group is used to realize multi-resolution decomposition process.Not sub sampled anisotropic filter group is used for Realize the decomposition of frequency domain direction；

In not sub sampled pyramid filter group, the not sub sampled pyramid filter of kth level can be obtained by following formula：

Not sub sampled pyramid filter also needs to meet Bezout identities：

H₀(z)G₀(z)+H₁(z)G₁(z)=1 (4)

Wherein H₀(z)、G₀Z () is the low pass resolution filter and composite filter of not sub sampled pyramid filter；H₁ (z)、G₁Z () is the high pass resolution filter and composite filter of not sub sampled pyramid filter：

Not sub sampled anisotropic filter group, is made up of single fan-filter, is that the direction for realizing high fdrequency component is put down Motion immovability, eliminates sampling element, and on pyramid high-subband, the directional response on relatively low and upper frequency easily draws Aliasing is sent out, up-sampling operation is carried out to not sub sampled anisotropic filter, can effectively eliminate aliasing.

According to not sub sampled pyramid and not sub sampled anisotropic filter group, many chis are carried out to luminance picture and infrared image Degree conversion, extracts corresponding high fdrequency componentAnd low frequency componentWherein, J >=d >=1, J represents image Total Decomposition order.Total Decomposition order is bigger, then the time of algorithm operation is longer, is the multiple dimensioned and multi-direction of guarantee picture breakdown Characteristic, makes J=2, k=[2,16]；

(5) using different fusion criterions, to high fdrequency componentAnd low frequency componentMelted respectively Close, the low frequency component and high fdrequency component after being merged；

Low frequency component reflects the profile information of original image, and it is the grain details information of image that high fdrequency component is represented, According to the characteristics of high fdrequency component and low frequency component, different criterions are respectively adopted and are merged, the high fdrequency component after being merged And low frequency component；

1) low frequency component fusion

For the corresponding low frequency component that step (4) is obtained, low frequency point after being merged using adaptive threshold fusion criterion Amount：

Wherein,For the low frequency component after fusion, I_THFor luminance threshold, w_thFor weight coefficient,It is respectively bright The corresponding low frequency component of degree image and infrared image；

For luminance picture and the low frequency component of infrared image, using the fusion criterion of adaptive threshold, merged, it is low Illumination visible images, ensemble average brightness value is relatively low, wherein, high luminance pixel point is most of to derive from bias light, such as：Automobile Light, street lamp or other lighting apparatus etc..By experiment, the maximum of correspondence low frequency component difference 0.13% derives from background Light, by formula (5) threshold value, takes w=0.75 in example.From formula (6), when the low frequency component of luminance picture is more than During threshold value, luminance picture low frequency component is selected to be the low frequency component after fusion, when the low frequency component of luminance picture is less than or equal to threshold During value, arithmetic average, the low frequency component that calculating can be arrived after fusion are carried out to the low frequency component of luminance picture and infrared image；

2) high fdrequency component fusion

For the corresponding high fdrequency component that step (4) is obtained, using Pulse Coupled Neural Network ignition times big fusion is selected Criterion, high fdrequency component after being merged, Pulse Coupled Neural Network is, by a feedback-type network, to be connected by several neurons Composition, each neuron is mainly made up of receiving portion, modulating part and pulse generator, and high fdrequency component coefficient is trigger pulse The feed back input of coupled neural network, as shown in formula (7)：

Wherein,It is the output of feedback fraction,It is input signal, (i, j) represents location of pixels, and k represents d floor heights The direction number of frequency component, n ' is current iteration number of times；

The coupling part of Pulse Coupled Neural Network, can be obtained by formula (8).

WhereinIt is the output of coupling part, m and n is the scope for connecting neuron, V_LFor normalization coefficient, W_ij,mn For the weight coefficient of other neurons of connection；

Pulse Coupled Neural Network internal arithmetic process, is calculated by formula (9) and formula (10)；

Wherein,It is internal state, β, a_θAnd V_θFor fixed coefficient,It is threshold value；

Every time iterative process is shown below：

Wherein, X represents original luminance picture or infrared image；N is total iterations；Represent total igniting time Number；

By comparing ignition times, the big corresponding high fdrequency component of ignition times is selected as high fdrequency component after fusion：

For luminance picture and the high fdrequency component of infrared image, adopt and taken greatly based on Pulse Coupled Neural Network ignition times Criterion merged, using formula (7), will correspondence high fdrequency component trigger pulse coupled neural network, and according to formula (9)- (12) corresponding ignition times are calculated, by formula (13) by comparing ignition times, select big ignition times corresponding High fdrequency component is used as the high fdrequency component after fusion；

(6), the low frequency component and high fdrequency component after fusion is carried out into NSCT inverse transformations, the gray level image after just being merged I_Gray-F；

For original image is low-light (level) gray scale visible images and infrared image, by multiple dimensioned inverse transformation, obtain Grayscale fusion image, maintains low-light (level) region Infrared Image Information, while the texture information of low-light (level) gray level image is remained, Improve primary visible light gray level image contrast and definition, be easy to human eye more accurately judge and recognize the people in scene or its His object；

(7) color visible image after the fusion for, being obtained by step (6) after gray level image and registration carries out weights and asks With obtain final color fusion image I_F；

Wherein, C represents R, G, B color channel；Represent the primary visible light pixel value of C Color Channels；Represent that C leads to The final fused images pixel value in road, w is weight coefficient, general w=0.5.

Under low light conditions, the average brightness value of color visible image is relatively low.To ensure that color fusion image retains more The color information of many visible images, can by the grayscale fusion image obtained in step 6 and original color by formula (14) See that light image carries out weights summation, obtain final color fusion image.

Combine concrete application example and accompanying drawing below to further illustrate the present invention.

(1), the collection of original low-light (level) visible images and infrared image.

Under low-light (level) environment, by the infrared camera and Visible Light Camera of Fig. 2 clouds terrace systems, gathering original low-light (level) can See light image and infrared image, its resolution ratio is 640*480.Such as Fig. 2；

(2), by the original infrared image for obtaining and color visible image, scale invariant feature Point matching is carried out；

(3) for the color visible image after registration, using HIS conversion its luminance picture is extracted；According to infrared sensing Device characteristic, strengthens the contrast of infrared image after registration；

(4) converted using NSCT, by luminance picture I_visibleWith infrared image I_IRMulti-resolution decomposition is carried out, it is right to respectively obtain Answer low frequency component and high fdrequency component；

(5) using different fusion criterions, to high fdrequency componentAnd low frequency componentMelted respectively Close, the low frequency component and high fdrequency component after being merged；

Low frequency component reflects the profile information of original image, and it is the grain details information of image that high fdrequency component is represented, According to the characteristics of high fdrequency component and low frequency component, different criterions are respectively adopted and are merged, the high fdrequency component after being merged And low frequency component；

For luminance picture and the corresponding low frequency component of infrared image, using the fusion criterion of adaptive threshold, melted Close, low-light (level) visible images, ensemble average brightness value is relatively low, can be obtained by formula (5) and formula (6), it is low when luminance picture When frequency component is more than threshold value, luminance picture low frequency component is selected to be the low frequency component after fusion, when the low frequency component of luminance picture During less than or equal to threshold value, arithmetic average is carried out to the low frequency component of luminance picture and infrared image, calculating can arrive low after fusion Frequency component；

For luminance picture and the corresponding high fdrequency component of infrared image, using based on Pulse Coupled Neural Network ignition times Take big criterion to be merged, using formula (7), by correspondence high fdrequency component trigger pulse coupled neural network, and according to formula (8)-(12) are calculated corresponding ignition times, by formula (13) by comparing ignition times, select big ignition times pair The high fdrequency component answered is used as the high fdrequency component after fusion；

(6) low frequency component and high fdrequency component after fusion is carried out into NSCT inverse transformations, the gray level image after just being merged I_Gray-F；、

(7) gray level image and the color visible image after registration carry out weights summation after the fusion for obtaining step (6), Obtain final color fusion image I_F；

Under low light conditions, the average brightness value of color visible image is relatively low, and dynamic range is little, and scene objects information is not Clearly, as shown in Fig. 3 (a) and Fig. 6 (a), it is difficult to observe the pedestrian in scene, the profile of vehicle, trunk etc.；And pass through scene The infrared image of the infra-red radiation collection of target, can be clearly observable the target information that visible light sensor cannot be seen, but Color information is a lack of, shown in such as Fig. 3 (b) and Fig. 6 (b)；

As shown in Figure 4, fused images had both effectively remained the color of street lamp in visible images, automobile tail light and sky areas Multimedia message, remains profile (such as wheel portion), pedestrian, the support bar grain details letter of street lamp of car in infrared image again Breath, as shown in fig. 6, fused images had both effectively remained the color informations such as light, street lamp, the building of automobile, remains red again The grain details information such as pedestrian, tree trunk in outer image；Therefore, by the method for the present invention, to infrared image and color visible Image carries out effective integration, not only can remain the color information of visible images, can also obtain infrared image scene mesh Target grain details information, can be widely applied to the fields such as video monitoring, intelligent transportation, national defense safety.

The preferred embodiment of the present invention is the foregoing is only, protection scope of the present invention is not limited in above-mentioned embodiment party Formula, the technical scheme of every principle for belonging to the present invention belongs to the protection domain of present aspect, for those skilled in the art For, some improvements and modifications for carrying out under the premise of not departing from the present invention, these improvements and modifications also should be regarded as the present invention Protection domain.

Claims (6)

1. a kind of adaptive low-light (level) visible images and infrared image fusion method, it is characterised in that comprise the following steps：

(1), the collection of original low-light (level) visible images and infrared image：

Under low light conditions, original color visible images are gathered respectively by the infrared camera and Visible Light Camera on head And infrared image,

(2), by the original infrared image for obtaining and color visible image, scale invariant feature Point matching is carried out：

According to original infrared image and the scale invariant feature of color visible image, image registration is carried out, it is ensured that in scene Arbitrarily the picture of same position is also at same position in the image of collection；

(3), for the color visible image after registration, using HIS conversion its luminance picture is extracted；It is special according to infrared sensor Property, strengthen the contrast of infrared image after registration：

(4), converted using NSCT, by luminance picture I_visibleWith infrared image I_IRMulti-resolution decomposition is carried out, correspondence is respectively obtained Low frequency component and high fdrequency component：

Including two parts：Not sub sampled pyramid filter and not sub sampled anisotropic filter group, not sub sampled pyramid filter Ripple device group is used to realize multi-resolution decomposition process that not sub sampled anisotropic filter group to be used to realize the decomposition of frequency domain direction；

(5) using different fusion criterions, to high fdrequency componentAnd low frequency componentMerged respectively, Low frequency component and high fdrequency component after being merged；

Low frequency component reflects the profile information of original image, and it is the grain details information of image that high fdrequency component is represented, according to The characteristics of high fdrequency component and low frequency component, different criterions are respectively adopted and are merged, high fdrequency component after being merged and low Frequency component；

(6), the low frequency component and high fdrequency component after fusion is carried out into NSCT inverse transformations, the gray level image after just being merged I_Gray-F；

For original image is low-light (level) gray scale visible images and infrared image, by multiple dimensioned inverse transformation, the gray scale for obtaining Fused images, maintain low-light (level) region Infrared Image Information, while remaining the texture information of low-light (level) gray level image, improve The contrast and definition of primary visible light gray level image, is easy to human eye more accurately to judge and recognize the people in scene or other mesh Mark thing；

(7) color visible image after the fusion for, being obtained by step (6) after gray level image and registration carries out weights summation, obtains To final color fusion image I_F；

$I_F^C=I_{Gray-F}+w\×I_{Color-F}^C---\left(14\right)$

Wherein, C represents R, G, B color channel；Represent the primary visible light pixel value of C Color Channels；Represent C-channel most Whole fused images pixel value, w is weight coefficient；

Under low light conditions, the average brightness value of color visible image is relatively low, is to ensure that color fusion image retains more The color information of visible images, by formula (14) by the grayscale fusion image obtained in step 6 and original color visible ray Image carries out weights summation, obtains final color fusion image.

2. a kind of adaptive low-light (level) visible images according to claim 1 and infrared image fusion method, it is special Levy and be：The resolution ratio of collection original color visible images and infrared image is 640*480 in step (1).

3. a kind of adaptive low-light (level) visible images according to claim 1 and infrared image fusion method, it is special Levy and be：In step (3), after registration, color visible image is 3-D view, and infrared image is two dimensional image, image The difference of space dimensionality, therefore, converted using color space HIS, extract the luminance picture I of color visible image_visible, its Computing formula is as shown in (1)：

$L_{visible}=\frac{I_r+I_g+I_b}{3}---\left(1\right)$

Wherein, I_r、I_g、I_bIt is expressed as the three-channel pixel values of RGB of low-light (level) visible images；

Infrared image is carried out into pixel brightness value to negate, is conducive to strengthening the contrast of infrared image；

I_IR=L-I_ir (2)

Wherein, I_IRFor the infrared image after going instead, I_irFor the infrared image after registration, L is the gray level of infrared image, when infrared When the pixel of image is 8bit, L=2^8=256.

4. a kind of adaptive low-light (level) visible images according to claim 1 and infrared image fusion method, it is special Levy and be, in step (4)：

In not sub sampled pyramid filter group, the not sub sampled pyramid filter of kth level can be obtained by following formula：

$H_s^{eq}\left(z\right)=\left\{\begin{array}{cc}{H}_1\left(z^{2^{s-1}}\right)\underset{j=0}{\overset{s-2}{\Π}}{H}_0\left(z^{2^j}\right)& 2^k>s\≥1\\ \underset{j=0}{\overset{s-2}{\Π}}{H}_0\left(z^{2^j}\right)& s=2^k\end{array}\right.---\left(3\right)$

Not sub sampled pyramid filter also needs to meet Bezout identities：

H₀(z)G₀(z)+H₁(z)G₁(z)=1 (4)

Wherein H₀(z)、G₀Z () is the low pass resolution filter and composite filter of not sub sampled pyramid filter；H₁(z)、G₁ Z () is the high pass resolution filter and composite filter of not sub sampled pyramid filter：

Not sub sampled anisotropic filter group, is made up of single fan-filter, and not sub sampled anisotropic filter is carried out Sampling operation, can effectively eliminate aliasing.

According to not sub sampled pyramid and not sub sampled anisotropic filter group, multiple dimensioned change is carried out to luminance picture and infrared image Change, extract corresponding high fdrequency componentAnd low frequency componentWherein, J >=d >=1, J represents the total score of image The solution number of plies.Total Decomposition order is bigger, then the time of algorithm operation is longer, is the multiple dimensioned and multi-direction spy for ensureing picture breakdown Property, make J=2, k=[2,16].

5. a kind of adaptive low-light (level) visible images according to claim 1 and infrared image fusion method, it is special Levy and be, step (5) low frequency component fusion method is as follows：

For the corresponding low frequency component that step (4) is obtained, low frequency component after being merged using adaptive threshold fusion criterion：

$I_{TH}=w_{th}\×max(I_{Visible}^J-I_{IR}^J)---\left(5\right)$

$I_F^{low}=\left\{\begin{array}{cc}{I}_{Visible}^J& I_{Visible}^J\≥I_{TH}\\ (I_{IR}^J+I_{Visible}^J)/2& otherwise\end{array}\right.---\left(6\right)$

Wherein,For the low frequency component after fusion, I_THFor luminance threshold, w_thFor weight coefficient,Respectively luminance graph The corresponding low frequency component of picture and infrared image；

For luminance picture and the low frequency component of infrared image, using the fusion criterion of adaptive threshold, merged, worked as brightness When the low frequency component of image is more than threshold value, luminance picture low frequency component is selected to be the low frequency component after fusion, when luminance picture When low frequency component is less than or equal to threshold value, arithmetic average is carried out to the low frequency component of luminance picture and infrared image, calculating can be arrived melts Low frequency component after conjunction.

6. a kind of adaptive low-light (level) visible images according to claim 1 and infrared image fusion method, it is special Levy and be, step (5) high fdrequency component fusion method is as follows：

For the corresponding high fdrequency component that step (4) is obtained, big fusion criterion is selected using Pulse Coupled Neural Network ignition times, High fdrequency component after being merged, Pulse Coupled Neural Network is, by a feedback-type network, to be made up of several neuron connections, Each neuron is mainly made up of receiving portion, modulating part and pulse generator, and high fdrequency component coefficient is trigger pulse coupling The feed back input of neutral net, as shown in formula (7)：

$F_{i,j}^{d,k}\left(n^{\′}\right)=I_{i,j}^{d,k}---\left(7\right)$

Wherein,It is the output of feedback fraction,It is input signal, (i, j) represents location of pixels, and k represents d floor height frequency divisions The direction number of amount, n ' is current iteration number of times；

The coupling part of Pulse Coupled Neural Network, can be obtained by formula (8).

$L_{i,j}^{d,k}\left(n^{\′}\right)=\exp (-a_L)L_{i,j}^{d,k}\left(n^{\′}\right)+V_L\underset{m,n}{\Σ}{W}_{ij,mn}{Y}_{ij,mn}^{d,k}(n^{\′}-1)---\left(8\right)$

WhereinIt is the output of coupling part, m and n is the scope for connecting neuron, V_LFor normalization coefficient, W_ij,mnFor even The weight coefficient of other neurons for connecing；

Pulse Coupled Neural Network internal arithmetic process, is calculated by formula (9) and formula (10)；

$U_{i,j}^{d,k}\left(n^{\′}\right)=E_{i,j}^{d,k}\left(n^{\′}\right)(1+{\mathrm{\βL}}_{i,j}^{d,k}(n^{\′}\left)\right)---\left(9\right)$

${\mathrm{\θ}}_{i,j}^{d,k}\left(n^{\′}\right)=\exp (-a_{\mathrm{\θ}}){\mathrm{\θ}}_{i,j}^{d,k}(n^{\′}-1)+V_{\mathrm{\θ}}{Y}_{i,j}^{d,k}(n^{\′}-1)---\left(10\right)$

Wherein,It is internal state, β, a_θAnd V_θFor fixed coefficient,It is threshold value；

Every time iterative process is shown below：

$Y_{i,j}^{d,k}\left(n^{\′}\right)=\left\{\begin{array}{cc}1& U_{i,j}^{d,k}\left(n^{\′}\right)>{\mathrm{\θ}}_{i,j}^{d,k}\left(n^{\′}\right)\\ 0& otherwise\end{array}\right.---\left(11\right)$

$\begin{array}{cccc}{T}_{X,ij}^{d,k}=\underset{n^{\′}=1}{\overset{N}{\Σ}}{Y}_{i,j}^{d,k}\left(n^{\′}\right)& X=Visible& or& IR\end{array}---\left(12\right)$

Wherein, X represents original luminance picture or infrared image；N is total iterations；Represent total ignition times；

By comparing ignition times, the big corresponding high fdrequency component of ignition times is selected as high fdrequency component after fusion：

$I_F^{high}=\left\{\begin{array}{cc}{I}_{Visible}^{d,k}& T_{Visible,ij}^{d,k}\≥T_{IR,ij}^{d,k}\\ I_{IR}^{d,k}& otherwise\end{array}\right.---\left(13\right)$

For luminance picture and the high fdrequency component of infrared image, adopt and take big standard based on Pulse Coupled Neural Network ignition times Then merged, using formula (7), by correspondence high fdrequency component trigger pulse coupled neural network, and according to formula (9)-(12) Corresponding ignition times are calculated, by formula (13) by comparing ignition times, the big corresponding high frequency of ignition times is selected Component is used as the high fdrequency component after fusion.