CN103034986A - Night vision image enhancement method based on exposure fusion - Google Patents

Abstract

The invention relates to a night vision image enhancement method based on exposure fusion. The night vision image enhancement method includes a first step of inputting an image, a second step of carrying out image histogram equalization processing to the image, a third step of carrying out color space conversion to the image after the equalization processing, a fourth step of changing illumination coefficient, generating m virtual exposure images, and then carrying out Laplacian pyramid decomposition, a fifth step of generating m weighted graphs, carrying out normalization processing, and then carrying out Gaussian pyramid decomposition, a sixth step of carrying out layered fusion to the results obtained through decomposition of the fourth step and the fifth step, and a seventh step of carrying out n-tier reconstruction to the obtained Laplacian pyramid to obtain an enhanced night vision image. The night vision image enhancement method achieves the purpose of enhancing details of light places and dark places, at the same time, effectively solves the problem of illumination compensation of the night vision image, and improves visual effects of the image.

Description

A kind of night vision image Enhancement Method that merges based on exposure

Technical field

The present invention relates to technical field of image processing, be specifically related to gather under the environment a kind of night the Enhancement Method of image.

Background technology

Night vision image is a kind of low-light (level) image, the characteristics such as have that tonal range is narrower, the spatial coherence of neighbor is high, grey scale change is not obvious, this so that the information such as the object in the image, background, details, noise be included in the narrower tonal range.Therefore, the source images that obtains must be converted to a kind of form that is more suitable for eye-observation and Computer Analysis processing.

The purpose that night vision image strengthens is to improve the contrast of image, reappears the picture rich in detail of taking under the perfect light source condition.Algorithm for image enhancement is divided two large classes at present: the overall situation strengthens and local enhancement. and it is to reach the purpose that contrast strengthens according to certain rule by changing overall brightness that the overall situation strengthens, and typical algorithm has histogram equalization, nonlinear transformation etc.Histogram equalization is conducive to the raising of picture contrast, but because the method is synthesized a new gray level with a plurality of gray levels in the original image, therefore, some details in the original image might be lost such as texture information; Some image, such as histogram the peak is arranged, the after treatment factitious undue enhancing of contrast. nonlinear transformation such as log-transformation, low gray level in the image partly can be stretched, and high grade grey level is partly carried out dynamic compression, can effectively carry out illumination compensation to image, but lose easily some marginal informations.Generally speaking, above-mentioned algorithm is more satisfactory to the low situation effect of overall contrast, weak effect for local soft image processing. the local enhancement of image can be realized the enhancing processing in any situation theoretically, effect is better than overall situation enhancing algorithm on the local soft image processing, but Local Enhanced Operator is realized difficult, is difficult to obtain to be fit to the Local Enhanced Operator of various images in the reality.

The exposure fusion refers to become a high-quality combination picture with one group about the different image sequences fusion of the depth of exposure of Same Scene.In high dynamic scene, the photo of taking out with digital camera is under-exposure or overexposure often, utilizes this technology can generate high dynamic range images.

Existing exposure integration technology generally needs the Same Scene exposure repeatedly, rebuild the image of sharpening with several exposure images, the present invention adopts another kind of novel manner, at first generate some virtual exposure images with algorithm, adopt again the fusion that exposes of the form of pyramid decomposition, and be applied to night vision image.

Summary of the invention

The object of the present invention is to provide a kind of night vision image Enhancement Method that merges based on exposure, solve not high, the visual poor problem of the picture contrast that gathers under the environment night.Can be widely used in the image pre-service link in the fields such as supervisory system, intelligent vehicle, target identification.

Concrete technical scheme is as follows:

A kind of night vision image Enhancement Method that merges based on exposure, adopt following steps:

(1) input picture;

(2) image is carried out histogrammic equalization processing;

(3) image after the equalization is carried out color space conversion;

(4) change luminosity coefficient, generate m virtual exposure image, carry out again Laplacian pyramid;

(5) generate m width of cloth weighted graph, and normalized, carry out again gaussian pyramid and decompose;

(6) step (4) and (5) two kinds of results that pyramid decomposition obtains are carried out hierarchical fusion;

(7) laplacian pyramid that obtains is carried out the n layer and rebuild night vision image after being enhanced.

Further, it is used for the image pre-service links such as supervisory system, intelligent vehicle, target identification.

Further, step (1) is specially: input the big or small M of a width of cloth rgb format * N color night vision image I (x, y); Image approximate after step (2) is processed evenly distributes, and has effectively expanded the dynamic range of image; Step (3) is specially: the image after the equalization is transformed into the HSV space by RGB, obtains tone H _Org, saturation degree S _Org, brightness V _OrgThree-component.

Further, step (4) is specially:

(4-1) with luminance component V _OrgImprove respectively n ₁, n ₂... n _mTimes illumination coefficient obtains V ₁, V ₂..., V _m, and carry out HSV to the spatial alternation of RGB, obtain m virtual colored exposure image I ₁, I ₂..., I _m

(4-2) m width of cloth virtual image is carried out n layer Laplacian pyramid

Pyramid is used for the multi-resolution representation of image, and l layer Laplacian pyramid is designated as

, 1≤i≤m.

Further, step (5) is specially:

(5-1) with luminance component V _OrgImprove respectively n ₁, n ₂... n _mTimes illumination coefficient obtains V ₁, V ₂..., V _m, and carry out HSV to the spatial alternation of RGB, obtain m virtual colored exposure image I ₁, I ₂..., I _m

(5-2) according to m virtual exposure image, calculate respectively contrast, saturation degree and three indexs of exposure quality of every width of cloth image, computer approach is specific as follows

A. contrast: the gray-scale map to every width of cloth input picture is used Laplace filter h _L, get the absolute value of wave filter output as the index of weighing contrast, be designated as C _k, 1≤k≤m

C _k=|V _k*h _L|,1≤k≤m

B. saturation degree: calculate the R of each pixel, G, the standard deviation of three Color Channels of B is designated as S as the index of weighing saturation degree _k, 1≤k≤m

$S_k=\sqrt{({(R-V_k)}^2+{(G-V_k)}^2+{(B-V_k)}^2)/3},1\≤k\≤m$

C. the quality of exposing: the original brightness of certain passage of pixel has reflected the exposure quality degree of pixel well.Generally, wish that pixel intensity is too near to 0 (under-exposure) and does not also wish its too close 1 (over-exposed).By weigh the departure degree of pixel intensity and 0.5 with Gaussian curve, concerning multichannel image, use Gaussian curve to act on respectively each Color Channel, then the result is multiplied each other as the tolerance of exposure quality, be designated as E _k, 1≤k≤m

$E_k=\exp (-\frac{{(R_k-0.5)}^2}{2{\mathrm{\σ}}^2})\×\exp (-\frac{{(G_k-0.5)}^2}{2{\mathrm{\σ}}^2})\×\exp (-\frac{{(B_k-0.5)}^2}{2{\mathrm{\σ}}^2}),1\≤k\≤m,\mathrm{\σ}=0.2$

With above three indexs formation weighted graph that multiplies each other:

W _k(x,y)=C _k(x,y)×S _k(x,y)×E _k(x,y)

And carry out normalized

${\hat{W}}_k(x,y)={\left[\underset{i}{\overset{m}{\mathrm{\Σ}}}{W}_i(x,y)\right]}^{-1}{W}_k(x,y)$ ；

(5-3) m width of cloth weighted graph is carried out this pyramid decomposition of n floor height, in the same step of n value, this pyramid decomposition of l floor height is designated as

, 1≤i≤m.

Further, step (6) is specially: the result of two kinds of pyramid decomposition is carried out hierarchical fusion,

$L\left\{R^l(x,y)\right\}=\underset{k=1}{\overset{m}{\mathrm{\Σ}}}G\left\{{\hat{W}}_k^l(x,y)\right\}L\left\{I_k^l(x,y)\right\}$

The l layer Laplacian pyramid of fused images R is designated as L{R ^l, it can regard the weighted mean value of m width of cloth virtual image as.

Further, step (7) is specially: the n layer fused images that obtains carried out inverse Laplace transform obtain fused images R (x, y), R (x, y)=L ^-1(L (R ^l(x, y)).

Compare with present prior art, the present invention changes the exposure image of luminosity coefficient generating virtual with software algorithm, and according to contrast, saturation degree, and three indexs of exposure quality adopt the fusion that exposes of the form of pyramid decomposition, have improved the visual effect of image.Specifically: the method need not the hardware parameters such as camera response curve, can automatically carry out, and has higher real-time; The method reaches the purpose that strengthens where there is light and dark place details, has effectively solved the illumination compensation problem of night vision image simultaneously, has improved the visual effect of image.

Description of drawings

Fig. 1 is system flowchart of the present invention

Embodiment

The below describes the present invention with reference to the accompanying drawings, and it is a kind of preferred embodiment in the numerous embodiments of the present invention.

Embodiment one:

(1) original image I (x, y) is carried out histogram equalization, and will and the result be transformed into the HSV space from rgb space, obtain three components: tone H _Org, saturation degree S _Org, brightness V _Org

(2) with luminance component V _OrgImprove respectively n ₁, n ₂... n _mTimes luminosity coefficient obtains the m width of cloth image V under the different illumination conditions ₁(x, y), V ₂(x, y) ..., V _m(x, y).

(3) by three components: tone H _Org, saturation degree S _Org, brightness V ₁, V ₂..., V _n, the HSV space is converted back rgb space, carry out the reconstruct of m width of cloth coloured image, form m virtual exposure image I _i, 1≤i≤m.

(4) with m virtual exposure image I _iCarry out respectively n layer Laplacian pyramid, l layer Laplacian pyramid is designated as

, 1≤i≤m.

(5) according to contrast, saturation degree, three indexs of exposure quality are calculated the weighted graph W of each width of cloth virtual image _k, 1≤k≤m,

W _k(x,y)=C _k(x,y)×S _k(x,y)×E _k(x,y)

Wherein, C _k, S _kAnd E _kRepresent respectively contrast, saturation degree and exposure quality.

(6) the weighted graph sequence is carried out normalization by following formula, making the weights sum of each pixel is 1, the normalization weighted graph that finally obtains

${\hat{W}}_k(x,y)={\left[\underset{i}{\overset{m}{\mathrm{\Σ}}}{W}_i(x,y)\right]}^{-1}{W}_k(x,y)$

(7) with normalization weighted graph sequence

Carry out this pyramid decomposition of n floor height, this pyramid decomposition of l floor height is designated as

, 1≤i≤m.

(8) m width of cloth virtual image and weighted graph are carried out hierarchical fusion, the amalgamation and expression formula is

$L\left\{R^l(x,y)\right\}=\underset{k=1}{\overset{m}{\mathrm{\Σ}}}G\left\{{\hat{W}}_k^l(x,y)\right\}L\left\{I_k^l(x,y)\right\}$

(9) image after merging is carried out n layer laplacian pyramid and rebuild the night vision image R (x, y) after being enhanced

Embodiment two:

Fig. 1 has provided the process flow diagram of the night vision image Enhancement Method that merges based on exposing of the present invention, and its key step is as follows:

(1) the big or small M of input one width of cloth rgb format * N color night vision image I (x, y);

(2) night vision image is carried out histogram equalization and process, the image approximate after the processing evenly distributes, and has effectively expanded the dynamic range of image.

(3) image after the equalization is transformed into the HSV space by RGB, obtains tone H _Org, saturation degree S _Org, brightness V _OrgThree-component.

(4) with luminance component V _OrgImprove respectively n ₁, n ₂... n _mTimes illumination coefficient obtains V ₁, V ₂..., V _m, and carry out HSV to the spatial alternation of RGB, obtain m virtual colored exposure image I ₁, I ₂..., I _m, purpose is the irradiation light of the varying strength of compensating images, thereby can obtain the detailed information of more dark place or where there is light, for follow-up processing provides abundanter low frequency and high-frequency information.

(5) according to m virtual exposure image, calculate respectively contrast, saturation degree and three indexs of exposure quality of every width of cloth image, computer approach is specific as follows

C. contrast: the gray-scale map to every width of cloth input picture is used Laplace filter h _L, get the absolute value of wave filter output as the index of weighing contrast, be designated as C _k, 1≤k≤m

C _k=|V _k*h _L|,1≤k≤m

D. saturation degree: calculate the R of each pixel, G, the standard deviation of three Color Channels of B is designated as S as the index of weighing saturation degree _k, 1≤k≤m

$S_k=\sqrt{({(R-V_k)}^2+{(G-V_k)}^2+{(B-V_k)}^2)/3},1\≤k\≤m$

C. the quality of exposing: the original brightness of certain passage of pixel has reflected the exposure quality degree of pixel well.Generally, wish that pixel intensity is too near to 0 (under-exposure) and does not also wish its too close 1 (over-exposed).By weigh the departure degree of pixel intensity and 0.5 with Gaussian curve, concerning multichannel image, use Gaussian curve to act on respectively each Color Channel, then the result is multiplied each other as the tolerance of exposure quality, be designated as E _k, 1≤k≤m

$E_k=\exp (-\frac{{(R_k-0.5)}^2}{2{\mathrm{\σ}}^2})\×\exp (-\frac{{(G_k-0.5)}^2}{2{\mathrm{\σ}}^2})\×\exp (-\frac{{(B_k-0.5)}^2}{2{\mathrm{\σ}}^2}),1\≤k\≤m,\mathrm{\σ}=0.2$

With above three indexs formation weighted graph that multiplies each other:

W _k(x,y)=C _k(x,y)×S _k(x,y)×E _k(x,y)

And carry out normalized

${\hat{W}}_k(x,y)={\left[\underset{i}{\overset{m}{\mathrm{\Σ}}}{W}_i(x,y)\right]}^{-1}{W}_k(x,y)$

(6) m width of cloth virtual image is carried out n layer Laplacian pyramid

Pyramid is used for the multi-resolution representation of image, and l layer Laplacian pyramid is designated as

, 1≤i≤m.

(7) m width of cloth weighted graph is carried out this pyramid decomposition of n floor height, in the same step of n value, this pyramid decomposition of l floor height is designated as

, 1≤i≤m.

(8) by following formula the result of two kinds of pyramid decomposition is carried out hierarchical fusion,

$L\left\{R^l(x,y)\right\}=\underset{k=1}{\overset{m}{\mathrm{\Σ}}}G\left\{{\hat{W}}_k^l(x,y)\right\}L\left\{I_k^l(x,y)\right\}$

The l layer Laplacian pyramid of fused images R is designated as

, it can regard the weighted mean value of m width of cloth virtual image as.

(9) laplacian pyramid that obtains is carried out the n layer and rebuild night vision image R after being enhanced.

This method merges the method that realizes a kind of night vision image sharpening based on exposure, and it need not manual intervention can carry out automatically, and preferably real-time is just arranged.Method has not only solved the illumination compensation problem of the low-light (level) images such as night vision, has kept preferably the details of image and has strengthened contrast simultaneously.

The above has carried out exemplary description to the present invention by reference to the accompanying drawings; obviously specific implementation of the present invention is not subjected to the restriction of aforesaid way; as long as the various improvement of having adopted method design of the present invention and technical scheme to carry out; or directly apply to other occasion without improvement, all within protection scope of the present invention.

Claims (7)

1. a night vision image Enhancement Method that merges based on exposure is characterized in that, adopts following steps:

(1) input picture;

(2) image is carried out histogrammic equalization processing;

(3) image after the equalization is carried out color space conversion;

(4) change luminosity coefficient, generate m virtual exposure image, carry out again Laplacian pyramid;

(5) generate m width of cloth weighted graph, and normalized, carry out again gaussian pyramid and decompose;

(6) step (4) and (5) two kinds of results that pyramid decomposition obtains are carried out hierarchical fusion;

(7) laplacian pyramid that obtains is carried out the n layer and rebuild night vision image after being enhanced.

2. the night vision image Enhancement Method that merges based on exposure as claimed in claim 1 is characterized in that, it is used for the image pre-service links such as supervisory system, intelligent vehicle, target identification.

3. the night vision image Enhancement Method that merges based on exposure as claimed in claim 1 or 2 is characterized in that step (1) is specially: input the big or small M of a width of cloth rgb format * N color night vision image I (x, y); Image approximate after step (2) is processed evenly distributes, and has effectively expanded the dynamic range of image; Step (3) is specially: the image after the equalization is transformed into the HSV space by RGB, obtains tone H _Org, saturation degree S _Org, brightness V _OrgThree-component.

4. such as each described night vision image Enhancement Method that merges based on exposure among the claim 1-3, it is characterized in that step (4) is specially:

(4-1) with luminance component V _OrgImprove respectively n ₁, n ₂... n _mTimes illumination coefficient obtains V ₁, V ₂..., V _m, and carry out HSV to the spatial alternation of RGB, obtain m virtual colored exposure image I ₁, I ₂..., I _m

(4-2) m width of cloth virtual image is carried out n layer Laplacian pyramid

Pyramid is used for the multi-resolution representation of image, and l layer Laplacian pyramid is designated as , 1≤i≤m.

5. such as each described night vision image Enhancement Method that merges based on exposure among the claim 1-4, it is characterized in that step (5) is specially:

(5-1) with luminance component V _OrgImprove respectively n ₁, n ₂... n _mTimes illumination coefficient obtains V ₁, V ₂..., V _m, and carry out HSV to the spatial alternation of RGB, obtain m virtual colored exposure image I ₁, I ₂..., I _m

(5-2) according to m virtual exposure image, calculate respectively contrast, saturation degree and three indexs of exposure quality of every width of cloth image, computer approach is specific as follows

A. contrast: the gray-scale map to every width of cloth input picture is used Laplace filter h _L, get the absolute value of wave filter output as the index of weighing contrast, be designated as C _k, 1≤k≤m

C _k=|V _k*?h _L|,1≤k≤m

B. saturation degree: calculate the R of each pixel, G, the standard deviation of three Color Channels of B is designated as S as the index of weighing saturation degree _k, 1≤k≤m

$S_k=\sqrt{({(R-V_k)}^2+{(G-V_k)}^2+{(B-V_k)}^2)/3},1\≤k\≤m$

C. the quality of exposing: the original brightness of certain passage of pixel has reflected the exposure quality degree of pixel well.Generally, wish that pixel intensity is too near to 0 (under-exposure) and does not also wish its too close 1 (over-exposed).By weigh the departure degree of pixel intensity and 0.5 with Gaussian curve, concerning multichannel image, use Gaussian curve to act on respectively each Color Channel, then the result is multiplied each other as the tolerance of exposure quality, be designated as E _k, 1≤k≤m

$E_k=\exp (-\frac{{(R_k-0.5)}^2}{2{\mathrm{\σ}}^2})\×\exp (-\frac{{(G_k-0.5)}^2}{2{\mathrm{\σ}}^2})\×\exp (-\frac{{(B_k-0.5)}^2}{2{\mathrm{\σ}}^2}),1\≤k\≤m,\mathrm{\σ}=0.2$

With above three indexs formation weighted graph that multiplies each other:

And carry out normalized

${\hat{W}}_k(x,y)={\left[\underset{i}{\overset{m}{\mathrm{\Σ}}}{W}_i(x,y)\right]}^{-1}{W}_k(x,y)$ ；

(5-3) m width of cloth weighted graph is carried out this pyramid decomposition of n floor height, in the same step of n value, this pyramid decomposition of l floor height is designated as

, 1≤i≤m.

6. such as each described night vision image Enhancement Method that merges based on exposure among the claim 1-5, it is characterized in that step (6) is specially: the result of two kinds of pyramid decomposition is carried out hierarchical fusion,

$L\left\{R^l(x,y)\right\}=\underset{k=1}{\overset{m}{\mathrm{\Σ}}}G\left\{{\hat{W}}_k^l(x,y)\right\}L\left\{I_k^l(x,y)\right\}$ 。

The l layer Laplacian pyramid of fused images R is designated as

, it can regard the weighted mean value of m width of cloth virtual image as.

7. the night vision image Enhancement Method that merges based on exposure as claimed in claim 4 is characterized in that, further, step (7) is specially: the n layer fused images that obtains carried out inverse Laplace transform obtain fused images R (x, y), R (x, y)=L ^-1(L (R ^l(x, y)).

Images