CN108492245B - Low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering - Google Patents

Low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering Download PDF

Info

Publication number
CN108492245B
CN108492245B CN201810119668.6A CN201810119668A CN108492245B CN 108492245 B CN108492245 B CN 108492245B CN 201810119668 A CN201810119668 A CN 201810119668A CN 108492245 B CN108492245 B CN 108492245B
Authority
CN
China
Prior art keywords
exposure image
subband
short
image
long
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201810119668.6A
Other languages
Chinese (zh)
Other versions
CN108492245A (en
Inventor
冯华君
王光霞
徐之海
李奇
陈跃庭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN201810119668.6A priority Critical patent/CN108492245B/en
Publication of CN108492245A publication Critical patent/CN108492245A/en
Application granted granted Critical
Publication of CN108492245B publication Critical patent/CN108492245B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Facsimile Image Signal Circuits (AREA)

Abstract

The invention discloses a low-photometric image pair fusion method based on wavelet decomposition and bilateral filtering. The method comprises the steps of shooting under the low-light condition to obtain two images of short exposure and long exposure, selecting the short exposure image as a reference image, adjusting brightness and color by utilizing global histogram matching, and simultaneously registering and aligning the long exposure image to the short exposure image. And performing wavelet decomposition on the two registered images respectively, performing bilateral filtering on a low-frequency subband of the short-exposure image, performing hard threshold filtering on a high-frequency subband to achieve the effect of reducing noise, then calculating a fusion weight map of each layer of the band according to a subband intensity difference value, performing subband reconstruction on each layer of the short-exposure image and each layer of the long-exposure image according to the fusion weight maps, and finally performing wavelet synthesis on the reconstructed subbands to obtain a result image. The method not only maintains the edge sharpness of the short-exposure image, but also maintains the brightness and the color of the long-exposure image, and effectively inhibits noise.

Description

基于小波分解和双边滤波的低光度图像对融合方法Low-light image pair fusion method based on wavelet decomposition and bilateral filtering

技术领域technical field

本发明属于数字图像处理领域，涉及一种基于小波分解和双边滤波的低光度图像对融合方法。The invention belongs to the field of digital image processing, and relates to a low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering.

背景技术Background technique

当在低光度条件下拍照时，由于光线不足，拍摄所得的图像经常包含大量噪声，且色彩暗淡，对比度较低，我们通过延长曝光时间可以获得色彩丰富且噪声较低的图像，但由于曝光时间较长，所摄物体的移动往往引起图像的局部模糊。图像示例如图1所示。我们希望通过适当融合得到一张质量较好的图像，使得结果图像中即保持短曝光图像的边缘锐度，又拥有长曝光图像的色彩与亮度并降低噪声。传统多帧曝光图像的高动态去鬼影融合技术往往需要多张连拍图像，且不考虑去噪问题。当输入图像只有长短曝光两张图像时，这类传统方法由于信息不足，容易出现鬼影及噪声遗留问题。When taking pictures in low light conditions, due to insufficient light, the resulting images often contain a lot of noise, with dull colors and low contrast. We can obtain images with rich colors and low noise by extending the exposure time. Longer, the movement of the photographed object tends to cause local blurring of the image. An example image is shown in Figure 1. We hope to obtain a better quality image through proper fusion, so that the resulting image not only maintains the edge sharpness of the short-exposure image, but also has the color and brightness of the long-exposure image and reduces noise. The traditional high-dynamic de-ghosting fusion technology of multi-frame exposure images often requires multiple continuous shooting images, and does not consider the problem of denoising. When the input image has only two images with long and short exposures, such traditional methods are prone to ghosting and noise due to insufficient information.

发明内容SUMMARY OF THE INVENTION

本发明的目的是通过将小波分解与多尺度双边滤波结合，在抑制噪声和鬼影的同时重建图像，使得结果图像中即保持短曝光图像的边缘锐度，又拥有长曝光图像的色彩与亮度并降低噪声。The purpose of the present invention is to reconstruct the image while suppressing noise and ghost images by combining wavelet decomposition with multi-scale bilateral filtering, so that the resulting image not only maintains the edge sharpness of the short-exposure image, but also has the color and brightness of the long-exposure image. and reduce noise.

为达到以上目的，本发明采用以下技术方案：一种基于小波分解和双边滤波的低光度图像对融合方法，该方法包括以下步骤：In order to achieve the above purpose, the present invention adopts the following technical solutions: a low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering, the method comprises the following steps:

1.对输入图像进行小波分解与多尺度双边滤波，包括以下步骤：1. Perform wavelet decomposition and multi-scale bilateral filtering on the input image, including the following steps:

1-1首先对短曝光图像进行直方图匹配生成图像Ih，然后参照直方图匹配后的短曝光图像对长曝光图像进行配准对齐生成图像Ir；1-1 First, perform histogram matching on the short-exposure image to generate an image Ih , and then register and align the long-exposure image with reference to the short-exposure image after histogram matching to generate an image Ir ;

1-2对步骤1-1处理后的两张图像Ih、Ir分别进行小波分解，生成相应的短曝光图像次带集

Figure BDA0001571704950000011
和长曝光图像次带集
Figure BDA0001571704950000012
1-2 Perform wavelet decomposition on the two images I h and I r processed in step 1-1, respectively, to generate corresponding subband sets of short exposure images
Figure BDA0001571704950000011
and long exposure image subband set
Figure BDA0001571704950000012

1-3估计步骤1-2获得的短曝光图像次带集

Figure BDA0001571704950000013
的噪声方差，公式为：1-3 Estimating the set of short exposure image subbands obtained in step 1-2
Figure BDA0001571704950000013
The noise variance of , the formula is:

Figure BDA0001571704950000014
Figure BDA0001571704950000014

其中，σL为第L层次带的噪声估计方差，median()表示取中值操作，HHL表示小波分解第L层的高频细节层，C1为常数参量，C1的大小决定估计的噪声方差的大小，C1越大得到的去噪图像越平滑，一般取值范围为2到4。Among them, σ L is the noise estimation variance of the L-th layer, median() represents the median operation, HHL represents the high-frequency detail layer of the L -th layer of wavelet decomposition, C 1 is a constant parameter, and the size of C 1 determines the estimated value The size of the noise variance, the larger C 1 is, the smoother the denoised image is, and the general value range is 2 to 4.

1-4为降低短曝光图像中噪声对后续鬼影检测的干扰，对步骤1-2获得的短曝光图像次带集

Figure BDA0001571704950000021
的低频次带进行双边滤波，公式为：1-4 In order to reduce the interference of noise in the short-exposure image to the subsequent ghost detection, the sub-band set of the short-exposure image obtained in step 1-2
Figure BDA0001571704950000021
The low frequency band of , is bilaterally filtered, and the formula is:

Figure BDA0001571704950000022
Figure BDA0001571704950000022

Figure BDA0001571704950000023
Figure BDA0001571704950000023

其中，

Figure BDA0001571704950000024
表示短曝光图像次带集第L层的的低频次带，
Figure BDA0001571704950000025
表示双边滤波后的低频次带，C为归一化函数，σd为滤波窗口参数，σL为第L层次带的噪声估计方差，N表示定义域，x,y,k,l表示像素位置坐标。in,
Figure BDA0001571704950000024
represents the low-frequency subband of the Lth layer of the short-exposure image subband set,
Figure BDA0001571704950000025
represents the low frequency band after bilateral filtering, C is the normalization function, σ d is the filter window parameter, σ L is the noise estimation variance of the L-th level band, N represents the domain of definition, and x, y, k, l represent the pixel position coordinate.

对步骤1-2获得的短曝光图像次带集

Figure BDA0001571704950000026
的高频次带集进行硬阈值滤波，公式为：Subband set of short exposure images obtained in steps 1-2
Figure BDA0001571704950000026
The high-frequency subband set of , is hard-threshold filtered, and the formula is:

Figure BDA0001571704950000027
Figure BDA0001571704950000027

其中，

Figure BDA0001571704950000028
表示步骤1-2获得的短曝光图像的第i个高频次带，
Figure BDA0001571704950000029
表示降噪滤波后的短曝光图像的第i个高频次带，σL为第L层次带的噪声估计方差。in,
Figure BDA0001571704950000028
represents the i-th high-frequency band of the short-exposure image obtained in steps 1-2,
Figure BDA0001571704950000029
represents the i-th high-frequency band of the short-exposure image after noise reduction filtering, and σ L is the noise estimation variance of the L-th level band.

2.利用步骤1-2获得的长曝光图像次带集与步骤1-4去噪处理后的短曝光图像次带集，得到对应次带差值，进而得到与次带对应的融合权重图，包括以下步骤：2. Using the long-exposure image sub-band set obtained in step 1-2 and the short-exposure image sub-band set after denoising in step 1-4 to obtain the corresponding sub-band difference value, and then obtain the fusion weight map corresponding to the sub-band, Include the following steps:

2-1计算步骤1-2得到的长曝光图像次带集与步骤1-4去噪处理后的短曝光图像次带集的相应次带差值的绝对值Di，计算公式为：2-1 Calculate the absolute value D i of the corresponding sub-band difference between the long-exposure image sub-band set obtained in step 1-2 and the corresponding sub-band set of the short-exposure image sub-band set after denoising in step 1-4, and the calculation formula is:

Figure BDA00015717049500000210
Figure BDA00015717049500000210

其中，

Figure BDA00015717049500000211
为步骤1-2得到的长曝光图像次带集，
Figure BDA00015717049500000212
为步骤1-4降噪处理后的短曝光图像次带集，Di为降噪后短曝光图像第i个次带与长曝光图像第i个次带的差值绝对值。in,
Figure BDA00015717049500000211
is the subband set of the long-exposure image obtained in steps 1-2,
Figure BDA00015717049500000212
is the set of sub-bands of the short-exposure image after noise reduction in steps 1-4, and D i is the absolute value of the difference between the i-th sub-band of the short-exposure image and the i-th sub-band of the long-exposure image after noise reduction.

2-2利用各层次带差值的中值估计鬼影检测阈值Ti，公式如下：2-2 The ghost detection threshold T i is estimated by using the median of the difference values at each level, and the formula is as follows:

Ti＝C2*median(Di) (14)T i =C 2 *median(D i ) (14)

其中，Ti为第i个次带的鬼影检测阈值，Di为降噪后短曝光图像第i个次带与长曝光图像第i个次带的差值绝对值，C2为常数参量，C2的大小决定鬼影检测程度，C2越大检测到的鬼影区域越小，一般取值范围为2到5。Among them, T i is the ghost detection threshold of the ith subband, D i is the absolute value of the difference between the ith subband of the short exposure image and the ith subband of the long exposure image after noise reduction, and C 2 is a constant parameter , the size of C 2 determines the degree of ghost detection, the larger the C 2 is, the smaller the detected ghost area, and the general value range is 2 to 5.

2-3利用各层次带差值的绝对值和估计的鬼影检测阈值得到融合权重图，为了实现自然融合，我们利用高斯函数特性构建平滑的权重函数，并利用导向滤波对融合权重图做平滑处理，计算公式如下：2-3 Use the absolute value of the band difference at each level and the estimated ghost detection threshold to obtain the fusion weight map. In order to achieve natural fusion, we use the Gaussian function to construct a smooth weight function, and use the guided filter to smooth the fusion weight map. processing, the calculation formula is as follows:

Figure BDA0001571704950000031
Figure BDA0001571704950000031

其中，Wi为第i个次带的融合权重图，Ti为估计的鬼影检测阈值，Di为降噪后短曝光图像第i个次带与长曝光图像第i个次带的差值绝对值，G()表示导向滤波平滑操作。Among them, Wi is the fusion weight map of the ith subband, T i is the estimated ghost detection threshold, and Di is the difference between the ith subband of the short exposure image and the ith subband of the long exposure image after noise reduction The absolute value of the value, G() represents the guided filter smoothing operation.

3.重建次带集得到结果图像，包括以下步骤：3. Reconstruct the secondary band set to obtain the resulting image, including the following steps:

3-1利用步骤1-2得到的长曝光图像次带集、步骤1-4去噪处理后的短曝光图像次带集、及步骤2-3平滑处理后的融合权重图构建新的次带集，计算公式为：3-1 Construct a new sub-band using the long-exposure image sub-band set obtained in step 1-2, the short-exposure image sub-band set after denoising in step 1-4, and the fusion weight map after smoothing in step 2-3 set, the calculation formula is:

Figure BDA0001571704950000032
Figure BDA0001571704950000032

其中，Fi为构建的新的次带集，

Figure BDA0001571704950000033
为步骤1-2得到的长曝光图像次带集，
Figure BDA0001571704950000034
为步骤1-4降噪处理后的短曝光图像次带集，Wi为第i个次带的融合权重图。Among them, F i is the new subband set constructed,
Figure BDA0001571704950000033
is the subband set of the long-exposure image obtained in steps 1-2,
Figure BDA0001571704950000034
is the sub-band set of the short-exposure image after denoising in steps 1-4, and Wi is the fusion weight map of the i -th sub-band.

3-2利用小波合成得到的次带集，得到最终图像。3-2 Use the subband set obtained by wavelet synthesis to obtain the final image.

本发明的有益效果：针对拍摄场景为低光度的情况，在低光度条件下拍摄获得短曝光和长曝光两张图像，短曝光图像由于曝光不足整体偏暗且存在大量噪声，长曝光图像由于所摄主体运动存在局部模糊。本发明利用拍摄所得的短曝光图像和长曝光图像，通过小波分解图像，运用双边滤波与硬阈值滤波抑制噪声，同时利用次带差值的绝对值估计鬼影检测阈值，进而利用高斯函数特性构建平滑的融合权重图，随后重建次带，使得最终结果图像中既保持了短曝光图像的锐利边缘，又保持长曝光图像的色彩与亮度，同时有效抑制噪声。本发明提高了图像成像质量与视觉效果。The beneficial effects of the present invention are as follows: for the situation where the shooting scene is low light, two images of short exposure and long exposure are obtained by shooting under low light conditions. The short exposure image is dark and has a lot of noise due to insufficient exposure. There is local blurring of subject motion. The invention utilizes the short-exposure image and long-exposure image obtained by shooting, decomposes the image by wavelet, uses bilateral filtering and hard threshold filtering to suppress noise, and at the same time uses the absolute value of the subband difference to estimate the ghost detection threshold, and then uses the Gaussian function characteristic to construct The smooth fusion weight map, followed by reconstruction of the subbands, keeps the sharp edges of the short-exposure image and the color and brightness of the long-exposure image in the final result image, while effectively suppressing noise. The invention improves image imaging quality and visual effect.

附图说明Description of drawings

图1为低光度条件下拍摄得到的图像示例，(a)为短曝光图像，(b)为长曝光图像。Figure 1 is an example of an image captured under low light conditions, (a) is a short exposure image and (b) is a long exposure image.

图2为本发明方法的流程示意图。Figure 2 is a schematic flow chart of the method of the present invention.

图3为全局直方图匹配图像示例，(a)为短曝光图像，(b)为长曝光图像，(c)为全局直方图匹配结果。Figure 3 is an example of a global histogram matching image, (a) is a short exposure image, (b) is a long exposure image, and (c) is the global histogram matching result.

图4为小波次带集某层对应的融合权重图W，(a)-(d)分别是子带LL、LH、HL、HH的权重图。Figure 4 is a fusion weight map W corresponding to a certain layer of the wavelet subband set, (a)-(d) are the weight maps of the subbands LL, LH, HL, and HH, respectively.

图5为实验结果图，(a)为短曝光图像，(b)为本发明方法的最终结果，(c)为长曝光图像。5 is a graph of experimental results, (a) is a short exposure image, (b) is the final result of the method of the present invention, and (c) is a long exposure image.

具体实施方式Detailed ways

以下结合附图和具体实施例对本发明作进一步说明。The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

本发明针对低光度条件下拍照的情况，此时由于光线不足，拍摄所得的图像经常包含大量噪声，且色彩暗淡，对比度较低，通过延长曝光时间可以获得色彩丰富且噪声较低的图像，但由于曝光时间较长，所摄物体的移动经常引起图像局部模糊。图像示例如图1所示。我们希望通过适当融合得到一张质量较好的图像，使得结果图像中即保持短曝光图像的边缘锐度，又拥有长曝光图像的颜色并降低噪声。本发明的流程如附图2所示，主要包括小波分解、多尺度双边滤波、利用次带差值计算权重图以及次带重建等几个步骤。The present invention is aimed at taking pictures under low light conditions. At this time, due to insufficient light, the obtained images often contain a lot of noise, and the colors are dim and the contrast is low. By prolonging the exposure time, images with rich colors and low noise can be obtained, but Due to the long exposure time, the movement of the photographed object often causes the image to be partially blurred. An example image is shown in Figure 1. We hope to obtain a better quality image through proper fusion, so that the resulting image not only maintains the edge sharpness of the short-exposure image, but also has the color of the long-exposure image and reduces noise. The process of the present invention is shown in FIG. 2 , which mainly includes several steps, such as wavelet decomposition, multi-scale bilateral filtering, calculating weight map using sub-band difference value, and sub-band reconstruction.

步骤1.对输入图像进行小波分解与多尺度双边滤波Step 1. Perform wavelet decomposition and multi-scale bilateral filtering on the input image

1-1首先对短曝光图像进行直方图匹配生成图像Ih，直方图匹配在R、G、B三个通道分别进行，图像示例如图3所示。对长曝光图像，我们采取快速角点检测配合RANSAC方法，以直方图匹配后的Ih为参照，对长曝光图像Il进行仿射变换，得到与短曝光图像Is配准的长曝光图像Ir；1-1 First, perform histogram matching on the short-exposure image to generate an image I h . The histogram matching is performed in three channels of R, G, and B, respectively. An example of the image is shown in Figure 3. For the long-exposure image, we adopt the fast corner detection combined with the RANSAC method, and take the histogram-matched Ih as a reference, perform affine transformation on the long-exposure image Il, and obtain a long-exposure image registered with the short-exposure image Is. I r ;

1-2对直方图匹配和配准处理后的输入图像Ih、Ir分别进行小波分解，生成短曝光次带集

Figure BDA0001571704950000041
和长曝光次带集
Figure BDA0001571704950000042
1-2 Perform wavelet decomposition on the input images I h and I r after the histogram matching and registration processing, respectively, to generate a short-exposure subband set
Figure BDA0001571704950000041
and long exposure subband set
Figure BDA0001571704950000042

1-3由于图像集

Figure BDA0001571704950000043
包含噪声，对其低通滤波的次带
Figure BDA0001571704950000044
进行双边滤波，对其高频次带
Figure BDA0001571704950000045
进行硬阈值滤波，我们定义降噪后的次带为
Figure BDA0001571704950000046
我们首先依据传统方法估计每层噪声水平如下：1-3 due to image set
Figure BDA0001571704950000043
subband containing noise, low pass filtered
Figure BDA0001571704950000044
Bilateral filtering is performed, and its high frequency band
Figure BDA0001571704950000045
For hard threshold filtering, we define the denoised subband as
Figure BDA0001571704950000046
We first estimate the noise level of each layer according to the traditional method as follows:

Figure BDA0001571704950000051
Figure BDA0001571704950000051

其中，σL为第L层次带的噪声估计方差，median()表示取中值操作，HHL表示小波分解第L层的高频细节层，C1为常数参量，C1的大小决定估计的噪声方差的大小，C1越大得到的去噪图像越平滑，一般取值范围为2到4。Among them, σ L is the noise estimation variance of the L-th layer, median() represents the median operation, HHL represents the high-frequency detail layer of the L -th layer of wavelet decomposition, C 1 is a constant parameter, and the size of C 1 determines the estimated value The size of the noise variance, the larger C 1 is, the smoother the denoised image is, and the general value range is 2 to 4.

为降低短曝光图像中噪声对后续鬼影检测的干扰，并且更好保存图像细节与边缘，我们参考多尺度双边滤波，对低频次带

Figure BDA0001571704950000052
进行双边滤波：In order to reduce the interference of noise in short-exposure images to subsequent ghost detection, and to better preserve image details and edges, we refer to multi-scale bilateral filtering.
Figure BDA0001571704950000052
Do bilateral filtering:

Figure BDA0001571704950000053
Figure BDA0001571704950000053

Figure BDA0001571704950000054
Figure BDA0001571704950000054

其中，

Figure BDA0001571704950000055
表示短曝光图像次带集第L层的的低频次带，
Figure BDA0001571704950000056
表示双边滤波后的低频次带，C为归一化函数，N表示定义域，x,y,k,l表示像素位置坐标，σd为滤波窗口参数，σL为第L层次带的噪声估计方差，median()表示取中值操作，HHL表示小波分解第L层的高频细节层，C1为常数参量，C1的大小决定估计的噪声方差的大小，C1越大得到的去噪图像越平滑，一般取值范围为2到4。in,
Figure BDA0001571704950000055
represents the low-frequency subband of the Lth layer of the short-exposure image subband set,
Figure BDA0001571704950000056
Represents the low frequency band after bilateral filtering, C is the normalization function, N represents the domain of definition, x, y, k, l represent the pixel position coordinates, σ d is the filter window parameter, σ L is the noise estimate of the L-th level band Variance, median() represents the median operation, HHL represents the high-frequency detail layer of the Lth layer of wavelet decomposition, C 1 is a constant parameter, and the size of C 1 determines the estimated noise variance. The smoother the noise image, the general value range is 2 to 4.

对于高频次带

Figure BDA0001571704950000057
我们进行硬阈值滤波，该方法利用的计算公式为：For high frequency band
Figure BDA0001571704950000057
We perform hard threshold filtering, and the calculation formula used by this method is:

Figure BDA0001571704950000058
Figure BDA0001571704950000058

其中，

Figure BDA0001571704950000059
表示短曝光图像的第i个高频次带，
Figure BDA00015717049500000510
表示降噪滤波后的短曝光图像的第i个高频次带，σL为第L层次带的噪声估计方差。in,
Figure BDA0001571704950000059
represents the ith high frequency band of the short exposure image,
Figure BDA00015717049500000510
represents the i-th high-frequency band of the short-exposure image after noise reduction filtering, and σ L is the noise estimation variance of the L-th level band.

步骤2.利用步骤1-2获得的长曝光图像次带集与步骤1-4去噪处理后的短曝光图像次带集，得到对应次带差值，进而得到与次带对应的融合权重图，包括以下步骤：Step 2. Use the long-exposure image sub-band set obtained in step 1-2 and the short-exposure image sub-band set after denoising in step 1-4 to obtain the corresponding sub-band difference value, and then obtain the fusion weight map corresponding to the sub-band , including the following steps:

2-1计算步骤1-2得到的长曝光图像次带集与步骤1-4去噪处理后的短曝光图像次带集的相应次带差值的绝对值Di，计算公式为：2-1 Calculate the absolute value D i of the corresponding sub-band difference between the long-exposure image sub-band set obtained in step 1-2 and the corresponding sub-band set of the short-exposure image sub-band set after denoising in step 1-4, and the calculation formula is:

Figure BDA00015717049500000511
Figure BDA00015717049500000511

其中，

Figure BDA0001571704950000061
为步骤1-2得到的长曝光图像次带集，
Figure BDA0001571704950000062
为步骤1-4降噪处理后的短曝光图像次带集，Di为降噪后短曝光图像第i个次带与长曝光图像第i个次带的差值绝对值。in,
Figure BDA0001571704950000061
is the subband set of the long-exposure image obtained in steps 1-2,
Figure BDA0001571704950000062
is the set of sub-bands of the short-exposure image after noise reduction in steps 1-4, and D i is the absolute value of the difference between the i-th sub-band of the short-exposure image and the i-th sub-band of the long-exposure image after noise reduction.

2-2降噪后的短曝光图像与长曝光图像虽然亮度接近但仍有微小差异，降噪过程中的双边滤波与硬阈值滤波虽然降低了短曝光图像中噪声对鬼影区域检测的影响，但也容易造成微小细节被过渡平滑或部分微弱噪声的残留。基于此，我们在考虑利用相应次带差值绝对值构建融合权重时，较大的差值可以判定为由鬼影引起，较小的差值则可能是由微弱的纹理、色彩偏差或残留噪声偏差引起，仍需判定为非鬼影区。因此我们需要一个差值阈值来划分鬼影区与非鬼影区，假定鬼影区在整幅图像中面积比重较小，我们可以选取图像差值中值估计鬼影检测阈值Ti，公式如下：2-2 Although the brightness of the short-exposure image and the long-exposure image after noise reduction is similar, there is still a slight difference. Although the bilateral filtering and hard threshold filtering in the noise reduction process reduce the influence of noise in the short-exposure image on the detection of ghost areas, But it is also easy to cause tiny details to be transitioned smoothly or some weak noise remains. Based on this, when we consider using the absolute value of the corresponding subband difference to construct the fusion weight, the larger difference can be determined to be caused by ghosts, and the smaller difference may be caused by weak texture, color deviation or residual noise. If the deviation is caused, it still needs to be judged as a non-ghosting area. Therefore, we need a difference threshold to divide the ghost area and the non-ghost area. Assuming that the ghost area has a small proportion in the entire image, we can select the median image difference to estimate the ghost detection threshold T i , the formula is as follows :

Ti＝C2*median(Di) (22)T i =C 2 *median(D i ) (22)

其中，Ti为第i个次带的鬼影检测阈值，Di为降噪后短曝光图像第i个次带与长曝光图像第i个次带的差值绝对值，C2为常数参量，C2的大小决定鬼影检测程度，C2越大检测到的鬼影区域越小，一般取值范围为2到5。Among them, T i is the ghost detection threshold of the ith subband, D i is the absolute value of the difference between the ith subband of the short exposure image and the ith subband of the long exposure image after noise reduction, and C 2 is a constant parameter , the size of C 2 determines the degree of ghost detection, the larger the C 2 is, the smaller the detected ghost area, and the general value range is 2 to 5.

2-3利用各层次带差值的绝对值和估计的鬼影检测阈值得到融合权重图，为了实现自然融合，我们利用高斯函数特性构建平滑的权重函数，并利用导向滤波对融合权重图做平滑处理，计算公式如下：2-3 Use the absolute value of the band difference at each level and the estimated ghost detection threshold to obtain the fusion weight map. In order to achieve natural fusion, we use the Gaussian function to construct a smooth weight function, and use the guided filter to smooth the fusion weight map. processing, the calculation formula is as follows:

Figure BDA0001571704950000063
Figure BDA0001571704950000063

其中，Wi为第i个次带的融合权重图，Ti为估计的鬼影检测阈值，Di为降噪后短曝光图像第i个次带与长曝光图像第i个次带的差值绝对值，G()表示导向滤波平滑操作。从上式可以看出，当两幅图像差值较小时，融合权重接近于1，当两幅图像差值远大于鬼影检测阈值时，融合权重趋于0。Among them, Wi is the fusion weight map of the ith subband, T i is the estimated ghost detection threshold, and Di is the difference between the ith subband of the short exposure image and the ith subband of the long exposure image after noise reduction The absolute value of the value, G() represents the guided filter smoothing operation. It can be seen from the above formula that when the difference between the two images is small, the fusion weight is close to 1, and when the difference between the two images is much larger than the ghost detection threshold, the fusion weight tends to 0.

3.重建次带集得到结果图像，包括以下步骤：3. Reconstruct the secondary band set to obtain the resulting image, including the following steps:

3-1利用步骤1-2得到的长曝光图像次带集、步骤1-4去噪处理后的短曝光图像次带集、及步骤2-3平滑处理后的融合权重图构建新的次带集，计算公式为：3-1 Construct a new sub-band using the long-exposure image sub-band set obtained in step 1-2, the short-exposure image sub-band set after denoising in step 1-4, and the fusion weight map after smoothing in step 2-3 set, the calculation formula is:

Figure BDA0001571704950000064
Figure BDA0001571704950000064

其中，Fi为构建的新的次带集，

Figure BDA0001571704950000071
为步骤1-2得到的长曝光图像次带集，
Figure BDA0001571704950000072
为步骤1-4降噪处理后的短曝光图像次带集，Wi为第i个次带的融合权重图。从上式可以看出，当两幅图像差值较小时，融合权重接近于1，结果图像信息更多来自长曝光图像，在鬼影区融合权重趋于0，结果图像信息更多来自于去噪后的短曝光图像。Among them, F i is the new subband set constructed,
Figure BDA0001571704950000071
is the subband set of the long-exposure image obtained in steps 1-2,
Figure BDA0001571704950000072
is the sub-band set of the short-exposure image after denoising in steps 1-4, and Wi is the fusion weight map of the i -th sub-band. It can be seen from the above formula that when the difference between the two images is small, the fusion weight is close to 1, and the resulting image information is more from the long-exposure image. In the ghost area, the fusion weight tends to 0, and the resulting image information is more from the A short exposure image after noise.

3-2利用小波合成得到的次带集，得到最终图像。3-2 Use the subband set obtained by wavelet synthesis to obtain the final image.

本实验小波分解中所涉及的小波基为降噪过程经常使用的‘db8’小波基，融合算法多尺度小波分解层数设置为2，融合算法中的常数参量C1与C2为3，导向滤波的窗口参数与快速双边滤波的窗口参数σd取值相同都为7。当图像分辨率增加时，可以参考经验值适当调整，以获取具有良好视觉效果的融合图像。The wavelet base involved in the wavelet decomposition in this experiment is the 'db8' wavelet base that is often used in the noise reduction process. The multi-scale wavelet decomposition layer number of the fusion algorithm is set to 2, and the constant parameters C 1 and C 2 in the fusion algorithm are set to 3. The filtering window parameter and the fast bilateral filtering window parameter σ d have the same value of 7. When the image resolution increases, it can be adjusted appropriately with reference to empirical values to obtain a fused image with good visual effects.

本发明测试部分图像得到实验效果，图像示例如图5所示，我们可以看到，结果图像在鬼影区域保留了短曝光图像的边缘锐度，如第一行图像中小男孩的面部区域、第二行图像中女士的眼睛；在背景等非鬼影区域结果图像整色彩亮度信息与长曝光图像接近，如图5(c)中地板区域。图3中全局直方图匹配虽然调整了短曝光图像的亮度与色彩，但是图像中存留大量噪声，以及部分色彩偏差，如图3(c)中地板区域噪声明显且有色差存在，但我们结果图像中的地板区域噪声和色彩偏差明显减少，更接近长曝光图像信息；且我们结果图像整体无明显噪声，如第三行图像中小女孩的衣服。本发明实验结果图像整体过渡自然，不存在明显色差、区域接缝等情况，达到预期效果。The test part of the image of the present invention obtains the experimental effect. An example of the image is shown in Figure 5. We can see that the resulting image retains the edge sharpness of the short-exposure image in the ghost area, such as the face area of the little boy in the first row of images, and the third row of images. The eyes of the lady in the two-line image; in the non-ghosting area such as the background, the color and brightness information of the resulting image is close to that of the long-exposure image, as shown in the floor area in Figure 5(c). Although the global histogram matching in Figure 3 adjusts the brightness and color of the short-exposure image, there is still a lot of noise and some color deviation in the image. In Figure 3(c), the floor area has obvious noise and color difference, but our result The noise and color deviation of the floor area in the image are significantly reduced, which is closer to the long-exposure image information; and our result image has no obvious noise as a whole, such as the little girl's clothes in the third row of images. As a result of the experiment of the present invention, the overall transition of the image is natural, and there is no obvious chromatic aberration, regional seams, etc., and the expected effect is achieved.

Claims (8)

1. A low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering is characterized by comprising the following steps:
(1) performing wavelet decomposition and multi-scale bilateral filtering on an input image, specifically:
(1.1) firstly, carrying out global histogram matching on the short-exposure image by referring to the long-exposure image, and then carrying out registration alignment on the long-exposure image by referring to the short-exposure image after histogram matching;
(1.2) respectively carrying out wavelet decomposition on the two images processed in the step (1.1) to generate a corresponding short-exposure image subband set and a long-exposure image subband set;
(1.3) estimating the noise variance of each hierarchical band by using the high-frequency sub-band in the sub-band set of the short-exposure image obtained in the step (1.2);
(1.4) carrying out bilateral filtering on the low-frequency subband of the short-exposure image subband set obtained in the step (1.2), and carrying out hard threshold filtering on the high-frequency subband set to achieve the purpose of noise suppression;
(2) obtaining a difference value of a corresponding subband by using the long exposure image subband set obtained in the step (1.2) and the short exposure image subband set subjected to noise reduction processing in the step (1.4), further estimating a ghost detection threshold value, and obtaining a fusion weight map corresponding to the subband, wherein the specific steps are as follows:
(2.1) calculating the absolute value of the corresponding subband difference value of the long-exposure image subband set obtained in the step (1.2) and the short-exposure image subband set subjected to noise reduction processing in the step (1.4);
(2.2) estimating a ghost detection threshold value by using the median value of the difference absolute value of each layer;
(2.3) obtaining a fusion weight map by using the absolute value of the difference value of each layer and the estimated ghost detection threshold, and smoothing the fusion weight map by using guide filtering;
(3) reconstructing the subband set to obtain a result image, specifically:
(3.1) constructing a new subband set by using the long-exposure image subband set obtained in the step (1.2), the short-exposure image subband set subjected to denoising in the step (1.4) and the fusion weight map subjected to smoothing in the step (2.3);
and (3.2) utilizing the subband set constructed in the wavelet synthesis step (3.1) to obtain a final result image.
2. The wavelet decomposition and bilateral filtering-based fusion method of low-photometric image pairs according to claim 1 wherein in step (1.3), the noise variance of each level band is estimated using the high-frequency subbands of the subband set of the short-exposure image obtained in step (1.2) according to the following formula:
Figure FDA0002413928630000011
wherein σLEstimate the variance for the noise of the L-th band, mean () represents the median operation, HHLHigh frequency detail layer, C, representing the L-th layer of the wavelet decomposition1Is a constant parameter, C1The magnitude of (C) determines the magnitude of the estimated noise variance, C1The larger the obtained denoised image is, the smoother the image is, and the value range is 2 to 4.
3. The wavelet decomposition and bilateral filtering-based low-photometric image pair fusion method according to claim 1 wherein in step (1.4), the formula for bilateral filtering of the low-frequency subbands of the subband set of the short-exposure image obtained in step (1.2) is:
Figure FDA0002413928630000021
Figure FDA0002413928630000022
wherein,
Figure FDA0002413928630000023
a low frequency subband representing the lth layer of the short exposure image subband set,
Figure FDA0002413928630000024
representing the low frequency sub-band after bilateral filtering, C is a normalization function, N represents a definition domain, x, y, k and l represent pixel position coordinates, and sigma represents the position coordinate of a pixeldAs filter window parameter, σLThe variance is estimated for the noise of the L-th hierarchical band.
4. The wavelet decomposition and bilateral filtering-based fusion method of low-photometric image pairs according to claim 1 wherein in step (1.4) the hard-threshold filtering is performed on the high frequency subband set of the short-exposure image subband set obtained in step (1.2) according to the formula:
Figure FDA0002413928630000025
wherein,
Figure FDA0002413928630000026
the i-th high frequency subband representing the short exposure image obtained in step (1.2),
Figure FDA0002413928630000027
representing noise-reduced filtered short exposure mapsI-th high-frequency sub-band of the image, σLThe variance is estimated for the noise of the L-th hierarchical band.
5. The wavelet decomposition and bilateral filtering-based low-photometric image pair fusion method according to claim 1, wherein in step (2.1), the formula for calculating the absolute value of the corresponding subband difference value by using the long-exposure image subband set obtained in step (1.2) and the short-exposure image subband set subjected to the noise reduction processing in step (1.4) is as follows:
Figure FDA0002413928630000028
wherein,
Figure FDA0002413928630000031
for the long exposure image subband set obtained in step (1.2),
Figure FDA0002413928630000032
for the short-exposure image subband set after the noise reduction processing in step (1.4), DiAnd the absolute value of the difference value of the ith subband of the short-exposure image after noise reduction and the ith subband of the long-exposure image is obtained.
6. The wavelet decomposition and bilateral filtering-based low photometric image pair fusion method according to claim 1 wherein in said step (2.2), the formula for estimating the ghost detection threshold using the median of the difference values of each level band is:
Ti＝C2*median(Di) (6)
wherein, TiThreshold for ghost detection for the ith subband, DiThe absolute value of the difference value of the ith subband of the short-exposure image after noise reduction and the ith subband of the long-exposure image C2Is a constant parameter, C2The size of (D) determines the degree of ghost detection (C)2The larger the detected ghost area, the smaller the value range is 2 to 5.
7. The wavelet decomposition and bilateral filtering-based low-photometric image pair fusion method according to claim 1, wherein in said step (2.3), the fusion weight map is obtained by using absolute values of band difference values of each layer and estimated ghost detection threshold, and the formula for smoothing the fusion weight map by using guided filtering is as follows:
Figure FDA0002413928630000033
wherein, WiA fused weight map for the ith subband, TiFor the estimated ghost detection threshold, DiG () represents a guided filtering smoothing operation for the absolute value of the difference between the ith subband of the short-exposure image after noise reduction and the ith subband of the long-exposure image.
8. The wavelet decomposition and bilateral filtering-based low-photometric image pair fusion method according to claim 1, wherein in the step (3.1), the formula for constructing a new subband set by using the long-exposure image subband set obtained in the step (1.2), the denoised short-exposure image subband set in the step (1.4), and the smoothed fusion weight map in the step (2.3) is as follows:
Figure FDA0002413928630000034
wherein, FiIn order to construct a new set of subbands,
Figure FDA0002413928630000035
for the long exposure image subband set obtained in step (1.2),
Figure FDA0002413928630000036
for the short-exposure image subband set, W, after the noise reduction processing in step (1.4)iIs the fusion weight map of the ith subband.
CN201810119668.6A 2018-02-06 2018-02-06 Low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering Expired - Fee Related CN108492245B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810119668.6A CN108492245B (en) 2018-02-06 2018-02-06 Low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810119668.6A CN108492245B (en) 2018-02-06 2018-02-06 Low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering

Publications (2)

Publication Number Publication Date
CN108492245A CN108492245A (en) 2018-09-04
CN108492245B true CN108492245B (en) 2020-06-30

Family

ID=63344607

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810119668.6A Expired - Fee Related CN108492245B (en) 2018-02-06 2018-02-06 Low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering

Country Status (1)

Country Link
CN (1) CN108492245B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110322409B (en) * 2019-06-14 2021-08-31 浙江大学 An Improved Wavelet Transform Image Fusion Method Based on Marker Map
CN110428391B (en) * 2019-08-02 2022-05-03 格兰菲智能科技有限公司 Image fusion method and device for removing ghost artifacts
CN111028152B (en) * 2019-12-02 2023-05-05 哈尔滨工程大学 A method for super-resolution reconstruction of sonar images based on terrain matching
CN111325694B (en) * 2020-02-25 2024-02-13 深圳市景阳科技股份有限公司 Image noise removing method and device
CN112689099B (en) * 2020-12-11 2022-03-22 北京邮电大学 A ghost-free high dynamic range imaging method and device for a dual-lens camera
CN113658065B (en) * 2021-08-09 2024-07-23 Oppo广东移动通信有限公司 Image noise reduction method and device, computer readable medium and electronic equipment
CN116095517B (en) * 2022-08-31 2024-04-09 荣耀终端有限公司 Blurring method, terminal device and readable storage medium
CN117611461B (en) * 2023-11-14 2025-02-14 国网江苏省电力有限公司淮安供电分公司 A multi-frame motion image fusion method and system for robot vision

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104240203A (en) * 2014-09-09 2014-12-24 浙江工业大学 Medical ultrasound image denoising method based on wavelet transform and quick bilateral filtering
CN105279746A (en) * 2014-05-30 2016-01-27 西安电子科技大学 Multi-exposure image integration method based on bilateral filtering
CN106127718A (en) * 2016-06-17 2016-11-16 中国人民解放军国防科学技术大学 A kind of many exposure images fusion method based on wavelet transformation
CN107220956A (en) * 2017-04-18 2017-09-29 天津大学 A kind of HDR image fusion method of the LDR image based on several with different exposures

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105279746A (en) * 2014-05-30 2016-01-27 西安电子科技大学 Multi-exposure image integration method based on bilateral filtering
CN104240203A (en) * 2014-09-09 2014-12-24 浙江工业大学 Medical ultrasound image denoising method based on wavelet transform and quick bilateral filtering
CN106127718A (en) * 2016-06-17 2016-11-16 中国人民解放军国防科学技术大学 A kind of many exposure images fusion method based on wavelet transformation
CN107220956A (en) * 2017-04-18 2017-09-29 天津大学 A kind of HDR image fusion method of the LDR image based on several with different exposures

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Variational Approach for the Fusion of Exposure Bracketed Pairs;Marcelo Bertalmío，Stacey Levine;《IEEE TRANSACTIONS ON IMAGE PROCESSING》;20130228;第22卷(第2期);712-723 *

Also Published As

Publication number Publication date
CN108492245A (en) 2018-09-04

Similar Documents

Publication Publication Date Title
CN108492245B (en) Low-luminosity image pair fusion method based on wavelet decomposition and bilateral filtering
Liu et al. Efficient single image dehazing and denoising: An efficient multi-scale correlated wavelet approach
CN104240194B (en) A kind of enhancement algorithm for low-illumination image based on parabolic function
CN112734650A (en) Virtual multi-exposure fusion based uneven illumination image enhancement method
US8878963B2 (en) Apparatus and method for noise removal in a digital photograph
Kim et al. A novel approach for denoising and enhancement of extremely low-light video
CN108765342A (en) A kind of underwater image restoration method based on improvement dark
CN109658447B (en) Night image defogging method based on edge detail preservation
CN113850741B (en) Image noise reduction method and device, electronic equipment and storage medium
CN114331939B (en) Detail enhancement multi-exposure image fusion method based on homomorphic filtering and storable medium
CN108133462B (en) A Single Image Restoration Method Based on Gradient Field Segmentation
WO2023273868A1 (en) Image denoising method and apparatus, terminal, and storage medium
CN116416175A (en) Image fusion method based on self-adaptive edge-preserving smooth pyramid
Dar et al. An enhanced adaptive histogram equalization based local contrast preserving technique for HDR images
Wu et al. Reflectance-guided, contrast-accumulated histogram equalization
CN115249211A (en) An Image Restoration Method Based on Underwater Non-Uniform Incident Light Model
GUAN et al. A dual-tree complex wavelet transform-based model for low-illumination image enhancement
Kaur et al. An improved adaptive bilateral filter to remove gaussian noise from color images
Kokro et al. Histogram matching and fusion for effective low-light image enhancement
Wang et al. Video enhancement using adaptive spatio-temporal connective filter and piecewise mapping
CN116385312A (en) Low-illumination image denoising method based on phase correlation
Lee et al. Efficient Low Light Video Enhancement Based on Improved Retinex Algorithms
Jiao et al. Attention-based multi-branch network for low-light image enhancement
Wen et al. A Robust Blind Deblurring Method for Natural Blurry Images
Sharma et al. Synthesis of flash and no-flash image pairs using guided image filtering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200630

CF01 Termination of patent right due to non-payment of annual fee