KR101408343B1 - Apparatus for processing wide dynamic rage - Google Patents

Abstract

In the wide-area backlight correction, the edge information is extracted from an image having a relatively short exposure time to improve the sharpness of the image, and the brightness of the list is adjusted by using the difference between the long-exposure image and the fused image, To a wide-area backlight compensation apparatus capable of obtaining an enhanced image. The wide-area backlight compensation apparatus includes edge information extracted from the image having the first exposure time, luminance fusion result of the image having the first and second exposure times, image having the second exposure time, And output the combined image of brightness difference information extracted from the second image.

Edge extraction, brightness difference, brightness fusion, sharpness improvement, contrast enhancement

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001]

BACKGROUND OF THE INVENTION Field of the Invention [0002] The present invention relates to a digital image processing apparatus, and more particularly, to a wide-area backlight correction apparatus for improving a contrast of a list portion.

Wide dynamic range (WDR) is a solution to the problem that the existing backlight compensation (BLC) simply brightens the dark areas and saturates the images in the bright areas. The WDR function keeps the bright areas as dark as possible So that a clearer image can be realized by processing the image so that it looks bright.

The reason for using the WDR function is that when shooting a mixed image of a list (usually outdoors) and dark areas (usually indoor) due to a large difference in illuminance, only one image is taken to optimize the exposure time and aperture, Is not able to acquire well visible images. Therefore, the WDR can produce clear images of both the list and dark areas by merging the exposure adjusted images and the different exposure adjusted images so that the list can be seen clearly.

These WDR-enabled products are widely used for security cameras for surveillance, general digital cameras, broadcasting cameras, automobile cameras, and portable cameras. For example, when a video is shot in a place where a backlight such as a window in a room is present, or when a list and a dark place are mixed together, such as a night view, a scene in which a list of sunshine, such as a sports game, When shooting, the WDR function allows you to shoot clearer images.

On the other hand, WDR implementation technology generally uses a block-based fusion method and a pixel-based fusion method.

In the block-based fusion method, the image fusion unit is set as a block, and the fusion ratio is different according to each block. As compared with the pixel-by-pixel processing, In order to solve this problem, anti-blocking processing is indispensably required. Further, when images are divided into upper and lower portions, images having different brightness may be output.

The pixel-based fusion method sets the image fusion unit as one pixel and fuses according to each pixel, and there is a pixel-based fusion using a weight-sum and a pixel-based fusion using a gradient domain. For example, US Pat. No. 6,501,504 entitled " Dynamic range enhancement for imaging sensors " by H. John Tatko et al. And Kzauhito Horiuchi, "Imaging processing apparatus for generating a wide dynamic range image "in U.S. Patent No. 6,825,884. In the latter case, the input image having a long exposure time and the input image having a short exposure time are converted into a gradient domain, and the respective gradient magnitudes are adjusted, and the fusion ratio is determined for each pixel. It is difficult to mount it in the DSP chip. Further details on "Gradient domain high dynamic range compression" such as ^{R. Fattal, Proceedings of the 29 th} annual conference on computer graphics and interactive techniques, pp. 249-256, 2002.

In the case of the conventional WDR, the brightness of the long-exposure image is almost used to maintain the brightness of the dark portion as much as possible. On the other hand, the brightness of the list is used by blending long images and short images. Basically, the brightness is larger than that of the dark parts. Therefore, it is possible to create a natural image which does not cause a reversal in brightness even when compared with actual illuminance. The range in which brightness can be expressed is inevitably limited. Therefore, contrast reduction in the list is inevitable, and complementing it is a key issue of WDR performance.

FIG. 9A shows an image taken with the exposure on the list side while the WDR is off, and FIG. 9B shows an image taken when the WDR is turned on. 9A and FIG. 9B, it can be seen that, in the WDR fusion, the detail of the dark part is lost compared to the image having a short exposure time, but the details of the building shape and the tree branches in the list are lost.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and it is an object of the present invention to provide an image processing apparatus and a method thereof, which can improve sharpness of an image by extracting edge information from an image having a relatively short exposure time, And to provide a wide-area backlight compensation apparatus capable of acquiring an image with improved contrast by adjusting the brightness of a list portion.

In order to solve the technical problem to be solved by the present invention, a wide-area backlight correction apparatus is provided with an edge-information extracting unit for extracting, from edge information extracted from an image having a first exposure time, a luminance fusion result of an image having first and second exposure times, And outputting the combined image of brightness difference information extracted from the first and second images subjected to the luminance fusion.

In the present invention, the first exposure time in the apparatus may be shorter than the second exposure time.

The apparatus may further include first conversion means for converting the image signals acquired at the first and second exposure times into luminance / color difference signals, respectively; A preprocessing means for normalizing the luminance signal having the first exposure time so as to correspond to the luminance signal having the second exposure time; A result of fusion of the edge information extracted from the luminance signal having the first exposure time, the luminance signal having the normalized first exposure time and the luminance signal having the second exposure time, And luminance fusion means for synthesizing the brightness difference extracted from the fusion result; A color difference fusing unit for summing the color difference signals having the first and second exposure times using a weight reflecting the output of the luminance fusing unit; And second converting means for converting the outputs of the luminance fusing means and the color difference fusing means into RGB signals.

In the present invention, the luminance fusing unit may include an edge extracting unit that extracts an edge from the luminance signal having the first exposure time; A luminance fusion unit for applying a weight to the luminance signal having the normalized first exposure time and the luminance signal having the second exposure time and for summing up; A subtraction unit for subtracting the luminance signal having the second exposure time and the output of the luminance fusion unit; And a post-processing unit for synthesizing outputs of the edge extracting unit, the luminance fusion unit, and the subtracting unit.

In the present invention, the apparatus may further include a first calculation unit for accumulating a predetermined gain value at the output of the edge extracting unit.

The apparatus may further include a second calculation unit for accumulating a predetermined gain value at the output of the subtraction unit.

The present invention may further comprise a blurring processor for blurring the output of the second calculator.

In the present invention, the apparatus may further include a limiter for limiting the output of the second calculator to a predetermined reference value.

As described above, according to the present invention, edge information is extracted from an image having a relatively short exposure time at the time of wide-area backlight correction to improve the sharpness of the image, and a list of images By adjusting the brightness, an image with improved contrast can be obtained.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The description of the drawings and the drawings is for illustrative purposes only and is not intended to limit the scope of the invention.

1 is a block diagram showing a configuration of a wide-area backlight compensation apparatus 100 according to the present invention.

1, a wide-angle backlight compensation apparatus 100 includes a Y / C conversion unit 110, a preprocessing unit 120, a luminance fusion unit 130, a color difference fusion unit 140, and RGB conversion unit 150, .

The Y / C conversion means 110 converts the image signals obtained under different exposure times into luminance / color difference signals. Here, different exposure times mean that two images are taken at different exposure times for the same scene for a predetermined time, for example, 1/60 second, for WDR image processing. One is a long exposure time image, which includes scene information of the dark area, mainly indoor, in the same scene. The other is a short exposure time image, which includes scene information in the same scene, mainly outdoor scene information. The Y / C conversion unit 110 converts image signals having two different exposure times acquired from an image acquisition unit (not shown), for example, a CCD / CMOS image sensor, To chroma of the image.

The preprocessing means 120 outputs the luminance signal having the first exposure time among the different exposure times output from the Y / C conversion means 110 to the luminance signal having the second exposure time longer than the first exposure time Normalize. Here, the first exposure time means an image having a relatively short exposure time (herein, defined as an SE image), and the second exposure time means an image having a relatively long exposure time (herein, defined as an LE image) . For example, the second exposure time may be about eight times the first exposure time, and the ratio may be adjusted to a suitable ratio for expressing the list and dark areas. In addition, since the human eye is more sensitive to the change in brightness than the change in color, the preprocessing process is performed using the luminance signal of each of the SE image and the LE image (herein, defined as YS and YL signals). In other words, by applying the normalization process so that YS matches the reference point of YL before merging the two images, optimal WDR result image can be obtained even when the imbalance of image brightness is minimized and the exposure time adjustment is restricted.

The luminance fusing unit 130 calculates the fusing result of the edge information extracted from the luminance signal having the first exposure time, the luminance signal having the normalized first exposure time and the luminance signal having the second exposure time, And the brightness difference extracted from the fusion result is synthesized. In the luminance fusion process, the luminance signal having the normalized first exposure time and the luminance signal having the second exposure time are added using the first weight. For WDR processing, luminance and chrominance components of image are divided and fused. This reduces the complexity of the hardware and reduces the color imbalance that occurs when adjusting R, G, and B, respectively, as compared to using a common RGB color space. In the luminance fusion, we apply the appropriate weights to the YS and YL signals and add them together to generate a luminance image that shows both the list and the dark areas. Also, edge information is extracted from the luminance signal having the first exposure time to improve the sharpness of the image, and the brightness of the list is adjusted using the brightness difference extracted from the image having the second exposure time and the fusion result. In addition, artifacts generated during fusion can be minimized by performing a post-processing that emphasizes a contrast ratio such as sharpness on an image obtained by combining an edge extracted image, a luminance fusion image, and a brightness difference image. The specific configuration and function of the luminance fusion unit 130 will be described below.

The color difference fusing unit 140 adds the color difference signal having the first exposure time and the color difference signal having the second exposure time using the second weight reflecting the fusion result of the luminance fusion unit 130. [ In addition, post-processing such as color saturation enhancement and color balance adjustment may be selectively performed.

The RGB conversion means 150 converts a signal output from the luminance fusion means 130 and a signal output from the color difference fusion means 140, that is, a Y / C signal, into an RGB signal and outputs a WDR resultant image signal.

FIG. 2 is a detailed block diagram of the luminance fusing unit 130 including the Y / C conversion unit 110 and the preprocessing unit 120 in FIG.

2, the luminance fusion unit 130 includes an edge extraction unit 131, a first calculation unit 132, a luminance fusion unit 133, a subtraction unit 134, a second calculation unit 135, (136).

The output signal Yout of the luminance fusing unit 130 can be obtained by the following equation (1).

First, we explain Post1. The edge extracting unit 131 extracts edge information from the luminance component of the SE image output from the Y / C converting unit 110. [ The edge extracting unit 131 extracts edge information from the original luminance component by applying a high pass filter (HPF) or a low pass filter (LPF). FIG. 3A shows an SE image, and FIG. 3B shows an edge image extracted from the FIG. 3A SE image. The first arithmetic unit 132 can adjust the sharpness only in the bright region by integrating and calculating the gain (parameter 1) on the extracted edge image. Depending on the situation, it is possible to add a threshold that allows the edge to be found only in a certain brightness or more in the SE image so that the edge component of the list is not included, thereby preventing the high frequency components such as the dark part from being extracted together with the edge.

In the present invention, in order to improve the clarity of the list, the edges of the SE image with relatively short exposure time are used, but the sharpness of the list can be adjusted by applying a high-pass filter to the LE image having a relatively long exposure time. Also, by applying a high-pass filter to the fused result as in the conventional method, the sharpness of the entire image can be adjusted. Therefore, it is possible to arbitrarily adjust the sharpness of the image list, the dark area, and the entire image with a total of three pieces of edge information.

FIG. 4A shows the luminance fusion result image, and FIG. 4B shows the image obtained by integrating the gain (parameter 1) on the edge extraction result. FIG. 4C shows an image enlarged only on the list in FIG. 4A, and FIG. 4D shows an image enlarged only on the list in FIG. 4B. It can be seen that the sharpness of the list is improved by edge extraction as described above.

The luminance fusion unit 133 adds the luminance signal having the normalized first exposure time and the luminance signal having the second exposure time by weighting.

Next, we explain Post 2. The subtraction section 134 subtracts the brightness component of the LE image output from the Y / C conversion means 110 and the fusion result output from the brightness fusion section 133 to extract the brightness variation of the name part, The second calculation unit 135 can process the brightness of the bright region only. The overall brightness of the list area is lowered, but the brightness change becomes larger as the amount of change due to fusion increases, so that the contrast is improved. As shown in FIG. 5, in the case of the dark part, since the brightness of the LE image having a long exposure time is used as it is, the change amount of the list is almost 0, so that it appears white. In the case of the list, , It is possible to obtain the change amount of the negative list by subtracting it from the LE image having a long exposure time. In FIG. 5, FIG. 5A shows the LE image with a long exposure time, FIG. 5B shows the image output from the brightness fusion unit 133, and FIG. 5C shows the subtraction result image of FIGS. 5A and 5B.

In case of subtraction, the brightness of the list is low and the maximum expression range can not be fully used. Therefore, the maximum value of the image is obtained after reflecting the change of the list, and then the maximum expression range is used The gain correction process can be additionally performed. At this time, the gain value (parameter 2) may be integrated with the subtraction result in the second operation unit 135 and output.

The following Equation 2 shows Post 2 'including a gain correction process.

Also, if the gain is too large, the brightness difference in the border region between the list and the dark region may become abrupt, resulting in fused side effects, or a slight side effect may be amplified. Therefore, a low pass filter (blurring processor) The brightness change amount may be blurred or the maximum value of the brightness change due to post-processing may be limited by using a threshold (Restriction).

FIG. 6A shows the luminance fusion result image, and FIG. 6B shows the brightness change amount image according to the subtraction result. FIG. 6C shows an image enlarged only on the list in FIG. 6A, and FIG. 6D shows an image enlarged only on the list in FIG. 6B. Thus, by subtracting the luminance component of the LE image and the luminance fusion result, it can be seen that the contrast of the list is improved.

The post-processing unit 136 combines the output of the first calculation unit 132, the output of the luminance fusion unit 133, and the output of the second calculation unit 135, and outputs it as a Yout signal. The post-processing unit 136 can improve the output of the first calculation unit 132, the output of the luminance fusion unit 133, and the output of the second calculation unit 135 in a specific region.

7A is a graph showing the luminance components of the LE image, FIG. 7B and FIG. 7C, and FIG. 7C is a graph showing luminance of the luminance fusion result (prior art) And Fig. 7D shows luminance components of the post-processing result image according to the present invention. Compared with FIG. 7C, FIG. 7D shows that the sharpness is improved.

7A is a color image of an LE color image, FIG. 7B is an SE color image, FIG. 7C is a color image of a luminance fusion result (prior art), FIG. And shows the resultant color image according to the present invention. Compared with FIG. 7C, FIG. 7D shows that the contrast is improved.

The present invention has been described with reference to the preferred embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

1 is a block diagram showing a configuration of a wide-area backlight compensation apparatus according to the present invention.

Fig. 2 is a detailed block diagram of luminance fusion means including Y / C conversion means and preprocessing means in Fig. 1. Fig.

FIG. 3 is a view showing an image for explaining the edge extracting unit in FIG.

Fig. 4 is a diagram showing an edge extracting effect in Fig.

FIG. 5 is a diagram showing an image for explaining a subtraction unit in FIG. 2. FIG.

FIG. 6 is a diagram showing an effect of improving the contrast according to the subtraction unit in FIG. 2;

FIG. 7 is a diagram showing luminance components of an image of each block in FIG. 2. FIG.

8 is a view showing a color image for each block of FIG.

9 is a view showing a WDR on / off image according to the related art.

Claims (8)

delete delete First conversion means for converting the image signals acquired at the first and second exposure times into luminance / color difference signals, respectively; A preprocessing means for normalizing the luminance signal having the first exposure time so as to correspond to the luminance signal having the second exposure time; A result of fusion of the edge information extracted from the luminance signal having the first exposure time, the luminance signal having the normalized first exposure time and the luminance signal having the second exposure time, And luminance fusion means for synthesizing the brightness difference extracted from the fusion result; A color difference fusing means for summing color difference signals having first and second exposure times using a weight reflecting the output of the luminance fusing means; And And second conversion means for converting the outputs of the luminance fusion means and the color difference fusion means into RGB signals, The luminance fusion means An edge extracting unit for extracting an edge from the luminance signal having the first exposure time; A luminance fusion unit for applying a weight to the luminance signal having the normalized first exposure time and the luminance signal having the second exposure time and for summing up; A subtraction unit for subtracting the luminance signal having the second exposure time and the output of the luminance fusion unit; And a post-processing unit for synthesizing the outputs of the edge extracting unit, the luminance fusion unit, and the subtracting unit. delete The method of claim 3, Further comprising a first calculating unit for multiplying an output of the edge extracting unit by a predetermined gain value. The method of claim 3, And a second arithmetic unit for multiplying an output of the subtraction unit by a predetermined gain value. Claim 7 has been abandoned due to the setting registration fee. The method according to claim 6, And a blurring processor for blurring the output of the second calculator. Claim 8 has been abandoned due to the setting registration fee. The method according to claim 6, And a limiter for limiting the output of the second calculator to a predetermined reference value.

Images