KR101468433B1 - Apparatus and method for extending dynamic range using combined color-channels transmission map - Google Patents

Abstract

Disclosed are an apparatus and a method for extending a dynamic range using a combined color channel transmission map. A color separating unit outputs an intermediate image for each color channel by separating an input image by predetermined color channel. A transmission map generating unit generates a color channel transmission map for each color channel based on the intermediate image for each color channel and air glow for each color channel inputted to an image obtaining device, and generates a combined color channel transmission map by combining each of the color channel transmission maps. A gain calculating unit calculates a gain to extend a dynamic range based on the combined color channel transmission map. A contrast improving unit improves the contrast of the input image by multiplying the input image and the gain. According to the present invention, the contrast of a low expose area can be improved using only one image while the background is preserved. Also, calculating time to improve a dynamic range can be significantly reduced, and an HDR image can be generated without color distortion.

Description

[0001] The present invention relates to an apparatus and method for extending a dynamic range using a combined color channel conversion map,

The present invention relates to a dynamic range expansion apparatus and method, and more particularly, to a dynamic range expansion apparatus and method using a combined color channel conversion map for maximizing a reproducible maximum exposure region in a range of an output level for an image .

The human eye is capable of recognizing brightness of up to 10,000 cd / m 2 from a real world landscape with a very wide dynamic range, while a substantial digital image acquisition and output device has a very reduced range of possibilities. The dynamic range is a term that indicates how big a difference in shade can be represented by the difference between two colors, which means the maximum reproducible exposure area in the range of the output level. This can be represented by the brightness range of a reproducible subject in a single shot. In addition, commercial imaging devices offer a limited dynamic range of only 800: 1. For this reason, existing digital imaging devices can only provide a partial area of the dynamic range, and thus the obtained image has a problem of low exposure or overexposure as shown in FIG. Therefore, a dynamic range expansion method is required to improve the contrast.

The most widely used contrast enhancement method is the standard histogram equalization (SHE) technique. However, in spite of the simplicity of the technique, the SHE can not avoid the problem of low exposure or overexposure and color distortion because it changes brightness overall. To solve this problem, a number of modified versions have been proposed. One of these is bi-histogram equalization (BHE), which uses intensity averages to separate the input histogram into two lower histograms. A similar technique is the dualistic sub-image histogram equalization (DSIHE) technique, which uses the median of intensity values to separate the input histogram into two lower histograms. However, these techniques can not provide a higher degree of brightness preservation and contrast enhancement if the histogram distribution is focused on a particular area.

On the other hand, a recursive mean-separate histogram equalization (RMSHE) technique has been proposed that recursively separates input histograms into lower histograms using averaging. The main disadvantage of RMSHE is its infinite processing time due to the reliance on numerous discrete histograms. In addition, a gain-controllable clipped histogram equalization (GC-CHE) technique has been proposed that cuts the histogram and adaptively controls the histogram's maximum value and the intensity of the transform function. Although these histogram equalization techniques can improve contrast while preserving brightness, they can not avoid color distortion or extend the dynamic range.

The most widely used dynamic range extension method is to combine multiple images with different exposures. However, this method requires at least two low dynamic range (LDR) images and results in ghost defects due to motion in the scene. Several techniques have been proposed to eliminate motion-induced ghosting defects. One proposed a flexible, high-dynamic-range (HDR) imaging method that aligns LDR images using repetitive affine transformations. However, this method can only compensate for global motion and causes high computational load due to iterative processing. To compensate for global and local motion, a single image-based non-ghosted HDR imaging technique has been proposed that generates LDR images with two virtual different exposures from one input image using weighted histogram separation. However, this technique still produces color distortion.

On the other hand, some techniques used dark channels before extending the dynamic range. These techniques first extract low-exposure areas using dark channels, and then improve the contrast of low-exposure areas while preserving the background. The concept of dark channel priority is that in most non-sky patches, at least one color channel has very low intensity and near zero pixels. This means that not only low exposed areas but also specific color areas have very low strength. Therefore, the extracted area may contain a background, which results in loss of specific information.

Korean Published Patent Application No. 2013-0040321 (published on April 24, 2013) Korean Published Patent Application No. 2014-0011943 (Disclosure Date: January 29, 2014)

High dynamic range for contrast enhancement (IEEE Trans. Consumer Electronics 52 (4), 2006) Uneven Background Illumination Calculation for High Dynamic Range Display (IEEE Conf. Int. Conf. Image Processing, 2009) Dark channel priority based spatially adaptive contrast enhancement for background illumination compensation (IEEE Conf. Int. Conf. Acoustics, Speech and Signal Processing, 2013) Night image enhancement using improved dark channel priority (IEEE Conf. Int. Conf. Image Processing, 2013) Contrast Enhancement Using Bias Histogram Equalization (IEEE Trans. Consumer Electronics 43 (1), 1997) Image Enhancement Using Equivalent Region Double Subimage Histogram Equalization (IEEE Trans. Consumer Electronics 49 (2), 2003) Contrast Enhancement Using Recursive Average Split Histogram Equalization for Scalable Brightness Conservation (IEEE Trans. Consumer Electronics 45 (1), 1999) Adaptive Contrast Enhancement Using Gain Adjustable Cropped Histogram Equalization (IEEE Trans. Consumer Electronics 54 (4), 2008) Reconstruction of high-dynamic-range brightness map from photographs (Proc. ACM SIGGRAPH, 1997) Improved Flexible Synthesis to Eliminate Ghost Defects in High Dynamic Images (IEEE Trans. Consumer Electronics 57 (2), 2011) High-dynamic-range image processing without a single image-based ghost using local histogram expansion and spatially adaptive noise reduction (IEEE Trans. Consumer Electronics 57 (4), 2011) Removal of Single Image Haze Using Dark Channel Priority (IEEE Trans. Pattern Analysis, Machine Intelligence 33 (12), 2011) Multi-scale Retinex (IEEE Trans. Image Processing 6 (7), 1997) to fill the gap between color image and human perception of landscape,

It is an object of the present invention to improve contrast of a low-exposure area while preserving a background with only one image, dramatically reduce computation time for improving dynamic range, and generate HDR image without color distortion And to provide a dynamic range expansion apparatus and method capable of performing dynamic range expansion.

Another object of the present invention is to provide a method and an apparatus for enhancing the contrast of a low-exposure area while preserving a background using only one image, dramatically reducing computation time for improving the dynamic range, There is provided a computer-readable recording medium on which a program for causing a computer to execute a dynamic range expansion method capable of being created can be recorded.

According to another aspect of the present invention, there is provided a dynamic range expansion apparatus comprising: a color separator for separating colors of input images into color channels and outputting an intermediate image for each color channel; Generating a color channel conversion map for each of the color channels based on the intermediate image for each color channel and the atmospheric light for each color channel input to the image capturing device, combining the respective color channel conversion maps to generate a combined color channel conversion map A conversion map generation unit for generating a conversion map; A gain calculation unit for calculating a gain for extending a dynamic range based on the combined color channel conversion map; And a contrast enhancing unit for enhancing the contrast of the input image by multiplying the input image by the gain.

According to another aspect of the present invention, there is provided a dynamic range expansion method comprising: (a) outputting an intermediate image for each color channel by separating colors of input images into preset color channels; step; (b) generating a color channel conversion map for each of the color channels based on the atmospheric light for each color channel input to the intermediate image and image acquiring device for each color channel, combining the respective color channel conversion maps, Generating a map; (c) calculating a gain for extending the dynamic range based on the combined color channel conversion map; And (d) enhancing the contrast of the input image by multiplying the input image by the gain.

According to the dynamic range expansion apparatus and method of the present invention, by enhancing the dynamic range using a color-channel transmission (CCT) map, the contrast of the low-exposure area can be improved while preserving the background using only one image The computation time for improving the dynamic range can be drastically reduced, and the HDR image can be generated without color distortion.

1 shows an example of an image obtained by a conventional digital imaging apparatus,
2 shows an image acquisition model for a reduced dynamic range environment,
3 is a view showing a preferred embodiment of a dynamic range expansion apparatus according to the present invention,
4 is a view showing an example of an image having not only a low exposure area but also a primary color area having very low intensity,
5 is a diagram showing an example of a color channel conversion map for each color channel,
6 is a diagram illustrating an example of various color channel conversion maps for an input image,
FIG. 7 is a flowchart illustrating a dynamic range expansion method according to an exemplary embodiment of the present invention. FIG.
FIG. 8 is a view showing a 1600 × 1066 size car image and a display image used in an experiment performed to verify the performance of the dynamic range expansion apparatus and method according to the present invention,
9 is a diagram showing a dynamic range expansion result for the automobile image shown in FIG. 8,
FIG. 10 is an enlarged view of a rectangular area of the image shown in FIG. 9,
FIG. 11 is a diagram showing a dynamic range expansion result for the display image shown in FIG. 8,
12 is an enlarged view of a rectangular area of the image shown in FIG. 11,
FIG. 13 is a graph comparing the computation time for the dynamic range extension of the present invention and the conventional techniques with respect to the images shown in FIG.

Hereinafter, preferred embodiments of a dynamic range expansion apparatus and method according to the present invention will be described in detail with reference to the accompanying drawings.

FIG. 2 shows an image acquisition model for a reduced dynamic range environment. The present invention detects low exposure areas using a combined color-channels transmission (CCT) map and changes the brightness of the detected area for contrast enhancement. Since the CCT map represents the amount of light reaching the image sensor from the landscape, the brightness in the low-exposure area can be adjusted in a spatially adaptive manner. The expansion of the dynamic range in the following description means improvement of the contrast of the image.

3 is a view showing a preferred embodiment of the dynamic range expansion apparatus according to the present invention.

3, the dynamic range expansion apparatus 100 includes a color separation unit 110, a transformation map generation unit 120, a gain calculation unit 130, and a contrast enhancement unit 140.

The color separator 110 separates the hue of the input image into the color channels set in advance. For example, when the colors of the input image are separated based on red (R), green (G), and blue (B), the color separator 110 separates the intermediate images of the three channels of R, G, and B Output.

The transformation map generator 120 generates a CCT map for each color channel, and then combines each CCT map to generate a combined CCT map for contrast enhancement. The present invention improves the contrast of a low-exposure area by a spatially adaptive technique, and can express concrete information of a background and a foreground area, so that the dynamic range can be expanded without a ghost defect.

In general, an image is obtained by combining atmospheric light expressed by the following equation and light reflected by the surface of the object.

(X, y) is the combined CCT map, and f (x, y) is the combined input image, (x, y) is the pixel coordinates of the input image, A is the atmospheric light, ) Represents the object surface.

Based on the image acquisition model shown in Figure 2, the atmospheric light is directly input to the image sensor and reflected from the surface of the object to the opposite side of the image sensor. Therefore, the object area appears darker than the acquired image, and approaches 0 as shown in the following equation.

Here, c denotes a color channel.

Modified histogram equalization and HDR imaging are widely used to solve this problem. However, the modified histogram equalization method can change the pixel intensity, which does not belong to the low exposure area because it only shows the intensity distribution. Conversely, HDR images need to acquire LDR images of a number of different exposures, and LDR images result in ghosting defects when mixed with relative motion between images. To solve this problem, the present invention improves the contrast of the low-exposure area while changing the brightness of the low-exposure area and preserving the brightness of the background. To this end, the present invention extracts a low-exposure area using a combined CCT map representing the amount of light input to the image sensor.

A small value of the CCT map means that less light is input, which can be considered as a low exposure area. This approach is similar to the dark channel prior-based backlighting compensation (DCPBLC) method, which primarily uses dark channels for haze removal or fog removal. The dark channel preference assumes that for most non-sky patches, at least one color channel has very low intensity and near zero pixels. This means that not only the low-exposure area but also the primary color area has very low intensity.

Fig. 4 is a view showing an example of such a case. In the input image shown in FIG. 4A, the sky area includes primary colors such as red and blue. FIG. 4B shows a transformation map using dark channel priority, and FIG. 4C shows a color-coded transformation map. The red areas need to be enhanced, while the blue areas need to maintain their original brightness. Since the primary color has a high value in only one or two channels, it belongs to the low exposure area as shown in Fig. 4 (c).

The conversion map generation unit 120 first generates a CCT map for each color channel by using the following equation to accurately extract only the low-exposure area.

Where, t ^c (x, y) is the transformation map by the color channels, (x, y) are the pixel coordinates of the input image, A ^c is the standby light maximum intensity in each color channel, g ^c is of the input image C is a color channel identifier, R is red, G is green, and B is blue.

As described above, since the reflective surface is opposite to the image sensor, the intensity of the area of the object is close to zero. Therefore, equation (3) can be summarized as follows.

A CCT map for each color channel is shown in Fig.

The high intensity value and the low intensity value respectively indicate the low exposure area and the brightness preservation area in the input image. In the red channel conversion map, the sky area is darker than the other conversion map due to the sunset color. This means that even if the area is not exposed low, it can be regarded as a low exposed area in the color channel. Therefore, the conversion map generator 120 combines the conversion maps of the three color channels to accurately extract the low-exposure area to generate a combined CCT map. This process can be expressed by the following equation.

는 겹합된 CCT 맵을 의미한다.here,

Means an overlapped CCT map.

The low exposure area can be successfully extracted because it has low intensity in the combined CCT map. In addition, the combined CCT map is generated by multiplying the CCT maps corresponding to the three color channels, thereby avoiding errors resulting from specific colors. FIG. 6 shows an example of various CCT maps for an input image. 6 (a) is an input image, (b) is a red channel conversion map, (c) is a green channel conversion map, and (d) is a blue channel conversion map.

Since the combined CCT map represented by Equation (5) includes a value between 0 and 1, it is necessary to scale the values for image intensity by the following equation.

Here, β represents a normalization factor, and β = 1.7 is used experimentally.

The gain calculating unit 130 calculates a gain for extending the dynamic range by the following equation based on the combined CCT map or the scaled combined CCT map.

Here, a (x, y) is a gain.

The contrast enhancing unit 140 enhances the contrast by multiplying the gain of the input image by the following equation in order to expand the dynamic range.

는 대비 향상된 영상이다.here,

Is a contrast enhanced image.

Since the gain includes the amount of light that is calculated from the combined CCT map and input to the image sensor from the landscape, the dynamic range extension device according to the present invention can change the brightness of the low- exposure area in a spatially adaptive manner without unnatural boundary defects have.

FIG. 7 is a flowchart illustrating a dynamic range expansion method according to an exemplary embodiment of the present invention. Referring to FIG.

The color separation unit 110 divides the color of the input image into the color channels according to the preset color channels to generate the intermediate images g ^R (x, y), g ^G (x, y), g ^B ) (S700). Next, the transformation map generation unit 120 generates a CCT map using Equation (4) based on the intermediate image for each color channel, and then generates a combined CCT map by multiplying each CCT map (S710). Next, the transformation map generation unit 120 scales the combined CCT map based on the minimum value and the maximum value (S720). Scaling of this combined CCT map is an optional process. Next, the gain calculating unit 130 calculates a gain for expanding the dynamic range by using the combined CCT map or the scaled combined CCT map as the exponent of e (S730). Next, in order to increase the dynamic range, the contrast enhancement unit 140 multiplies the gain of the input image by the following equation to improve contrast (S740).

In order to verify the performance of the dynamic range expansion apparatus and method according to the present invention, a car and a display image of 1600 × 1066 size were used. As shown in Fig. 8, each image was obtained in a reduced dynamic range state. The results of the present invention were compared with results by DCPBLC, DSIHE, GC-CHE and multiscale retinex (MSR) techniques.

Fig. 9 shows the dynamic range expansion result. It can be seen that Figures 9 (b) to 9 (d) include color distortion near the wall. Although the MSR improves the color distortion near the wall, the brightness enhancement is not sufficient as shown in Fig. 9 (e). Alternatively, the result of the present invention can successfully extend the dynamic range without color distortion, as can be seen in FIG. 9 (f). For a clearer comparison, the rectangular area of FIG. 9 (a) is cut out and expanded to be shown in FIG.

Since the shutter speed for acquiring the input image is set to the normal exposure with respect to the headlight area, the other area is exposed low as shown in Fig. 10 (a). Although the DSIHE and GC-CHE techniques improve the brightness, the headlight area also saturates, as shown in Figures 10 (c) and 10 (d). The result of DCPBLC and MSR preserves the brightness of the headlight area, but does not improve the brightness enough to recognize the car's logo. In contrast, the present invention detects and improves low-exposure areas for expansion of the dynamic range and improves the brightness of the low-exposure areas while correcting the brightness of the headlight area as shown in Fig. 10 (f).

As shown in FIG. 11 (a), the background display panel in the display image serves as a backlight source. As can be seen from (b) to (e) of FIG. 11, existing methods can not simultaneously enhance objects and background. Alternatively, as shown in FIG. 11 (f), the present invention improves only the low-exposure area while preserving the background. For a clearer comparison, the rectangular region of FIG. 11 (a) is cut out and expanded to be shown in FIG.

As can be seen in FIG. 12 (b), although the DCPBLC scheme improves the contrast of the object region, the lattice monitor boundaries in the background become blurred due to saturation. Conversely, the monitor boundaries become thicker in the results of DSIHE and GC-CHE, and as a result, the contrast of the object region is not sufficiently improved as can be seen in Figures 12 (c) and 12 (d). Although the MSR partially improves the contrast of the object region, the object region is still exposed to low light. As shown in FIG. 12 (e), the contrast of the display panel deteriorates because the display panel becomes brighter. In contrast, the present invention improves the contrast of low-exposure areas while preserving the background. A successful result of the dynamic range expansion using the present invention is shown in Fig. 12 (f).

FIG. 13 is a graph comparing the computation time for the dynamic range extension of the present invention and the conventional techniques with respect to the images shown in FIG. Since DCPBLC and DSIHE generate two images for contrast enhancement, calculation time can not be reduced. The GC-CHE must calculate the total intensity density of the input image to set the shear gain, which is a major factor in increasing the computation time. The MST estimates the light source and reflection in each color channel, resulting in the longest calculation time. In contrast, the present invention improves the contrast of a low-exposure area in order to expand a dynamic range by using a single image, so that the calculation time can be drastically reduced.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer-readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation in the embodiment in which said invention is directed. It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the appended claims.

Claims (10)

delete delete A color separator for separating colors of an input image into color channels which are set in advance and outputting an intermediate image for each color channel;
Generating a color channel conversion map for each of the color channels based on the intermediate image for each color channel and the atmospheric light for each color channel input to the image capturing device, combining the respective color channel conversion maps to generate a combined color channel conversion map A conversion map generation unit for generating a conversion map;
A gain calculation unit for calculating a gain for extending a dynamic range based on the combined color channel conversion map; And
And a contrast enhancing unit for enhancing the contrast of the input image by multiplying the input image by the gain,
Wherein the conversion map generator scales the combined color channel conversion map based on the minimum value and the maximum value of the combined color channel conversion map,
Wherein the gain calculator calculates the gain based on the scaled combined color channel conversion map,
The transformation map generator scales the combined color channel transformation map by the following equation A,
Wherein the gain calculating unit calculates the gain by the following expression (B): " (1) "
[Mathematical formula A]

[Mathematical expression B]

Where n (x, y) is the scaled combined color channel conversion map,

The combined color channel conversion map,

Is a minimum value of the combined color channel conversion map,

(X, y) is a maximum value of the combined color channel conversion map, (x, y) is each pixel coordinate of the input image,? Is a normalization factor experimentally determined, and a (x, y) is the gain. The method of claim 3,
Wherein the conversion map generator generates the color channel-specific conversion map according to Equation (C): < EMI ID =
[Mathematical expression C]

Where, t ^c (x, y) is the transformation map by the color channels, (x, y) are the pixel coordinates of the input image, A ^c is the standby light maximum intensity in each color channel, g ^c is of the input image C is a color channel identifier, R is red, G is green, and B is blue. A color separator for separating colors of an input image into color channels which are set in advance and outputting an intermediate image for each color channel;
Generating a color channel conversion map for each of the color channels based on the intermediate image for each color channel and the atmospheric light for each color channel input to the image capturing device, combining the respective color channel conversion maps to generate a combined color channel conversion map A conversion map generation unit for generating a conversion map;
A gain calculation unit for calculating a gain for extending a dynamic range based on the combined color channel conversion map; And
And a contrast enhancing unit for enhancing the contrast of the input image by multiplying the input image by the gain,
Wherein the gain calculating unit calculates the gain by the following equation (C): "
[Mathematical expression C]

Where a (x, y) is the gain,

Is the combined color channel conversion map, and (x, y) is the pixel coordinates of the input image. delete delete (a) outputting an intermediate image for each color channel by separating the hue of the input image for each color channel set in advance;
(b) generating a color channel conversion map for each of the color channels based on the atmospheric light for each color channel input to the intermediate image and image acquiring device for each color channel, combining the respective color channel conversion maps, Generating a map;
(c) calculating a gain for extending the dynamic range based on the combined color channel conversion map; And
(d) enhancing the contrast of the input image by multiplying the input image by the gain,
Wherein in the step (b), the combined color channel conversion map is scaled based on a minimum value and a maximum value of the combined color channel conversion map,
In the step (c), the gain is calculated on the basis of the scaled combined color channel conversion map,
In the step (b), the combined color channel conversion map is scaled by the following equation (A)
And in the step (c), the gain is calculated by the following equation (B): "
[Mathematical formula A]

[Mathematical expression B]

Where n (x, y) is the scaled combined color channel conversion map,

The combined color channel conversion map,

Is a minimum value of the combined color channel conversion map,

(X, y) is a maximum value of the combined color channel conversion map, (x, y) is each pixel coordinate of the input image,? Is a normalization factor experimentally determined, and a (x, y) is the gain. 9. The method of claim 8,
Wherein, in the step (b), the color channel-specific conversion map is generated by the following equation (C): < EMI ID =
[Mathematical expression C]

Where, t ^c (x, y) is the transformation map by the color channels, (x, y) are the pixel coordinates of the input image, A ^c is the standby light maximum intensity in each color channel, g ^c is of the input image C is a color channel identifier, R is red, G is green, and B is blue. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the dynamic range expansion method according to claim 8 or 9.

Images