KR20110048811A - Method and apparatus for converting dynamic ranges of input images - Google Patents

Abstract

PURPOSE: A device and method for converting a dynamic range of an input image are provided to obtain an LDR(Low Dynamic Range) image suitable for visual system of a human from a HDR(High Dynamic Range) image. CONSTITUTION: A global brightness controller(110) globally controls a luminance of an input image with a scale factor. A local chromaticity controller(120) controls a color of the input image. A local brightness controller(130) regionally controls the luminance of the input image. A tone mapping unit(140) outputs a displayable colo in a display device through the combination of a globally controlled brightness and a brightness control function.

Description

Method and Apparatus for converting dynamic ranges of input images}

TECHNICAL FIELD The present invention relates to image processing, and more particularly, to a technique for converting a high dynamic range of an image existing in nature into a low dynamic range that can be reproduced in a conventional display device.

High dynamic range (HDR) imaging is an attractive technique that can mimic human visual capabilities. HDR imaging is used in a variety of applications, including scientific and medical visualization, satellite photography, digital cameras, and digital cinemas. In particular, the ability to capture luminance in real world scenes has been recognized as an essential function of digital cameras.

As shown in FIG. 1, the luminance of the real world having a range of 10 ⁸ : 1 can be expressed only by HDR imaging that covers from sunlight to starlight. However, in general, display devices such as plasma, CRT, LCD, and projectors used for image display have a low dynamic range (LDR) of about 10 ² to 10 ³ : 1.

HDR monitors may be widely used in the near future, but they are very rare and expensive at the moment. Therefore, when displaying an HDR image in conventional display devices, it is necessary to go through a process of significantly reducing the range of values (called tone reproduction or tone mapping).

The HDR image is lost as the details of the very dark or very bright areas are converted to LDR images during the tone reproduction process. Therefore, it may not be a suitable image in the human visual system (HVS).

For this reason, there is a need for a tone reproducing technique capable of preserving the detail included in the HDR image while converting the HDR image into a displayable range. Most conventional tone reproduction techniques have focused on compressing HDR images and evaluating their quality.

However, this conventional tone reproduction algorithm simply converts the HDR pixels into LDR values, and thus fails to properly preserve details such as edges and the like, especially in very dark or bright areas. This is because these techniques only care about preserving all the contrast locally, without creating unnecessary artifacts.

On the other hand, among these conventional techniques, there is a local operator technique that uses the spatial modeling function of the surrounding pixels. Here, the operator functions use the average of the luminance value of the surrounding pixel of the value of each pixel to be converted. However, it is difficult to accurately determine the size of the surrounding pixel region and the computation amount is required for such determination. In addition, this method has a problem in that various kinds of artificiality are generated as the prediction error increases.

Therefore, there is a need to develop a dynamic range conversion technique that is suitable for human visual systems and can preserve the details of an image such as an edge.

An object of the present invention is to provide a dynamic range conversion technology capable of converting into an LDR image while preserving details of an image such as an edge of an HDR image.

Technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided an apparatus for converting a dynamic range of an input image, the global luminance controller configured to globally adjust the luminance of the input image by a scale factor; A local chromaticity adjuster for adjusting the chromaticity of the input image using a chromaticity control coefficient; A local luminance controller for obtaining a luminance adjustment function having different characteristics for each section according to characteristics of each section of the luminance of the input image to locally adjust the luminance of the input image; And a tone mapping unit configured to output a chromaticity that can be displayed on a display device by combining the globally adjusted luminance, the adjusted chromaticity, and the luminance adjustment function.

According to an aspect of the present invention, there is provided a method of converting a dynamic range of an input image, the method comprising: globally adjusting the brightness of the input image by a scale factor; (b) adjusting the chromaticity of the input image using a chromaticity control coefficient; (c) obtaining a brightness adjustment function having different characteristics for each section according to the characteristics of each section of the brightness of the input image to locally adjust the brightness of the input image; And (d) outputting a chromaticity that can be displayed on a display device by combining the globally adjusted luminance, the adjusted chromaticity, and the luminance adjustment function.

As described above, according to the dynamic range conversion technique of the present invention, an LDR image suitable for a human visual system can be obtained from an HDR image without excessively increasing the amount of computation.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms, and only the present embodiments are intended to complete the disclosure of the present invention, and the general knowledge in the art to which the present invention pertains. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

2 shows the concept of an ideal LDR display.

The scene of the real world is converted into a captured image by the imaging device. These captured images are converted to LDR images by tone mapping and displayed on the LDR display. The observer eventually observes the image as it appears on the LDR display, not the scene in the real world.

On the other hand, if the observer can observe the scene of the real world, it will not be recognized that the image displayed by the display and the image observed by the naked eye are the same. However, the key is how close the image displayed by the LDR display can be to the image observed in the real world with the observer's naked eye. As such, the implementation of the image by the display device that is more similar to the scene of the real world depends on how accurate tone mapping is performed separately from improving the hardware performance of the display device.

FIG. 3 compares an LDR image (a) obtained using a conventional linear scaling technique with an LDR image (b) obtained by lowering a luminance of a high luminance portion. The images of FIG. 3 are good examples showing how detailed images should be preserved in the high luminance portion. The observer will not at least think of the image on the left as a realistic image.

Accordingly, the present invention seeks to obtain a more natural image in the human visual system while preserving the detail of the LDR image obtained after the dynamic range conversion by using global adjustment and local adjustment together.

4 shows a configuration of a dynamic range conversion apparatus 100 according to an embodiment of the present invention. The dynamic range converting apparatus 100 may include a color space converter 105, a global luminance adjuster 110, a local chroma adjustment unit 120, and a local luminance adjuster 130. and a luminance mapping unit 140 and a tone mapping unit 140.

The color space converter 105 converts the input RGB image into a tristimulus values image, for example, an XYZ image of a Commission Internationale de l'Eclairage (CIE). The standard matrix defined by ITU-R BT.709 is represented by Equation 1 below.

The global luminance controller 110 performs a "global adjustment" on the luminance component of the converted XYZ image. The global adjustment is to reduce the overall dynamic range of the image, and it is necessary to reduce the range to an appropriate range according to the characteristics of the image. In other words, the width of the reduction of the dynamic range is determined uniformly according to the characteristics of the image.

First, the global luminance controller 110 may calculate an average luminance, in particular, a logarithmic average luminance Y _av , based on a range of total luminance values of an image as shown in Equation 2 below.

Here, N is the number of pixels in the image, and Y (x, y) is the input luminance value of the pixel at the x and y positions of the image. However, when Y (x, y) is 0, a very small constant value δ may be added in order to prevent an error of a log function during operation.

The global luminance controller 110 may calculate the scale factor a as shown in Equation 3 using the average luminance and the maximum and minimum luminance of the input image.

Here, Y _max is the maximum luminance of the input image, and Y _min is the minimum luminance of the input image. As a result, f may be regarded as a distance (deviation) between the maximum luminance and the minimum luminance and the average luminance of the input image in the image. However, since f may have both a positive value and a negative value, the scale factor (a) may be represented by 2 ^f as shown in Equation 3 so that the scale factor (a) may be always positive.

On the other hand, in order to be able to obtain the f value more stably in the equation (3), it is necessary to exclude the case of having a very low luminance and the case of having a very high luminance for reasons such as noise. Experimentally, it is desirable to exclude pixels corresponding to 3% of the total, which belong to a very dark or very bright part of the input image. In this case, the calculations of Equations 2 and 3 will naturally be made for the remaining 97% of pixels.

The global luminance controller 110 eventually adjusts the luminance components of the input image uniformly, that is, globally, according to Equation 4 below.

A is a value having a high correlation with a dark or light part of an image.

As such, the luminance of the input image globally adjusted by the global luminance controller 110 is provided to the local chromaticity controller 120.

The local chromaticity adjusting unit 120 obtains a chromaticity adjustment coefficient for determining the degree of adjustment of the chromaticity, and applies the chromaticity adjustment coefficient to each of the RGB components of the input image to obtain the adjusted RGB components.

In this case, although the local chromaticity adjuster 120 may directly use the RGB component of the input image, preferably, the global luminance adjuster 110 uses the Y 'which is the result of the global adjustment of the luminance component. It is desirable to restore RGB. In other words, the local chromaticity adjusting unit 120 restores the R, G, and B components of the input image by using Equations 5 and 6 below.

Of course, R, G, and B in Equation 5 may be different from R, G, and B of the first input image since Y is changed to Y '.

Meanwhile, the local chromaticity adjusting unit 120 calculates a chromaticity adjustment coefficient D for applying to each of the color components R, G, and B as shown in Equation 7 below.

Where F is a constant proportional to the chromaticity adjustment coefficient and L _A is the luminance value of the adapted field.

The proportional constant F is a value that can be determined empirically and has a value of 1.0 for an ambient image of average brightness, 0.9 for a slightly darker ambient image, and 0.8 for a darker ambient image. In addition, L _A means a luminance value located at the top 20% of the luminance value of white in the adapted field, that is, the upper fifth quartile luminance value.

As such, after calculating the chromaticity adjustment coefficient as described above, the local chromaticity adjustment unit 120 applies the chromaticity adjustment coefficient to the restored color components R, G, and B and is adjusted as in Equation 8 below. The color components (R _c , G _c , B _c ) are obtained.

Where R _w , G _w , and B _w are the values in the RGB color space calculated using the tristimulus values of white described in any one of the environments listed in Table 1, that is, X _w , Y _w , and Z _w , respectively. . In from X _w, Y _w, Z _w in the calculation to obtain the R _w, G _w, B _w described above can be used to Equation 5, and 6.

Meanwhile, although the local chromaticity adjusting unit 120 may provide the obtained R _c , G _c, and B _c to the tone mapping unit 140 with the adjusted final chromaticity value, preferably, the values are again represented by the following equation. The final chromaticity values R ', G' and B 'adjusted by the conversion by 9 and 10 are obtained and provided to the tone mapping unit 140.

The local luminance controller 130 performs the characteristic of each section constituting the dynamic range of the luminance of the image separately from the global luminance controller 110 uniformly adjusting the entire dynamic range of the luminance according to the characteristics of the image. In consideration of the (regional characteristic), a luminance control function for individual luminance adjustment for each section is obtained. To this end, the local luminance controller 130 uses the luminance Y provided by the color space converter 105.

First, the local luminance controller 130 calculates a pixel count of the luminance histogram by dividing the pixel count into a plurality of bins. For example, the local luminance controller 130 may determine the size Δb of the section Bin as shown in Equation 11 below.

Here, Y _max is the maximum luminance of the input image, Y _min is the minimum luminance of the image, and NB (number of bins) is the number of divided sections.

For example, the probability distribution function when the representative image of "Memorial church" is divided into ten luminance sections may be illustrated as shown in FIG. 5. In this probability distribution function, the value in each section Bin has a value between 0 and 1, and the value of all the following sections is added to 1.

This probability distribution function t may be calculated as in Equation 12 below.

As such, when the probability distribution function based on the luminance histogram is determined, a weight w corresponding to the value of the determined probability distribution function may be determined. This weight should be set to have a larger value as the probability distribution function has a larger value (ie, a bin having a higher frequency). This means that a high weight is given to a section having a high frequency and a low weight is given to a section having a low frequency among all dynamic ranges of the input image. In other words, when the input image is converted by tone mapping, it is more likely that the details belonging to the frequency-prone section are preserved more than the frequency-prone section. If the image in which the frequency is concentrated only in a specific section is mapped with the same weight as other sections, the dynamic range is reduced, and the detail in the specific section is likely to be lost. Therefore, it is necessary to vary the weight according to the frequency of the interval.

6 shows a graph for obtaining a weight according to an embodiment of the present invention. As a graph showing the relationship between the probability distribution t and the weight w, various relation graphs such as a straight line can be considered. However, in the present invention, a quadratic curve as shown in FIG. 6 is used. In FIG. 6, when the probability distribution t increases from 0 to 0.6, the weight w has a value between approximately 0.8 and 1.6. For example, since the probability distribution of Bin 4 in FIG. 5 is 0.38, the weight w is approximately 1.5 when this value is substituted in the horizontal axis of FIG. 6. This graph in FIG. 6 can be obtained empirically and various modifications are possible as required by those skilled in the art.

The local luminance controller 130 reflects the obtained weight and calculates a luminance adjustment function for an arbitrary luminance value. The brightness control function provides an appropriate reflection ratio when obtaining the output brightness from the input brightness, which is changed by a variable weight for each section. The luminance adjustment function L _d ′ may be calculated by, for example, Equation 13 below.

Here, Y _i represents the actual input luminance in the section i, Y _{max (i)} represents the maximum luminance in the section i, Y _{min (i)} represents the minimum luminance in the section i.

In Equation 13, the weight w _i is a value that varies for each section (i.e., the same weight is applied in the same section), and as the value increases, the weight w _i increases in the corresponding section. Therefore, as the weight increases, the corresponding section is enlarged as compared with other sections. In addition, it can be seen that the luminance adjustment function L _d ′ is not a fixed constant value but a function that varies according to the input luminance Y _i . As such, the brightness control function calculated by the local brightness controller 130 is provided to the tone mapping unit 140.

Again, referring to FIG. 4, the tone mapping unit 140 may include a globally adjusted luminance value provided by the global luminance controller 110, an adjusted chromaticity value provided by the local chromaticity adjuster 120, and a local luminance adjuster ( 130, the display device displays the chromaticity values (R _d , G _d , B _d ) that can be displayed on the display device, that is, the LDR image.

The tone mapping unit 140 may obtain chromaticity values R _d , G _d , and B _d of the LDR image according to Equation 14, for example. As a result, the first HDR image (R, G, B) is input to the dynamic range converter 100, and finally the LDR image (R _d , G _d , B _d ) is output.

In Equation 14, R ', G', and B 'are chromaticity values provided by the local chromaticity adjusting unit 120, and L' _d is a luminance adjustment function provided by the local luminance adjusting unit 130 (a function of input luminance). Y 'is a luminance value provided by the global luminance controller 110. In addition, s may be a value of approximately 0.45 as a user parameter used for gamma correction.

Up to now, each component of FIG. 4 is a software such as a task, a class, a subroutine, a process, an object, an execution thread, a program, a field-programmable gate array (FPGA), or an ASIC (application) performed in a predetermined area of memory. It may be implemented in hardware such as a specific integrated circuit, or may be a combination of the software and the hardware. The components may be included in a computer readable storage medium or a part of the components may be distributed over a plurality of computers.

In addition, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical functions. It is also possible in some alternative implementations that the functions mentioned in the blocks occur out of order. For example, the two blocks shown in succession may in fact be executed substantially concurrently, and the blocks may sometimes be executed in the reverse order, depending on the corresponding function.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

1 shows the luminance range (HDR) of scenes in the real world.

2 shows the concept of an ideal LDR display.

3 is a view comparing an LDR image obtained using a conventional linear scaling technique with an LDR image obtained by lowering a luminance of a high luminance portion.

Figure 4 is a block diagram showing the configuration of a dynamic range conversion apparatus according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating a probability distribution function when a “memorial church” image is divided into ten luminance sections. FIG.

6 is a diagram showing a graph for obtaining a weight according to an embodiment of the present invention.

 (Symbol description of main part of drawing)

100: dynamic range conversion unit 105: color space conversion unit

110: global luminance adjusting unit 120: local chromaticity adjusting unit

130: local luminance control unit 140: tone mapping unit

Claims (16)

A global luminance controller configured to globally adjust the luminance of the input image by the scale factor; A local chromaticity adjuster for adjusting the chromaticity of the input image using a chromaticity control coefficient; A local luminance controller for obtaining a luminance adjustment function having different characteristics for each section according to characteristics of each section of the luminance of the input image to locally adjust the luminance of the input image; And And a tone mapping unit configured to output the chromaticity displayable on a display device by combining the globally adjusted luminance, the adjusted chromaticity, and the luminance adjustment function. The method of claim 1 wherein the scale factor is And converting the dynamic range of the input image, the value representing the distance between the maximum luminance and the minimum luminance of the input image and the average luminance. The method of claim 1, wherein the chromaticity adjustment coefficient And a luminance value located at a top 20% of the white luminance value of the input image as a variable. The method of claim 1, And a color space converter for converting the chromaticity of the input image into an XYZ image of a Commission Internationale de l'Eclairage (CIE). The method of claim 1, wherein the adjustment function comprises a weight for each interval, The weight for each section has a larger value as the frequency of the luminance value of the input image increases in a corresponding section, and has a smaller value as the frequency decreases. The method of claim 5, wherein the weighted intervals and the frequency A device for converting a dynamic range of an input image having a relation according to a quadratic curve graph. The method of claim 5, wherein the weight is An apparatus for converting a dynamic range of an input image having a range between 0.8 and 1.6. According to claim 1, wherein the tone mapping unit The greater the luminance control function, the greater the proportion of the output chromaticity, the apparatus for converting a dynamic range of the input image. (a) globally adjusting the brightness of the input image by the scale factor; (b) adjusting the chromaticity of the input image using a chromaticity control coefficient; (c) obtaining a brightness adjustment function having different characteristics for each section according to the characteristics of each section of the brightness of the input image to locally adjust the brightness of the input image; And and (d) combining the globally adjusted luminance, the adjusted chromaticity, and the luminance adjustment function to output a chromaticity that can be displayed on a display device. The method of claim 9, wherein the scale factor is And converting the dynamic range of the input image to a value representing the distance between the maximum luminance and the minimum luminance of the input image and the average luminance. The method of claim 9, wherein the chromaticity adjustment coefficient And a luminance value positioned at an upper 20% of the white luminance value of the input image as a variable. 10. The method of claim 9, prior to step (a), And converting the chromaticity of the input image into an XYZ image of a Commission Internationale de l'Eclairage (CIE). The method of claim 9, wherein the adjustment function comprises a weight for each interval, The weight for each section has a larger value as the frequency of the luminance value of the input image is larger in a corresponding section, and has a smaller value as the frequency is smaller, and converts a dynamic range of the input image. The method of claim 13, wherein the weighted intervals and the frequency A method for transforming a dynamic range of an input image having a relationship according to a quadratic curve graph. The method of claim 13, wherein the weight is A method for converting a dynamic range of an input image having a range between 0.8 and 1.6. The method of claim 9, wherein step (d) And increasing the output chromaticity in proportion to the larger the luminance control function.

Images