KR20110064631A - Method and apparatus for converting dynamic ranges of input images - Google Patents

Abstract

PURPOSE: A method and an apparatus for changing the dynamic range of an input image are provided to acquire an LDR(Low Dynamic Range) image suitable for the visual system of the human from an HDR(High Dynamic Range) image. CONSTITUTION: According to the section feature of the brightness of an input image, a parameter generator(110) creates control parameters according to a section. A brightness control unit(120) controls the brightness of the input image according to the section. A chromaticity control unit(130) controls the chromaticity of the input image. Using the combination of the controlled chromaticity and the controlled brightness, a tone mapping unit(140) performs mapping to the color which can be displayed on a display device.

Description

Method and Apparatus for converting dynamic ranges of input images}

TECHNICAL FIELD The present invention relates to image processing, and more particularly, to a technique for converting a high dynamic range of an image existing in nature into a low dynamic range that can be reproduced in a conventional display device.

High dynamic range (HDR) imaging is an attractive technique that can mimic human visual capabilities. HDR imaging is used in a variety of applications, including scientific and medical visualization, satellite photography, digital cameras, and digital cinemas. In particular, the ability to capture luminance in real world scenes has been recognized as an essential function of digital cameras.

As shown in FIG. 1, the luminance of the real world having a range of 10 ⁸ : 1 can be expressed only by HDR imaging that covers from sunlight to starlight. However, in general, display devices such as plasma, CRT, LCD, and projectors used for image display have a low dynamic range (LDR) of about 10 ² to 10 ³ : 1.

HDR monitors may be widely used in the near future, but they are very rare and expensive at the moment. Therefore, when displaying an HDR image in conventional display devices, it is necessary to go through a process of significantly reducing the range of values (called tone reproduction or tone mapping).

The HDR image is lost as the details of the very dark or very bright areas are converted to LDR images during the tone reproduction process. Therefore, it may not be a suitable image in the human visual system (HVS).

For this reason, there is a need for a tone reproducing technique capable of preserving the detail included in the HDR image while converting the HDR image into a displayable range. Most conventional tone reproduction techniques have focused on compressing HDR images and evaluating their quality.

However, this conventional tone reproduction algorithm simply converts the HDR pixels into LDR values, and thus fails to properly preserve details such as edges and the like, especially in very dark or bright areas. This is because these techniques only care about preserving all the contrast locally, without creating unnecessary artifacts.

On the other hand, among these conventional techniques, there is a local operator technique that uses the spatial modeling function of the surrounding pixels. Here, the operator functions use the average of the luminance value of the surrounding pixel of the value of each pixel to be converted. However, it is difficult to accurately determine the size of the surrounding pixel region and the computation amount is required for such determination. In addition, this method has a problem in that various kinds of artificiality are generated as the prediction error increases.

Therefore, there is a need to develop a dynamic range conversion technique that is suitable for human visual systems and can preserve the details of an image such as an edge.

An object of the present invention is to provide a dynamic range conversion technology capable of converting into an LDR image while preserving details of an image such as an edge of an HDR image.

Technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

An apparatus for converting a dynamic range of an input image according to an embodiment of the present invention for achieving the technical problem, the control parameter used to adjust the brightness of the input image characteristics of each section of the brightness of the input image A parameter generator for generating sections according to the intervals; A brightness controller which adjusts the brightness of the input image for each section by using the generated control parameter to obtain a final brightness; A chromaticity controller configured to adjust chromaticity of the input image based on the luminance adjusted for each section; And a tone mapping unit for combining the adjusted luminance and the adjusted chromaticity for each section to map the chromaticities that can be displayed on the display device.

According to an aspect of the present invention, a method for converting a dynamic range of an input image includes: (a) a control parameter used to adjust the brightness of the input image; Generating each section according to the characteristics of each section of the section; (b) obtaining final luminance by adjusting the luminance of the input image for each section using the generated control parameter; (c) adjusting chromaticity of the input image based on the luminance adjusted for each section; And (d) combining the adjusted luminance and the adjusted chromaticity for each of the sections to map the chromaticities that can be displayed on the display device.

As described above, according to the dynamic range conversion technique of the present invention, an LDR image suitable for a human visual system can be obtained from an HDR image without excessively increasing the amount of computation.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms, and only the present embodiments are intended to complete the disclosure of the present invention, and the general knowledge in the art to which the present invention pertains. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

2 shows the concept of an ideal LDR display.

The scene of the real world is converted into a captured image by the imaging device. These captured images are converted to LDR images by tone mapping and displayed on the LDR display. The observer eventually observes the image as it appears on the LDR display, not the scene in the real world.

On the other hand, if the observer can observe the scene of the real world, it will not be recognized that the image displayed by the display and the image observed by the naked eye are the same. However, the key is how close the image displayed by the LDR display can be to the image observed in the real world with the observer's naked eye. As such, the implementation of the image by the display device that is more similar to the scene of the real world depends on how accurate tone mapping is performed separately from improving the hardware performance of the display device.

FIG. 3 compares an LDR image (a) obtained using a conventional linear scaling technique with an LDR image (b) obtained by lowering a luminance of a high luminance portion. The images of FIG. 3 are good examples showing how detailed images should be preserved in the high luminance portion. The observer will not at least think of the image on the left as a realistic image.

Accordingly, the present invention seeks to obtain a more natural image in the human visual system while preserving the detail of the LDR image obtained after the dynamic range conversion by using global adjustment and local adjustment together.

4 shows a configuration of a dynamic range conversion apparatus 100 according to an embodiment of the present invention. The dynamic range converting apparatus 100 includes a color space converting unit 105, a parameter generating unit 110, a luminance adjusting unit 120, a chromaticity adjusting unit 130, and a tone mapping unit 140. Can be.

The color space converter 105 converts the input RGB image into a tristimulus values image, for example, an XYZ image of a Commission Internationale de l'Eclairage (CIE). The standard matrix defined by ITU-R BT.709 is represented by Equation 1 below.

The parameter generator 110 generates a control parameter that enables to adjust the brightness of the brightness controller 120. In particular, the control parameter is generated for each section according to the characteristics of each section of the luminance of the input image. The parameter may include a weight w _i representing a characteristic of each section, that is, a bin and an average value Y _ai of luminance of each section. The subscript i is an index for identifying each section Bin, and the subscript a is an average value. Hereinafter, a specific embodiment of calculating the parameters, that is, the weights w _i and Y _ai will be described.

First, the parameter generator 110 generates a luminance histogram based on the pixel count of the luminance Y obtained as in Equation 1 above. In this case, the luminance histogram is displayed by dividing the interval Bin. For example, the parameter generator 110 may determine the size Δb of the section Bin as shown in Equation 2 below.

Here, Y _max represents the maximum luminance of the input image, Y _min represents the minimum luminance of the image, and NB represents the number of divided sections. For example, the probability distribution function when the representative image of "Memorial church" is divided into ten luminance sections may be illustrated as shown in FIG. 5. In this probability distribution function, the value in each section Bin has a value between 0 and 1, and the value of all the following sections is added to 1.

This probability distribution function t may be calculated as in Equation 3 below.

As such, when the probability distribution function based on the luminance histogram is determined, a weight w _i corresponding to the value of the determined probability distribution function may be determined. This weight should be set to have a larger value as the probability distribution function has a larger value (ie, a bin having a higher frequency). This means that a high weight is given to a section having a high frequency and a low weight is given to a section having a low frequency among all dynamic ranges of the input image.

In other words, when the input image is converted by tone mapping, it is more likely that the details belonging to the frequency-prone section are preserved more than the frequency-prone section. If the image in which the frequency is concentrated only in a specific section is mapped with the same weight as other sections, the dynamic range is reduced, and the detail in the specific section is likely to be lost. Therefore, it is necessary to vary the weight according to the frequency of the interval.

6 shows a graph for obtaining a weight according to an embodiment of the present invention. As a graph showing the relationship between the probability distribution t and the weight w, various relation graphs such as a straight line can be considered. However, in the present invention, a quadratic curve as shown in FIG. 6 is used. In FIG. 6, when the probability distribution t increases from 0 to 0.6, the weight w has a value between approximately 0.8 and 1.6. For example, since the probability distribution of Bin 4 in FIG. 5 is 0.38, the weight w is approximately 1.5 when this value is substituted in the horizontal axis of FIG. 6. The specific form of the graph in FIG. 6 may be selected empirically and various modifications may be made according to the needs of those skilled in the art.

Meanwhile, the parameter generator 110 calculates an average value of luminance for each section. The average value is, for example, a logarithmic average value with respect to the luminance value, as shown in Equation 4 below. This is taken into account that the brightness of the pixel is represented in binary (e.g., 2 ⁸ , or 8 bits, for 256 gray scales).

Here, N _i is the number of pixels included in the i-th bin, and Y _i (x, y) is the input luminance value of the pixel at the x and y positions of the image. However, when Y _i (x, y) is 0, a very small constant value δ may be added to prevent an error of a log function during operation.

The parameter generator 110 provides the brightness controller 120 with the control parameters w _i and Y _ai for the brightness adjustment, obtained as described above.

Referring back to FIG. 4, the brightness controller 120 adjusts the adjustment of the brightness Y among the input images X, Y, and Z by using the control parameter provided from the parameter generator 110. To perform. Such adjustment includes "global adjustment" in consideration of the characteristics of the entire image, and "local adjustment" in consideration of the characteristics of each section of the image. However, according to an embodiment, only regional adjustment may be used and the global adjustment may be additionally performed.

The global adjustment is to reduce the overall dynamic range of the image, and it is necessary to reduce the range to an appropriate range according to the characteristics of the image. In other words, the width of the reduction of the dynamic range is determined uniformly according to the characteristics of the image.

First, the luminance controller 120 performs global adjustment on the input luminance component.

The brightness controller 120 may calculate the global scale factor α as shown in Equation 5 by using the average brightness and the maximum brightness and the minimum brightness of the input image.

Here, Y _max is the maximum luminance of the input image, Y _min is the minimum luminance of the input image, and Y _a is the average value of the input luminance in the entire section. The Y _a may be simply calculated according to Equation 6 below using the average value Y _ai of the luminance for each section provided from the parameter generator 110.

The f may be regarded as a distance (deviation) between the maximum luminance and the minimum luminance and the average luminance of the input image in the image. However, since f may have both a positive value and a negative value, ^f may be represented by 2 ^f so that the global scale factor α may always be positive. The global scale factor α may be regarded as a value having a high correlation with dark or light portions of an image.

On the other hand, in order to be able to obtain the f value more stably in Equation 5, it is necessary to exclude the case of having a very low luminance and the case of having a very high luminance for reasons such as noise. Experimentally, it is desirable to exclude pixels corresponding to 3% of the total, which belong to a very dark or very bright part of the input image. In this case, the calculations of Equations 4 and 5 will be made for the remaining 97% of pixels.

The brightness controller 120 uniformly adjusts all brightness components of the input image according to Equation 7 below, that is, globally.

Next, the brightness controller 120 performs area control on the brightness Y _g on which the global control is performed. In order to perform such local adjustment, the control parameters w _i and Y _ai provided from the parameter generator 110 are used.

In detail, the brightness controller 120 may calculate the local scale factor α _i according to Equation 8 below.

Equation 8 is basically similar to Equation 5 except that it should be calculated for each interval (i). Therefore, Y _{max (i)} is the maximum value among the luminance belonging to the i th section Bin of the input image, Y _{min (i)} is the minimum value among the luminance belonging to the i th section Bin of the input image, Y _ai is an average value of the volutes in the corresponding interval as in Equation 4. Therefore, unlike the global scale factor α, the local scale factor α _i obtained as described above is obtained by the number of scales NB.

The luminance adjusting unit 120 uses the local scale factor α _{i to} determine the finally adjusted luminance Y _{gl (i} ) (hereinafter, referred to as “final luminance”) as shown in Equation 9 below. Obtain

The subscript gl means a result of global control and local control, and i is an index for each section. Therefore, the final luminance has a different value for each section (i). In this way, by adjusting the luminance in a different manner according to each section according to the image, it is possible to obtain a more natural image to the human visual system while preserving the detail of the final image (LDR image).

The final luminance obtained by the luminance controller 120 may be used to perform chromaticity adjustment in the chromaticity controller 130.

The chromaticity adjusting unit 130 obtains chromaticity adjustment coefficients for determining the degree of adjustment of chromaticity, and applies the chromaticity adjustment coefficients to the RGB components of the input image, respectively, to obtain adjusted RGB components.

한다.In this case, although the chromaticity adjuster 130 may directly use the RGB component of the input image, preferably, the luminance adjuster 120 uses Y _{gl (i)} which is the result of global and local adjustment of the luminance component. It is desirable to restore the RGB of the input image. In other words, the chromaticity adjusting unit 130 restores the R, G, and B components of the input image using the following equations (10) and (11).

do.

Of course, in Equation 10, since R is changed to Y _{gl (i)} , R, G, and B may have different values from R, G, and B of the first input image. Meanwhile, the chromaticity adjusting unit 130 calculates a chromaticity adjustment coefficient D for applying to each of the color components R, G, and B as shown in Equation 12 below.

Where F is a constant proportional to the chromaticity adjustment coefficient and L _A is the luminance value of the adapted field.

The proportional constant F is a value that can be determined empirically and has a value of 1.0 for an ambient image of average brightness, 0.9 for a slightly darker ambient image, and 0.8 for a darker ambient image. In addition, L _A means a luminance value located at the top 20% of the luminance value of white in the adapted field, that is, the upper fifth quartile luminance value.

As described above, after the chromaticity adjusting unit 130 calculates the chromaticity adjusting coefficient as described above, the chromaticity adjusting unit 130 applies the chromaticity adjustment coefficient to the restored color components R, G, and B, and adjusts the color as shown in Equation 13 below. Obtain the components (R _c , G _c , B _c ).

Where R _w , G _w , and B _w are the values in the RGB color space calculated using the tristimulus values of white described in any one of the environments listed in Table 1, that is, X _w , Y _w , and Z _w , respectively. . In from X _w, Y _w, Z _w in the calculation to obtain the R _w, G _w, B _w described above can be used to equation (10) and 11.

Meanwhile, although the chromaticity adjusting unit 130 may provide the obtained R _c , G _c, and B _c to the tone mapping unit 140 with the adjusted final chromaticity value, preferably, the values are again represented by Equation 14 below. And the final chromaticity values R ', G', and B 'adjusted by the conversion by 15 and provided to the tone mapping unit 140.

Again, referring to FIG. 4, the tone mapping unit 140 displays chromaticity values that can be displayed on the display device using the globally adjusted final luminance provided by the luminance adjusting unit 120 and the adjusted chromaticity provided by the chromaticity adjusting unit 130. (R _d , G _d , B _d ), that is, LDR images are output.

The tone mapping unit 140 may obtain chromaticity values R _d , G _d , and B _d of the LDR image, for example, according to Equation 16 below. As a result, the first HDR image (R, G, B) is input to the dynamic range converter 100, and finally the LDR image (R _d , G _d , B _d ) is output.

In Equation 16, R ', G', and B 'are chromatic values provided by the chromaticity adjusting unit 130, and Y' is a luminance value that can be obtained from R ', G', and B '. In addition, s may be a value of approximately 0.45 as a user parameter used for gamma correction.

Meanwhile, since the final luminance Y _{gl (i)} adjusted by the luminance controller 120 has an arbitrary range, it is necessary to normalize it before applying it to Equation 16. That is, Y _{d (i)} means the luminance normalized to 0 to 1 from the final luminance adjusted by the luminance adjusting unit 120.

 Up to now, each component of FIG. 4 is a software such as a task, a class, a subroutine, a process, an object, an execution thread, a program, or a field-programmable gate array (ASGA) or an ASIC (application) performed in a predetermined area of memory. It may be implemented in hardware, such as a -specific integrated circuit, or may be a combination of the above software and hardware. The components may be included in a computer readable storage medium, or a part of the components may be distributed and distributed over a plurality of computers.

In addition, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical functions. It is also possible in some alternative implementations that the functions mentioned in the blocks occur out of order. For example, the two blocks shown in succession may in fact be executed substantially concurrently, and the blocks may sometimes be executed in the reverse order, depending on the corresponding function.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

1 shows the luminance range (HDR) of scenes in the real world.

2 shows the concept of an ideal LDR display.

3 is a view comparing an LDR image obtained using a conventional linear scaling technique with an LDR image obtained by lowering a luminance of a high luminance portion.

Figure 4 is a block diagram showing the configuration of a dynamic range conversion apparatus according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating a probability distribution function when a “memorial church” image is divided into ten luminance sections. FIG.

6 is a diagram showing a graph for obtaining a weight according to an embodiment of the present invention.

 (Symbol description of main part of drawing)

100: dynamic range conversion unit 105: color space conversion unit

110: parameter generator 120: luminance controller

130: chromaticity control unit 140: tone mapping unit

Claims (18)

An apparatus for converting a dynamic range of an input image, A parameter generator configured to generate control parameters used to adjust the luminance of the input image for each section according to characteristics of each of the luminance of the input image; A brightness controller which adjusts the brightness of the input image for each section by using the generated control parameter to obtain a final brightness; A chromaticity controller configured to adjust chromaticity of the input image based on the luminance adjusted for each section; And A tone mapping unit for combining the adjusted luminance and the adjusted chromaticity for each section to map the chromaticities that can be displayed on a display device; / RTI > The method of claim 1, And a color space converter configured to convert the chromaticity of the input image into an XYZ image of a Commission Internationale de l'Eclairage (CIE). The method of claim 1, wherein the control parameter At least a weight generated for each section. The apparatus of claim 3, wherein the weight for each section has a larger value as the frequency of the luminance value of the input image increases in a corresponding section, and has a smaller value as the frequency becomes smaller. The method of claim 4, wherein the weights and frequency of each section are Devices with relationships according to quadratic curve graphs. The method of claim 3, wherein the brightness control unit A local scale factor indicating a distance between a maximum luminance and a minimum luminance for each interval of the input image and an average luminance for each interval; And Weight for each section; And obtaining the final luminance by multiplying by the luminance of the input image. The method of claim 3, wherein the brightness control unit A global scale factor representing a distance between maximum and minimum luminance and an average luminance of the input image; A local scale factor indicating a distance between a maximum luminance and a minimum luminance for each interval of the input image and an average luminance for each interval; And And obtaining the final luminance by multiplying the interval-specific weights by the luminance of the input image. The method of claim 1, wherein the tone mapping unit before outputting the chromaticity, And normalize the final luminance to have a value between 0 and 1. The method of claim 8, wherein the tone mapping unit And mapping the output chromaticity in proportion to the normalized final luminance. In the method for converting the dynamic range of the input image, (a) generating a control parameter used to adjust the brightness of the input image for each section according to the section-specific characteristics of the brightness of the input image; (b) obtaining final luminance by adjusting the luminance of the input image for each section using the generated control parameter; (c) adjusting chromaticity of the input image based on the luminance adjusted for each section; And (d) combining the adjusted luminance and the adjusted chromaticity for each of the sections and mapping them to a chromaticity that can be displayed on a display device; How to include. The method according to claim 10, wherein before step (a) And converting the chromaticity of the input image into an XYZ image of a Commission Internationale de l'Eclairage (CIE). The method of claim 10, wherein the control parameter At least a weight generated for each interval. The method of claim 12, wherein the weight for each section has a larger value as the frequency of the luminance value of the input image increases in a corresponding section, and has a smaller value as the frequency becomes smaller. The method of claim 13, wherein the interval-weighted weight and the frequency Method with relation according to quadratic curve graph. The method of claim 12, wherein step (b) A local scale factor indicating a distance between a maximum luminance and a minimum luminance for each interval of the input image and an average luminance for each interval; And Weight for each section; And obtaining the final luminance by multiplying by the luminance of the input image. The method of claim 12, wherein step (b) A global scale factor representing a distance between maximum and minimum luminance and an average luminance of the input image; A local scale factor indicating a distance between a maximum luminance and a minimum luminance for each interval of the input image and an average luminance for each interval; And And obtaining the final luminance by multiplying the interval weights by the luminance of the input image. The method of claim 10, wherein the step (d) before the output of the chromaticity, Normalizing the final luminance to have a value between 0 and 1. 18. The method of claim 17, wherein step (d) And mapping the output chromaticity in proportion to the normalized final luminance.

Images