JP2005284534A - Method for converting image of high dynamic range into image of low dynamic range and device related to the method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a device for converting an image of a high dynamic range into an image of a low dynamic range. <P>SOLUTION: The method for converting a high dynamic range image into a low dynamic range image comprises a process (a) for converting a first luminance value corresponding to a pixel into a second luminance value having a second luminance range smaller than the first luminance range of the first luminance value and a process (b) for mapping the second luminance value corresponding to the pixel to a plurality of third luminance values and generating a low dynamic range image by utilizing a film duplication function in which a visual effect is not attached to the low dynamic range image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method and related devices, and more particularly to a method of converting a high dynamic range image into a low dynamic range image and related devices.

  The dynamic range is the ratio between the maximum value and the minimum value of the screen brightness. The dynamic range of a CRT or liquid crystal display is approximately 250: 1 and occupies half of the visible color gamut. However, the human visual system has a dynamic range exceeding 10,000: 1. If there is a condition of constant brightness, humans can distinguish 10,000 colors. Note that the maximum and minimum image intensity values of CGI (computer image formation) are 3000: 1.

  The image displayed by the normal display device corresponds to 256 gray levels. That is, each color channel (R, G, B) is defined by 8 bits. Therefore, the minimum value of the gray level is 0 and the maximum value is 255. According to the foregoing, it can be seen that the dynamic range of the actual scene far exceeds the dynamic range that can be displayed by the display device.

  An image with a high dynamic range is usually called a high dynamic range image (hereinafter abbreviated as HDR image). According to the prior art, there is no method for directly capturing and displaying an HDR image. Although a technique for capturing an HDR image currently exists, since multiple exposures must be performed on the same screen, a plurality of different images are taken even on the same screen (the images differ depending on the exposure conditions). Subsequently, by combining a plurality of images with a conventional algorithm, the true radiance level of the screen is obtained and the HDR image is captured.

  After capturing an HDR image, storing the image becomes a problem. Existing image formats usually apply to conventional display devices. As described above, the display device is driven accurately by storing the color channel of the image in 8 bits. However, since conventional image formats are not suitable for HDR images, image formats suitable for HDR images are developed one after another. To increase the dynamic range of a digital image, it is common to increase the number of bits for each color channel. For example, LogLuv uses 4 bytes per pixel. Of these, 2 bytes are used to encrypt the logarithmic value of luminance Y, and 2 bytes are used to encrypt the U channel and V channel of the LUV color space.

  Next, tone mapping for reproducing the screen on the display device using the HDR image is necessary. Tone mapping converts a luminance value recorded in an HDR image into a luminance value of a low dynamic range and drives a display device of a low dynamic range. Thus, the main purpose of tone mapping is to compress the dynamic range to suit the display device. As is well known, either the tone reproduction curve (TRC) or the tone reproduction operator (TRO) is applied to the HDR image. Taking the tone reproduction curve as an example, each pixel changes from the current luminance value to a display intensity within a dynamic range that can be displayed by the display device. The tone reproduction curve corresponds to a transformation function that is independent of the spatial distribution, and every pixel in the HDR image is processed by the same transformation function. On the other hand, the tone reproduction operator adjusts the luminance of each pixel using a spatial relationship. That is, even if the luminance of the pixels is different, there is a possibility that the same display intensity can be accommodated within the dynamic range of the display device.

  Tone mapping can also adjust the display quality of the HDR image. For example, tone mapping can mimic the reaction of the naked eye. The HDR image is a combination of a plurality of images with different exposure conditions and does not include human visual effects. However, the human visual system can detect various visual effects, for example, the brighter the scene becomes dazzling and the darker the scene becomes blurred. Therefore, the conventional tone mapping adds a human visual effect to a low dynamic range image (hereinafter referred to as an LDR image) processed from an HDR image. The LDR image displayed on the display device corresponds to a real scene, but by adding such a visual effect, the image quality is inevitably deteriorated. In addition, the addition of visual effects requires complex computations, thus slowing down the processing speed.

  In order to solve the above-described problems, an object of the present invention is to provide a method and apparatus for converting an HDR image into a corresponding LDR image.

  The present invention provides a method for converting an HDR image into an LDR image. Each HDR image has a plurality of pixels corresponding to the first luminance value. The method includes (a) converting a corresponding first luminance value of a pixel to a second luminance value having a second luminance range that is less than the first luminance range of the first luminance value, and (b) converting the visual effect into an LDR image. Using a film copy function that is not added to the image, mapping the corresponding second luminance value of the pixel to a plurality of third luminance values to produce an LDR image, and the like.

  The present invention further provides an image processing system. The system includes an image generator for generating an HDR image having a plurality of pixels each corresponding to a first luminance value, and a corresponding first luminance value for the pixel is less than a first luminance range of the first luminance value. At the same time as converting to a second luminance value having a second luminance range, the film copying function is used to map the corresponding second luminance value of the pixel to a plurality of third luminance values, and LDR without adding visual effects. Image processing logic for generating an image.

  The present invention utilizes film copy S curves to produce realistic LDR images that do not include visual effects. By using the film copy S curve, the contrast of pixels whose luminance value is in the intermediate luminance range is optimized. Since the film copy S curve is defined in advance and does not need to be calculated during tone mapping, the image processing efficiency is improved.

  In order to detail the features of such a method and apparatus, a specific example is given and described below with reference to the drawings.

  Please refer to FIG. FIG. 1 is a flowchart of tone mapping according to the present invention. According to FIG. 1, in step 100, an HDR image is loaded. As described above, since the HDR image is composed of a plurality of images having different exposure conditions, the dynamic range exceeds the dynamic range that can be displayed by the display device. Therefore, the dynamic range of the HDR image must be compressed so as to be suitable for the display device. Since the human visual system is more sensitive to local luminance changes than absolute luminance, the dynamic range is compressed by overall gradation compression in step 102 to reduce the overall illuminance difference. In order to reduce the computational complexity, the following space invariant operators can be used.

分母Ｌ_ｗ（ｘ，ｙ）はＨＤＲ画像におけるピクセル（ｘ，ｙ）の初期輝度（ｒａｗ ｌｕｍｉｎａｎｃｅ）を示し、Ｌ_ｄ（ｘ，ｙ）はピクセル（ｘ，ｙ）の拡縮後輝度（ｓｃａｌｅｄ ｌｕｍｉｎａｎｃｅ）を示す。図２を参照する。図２はこの発明による全体階調圧縮を表す説明図である。図２によれば、Ｌ_ｄ（ｘ，ｙ）は表示装置に適用する正規化グレイレベルを示し、言い換えれば、Ｌ_ｄ（ｘ，ｙ）は０と１との間に介する。Ｌ_ｗ（ｘ，ｙ）は高い輝度値（Ｌ_ｗ（ｘ，ｙ）に正規化される）に対応しながら、Ｌ_ｄ（ｘ，ｙ）は低い輝度値（「１」に正規化される）に対応する。方程式（１）の分母はＬ_ｄ（ｘ，ｙ）が０と１との間に介するようにさせ、方程式（１）は初期輝度値を表示可能のダイナミックレンジ内に変換することを確保する。例えば、初期輝度値は表示装置に適する輝度値に圧縮される。

The denominator L _w (x, y) represents the initial luminance (raw luminance) of the pixel (x, y) in the HDR image, and L _d (x, y) represents the luminance after scaling (scaled luminance) of the pixel (x, y). Indicates. Please refer to FIG. FIG. 2 is an explanatory view showing the whole gradation compression according to the present invention. According to FIG. 2, L _d (x, y) represents the normalized gray level applied to the display device, in other words, L _d (x, y) is interposed between 0 and 1. L _w (x, y) while corresponding to the high brightness values _(L w (x, y) is normalized _{to), L d (x, y} ) is normalized to a low luminance value ( "1" ). The denominator of equation (1) causes L _d (x, y) to be between 0 and 1, and equation (1) ensures that the initial luminance value is converted into a displayable dynamic range. For example, the initial luminance value is compressed to a luminance value suitable for the display device.

  Subsequently, in step 104, histogram equalization is performed to improve the contrast of the image. Histogram equalization converts the histogram of the adjusted image into a uniform histogram. If the cumulative frequency distribution P (b) is necessary to equalize the histogram, the definition is as follows.

そのうち、Ｔはヒストグラムのエントリーの総数を示す。したがって、Ｐ（ｂ_ｉ）はヒストグラムにある升目（ｂｉｎ）ｂ_ｉの度数分布であり、ｆ（ｂ_ｉ）はビンｂ_ｉの度数カウントである。ヒストグラム等化を行うと、輝度値が同じ確率分布を有する出力画像は得られる。等化用の方程式は以下の通りである。

Of these, T indicates the total number of entries in the histogram. Therefore, P _{(b i)} is the frequency distribution of the squares (bin) _{b i} in the histogram, f _{(b i)} is the frequency count of the bin _{b i.} When histogram equalization is performed, an output image having a probability distribution with the same luminance value is obtained. The equation for equalization is as follows.

そのうち、Ｂ_ｄは調整された表示輝度を示し、Ｂは初期輝度を示す。ｌｏｇ（Ｌ_ｄｍｉｎ）は表示装置の最小輝度値を示し、ｌｏｇ（Ｌ_ｄｍａｘ）は表示装置の最大輝度値を示す。このように入力画像のピクセル輝度値を再配分した後、出力画像のコントラストは大幅に向上される。

Among these, _Bd indicates the adjusted display luminance, and B indicates the initial luminance. log (L _dmin ) indicates the minimum luminance value of the display device, and log (L _dmax ) indicates the maximum luminance value of the display device. After redistributing the pixel luminance values of the input image in this way, the contrast of the output image is greatly improved.

However, if more than half of the pixel luminance values are below 0.2 * L _dmax , the small luminance range [0, 0.2 * L _dmax ] occupies only half of the display luminance value of the display device, so the adjusted image The contrast may be excessively emphasized. For example, the luminance difference between pixels is enlarged by histogram equalization, and the image becomes unnatural. Therefore, in step 106, histogram adjustment for preventing abnormal enlargement of image contrast is performed. The histogram adjustment limits the image contrast under the upper limit condition. The upper limit conditions are as follows.

方程式（４）において、画像コントラストは従来の線形正規化演算子（その傾きはＬ_ｄ／Ｌ）から得られる画像コントラストを超えることない。前述の方程式（２）から以下の通りの不等式は得られる。

In equation (4), the image contrast does not exceed the image contrast obtained from the conventional linear normalization operator (whose slope is L _d / L). From the above equation (2), the following inequality is obtained.

そのうち、Δｂは［ｌｏｇ（Ｌ_ｍａｘ）−ｌｏｇ（Ｌ_ｍｉｎ）］／Ｎである。Ｎはヒストグラムにある升目の数量であり、ｌｏｇ（Ｌ_ｍａｘ）は初期画像の最大輝度であり、ｌｏｇ（Ｌ_ｍｉｎ）は初期画像の最小輝度である。よってΔｂは升目の大きさに対応する。方程式（５）によれば、ヒストグラムにある升目の度数カウントが上限（Ｌ_ｄ／Ｌ）を超えなければ、調整された画像のコントラストは異常に拡大されることがない。

Among them, Δb is [log (L _max ) −log (L _min )] / N. N is the number of cells in the histogram, log (L _max ) is the maximum luminance of the initial image, and log (L _min ) is the minimum luminance of the initial image. Therefore, Δb corresponds to the size of the cell. According to equation (5), the contrast of the adjusted image is not abnormally enlarged unless the frequency count of the cells in the histogram exceeds the upper limit (L _d / L).

  In step 108, a final LDR image is produced in step 110 by performing a mapping operation on the film copy S curve. Please refer to FIG. FIG. 3 is an explanatory diagram showing an example of a film copy S curve according to the present invention. The S curve 10 corresponds to the photosensitive reaction curve of the photographic film. According to FIG. 3, the input luminance value in the intermediate range (10 to 1000) occupies almost the entire output luminance value. In other words, the film copy S-curve 10 increases the contrast in the mid-range luminance region of the image. Therefore, the contrast between the high luminance region and the low luminance region is reduced, and the quality of the final LDR image is improved.

  The tone mapping according to the present invention is applied to an image processing system that converts an HDR image into an LDR image. Please refer to FIG. FIG. 4 is a block diagram of an image processing system according to the present invention. Image processing system 20 includes an image generator 22 and image processing logic 24. The image generator 22 generates a high dynamic range image, and the image processing logic 24 performs tone mapping to convert the HDR image into an LDR image. For example, the image processing system 20 is a digital camera, and the image generator 22 includes a CCD (Charge Coupled Device) module that captures incident light and produces an image, and a shutter that controls exposure of the CCD module. The image generator 22 captures images with the CCD module, and then generates a plurality of images with different exposure states by the shutter. Subsequently, the image generator 22 combines a plurality of images as an HDR image by a conventional technique.

  Subsequently, the HDR image is processed by the image processing logic 24. Image processing logic 24 performs steps 102, 104, 106, and 108 described above to produce the requested LDR image. Since the image processing logic 24 is a digital signal processor (DSP) that uses a simple film copying function, the associated operations are not complex. Generally, a digital camera includes a small LCD display for previewing images. Since the image processing logic 24 uses a simple film copying function, it not only has excellent image processing capabilities, but also allows the user to preview LDR images on the LCD display immediately. A high quality LDR image that does not include visual effects can be obtained.

  In another embodiment of the invention, the image generator 22 only captures and produces a plurality of images with different exposure conditions, and the conversion to an HDR image is performed by the image processing logic 24. Similarly, the image processing logic 24 performs the above-described steps 102, 104, 106, and 108 to produce the requested LDR image.

  As in the first embodiment, the image processing system 20 is a digital camera, and the image generator 22 and the image processing logic 24 are in the same housing. However, the image processing system 20 can be composed of a plurality of independent devices. For example, the image generator 22 is a digital camera, and the image processing logic 24 is a host computer. In this case, the output image data of the image generator 22 is transmitted to the external image processing logic 24, and subsequent image processing is executed.

  The above is a preferred embodiment of the present invention and does not limit the scope of the present invention. Therefore, any modifications or changes that can be made by those skilled in the art, which are made within the spirit of the present invention and have an equivalent effect on the present invention, shall belong to the scope of the claims of the present invention. To do.

  The present invention uses a film copy S curve to produce a realistic LDR image that does not include visual effects. The film copy S curve not only optimizes the contrast of the image, but also improves image processing effectiveness. .

3 is a flowchart of tone mapping according to the present invention. It is explanatory drawing showing the whole gradation compression by this invention. It is explanatory drawing showing an example of the film copy S curve by this invention. 1 is a block diagram of an image processing system according to the present invention.

Explanation of symbols

10 Film Copy S Curve 20 Image Processing System 22 Image Generator 24 Image Processing Logic

Claims (10)

In a method for converting a high dynamic range (HDR) image to a low dynamic range (LDR) image, the HDR image has a plurality of pixels each corresponding to a first luminance value, the method comprising:
(A) converting the corresponding first luminance value of the pixel into a second luminance value having a second luminance range smaller than the first luminance range of the first luminance value;
(B) using a film copying function that does not add a visual effect to the LDR image, and mapping the corresponding second luminance value of the pixel to a plurality of third luminance values to produce an LDR image. A method of converting a characteristic HDR image into an LDR image. The method further comprises:
2. The method of converting an HDR image into an LDR image according to claim 1, further comprising the step of: (c) adjusting the second luminance value of the pixel by performing histogram equalization. The step (c) further includes
3. The method of converting an HDR image into an LDR image according to claim 2, comprising the step of preventing the total number of pixels corresponding to the second luminance value from exceeding a predetermined limit.   2. The method of converting an HDR image into an LDR image according to claim 1, wherein the step (a) is performed by a whole tone compression method. The image processing system
An image generator for producing an HDR image having a plurality of pixels each corresponding to a first luminance value;
Converting the corresponding first luminance value of the pixel into a second luminance value having a second luminance range that is less than the first luminance range of the first luminance value, and simultaneously utilizing the film copy function, And an image processing logic for mapping the luminance value to a plurality of third luminance values and generating an LDR image without adding a visual effect.   6. The image processing system according to claim 5, wherein the image processing system is a digital camera.   6. The image processing system according to claim 5, wherein the image generator captures a plurality of images having different exposure conditions to generate an HDR image.   6. The image processing system according to claim 5, wherein the image processing logic performs histogram equalization to adjust the second luminance value of the pixel.   9. The image processing system according to claim 8, wherein the image processing logic prevents the total number of pixels corresponding to the second luminance value from exceeding a predetermined limit.   6. The image processing system according to claim 5, wherein the image processing logic performs overall gradation compression to convert the first luminance value into the second luminance value.

Images