TWI235608B - Method and apparatus for transforming a high dynamic range image into a low dynamic range image - Google Patents

Abstract

A method and an apparatus for transforming a high dynamic range image into a low dynamic range image. The method includes converting first luminance values associated with pixels into a plurality of second luminance values, and utilizing a film transfer function for mapping the second luminance values associated with the pixels into a plurality of third luminance values to generate the low dynamic range image. A second luminance range of the second luminance values is smaller than a first luminance range of the first luminance values, and the film transfer function adds no visual artifact to the low dynamic range image.

Description

1235608 V. Description of the invention (1) [Technical field to which the invention belongs] The present invention provides an image processing method and related device, and more particularly, a method and related device for converting a high dynamic range image into a low dynamic range image. [Prior Art] The dynamic range of a scene (scene) is defined as the ratio between the highest scene brightness and the lowest scene brightness. In general, the dynamic range of a standard display device (such as a CRT display or an LCD display) is approximately 250: 1, which covers approximately half of the entire visible color range. However, the human visual system (humanvisi ο nsystem, Η VS) usually has a dynamic range greater than 10,000: 1, and can distinguish about 10, 000 different colors at a specific brightness. . In addition, a computer-generated image (CGI) usually has a proportional relationship between the highest and lowest intensity values, which is approximately equal to 3000 times. Generally, an image displayed on a standard display device can correspond to 256 different gray levels. That is, each color channel (R, G, B) is determined by 8 bits. Therefore, the minimum grayscale value is 0 and the maximum grayscale value is 255. It can be known from the above that the dynamic range of the scene in the real world is usually beyond the display of standard display devices.

1235608 V. Description of the invention (2) The dynamic range of an image with a high dynamic normal range is not any simple and (capture a more complex) (the following brief need to be the same exposure) the same image (therefore, different degrees of data. Then Algorithm, combination (radiance of the HDR image and depicting the miscellaneous technique, the scene is performed. The result is that different image exposure conditions are obtained, and different levels are to be captured.) The surrounding image can usually be called "high motion is called HDR image). In current technology, direct methods can be used to capture (high dynamic range) images. Some can be used to capture HDR images, however, multiple exposures (mul tiple for the same scene will result in multiple photos). It is not derived using different exposure conditions.) Different images can provide different bright scenes.> The scene can be shot with the conventional technology. The real light shot level is 10, so you can get the scene you want to capture. Corresponding to / After the HDR image is recorded correctly, the next scene is stored. Various digital images commonly used today 袼Large handle: used on traditional display devices. As mentioned earlier, an m-bit 70 stores each color channel (color ^ 疋 C, MM1) of the image to properly drive standard display devices. Gu also said : The dynamic range of the known image format is not enough. | :: The above problem. To increase the number of digits .. + Μ to solve 1235608 V. The method of invention description (3) is to increase the number of bits per color channel, L〇 The gLuv coding technique is an example in the following example: 'Each pixel uses 4 bytes to represent. There are 2 bytes used to encode the logarithm of the brightness γ (logarithm), and the other 2 Bytes are used to encode the U and V channels in the LUV color space. After the HDR image is generated correctly, a tone mapping procedure must be used to use the resulting HDr image 'reproduces the captured scene on a standard display device. In other words, the tone mapping program converts the brightness values recorded in the HDR image into brightness values with a lower dynamic range to drive Standard display device with low dynamic range. Therefore, the main purpose of the above-mentioned tone mapping program is to compress a larger dynamic range into a dynamic range suitable for standard display devices. It is well known that a tone reproduction curve, TRC) or a tone reproduction operator (TR0) can be applied to the image data in the HDR image. Taking the tone reproduction curve as an example, each pixel will be converted from its current brightness value to a display intensity within the dynamic range of a standard display device. The above-mentioned tone reproduction curve corresponds to a transformation function independent of the spatial distribution. Each pixel in the HDR image is processed using the same transformation function. As for the hue reproduction operator, the spatial relationship (s p a t i a 1 c ο n t e X t) is used to adjust the brightness of each pixel. That is, two pixels with the same brightness value may correspond to different display intensities in the dynamic range of a standard display device.

Page 8 1235608 V. Description of the invention (4) Two pixels with different brightness values may also correspond to the same display intensity in the dynamic range of a standard display device. Tone mapping technology can also be used to adjust the display quality of captured HDR images. For example, the hue mapping program of the prior art can simulate the response of the human eye. Please note that the HD image is obtained by combining different images under different exposure conditions, that is, the transmitted HDR image does not originally contain any human visual effects (ν i s u a 1 artifact). However, the human visual system is full of various visual effects, such as a glare effect at high brightness, or a blur effect at low brightness. Therefore, the conventional tone mapping program will add the visual effects of the human visual system to the LDR image after receiving the HDR image. Although the final LDR image is displayed on a standard display device, it will correspond to the situation where the observer sees the captured scene, but due to the added visual effects, the image quality will still be reduced. In addition, this method must be combined with complex real-time operations to add the desired visual effects to the LDR image. Such time-consuming and complicated operations will eventually lead to poor image processing efficiency. [Summary of the Invention] One of the present invention The purpose is to provide a method and a device for converting a high dynamic range image into a corresponding low dynamic range.

1235608

5. Description of the invention (5) Video. ^ According to an embodiment proposed below, the method disclosed in the present invention is to convert a high dynamic range image into a low dynamic range image. The high dynamic range image has a plurality of pixels, and the pixels respectively have a plurality of first brightness values. The method includes the following steps :) The first brightness values corresponding to the first pixels are converted into a plurality of second brightness values. True f ^, a second brightness range of the second brightness values is less than a first brightness range of the (i) -shell value; and (b) a film conversion number is used to convert the pixels The corresponding second brightness values are mapped to a plurality of third brightness values to generate the low dynamic range image, wherein the film conversion function does not add a visual effect to the middle of the low dynamic range image. 0 According to the following In another embodiment, a video processing system disclosed in the present invention includes an image generator, which is used to generate a speech, range image, the dynamic range image has a plurality of pixels, etc. The pixels are respectively corresponding to a plurality of first brightness values; and a $ = logical logic is used for the first brightness values 2 ^ corresponding to the pixels, the plurality of second brightness values, and a film conversion is used Letter g, with J The 5 degrees corresponding to the pixels corresponding to the pixels are mapped to a plurality of second brightness values, and a low value is generated without adding visual effects = the range is less than the image; where the second brightness values are A second brightness is a first brightness range of the first brightness values.

1235608 V. Description of the invention (6) Since the present invention uses a film to transform the S curve, the final LDR image is a more realistic image without other visual effects. The LDR image displayed on a standard display device will be More clear. By using the film to convert the S-curve, pixels with brightness values in the middle brightness range will get better image contrast, and more subtle changes will become more clear. And the film conversion S curve can be defined in advance and does not need to be calculated dynamically in the tone processing program. Therefore, in practice, the tone mapping procedure proposed by the present invention will be very simple, and the image processing performance will become better. [Embodiment] Please refer to FIG. 1. FIG. 1 is a flowchart of an embodiment of a tone mapping process. The steps involved in Figure 1 are detailed below. First, an HDR image is loaded before step 100. The loaded HDR image is generated by a plurality of images obtained under different exposure conditions, as described above, and the corresponding dynamic range is larger than that of a standard display device. Therefore, the original dynamic range of the input HDR image must be compressed to a low dynamic range suitable for standard display devices. In addition, the human visual system is not sensitive to absolute luminance, so the floor bath p has a greater response to the local luminance change of [5], and it will reduce the larger overall luminance. Difference

1235608 V. Description of the invention (7) Effects produced by illumination difference). Therefore, in step 102, a global gradient compression (g 1 0 b a 1 g r a d i e n t c o m p r e s s i ο n) procedure is performed to compress the original dynamic range and reduce the overall brightness difference. In order to reduce the complexity of operation and implementation, a spatially invariant operator can be used here, as shown in the following equation:

Lrf (x, y) = lf ^ Equation 的 Lw (x, y) in the denominator represents a primitive luminance (world luminance or raw luminance) of the pixel (x, y) in the HDR image. As for Ld (x, y ) Represents the scaled 1 uminance of the pixel (x, y). Please refer to FIG. 2. FIG. 2 is a schematic diagram of a general gradient compression program used in the present invention. As shown in FIG. 2, Ld. (X, y) represents a normalized gray level value (normalized gray 1 eve 1) suitable for a standard display device, that is, Ld (x, y) will be between 0 and 1 between. obviously,

Lw (X, y) will correspond to a higher brightness value (normalized by lw (x, y) itself), as for Lw (X, y) will correspond to a lower brightness value (normalized by 'Γ Into). The denominator in equation (1) allows the normalization method to fall between the two, and equation (1) can ensure that all the original brightness values can be transformed into an acceptable dynamic range. For example, the original brightness value is compressed into a display brightness value suitable for a particular display device.

Page 12 1235608 V. Description of the invention (8) Then, step 104 executes a histogram equalization program to improve the contrast of the image in the normalized brightness value of the multi-image after adjustment. . The exposure analysis image equalization here will transform the exposure analysis image of the adjusted image into a uniform exposure histogram. Assume that a cumulative frequency distribution P (b) is needed to equalize an exposure analysis chart, and its definition is shown in the following equation:

(PΦ) = ^ bi < bTf ^ -Equation (2) where T represents the total number of fields (e n t r y) in the exposure analysis chart. Therefore, P (bi) is the frequency distribution of bins in 'bi in the exposure analysis graph, and f (bjJ is the frequency count) of bins in bi in the exposure analysis graph. Applying this equalization program of an exposure analysis map to an input image will produce an output image whose brightness value has equal probability distribution. The equation for equalization can be as follows:

Bd-l0g () + [l0g (Ldmax)-l0g (Ldmin)] * P (B) Equation (3)

Page 13 1235608 V. Description of the invention (9) Among them, Bd represents the adjusted display brightness, and B represents the original image brightness. log ^ Ldmin) is the minimum brightness value of the standard display device, and 10 g (Ldmax) is the maximum brightness value of the standard display device. After re-distributing the brightness values of the pixels in the input image, the output image can have better image contrast, so it can be displayed more clearly. However, if half of the pixels have a brightness value lower than 0 · 2 + Ldmax, that is, a small brightness range [0 '〇 2 * Ldmax] will be used to display half of the brightness value of the standard display device. Just the scope. Therefore, it is possible that the image contrast of the adjusted image will be overemphasize. For example, suppose the two adjusted pixels in the original image have a small difference in brightness. After the above-mentioned equalization procedure of the exposure analysis chart, the brightness difference will be enlarged. This causes an image to appear unnatural, so the display quality becomes lower. Therefore, an exposure analysis map adjustment program is used in step 106 to prevent the image contrast in the final image from being abnormally enlarged. The aforementioned exposure analysis map adjustment procedure limits the image contrast by using a maximum limit condition (c e i 1 i n g condition). An example is proposed below, and the highest limit condition can be shown by the following equation:

Page 14 1235608 V. Explanation of the invention (ίο) Equation (4) indicates that the image contrast cannot exceed the image contrast obtained using a conventional linear scaling operation (the slope of which is equal to Ld / L). From the foregoing equation (2), the following inequality can be derived: f (h \ r * Afo equation (5) 10 g (Aw)-10 g (A / min) where △!) Is [i〇 g (Lmax) -log (Lmin)] / N, where N is the number of grids in the exposure analysis graph, and log (LinaxV is the maximum brightness of the original image '10 g (Lmin) is the minimum brightness of the original image. Therefore, the symbol △ b will correspond to the size of each grid. As shown in equation (5), it can be said that 'as long as the total frequency of no grid in the exposure analysis graph is greater than the upper limit (that is, Ld' / L) , Then the exposure analysis chart of the adjusted image will not abnormally enlarge the contrast of the image. In this way, the output image will not contain the grid of the overpopulated exposure analysis chart. After completing step 1 0 6 After that, in step 108, a film transfer S-curve is used to start — mapping operation # (mapping operation), and the final LDR image is generated in step 11 and step 10. Please refer to Fig. 3 'Fig. 3 is an example of the S curve stomach of the present invention. The horizontal axis of Fig. 3 represents the input brightness, and the vertical axis It means that the production is in succession. The film conversion S curve 10 in this example corresponds to one-phase $ 片 胶 $ 卷 ^ °

Page 15 1235608 Fifth, the invention (11) (photographic film) a photosensitive response (sensitization response) curve. As shown in Figure 3, the input brightness value in the middle range (10 ~ 1000) occupies approximately the entire range of the output brightness value. In other words, the film conversion S curve 10 will increase the contrast of the image in the middle range and brightness area. Therefore, the image contrast corresponding to the higher brightness area and the lower brightness area is greatly suppressed. Therefore, the visual quality of the final LDR image can be greatly improved. The tone mapping program proposed by the present invention can also be used in an image processing system to convert an H D R image into an L D R image. Please refer to FIG. 4, which is a schematic diagram of an image processing system according to an embodiment of the present invention. The image processing system 20 in this embodiment includes an image generator 22 and an image processing logic 24. The image generator 22 can generate a high dynamic range image. As for the image processing logic 24, a tone mapping program can be executed to transform the HDR image into a corresponding LD image. For example, the image processing system 20 may be a digital camera, and the image generator 22 includes a CCD module for capturing incident light to generate a corresponding image and a camera shutter (camerashu 11 er). Used to control the exposure status of the CCD module. Therefore, the image generator 22 can capture a scene by the ccj) module, and by appropriately controlling the camera shutter, 9 can generate a plurality of images with different exposure conditions. Next, the image = Xi Erfang's silly style, combines the obtained images into an HDR image corresponding to the scene to be captured.

Page 16 1235608 V. Description of the invention (12) After that, the image processing logic 24 is started to process the HDR image. The image processing logic 24 in this embodiment may perform the above steps 102, 104, 106, and 108 to generate a desired LDR image. Since the image processing logic 24 can be a digital signal processor (DSP) and a simple film conversion function is used, the related calculations are not too complicated. Generally speaking, most digital cameras have a small LCD display for previewing the captured scene. Since the image processing logic 24 uses a simple film conversion function, it will have good image processing performance, and users can quickly preview LDR images on the LCD display. In addition, the generated LDR image will also have better image quality (because no visual effects are added to the LDR image). As for another embodiment, the image generator 22 can only be used to capture a scene to generate multiple images under different exposure conditions. The image processing logic 24 is then responsible for generating the HDR image. Similarly, the image processing logic 24 can also perform the steps 10, 102, 104, 106, and 08 mentioned above to generate the final LDR image. As described above, the image processing system 20 may be a digital camera, so the image generator 22 and the image processing logic 24 are both disposed in the same casing. However, the image processing system 20 may also be composed of a plurality of independent devices. For example, the image generator 2 2 itself can be a digital camera, and the image processing logic 24 can be a computer host. in

Page 17 1235608 V. Description of the invention (13) Under this structure, the image data output by the image generator 2 2 will be transmitted to the external image processing logic 24 for subsequent image processing procedures. As mentioned above, in the practice of the conventional technology, visual effects are added to the final L DR image in order to emphasize the visual correctness (v i s u a 1 accuracy). However, due to the added visual effects, the image quality of LDR images will become worse. In addition, to add the visual effects of the human visual system, it will require a lot of time-consuming image reproduction work, such as adding a glare effect at a higher brightness and a blur effect at a lower brightness. Performance of image processing. Compared with the prior art, the present invention uses a film conversion S curve. The final LDR image is a more realistic image without other visual effects, and the final LDR image displayed on a standard display device will be clearer. By using the film to convert the S-curve, pixels with brightness values in the middle brightness range will get better image contrast, and more subtle changes will become clearer. And the film conversion S curve can be predefined, and it does not need to be calculated dynamically in the tone processing program. Therefore, in practice, the tone mapping procedure proposed by the present invention will be very simple, and the image processing performance will become better. The above description is only a preferred embodiment of the present invention, and any equivalent changes and modifications made in accordance with the scope of the patent application of the present invention shall fall within the scope of the patent of the present invention.

Page 18 1235608 Brief description of the diagram Brief description of the diagram Figure 1 is a flowchart of an embodiment of the tone mapping program of the present invention. Figure 2 is a schematic diagram of the overall gradient compression program used in the present invention. Figure 2 is an example of the S curve of the present invention. Figure 4 is a schematic diagram of an embodiment of the image processing system of the present invention. The symbol of the pattern indicates the 10 S curve 20 the image processing system 22 the image generator 24 the image processing logic

Page 19

Claims (1)

1235608 VI. Application for patent scope 5. — An image processing system including: an image generator for generating a high dynamic range image, the high dynamic range image has a plurality of pixels, and the pixels correspond to a plurality of pixels, respectively. A first brightness value; and an image processing logic for converting the first brightness values corresponding to the pixels into a plurality of second brightness values, and using a film conversion function to correspond to the pixels The obtained second brightness values are mapped to a plurality of third brightness values, and a low dynamic range image is generated without adding visual effects; wherein a second brightness range of the second brightness values is smaller than the A first brightness range equal to the first brightness value. 6. The image processing system described in item 5 of the scope of patent application, which is a digital camera. 7. The image processing system according to item 5 of the scope of patent application, wherein the image generator is capable of capturing a plurality of images under different exposure situations to generate the high dynamic range image. 8. The image processing system described in item 5 of the scope of patent application, wherein the image processing logic can execute an exposure analysis map equalization program to adjust the second brightness values of the pixels. 9 · The image processing system described in item 8 of the scope of patent application, wherein the image 1235608 VI. Patent Application Range The image processing logic can prevent a total number of preset pixels corresponding to a second brightness value from exceeding a preset limit. 10. The image processing system according to item 5 of the scope of patent application, wherein the image processing logic is capable of executing an overall gradient compression program to convert the first brightness values into the second brightness values. Page 22