WO2012015020A1

WO2012015020A1 - Method and device for image enhancement

Info

Publication number: WO2012015020A1
Application number: PCT/JP2011/067381
Authority: WO
Inventors: Xiaojun Zhang; Zhigang Fan
Original assignee: Sharp Kabushiki Kaisha
Priority date: 2010-07-27
Filing date: 2011-07-22
Publication date: 2012-02-02
Also published as: CN102339461A

Abstract

A method and a device for image enhancement are provided. The method according to an embodiment of the present invention comprises calculating a local luminance value and a local contrast value for each pixel in an input digital image; determining a gain factor for each pixel based on the local luminance value and the local contrast value; and adjusting a value of a pixel based on the gain factor determined for the pixel. With the method and device according to the embodiments of the present invention, both luminance and contrast of an image can be enhanced without introducing any artifacts. According to further embodiments, the computational efficiency can be improved. The present invention is applicable in a variety of image processing systems, such as image/video capture device, display device, print device and the like.

Description

DESCRIPTION

TITLE OF INVENTION

METHOD AND DEVICE FOR IMAGE ENHANCEMENT

TECHNICAL FIELD

The invention relates to digital image processing, and more particularly, to a technique for image enhancement and/ or dynamic range reduction, capable of generating a clearer and brighter image without any visible artifacts based on an input image .

BACKGROUND ART

It is very easy for human eyes to perceive a snow scene in the sun or a scene in a dim cave . Human visual system can accommodate various environments and perceive scenes in the respective environments without any difficulty. However, various man-made image sensors do not have such capability of flexibly accommodating various scenes, but only can capture a scene with a fixed dynamic range, and this range is limited. For example, a person in a room can identify furniture or carpet at a corner of the room, and also a cloud in the sun at the same time . However, it is difficult for one to shoot all the scenes he or she sees with a camera. If a long exposure time is set in order to capture details of a scene in a dim room, the sky out of window cannot be shot. If a short exposure time is set to capture details of the sky, no details in the room can be obtained. These are problems which may be encountered in our daily life . A camera or video recorder cannot shoot exactly what we see with our eyes . Thus, how to reduce a large dynamic range of an image without producing any artifact is a problem to be solved in digital image processing.

Patent Document 1 (US20060 1 10052A 1 ) discloses an image processing technique. An image is first transformed to the logarithmic space . Subtractions are carried out between different color channels to eliminate chroma. Then, standard coordinates are calculated based on local mean values and variances. The local mean values are obtained by convoluting the image with an average filter such as Gaussian filter. The local variances are calculated based on the definition of the conventional standard variance in combination with weighting operation. The standard coordinates are obtained by subtracting pixel values from the mean values and then dividing by the local variances. Patent Document 1 also gives two other approaches for calculating local standard coordinates. At last, several schemes are provided for synthesizing the standard coordinates and the original image to obtain an enhance output image . One of these schemes consists in scaling the three channels of the original pixels with the same factor, which is the ratio of the standard coordinates to the original luminance. Another scheme relates to the inverse operation of the above first step. However, Patent Document 1 does not give any details on how to obtain an ideal output image.

According to the above Patent Document 1 , it is required to calculate the local mean values and the local variances in a relatively large neighborhood, which imposes an increased computational load. In addition, in Patent Document 1 , it is impossible to ensure that no artificial color will be introduced.

In other words, the technique according to Patent Document 1 may produce visible artifacts.

SUMMARY OF INVENTION

An object of the present invention is to provide a method and device for image enhancement, capable of generating a clearer and brighter image without any visible artifacts based on an input image.

According to an aspect of the present invention, a method for digital image enhancement is provided, which comprises steps of: calculating a local luminance value and a local contrast value for each pixel in an input digital image ; determining a gain factor for each pixel based on the local luminance value and the local contrast value; and adjusting a value of a pixel based on the gain factor determined for the pixel.

According to another aspect of the present invention, a device for digital image enhancement is provided, which comprises: a luminance calculation unit configured to calculate a local luminance value for each pixel in an input digital image; a contrast calculation unit configured to calculate a local contrast value for each pixel in the input digital image; a gain calculation unit configured to determine a gain factor for each pixel based on the local luminance value and the local contrast value; and a mapping unit configured to adjust a value of a pixel based on the gain factor determined for the pixel. With the above solution, it is possible to enhance both luminance and contrast of an image without losing any details, even if the image has a very large dynamic range.

BRIEF DESCRIPTION OF DRAWINGS

The above features and advantages of the present invention will be more apparent from the following detailed descriptions taken in conjunction with the figures.

Fig. 1 is a block diagram of a device for digital image enhancement according to an embodiment of the present invention. Fig. 2 is a flowchart illustrating a method for digital image enhancement according to an embodiment of the present invention.

Fig. 3 is a schematic diagram illustrating a process of calculating luminance and contrast according to an embodiment of the present invention.

Fig. 4 shows an example of application of the device according to an embodiment of the present invention.

Fig. 5 shows an example of application of the method and device according to embodiments of the present invention.

Fig. 6 shows an example of application of the method and device according to embodiments of the present invention.

Fig. 7 shows an example of application of the method and device according to embodiments of the present invention.

DESCRIPTION OF EMBODIMENTS

The preferred embodiments of the present invention will be detailed in the following with reference to the figures. Throughout the figures, the same or similar elements, though shown in different figures, are represented by the same reference sign. For clarity and conciseness purpose, details for known functions and structures contained herein will be omitted, so as not to obscure the concept of the present invention. According to the embodiments of the present invention, a method and a device for digital image enhancement are provided. In the embodiments of the present invention, a captured image is enhanced taking into account the ambient luminance of a scene during the image shooting process as well as the contrast between colors of the individual parts of the shot subject. According to the embodiments of the present invention, a digital image with a large dynamic range can be processed into an image with a dynamic range to which human eyes are sensitive . Further, the luminance and the contrast of the image can be improved while reducing the dynamic range.

Fig. 1 shows a block diagram of a device for digital image enhancement according to an embodiment of the present invention. The device 100 of this embodiment can be applied in a camera for a real-time processing on a captured image . As an alternative, a digital image can be read from a memory 1 10 which stores digital images shot in advance, processed according to the method of the present invention, and then stored in a memory or output from an output device 140 such as a printer or a display.

As shown in Fig. 1 , the image enhancement device 100 according the embodiment of the present invention comprises : a gray scaling unit 101 configured to gray-scale a color digital image; a luminance calculation unit 105 configured to calculate a luminance value for each pixel of the image ; a contrast calculation unit 102 configured to calculate a contrast value for each pixel of the image ; a gain calculation unit 103 configured to determine a gain factor for each pixel based on the calculated luminance value and contrast value; and a mapping unit 104 configured to process each pixel based on the determined gain factor. The image having been processed by the mapping unit 104 is stored in a memory 1 10 or output from an output device 140 such as a printer or a display.

Next, the specific functions and operation processes of the individual units will be explained in detail with reference to Figs. 2 and 3. Fig. 2 is a flowchart illustrating a method for digital image enhancement according to an embodiment of the present invention.

At steps S l l , S 12 and S I 3 , digital image data are input from the memory 1 10. According to the embodiment of the present invention, the color digital image read from the memory

1 10 is an image having three channels, R, G and B, which is then converted into a gray scale image by the gray scaling unit 101 . According to the embodiment of the present invention, the gray scaling unit 10 1 can be configured to obtain a gray scale value either by averaging the pixel values from the three channels, by selecting a pixel value of one out of the three channels, or by calculating the weighted sum of the pixel values from the three channels. For example, the gray scaling unit 101 can be configured to determine the gray scale values of the pixels according to the following equations :

grey(x) = (r (?;} - g (x) - b (*)) /3 · · · ( 1 )

or

grey(x) = max r{x),g{x), b{x)) _^ where the variable x denotes the location of a pixel in the input image . Since x can be a two-dimension vector, it can be used to represent the location of the pixel in the input image in a two-dimension coordinate system. The above r(x) , g(x) and b(x) denotes the R, G and B components of the color pixel, respectively, and max(-) denotes an operation of taking maximum.

According to an embodiment of the present invention, no gray scaling process is required if the input image is a single channel image .

Then, at steps S 14 and S 15, the luminance calculation unit 105 and the contrast calculation unit 102 calculate the luminance values and the contrast values of the pixels. According to an embodiment of the present invention, the luminance value and the contrast value of a pixel can be calculated according to luminance 00 = mean, ( )

... (3)

contrast w =

where x denotes the location of the pixel in the input image, G{x) denotes the gray scale value of the pixel (equivalent to grey(x) in the above equations (1) or (2)), mean_t x) denotes the local mean value on the i-th scale, and W. denotes the weight coefficient on the i-th scale. According to an embodiment of the present invention, in order to effectively calculate these parameters, as shown in Fig. 3, an integral image is first calculated according to

... (5)

That is, the sum of the gray scale values of all the pixels at the top left corner is calculated. Then, mean^x) is calculated by four addition/ subtraction operations. After obtaining the local mean value mean.,{x) on each scale, the luminance value and the contrast value can be calculated according to the above equations (3) and (4). After obtaining the luminance value and the contrast value , at step S 16 , the gain calculation unit 103 calculates a gain factor for each pixel in the image based on the calculated luminance value and contrast value . According to an embodiment of the pre sent invention , the gain calculation unit

103 can be configured to calculate the gain factor for each pixel according to upper _bound{x)

gainix) = - ;— - ~ ;— . . .

1 + a * exp (—strength » (contrast(xj— cross_pomt(xj)) ^ where a = upper _bound(x)— 1 , such that gain(x) = 1 when contrast(x) = cross_point(x) , upper _bound(x^~) is set to be a fixed value for the entire image or a value adaptive to each pixel. According to an embodiment of the present invention, upper _bound(x) can be set according to

upper _bound(x) = lfmean(x) ^yj

or

upper _bound{x) = l/(max (r(x),£(x), i?(x)) ^gj where mean(x) can be a mean value on any of the scale or a weighted sum of mean value s on the re spective scales . Similarly, cross_point(x) can be set as a fixed value for the entire image or as a value adaptive to each pixel. According to an embodiment of the pre sent invention , cross_point(x) can be set according to cross_point(x) = 1.0 ^gj

or

cross_point(x) = 1— luminance (x) ( 1 0) The parameter strength in the above equation (6) is reserved for a user's interactive adjustment. The user can control the intensity of image enhancement by adjusting this parameter.

After the gain for each pixel is calculated, the mapping unit 104 carries out an enhancement process on each pixel at step S 17. According to an embodiment of the present invention, the mapping unit 104 multiplies the values of the respective channels of each pixel with the same gain factor, such that the hue of the input image can be maintained. The mapping unit 104 performs its operation in such a manner that the mapping operation can be carried out according to

(r', g',b') = gain{x) * (r,g, b) _

At step S 18 , the enhanced image can be printed or displayed on a display screen by the output device 140 such as a printer or a display.

While the specific embodiment of the device according to the present invention has been described having separate functional modules, it will be appreciated by those skilled in the art that these modules can be embodied in a single device, such as a CPU, or as a number of components. For example, the device according to the embodiment of the present invention can be implemented in a camera, a mobile phone , a PC , or a medical apparatus, and can be implemented in software or hardware . Fig. 4 shows an example in which the device according to an embodiment of the present invention is applied . It can be implemented in a digital image capture and display system . A digital image captured by a CC D camera is enhanced by the device according to the present invention , so as to obtain an enhanced image for displaying on a display.

As shown in Fig. 5 , the method and device according to the embodiments of the present invention are also applicable to video image enhancement. In this case, the method according to the embodiments of the present invention is applied to each frame of a video . Then , the enhanced digital images can be displayed on a display device .

As shown in Fig. 6 , the method and device according to the embodiments of the present invention are also applicable to a print system . In this case , an image can be enhanced using the method according to the embodiments of the present invention , before it is printed . Then , the enhanced image can be printed and output from a printer. As shown in Fig. 7, an image enhanced by the method and device according to the embodiments of the present invention can be stored in a memory for subsequent processing. With the method and device according to the embodiments of the present invention, both luminance and contrast of an image can be enhanced without introducing any artifacts . According to further embodiments, the computational efficiency can be improved . The present invention is applicable in a variety of image processing systems, such as image / video capture device, display device, print device and the like.

The foregoing only describes the exemplary embodiments for implementing the present invention . It can be appreciated by those skilled in the art that any modifications and partial alternations can be made without departing from the scope of the present invention. These modifications and alternations should be encompassed by the scope of the present invention which is only defined by the claims as attached.

Claims

Claim 1

A method for digital image enhancement, comprising steps of:

-calculating a local luminance value and a local contrast value for each pixel in an input digital image ;

-determining a gain factor for each pixel based on the local luminance value and the local contrast value; and

-adjusting a value of a pixel based on the gain factor determined for the pixel.

Claim 2

The method according to claim 1 , wherein the input digital image is a color image containing a plurality of color channels, and the method further comprises a step of gray scaling the input digital image prior to the step of calculating a local luminance value and a local contrast value for each pixel in an input digital image .

Claim 3

The method according to claim 1 or 2 , wherein the step of calculating a local luminance value and a local contrast value for each pixel in an input digital image comprises:

-calculating a weighted sum of local luminance values on different scales ; and

-calculating a weighted sum of local contrast value s on different scales .

Claim 4

The method according to claim 3 , wherein the luminance value and the contrast value are calculated according to

Λ"

luminance (A.) = ^ W. * mean, (x)

∑ ( G (x) \

W; * loglO , ,

\mean. (x) where x denotes the location of the pixel in the input image , denotes the gray scale value of the pixel, mean. (x) denotes the local mean value on the i-th scale, and W_t denotes the weight coefficient on the i-th scale .

Claim 5

The method according to claim 1 or 2 , wherein the step of determining a gain factor for each pixel based on the local luminance value and the local contrast value comprises : calculating the gain factor according to upp e r_b o nd (,τ)

gain(x) = — T-T ;——— -

1— a · exp (—strength * (contrast(x)— cross_point(x))j where a = upper_baund(x — 1 , such that gain(x) = 1 when contrast(x)— cross_point(x) ; upper _bound{x) is set as a fixed value for the entire image, or as a value adaptive to each pixel; cross_pomt(x) = l - luminance (s) _{^ and the} parameter strength is reserved for a user in performing interactive adjustment. Claim 6

The method according to claim 5 , wherein

upper _bound(x) = lfniean{x^~) _Qr

upper _bound(x) = 1 / (max (r(x),g(x), b(x))

where mean(x) can be a mean value on any one of the scales or a weighted sum of mean values on the respective scales.

Claim 7

A device for digital image enhancement, comprising:

-a luminance calculation unit configured to calculate a local luminance value for each pixel in an input digital image;

-a contrast calculation unit configured to calculate a local contrast value for each pixel in the input digital image;

-a gain calculation unit configured to determine a gain factor for each pixel based on the local luminance value and the local contrast value; and

-a mapping unit configured to adjust a value of a pixel based on the gain factor determined for the pixel.

Claim 8 The device according to claim 7, wherein the input digital image is a color image containing a plurality of color channels, and the device further comprises :

a gray scaling unit configured to gray- scale the input digital image prior to calculating a local luminance value and a local contrast value for each pixel in an input digital image .

Claim 9

The device according to claim 7 or 8, wherein the luminance calculation unit calculates the local luminance values for each pixel by calculating a weighted sum of local luminance values on different scales; and the contrast calculation unit calculates the local contrast value for each pixel by calculating a weighted sum of local contrast values on different scales.

Claim 10

The device according to claim 9 , wherein the luminance value and the contrast value are calculated according to

where x denotes the location of the pixel in the input image, G(A) denotes the gray scale value of the pixel, mean. jV) denotes the local mean value on the i-th scale , and W. denotes the weight coefficient on the i-th scale .

Claim 1 1

The device according to claim 7 or 8 , wherein the gain calculation unit is configured for determining a gain factor for each pixel based on the local luminance value and the local contrast value by calculating the gain factor according to upp erjh o und(p )

g in{x) =

1— a · exp (—strength * (contrast(x)— cross_point(x))) where a— upper _bound(x)— 1 , such that gain(x) = 1 when contrast(x) = cross_pomt(x) ; upper _bound{x) is set as a fixed value for the entire image or as a value adaptive to each pixel; cross_point(x) = 1 - luminance (x) _{} and th e} parameter strength is reserved for a user in performing interactive adj ustment.

Claim 1 2

The device according to claim 1 1 , wherein

upper _bound(x) = l/mean(x) _QJ. upper _bound(x) = l/(max (r(x)_fg(x)_lb(x^')) where mean(x) can be a mean value on any one of the scales or a weighted sum of mean value s on the respective scales .

Claim 1 3 A control program which causes a computer to perform a method for digital image enhancement, the method comprising steps of:

-calculating a local luminance value and a local contrast value for each pixel in an input digital image;

Claim 14

A computer-readable storage medium storing a control program which causes a computer to perform a method for digital image enhancement, the method comprising steps of: