WO2008142641A2 - Image enhancement - Google Patents

Image enhancement Download PDF

Info

Publication number
WO2008142641A2
WO2008142641A2 PCT/IB2008/051975 IB2008051975W WO2008142641A2 WO 2008142641 A2 WO2008142641 A2 WO 2008142641A2 IB 2008051975 W IB2008051975 W IB 2008051975W WO 2008142641 A2 WO2008142641 A2 WO 2008142641A2
Authority
WO
WIPO (PCT)
Prior art keywords
signal
flf
pixel value
average pixel
fma
Prior art date
Application number
PCT/IB2008/051975
Other languages
French (fr)
Other versions
WO2008142641A3 (en
Inventor
Meng Zhao
Claus N. Cordes
Henricus W. P. Van Der Heijden
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Publication of WO2008142641A2 publication Critical patent/WO2008142641A2/en
Publication of WO2008142641A3 publication Critical patent/WO2008142641A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/90Dynamic range modification of images or parts thereof

Definitions

  • the invention relates to a method of image enhancement, an image enhancement apparatus, a display apparatus comprising the image enhancement apparatus, a camera comprising the image enhancement apparatus, a portable device comprising the image enhancement apparatus, and to a computer program product comprising a processor readable code to execute the method of image enhancement.
  • Contrast enhancement is one of the most important image enhancement techniques because it makes not yet visible details in the image visible. Contrast enhancement can be achieved, for example, by using histogram stretching, gamma curve adjustments, or frequency band boosting. All of these approaches can be applied with either a fixed gain for the whole image (global enhancement) or with a locally varying gain (local enhancement) .
  • gamma correction which modulates the luminance with a locally adaptive curve which boosts the details for the low luminance values and/or the high luminance values.
  • European application 05109066.0 (also PCT IB2006/053414) discloses an example of a local gamma correction.
  • Fig. 3 of this prior art shows that gamma correction is a non-linear contrast enhancement.
  • An advantage of gamma correction is that the gamma corrected signal is still within the valid range of values, or said differently, no loss of detail occurs due to clipping. However, the gamma correction only preserves and emphasizes image detail in low and/or high luminance areas and has no effect in mid luminance areas.
  • the boosting of the luminance details by the local gamma correction either raises the low luminance values (near the black level) at the low luminance part of the image or decreases the high luminance values (near the white level) at a bright part of the image. Consequently, the contrast decreases.
  • the valid range of values is defined by the actual implementation. In an analog implementation the valid range is limited by the power supply voltages and in a digital implementation the valid range is defined by the number of bits used to represent the digital values.
  • the local contrast boosting determines a difference of the input signal and a low-pass filtered input signal, multiplies this difference by a factor larger than one, and adds the low-pass filtered input signal to the multiplied difference.
  • the local contrast boosting enhances the contrast on all frequency bands by linearly increasing the difference between input pixel values according to the luminance value of the low-pass filtered input image.
  • Local contrast boosting does not change the average value of the image but usually causes clipping, especially for low and high values of the low-pass filtered input image.
  • a first aspect of the invention provides a method of image enhancement as claimed in claim 1.
  • a second aspect of the invention provides a contrast enhancement apparatus as claimed in claim 13.
  • a third aspect of the invention provides a display apparatus as claimed in claim 14.
  • a fourth aspect of the invention provides a camera as claimed in claim 15.
  • a fifth aspect of the invention provides a portable device as claimed in claim 16.
  • a sixth aspect of the invention provides a computer program product as claimed in claim 17. Advantageous embodiments are defined in the dependent claims.
  • the method of image enhancement in accordance with the first aspect of the invention performs a linear contrast enhancement of an input image signal to obtain a first enhanced signal and a non-linear contrast enhancement the input image signal to obtain a second enhanced signal.
  • the linear contrast enhancement is, for example, a prior art contrast boosting, such as for example, a histogram stretching.
  • the non-linear contrast enhancement is, for example, a prior art gamma correction.
  • the input image signal represent an image which may be picture or a sequence of pictures (video), computer generated information such as text and/or graphics, or any other visual material.
  • the first enhanced signal and the second enhanced signal are mixed in response to an average pixel value of the input signal to obtain a mixed signal.
  • the contribution of the first enhanced signal to the mixed signal is lower for low and high values of the average pixel value than for mid values of the average pixel value.
  • the contribution of the second enhanced signal to the mixed signal is higher for low and high values of the average pixel value than for mid values of the average pixel value. Due to this special way of mixing, the positive aspects of both the linear and non-linear contrast enhancement approaches are used. Unexpectedly, the drawbacks of both enhancement approaches are not summed.
  • the average pixel value may be the average luminance, or the average chrominance, or the average color, which is a mixture of average luminance and average chrominance.
  • the average pixel value is the average luminance
  • the different approaches are used in the luminance areas for which they provide the best result. Because the contribution of the linear contrast enhancement in the areas where it has negative effects and the non-linear contrast enhancement provides better results is lower, consequently, the drawbacks of the linear contrast enhancement is less pronounced.
  • This mixing in response to the average luminance of the input image signal has an unexpectedly better result than using the two approaches in series or in parallel.
  • the average luminance of the input signal may be the luminance level of the total input image, or portions of the input image.
  • the average luminance is also referred to as the local average luminance which is the average luminance of a sub-area of the image.
  • the average luminance may be obtained by low-pass filtering the input signal.
  • the linear contrast enhancement is a local linear contrast enhancement
  • the non-linear contrast enhancement is a local non-linear contrast enhancement
  • the average luminance is a local average luminance.
  • Local means that the actual enhancement depends on the location in the input image and thus may be different for different input pixels of the input image.
  • An example of a local linear contrast enhancement is a local histogram stretching.
  • An example of a local non- linear contrast enhancement is a local gamma correction.
  • the local average luminance may be generated with a low-pass filter which filters over at least a local area of the presently processed input pixel. Such a local approach usually provides better enhancement results than an overall enhancement.
  • the mixing determines the mixed signal as a weighted linear combination of the first enhanced signal and the second enhanced signal.
  • a weighting factor of the first enhanced signal is smaller for low and high values than for mid values of the average luminance.
  • the weighting factor of the second enhanced signal is larger for low and high values than for mid values of the average luminance.
  • the weighted linear combination is defined by
  • FMA kl * FGA + (1 - kl) * FLC
  • FMA is the mixed signal
  • 1 - kl is the weighting factor for the first enhanced signal FLC
  • kl is the weighting factor for the second enhanced signal FGA.
  • the weighting factors kl and 1-kl depend on the local average luminance and have a value in the range 0 to 1 including the border values. For kl equal to zero, only the non-linear contrast enhancement contributes to the mixed signal, for kl equal to one half, both the linear and the non-linear contrast enhancement contribute to the mixed signal with equal intensity. If the factor kl varies gradually with the local average luminance, the contributions of the linear and the nonlinear contrast enhancement also change gradually with respect to each other and discontinuities in the contrast enhanced image are prevented.
  • both the first and second weighting factors are a function of the average luminance. Both these function are symmetrical around the mid value of a valid range of the average luminance. Such a symmetrical function has the advantage that the contribution of the non-linear contrast enhancement has the same behavior for both low and high average luminance levels.
  • the method further comprises a mixing of the input image signal and the mixed signal.
  • This mixing is performed in response to a control signal.
  • the signal obtained by the mixing of the input image signal and the mixed signal is referred to as the output signal.
  • the control signal is generated in response to both the local detail of the input pixel being processed and the average luminance of the input image signal.
  • the local detail is defined by the local structure of the image in the neighborhood of the presently processed input pixel. The amount of local detail is high if much local structure is present and/or if much high frequent detail is present.
  • the control signal is generated such that (i) for values of the average luminance below a first threshold value or above a second threshold value a contribution of the mixed signal to the output signal increases for increasing local detail, and (ii) a contribution of the input image signal to the output signal decreases for increasing local detail. For values of the average luminance in between the first threshold value and the second threshold value, only the mixed signal contributes to the output signal.
  • the output signal (FO) is defined by:
  • This method can be applied to the input signal at either a single frequency band or multi- frequency bands.
  • the method can be used within each frequency band of the signal, and the enhanced signals from each band are summed to obtain the final output.
  • Fig. 1 schematically shows a block diagram of a contrast enhancement circuit in accordance with the invention
  • Fig. 2 schematically shows a weighting factor for mixing the linear and the non-linear contrast enhanced signals in accordance with an embodiment
  • Fig. 3 schematically shows a weighting factor for mixing the mixed linear and non-linear contrast enhanced signal with the input image in accordance with an embodiment
  • Fig. 4 schematically shows an embodiment of a circuit for controlling the mixing of the mixed linear and non- linear contrast enhanced signal with the input image
  • Fig. 5 schematically shows an embodiment of a portable apparatus comprising a display
  • Fig. 6 schematically shows a block diagram of a processor system.
  • Fig. 1 schematically shows a block diagram of a contrast enhancement circuit in accordance with the invention.
  • the linear contrast enhancement 1 receives the input image signal FI and supplies the linear contrast enhanced image signal FLC.
  • the non-linear contrast enhancement 2 receives the input image signal FI and supplies the non-linear contrast enhanced image signal FGA.
  • the mixing 4 mixes the linear contrast enhanced image signal FLC with the non-linear contrast enhanced image signal FGA in response to the average image signal FLF to obtain the mixed image signal FMA.
  • the average image signal FLF may be obtained by a low-pass filtering of the input image signal FI.
  • the mixed image signal FMA may be the output signal of the contrast enhancement circuit.
  • the linear contrast enhancement is a local histogram stretching which provides a contrast boost enhancing the mid and high frequencies of the input image signal FI.
  • the nonlinear contrast enhancement is a local gamma correction which enhances the detail in the low and/or high luminance areas of the input image.
  • the mixing 4 of the linear contrast enhanced signal FLC and the non- linear contrast enhanced signal FGA in response to the average luminance FLF is performed to obtain the mixed signal FMA wherein a contribution of the linear contrast enhanced signal FLC is lower for low and high values of the average luminance FLF than for mid values of the average luminance FLF, and wherein a contribution of the non- linear contrast enhanced signal FGA is higher for low and high values of the average luminance FLF than for mid values of the average luminance FLF. Consequently, both contrast enhancement techniques provide the largest contribution to the mixed signal FMA where they have the highest positive effect, and provide the smallest contribution where they have the largest artifact contribution.
  • the mixing 4 may be a linear weighted sum of the linear contrast enhanced signal FLC and the non-linear contrast enhanced signal FGA.
  • the coefficient determining the contribution of the linear contrast enhanced signal has a smaller value for low and high values of the average luminance FLF than for mid values of the average luminance FLF.
  • the coefficient determining the contribution of the non- linear contrast enhanced signal FGA has a higher value for low and high values of the average luminance FLF than for mid values of the average luminance FLF. Both coefficients may be normalized in a range from zero to one (including the border values zero and one). In an example, if the coefficient for the non- linear contrast enhanced signal FGA is kl, the coefficient for the linear contrast enhanced signals FLC is 1-kl.
  • the optional mixing 6 receives the input image signal FI, the mixed image signal FMA and a high-pass filtered input image signal FHF, and supplies the output image signal FO.
  • the mixing of the input image signal FI and the mixed image signal FMA depends both on the level of the input image signal FI and the local detail in the input image. For mid luminance values of the input image the contribution of the mixed image signal FMA prevails over the contribution of the input image signal FI. In an embodiment, for the mid luminance values, the mixed image signal FMA fully contributes to the output image signal FO, while the input image signal FI does not contribute at all to the output image signal FO.
  • the contributions of the mixed image signal FMA and the input image FI to the output image signal FO depend on the detected amount of local detail FHF.
  • the contribution of the mixed image signal is higher if more local detail FHF is detected, and consequently, the contribution of the input image signal FI decreases with increasing local detail FHF.
  • the output image signal FO is defined by:
  • FO FMA for THK FK TH2
  • FMA the mixed signal
  • FI the input image signal
  • THl the first threshold value
  • TH2 the second threshold value.
  • a suitable value for the threshold values THl, TH2 is, for example, about 10% of the range from the minimum and maximum values of the range, respectively.
  • FI the input image signal
  • THl the first threshold value
  • TH2 the second threshold value.
  • a suitable value for the threshold values THl, TH2 is, for example, about 10% of the range from the minimum and maximum values of the range, respectively.
  • FI the minimum and maximum values of the range
  • FI the input image signal
  • THl the first threshold value
  • TH2 the second threshold value.
  • a suitable value for the threshold values THl, TH2 is, for example, about 10% of the range from the minimum and maximum values of the range, respectively.
  • FI the input image signal
  • THl the first threshold value
  • TH2 the second threshold value.
  • any other circuit may be used which provides a measure for the amount of detail detected in the input image signal FI.
  • the amount of detail may depend on the amount of structure and/or the amount of high- frequent information in the input image signal FI.
  • Fig. 2 schematically shows a weighting factor for mixing the linear and the non-linear contrast enhanced signals in accordance with an embodiment.
  • the factor kl is depicted as a function of the average level FLF of the input image signal FI for two embodiments. In both embodiments, the factor kl is symmetrical around the center value of the valid range of the average level FLF.
  • the input image signal FI and the average level FLF are defined by 8 bit digital words which have a value in the valid range from 0 to 255. Values outside this valid range can not be represented.
  • the factor kla is zero for the center value (which is 128 in this example for 8 bit words) of the valid range of the average level FLF and increases for lower and higher values of the average level FLF.
  • the parabola shape shown is just by way of example; any monotonously increasing function can be used. This function may change levels stepwise.
  • the center of the valid range the nonlinear contrast enhancement 2 has no contribution at all, while its contribution to the mixed signal FMA increases for lower and higher values in the valid range.
  • the maximum contribution of the non-linear contrast enhancement 2 may be limited. In the example shown, the maximum contribution is limited to the factor 0.25.
  • the contribution of the linear enhanced signal FLC to the mixed signal FMA may be the reciprocal (1-kl) of the contribution of the non-linear contrast enhanced signal FGA.
  • the weighting factor for the linear contrast enhanced signal FLC is 0.75 and the weighting factor for the non-linear contrast enhanced signal FLC is 0.25.
  • the drawback of the non- linear contrast enhancement that the overall contrast is decreased is compensated by the linear contrast enhancement 1. But, still the non- linear contrast enhancement 2 enhances the contrast in low and high luminance areas of the input image.
  • the factor klb has a non-zero value (arbitrary selected to be 0.25 in the example shown). The effect is that some non-linear contrast enhancement is performed for mid average luminance levels.
  • the n is the offset which in the embodiment shown is 0.25. It has to be noted that this offset n may be negative.
  • the offset n may be a predetermined value, which may be 0.
  • the offset n may have an inverse relation with the local average luminance FLF of the input image signal FI.
  • CLIP()0,l is meant that the value of what is within the brackets if smaller than 0 is clipped to 0, and if larger than 1 is clipped to 1.
  • An example of the symmetrical function for an 8 bit average luminance FLF is:
  • Fsym(FLF) ((FLF/255 - 0.5) 2 + n) wherein the 0.5 causes the symmetry.
  • Fig. 3 schematically shows a weighting factor for mixing the mixed linear and non-linear contrast enhanced signal with the input image in accordance with an embodiment.
  • the factor k2 is shown as function of the level of the input image signal FI which is represented by 8 bit words.
  • the factor k2 is 1 in the range of input values from the threshold value THl to the threshold value TH2.
  • the threshold value THl may 25 and the threshold value of TH2 may be 230.
  • these threshold values THl and TH2 are arbitrary and may be selected at a different distance from the border values 0 and 255.
  • the distance of the threshold value THl from 0 and the distance of the threshold value TH2 from 255 may differ.
  • the factor k2 depends on the amount of local detail FHF which dependence is indicated by the doted crosses. An embodiment of this dependence has already been discussed with respect to Fig. 2.
  • Fig. 4 schematically shows an embodiment of a circuit for controlling the mixing of the mixed linear and non- linear contrast enhanced signal with the input image.
  • the comparator 60 compares the input image signal FI with the threshold levels THl and TH2 and outputs a window signal WI indicating whether the level of the input image signal FI is or is not within the range from the threshold level THl to the threshold level TH2.
  • the window signal WI is one from zero to the threshold level THl, zero from the threshold level THl to the threshold level TH2, and one again from the threshold level TH2 to 255.
  • the multiplier 61 multiplies the local detail FHF with the constant C to obtain a factor k2'.
  • the multiplier 62 multiplies the window signal WI with the factor k2' to obtain a factor k2" which is equal to the factor k2' if the level of the input image signal FI is below the threshold level THl or above the threshold level TH2.
  • the factor k2" is zero in the range from the threshold level THl to the threshold level TH2.
  • the unit 63 replaces the zero value of the factor k2" in the range from the threshold level THl to the threshold level TH2 by 1, and supplies the factor k2.
  • the factor k2 can be generated in many other ways.
  • the circuit 60 may directly output the level 1 in the range from the threshold level THl to the threshold level TH2, and receives the factor k2' to output this factor outside this range.
  • Fig. 5 schematically shows an embodiment of a portable apparatus comprising a display 10, a camera 7 and buttons 25.
  • the image sensor of the camera 7 supplies an image sensor signal to a processing circuit (not shown) which supplies the input image signal FI.
  • the portable device may be a PDA (Personal Digital Assistant), a mobile phone, a digital camera or the like.
  • the camera 7 may alternatively be positioned at the other side of the portable apparatus than the display 10.
  • Fig. 6 schematically shows a block diagram of a processor system.
  • the processor executes the method of contrast enhancement in accordance with the present invention.
  • the processor PR receives the input image signal FI and supplies the output image signal FO.
  • the processor PR executes a program stored in the storage medium SM which, for example, is a memory or an optical storage medium.
  • the storage medium SM may be present in the processor PR.
  • the processor PR may process the input image signal FI to supply the mixed image signal FMA.
  • any reference signs placed between parentheses shall not be construed as limiting the claim.
  • Use of the verb "comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim.
  • the article "a” or “an” preceding an element does not exclude the presence of a plurality of such elements.
  • the invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Picture Signal Circuits (AREA)

Abstract

A method of image enhancement linear contrast enhances(l) an input image signal (FI) to obtain a first enhanced signal (FLC), and non- linear contrast enhances (2) the input image signal (FI) to obtain a second enhanced signal (FGA). An average pixel value (FLF) of pixels of the input image signal (FI) is determined (3). The first enhanced signal (FLC) and the second enhanced signal (FGA) are mixed (4) in response to the average pixel value (FLF) to obtain a mixed signal (FMA). A contribution of the first enhanced signal (FLC) to the mixed signal (FMA) is lower for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF), and a contribution of the second enhanced signal (FGA) to the mixed signal (FMA) is higher for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF).

Description

Image enhancement
FIELD OF THE INVENTION
The invention relates to a method of image enhancement, an image enhancement apparatus, a display apparatus comprising the image enhancement apparatus, a camera comprising the image enhancement apparatus, a portable device comprising the image enhancement apparatus, and to a computer program product comprising a processor readable code to execute the method of image enhancement.
BACKGROUND OF THE INVENTION
Image enhancement techniques are widely used to improve the perceived image quality. Contrast enhancement is one of the most important image enhancement techniques because it makes not yet visible details in the image visible. Contrast enhancement can be achieved, for example, by using histogram stretching, gamma curve adjustments, or frequency band boosting. All of these approaches can be applied with either a fixed gain for the whole image (global enhancement) or with a locally varying gain (local enhancement) .
One of these local enhancement approaches is the local gamma correction which modulates the luminance with a locally adaptive curve which boosts the details for the low luminance values and/or the high luminance values. European application 05109066.0 (also PCT IB2006/053414) discloses an example of a local gamma correction. Fig. 3 of this prior art shows that gamma correction is a non-linear contrast enhancement. An advantage of gamma correction is that the gamma corrected signal is still within the valid range of values, or said differently, no loss of detail occurs due to clipping. However, the gamma correction only preserves and emphasizes image detail in low and/or high luminance areas and has no effect in mid luminance areas. Further, the boosting of the luminance details by the local gamma correction either raises the low luminance values (near the black level) at the low luminance part of the image or decreases the high luminance values (near the white level) at a bright part of the image. Consequently, the contrast decreases. The valid range of values is defined by the actual implementation. In an analog implementation the valid range is limited by the power supply voltages and in a digital implementation the valid range is defined by the number of bits used to represent the digital values.
Another local enhancement technique is the local contrast boosting which enhances mid and high frequency band details. For example, the local contrast boosting determines a difference of the input signal and a low-pass filtered input signal, multiplies this difference by a factor larger than one, and adds the low-pass filtered input signal to the multiplied difference. The local contrast boosting enhances the contrast on all frequency bands by linearly increasing the difference between input pixel values according to the luminance value of the low-pass filtered input image. Local contrast boosting does not change the average value of the image but usually causes clipping, especially for low and high values of the low-pass filtered input image.
SUMMARY OF THE INVENTION
It is an object of the invention to provide an improved contrast enhancement. A first aspect of the invention provides a method of image enhancement as claimed in claim 1. A second aspect of the invention provides a contrast enhancement apparatus as claimed in claim 13. A third aspect of the invention provides a display apparatus as claimed in claim 14. A fourth aspect of the invention provides a camera as claimed in claim 15. A fifth aspect of the invention provides a portable device as claimed in claim 16. A sixth aspect of the invention provides a computer program product as claimed in claim 17. Advantageous embodiments are defined in the dependent claims.
The method of image enhancement in accordance with the first aspect of the invention performs a linear contrast enhancement of an input image signal to obtain a first enhanced signal and a non-linear contrast enhancement the input image signal to obtain a second enhanced signal. The linear contrast enhancement is, for example, a prior art contrast boosting, such as for example, a histogram stretching. The non-linear contrast enhancement is, for example, a prior art gamma correction. The input image signal represent an image which may be picture or a sequence of pictures (video), computer generated information such as text and/or graphics, or any other visual material. In accordance with the present invention, the first enhanced signal and the second enhanced signal are mixed in response to an average pixel value of the input signal to obtain a mixed signal. The contribution of the first enhanced signal to the mixed signal is lower for low and high values of the average pixel value than for mid values of the average pixel value. The contribution of the second enhanced signal to the mixed signal is higher for low and high values of the average pixel value than for mid values of the average pixel value. Due to this special way of mixing, the positive aspects of both the linear and non-linear contrast enhancement approaches are used. Unexpectedly, the drawbacks of both enhancement approaches are not summed. The average pixel value may be the average luminance, or the average chrominance, or the average color, which is a mixture of average luminance and average chrominance.
Thus, for example, if the average pixel value is the average luminance, in the combination of linear contrast enhancement and the non- linear contrast enhancement, the different approaches are used in the luminance areas for which they provide the best result. Because the contribution of the linear contrast enhancement in the areas where it has negative effects and the non-linear contrast enhancement provides better results is lower, consequently, the drawbacks of the linear contrast enhancement is less pronounced. This mixing in response to the average luminance of the input image signal has an unexpectedly better result than using the two approaches in series or in parallel. The average luminance of the input signal may be the luminance level of the total input image, or portions of the input image. In the last mentioned embodiment, the average luminance is also referred to as the local average luminance which is the average luminance of a sub-area of the image. The average luminance may be obtained by low-pass filtering the input signal.
In an embodiment, the linear contrast enhancement is a local linear contrast enhancement, the non-linear contrast enhancement is a local non-linear contrast enhancement, and the average luminance is a local average luminance. Local means that the actual enhancement depends on the location in the input image and thus may be different for different input pixels of the input image. An example of a local linear contrast enhancement is a local histogram stretching. An example of a local non- linear contrast enhancement is a local gamma correction. The local average luminance may be generated with a low-pass filter which filters over at least a local area of the presently processed input pixel. Such a local approach usually provides better enhancement results than an overall enhancement.
In an embodiment, the mixing determines the mixed signal as a weighted linear combination of the first enhanced signal and the second enhanced signal. A weighting factor of the first enhanced signal is smaller for low and high values than for mid values of the average luminance. The weighting factor of the second enhanced signal is larger for low and high values than for mid values of the average luminance. Such a linear combination is easy to compute and provides a pleasant phasing between the linear and the non-linear contrast enhanced signals such that discontinuities in the contrast enhanced image are prevented.
In an embodiment, the weighted linear combination is defined by
FMA = kl * FGA + (1 - kl) * FLC wherein FMA is the mixed signal, 1 - kl is the weighting factor for the first enhanced signal FLC and kl is the weighting factor for the second enhanced signal FGA. The weighting factors kl and 1-kl depend on the local average luminance and have a value in the range 0 to 1 including the border values. For kl equal to zero, only the non-linear contrast enhancement contributes to the mixed signal, for kl equal to one half, both the linear and the non-linear contrast enhancement contribute to the mixed signal with equal intensity. If the factor kl varies gradually with the local average luminance, the contributions of the linear and the nonlinear contrast enhancement also change gradually with respect to each other and discontinuities in the contrast enhanced image are prevented.
In an embodiment, both the first and second weighting factors are a function of the average luminance. Both these function are symmetrical around the mid value of a valid range of the average luminance. Such a symmetrical function has the advantage that the contribution of the non-linear contrast enhancement has the same behavior for both low and high average luminance levels.
In an embodiment, the method further comprises a mixing of the input image signal and the mixed signal. This mixing is performed in response to a control signal. The signal obtained by the mixing of the input image signal and the mixed signal is referred to as the output signal. The control signal is generated in response to both the local detail of the input pixel being processed and the average luminance of the input image signal. The local detail is defined by the local structure of the image in the neighborhood of the presently processed input pixel. The amount of local detail is high if much local structure is present and/or if much high frequent detail is present. The control signal is generated such that (i) for values of the average luminance below a first threshold value or above a second threshold value a contribution of the mixed signal to the output signal increases for increasing local detail, and (ii) a contribution of the input image signal to the output signal decreases for increasing local detail. For values of the average luminance in between the first threshold value and the second threshold value, only the mixed signal contributes to the output signal.
In an embodiment, the output signal (FO) is defined by:
FO = k2 * FMA + (1 - k2) * FI for FI < THl or FI > TH2
FO = FMA for THK FK TH2 wherein FO is the output signal, FMA is the mixed signal, FI is the input image signal, THl is the first threshold value, TH2 is the second threshold value, and wherein k2 = C * FHF wherein C is constant and FHF is the value of the local detail.
This method can be applied to the input signal at either a single frequency band or multi- frequency bands. In the multi- frequency band approach, the method can be used within each frequency band of the signal, and the enhanced signals from each band are summed to obtain the final output.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.
BRIEF DESCRIPTION OF THE DRAWINGS
In the drawings:
Fig. 1 schematically shows a block diagram of a contrast enhancement circuit in accordance with the invention, Fig. 2 schematically shows a weighting factor for mixing the linear and the non-linear contrast enhanced signals in accordance with an embodiment,
Fig. 3 schematically shows a weighting factor for mixing the mixed linear and non-linear contrast enhanced signal with the input image in accordance with an embodiment,
Fig. 4 schematically shows an embodiment of a circuit for controlling the mixing of the mixed linear and non- linear contrast enhanced signal with the input image,
Fig. 5 schematically shows an embodiment of a portable apparatus comprising a display, and
Fig. 6 schematically shows a block diagram of a processor system.
It should be noted that items which have the same reference numbers in different Figures, have the same structural features and the same functions, or are the same signals. Where the function and/or structure of such an item has been explained, there is no necessity for repeated explanation thereof in the detailed description.
DETAILED DESCRIPTION OF THE EMBODIMENTS Fig. 1 schematically shows a block diagram of a contrast enhancement circuit in accordance with the invention. The linear contrast enhancement 1 receives the input image signal FI and supplies the linear contrast enhanced image signal FLC. The non-linear contrast enhancement 2 receives the input image signal FI and supplies the non-linear contrast enhanced image signal FGA. The mixing 4 mixes the linear contrast enhanced image signal FLC with the non-linear contrast enhanced image signal FGA in response to the average image signal FLF to obtain the mixed image signal FMA. The average image signal FLF may be obtained by a low-pass filtering of the input image signal FI. The mixed image signal FMA may be the output signal of the contrast enhancement circuit. For example, the linear contrast enhancement is a local histogram stretching which provides a contrast boost enhancing the mid and high frequencies of the input image signal FI. For example, the nonlinear contrast enhancement is a local gamma correction which enhances the detail in the low and/or high luminance areas of the input image.
The mixing 4 of the linear contrast enhanced signal FLC and the non- linear contrast enhanced signal FGA in response to the average luminance FLF is performed to obtain the mixed signal FMA wherein a contribution of the linear contrast enhanced signal FLC is lower for low and high values of the average luminance FLF than for mid values of the average luminance FLF, and wherein a contribution of the non- linear contrast enhanced signal FGA is higher for low and high values of the average luminance FLF than for mid values of the average luminance FLF. Consequently, both contrast enhancement techniques provide the largest contribution to the mixed signal FMA where they have the highest positive effect, and provide the smallest contribution where they have the largest artifact contribution.
For example, the mixing 4 may be a linear weighted sum of the linear contrast enhanced signal FLC and the non-linear contrast enhanced signal FGA. The coefficient determining the contribution of the linear contrast enhanced signal has a smaller value for low and high values of the average luminance FLF than for mid values of the average luminance FLF. The coefficient determining the contribution of the non- linear contrast enhanced signal FGA has a higher value for low and high values of the average luminance FLF than for mid values of the average luminance FLF. Both coefficients may be normalized in a range from zero to one (including the border values zero and one). In an example, if the coefficient for the non- linear contrast enhanced signal FGA is kl, the coefficient for the linear contrast enhanced signals FLC is 1-kl. Two examples of the factor kl (kla and klb) will be discussed with respect to Fig. 2. The optional mixing 6 receives the input image signal FI, the mixed image signal FMA and a high-pass filtered input image signal FHF, and supplies the output image signal FO. The mixing of the input image signal FI and the mixed image signal FMA depends both on the level of the input image signal FI and the local detail in the input image. For mid luminance values of the input image the contribution of the mixed image signal FMA prevails over the contribution of the input image signal FI. In an embodiment, for the mid luminance values, the mixed image signal FMA fully contributes to the output image signal FO, while the input image signal FI does not contribute at all to the output image signal FO. For luminance values of the input image signal below a threshold value THl or above a threshold value TH2, the contributions of the mixed image signal FMA and the input image FI to the output image signal FO depend on the detected amount of local detail FHF. The contribution of the mixed image signal is higher if more local detail FHF is detected, and consequently, the contribution of the input image signal FI decreases with increasing local detail FHF. In an embodiment, the output image signal FO is defined by:
FO = k2 * FMA + (1 - k2) * FI for FI < THl or FI > TH2
FO = FMA for THK FK TH2 wherein FO is the output image signal, FMA is the mixed signal, FI is the input image signal, THl is the first threshold value, TH2 is the second threshold value. A suitable value for the threshold values THl, TH2 is, for example, about 10% of the range from the minimum and maximum values of the range, respectively. Thus, for an 8 bit representation of the input image signal FI at about 25 and 230, respectively. The factor k2 is determined by: k2 = C * FHF wherein C is constant and FHF is the amount of the local detail. An example of the factor k2 is shown in Fig. 3. The constant C determines the sensitivity for the local detail FHF. In a practical embodiment C has the value 10.
Instead of the high-pass filter 5 any other circuit may be used which provides a measure for the amount of detail detected in the input image signal FI. Again, it has to be noted that the amount of detail may depend on the amount of structure and/or the amount of high- frequent information in the input image signal FI.
Fig. 2 schematically shows a weighting factor for mixing the linear and the non-linear contrast enhanced signals in accordance with an embodiment. The factor kl is depicted as a function of the average level FLF of the input image signal FI for two embodiments. In both embodiments, the factor kl is symmetrical around the center value of the valid range of the average level FLF. In the example shown the input image signal FI and the average level FLF are defined by 8 bit digital words which have a value in the valid range from 0 to 255. Values outside this valid range can not be represented.
In the first embodiment, the factor kla is zero for the center value (which is 128 in this example for 8 bit words) of the valid range of the average level FLF and increases for lower and higher values of the average level FLF. The parabola shape shown is just by way of example; any monotonously increasing function can be used. This function may change levels stepwise. Thus, in the example shown, the center of the valid range the nonlinear contrast enhancement 2 has no contribution at all, while its contribution to the mixed signal FMA increases for lower and higher values in the valid range. The maximum contribution of the non-linear contrast enhancement 2 may be limited. In the example shown, the maximum contribution is limited to the factor 0.25. The contribution of the linear enhanced signal FLC to the mixed signal FMA may be the reciprocal (1-kl) of the contribution of the non-linear contrast enhanced signal FGA. Thus in the example shown, for a very low average luminance FLF, the weighting factor for the linear contrast enhanced signal FLC is 0.75 and the weighting factor for the non-linear contrast enhanced signal FLC is 0.25. The drawback of the non- linear contrast enhancement that the overall contrast is decreased is compensated by the linear contrast enhancement 1. But, still the non- linear contrast enhancement 2 enhances the contrast in low and high luminance areas of the input image.
In the second example at the center of the valid range for the average luminance FLF, the factor klb has a non-zero value (arbitrary selected to be 0.25 in the example shown). The effect is that some non-linear contrast enhancement is performed for mid average luminance levels. The factor kl can be defined as: kl = CLIP(fsym(FLF) + n))0,l wherein fsym(FLF) is a symmetrical function of the average luminance FLF. The symmetrical function is symmetrical with respect to the center value of the valid range of the average luminance FLF. The n is the offset which in the embodiment shown is 0.25. It has to be noted that this offset n may be negative. The offset n may be a predetermined value, which may be 0. Alternatively, the offset n may have an inverse relation with the local average luminance FLF of the input image signal FI. Thus, the higher the local average luminance FLF is, the lower the value of the offset n is. With CLIP()0,l is meant that the value of what is within the brackets if smaller than 0 is clipped to 0, and if larger than 1 is clipped to 1. An example of the symmetrical function for an 8 bit average luminance FLF is:
Fsym(FLF) = ((FLF/255 - 0.5)2 + n) wherein the 0.5 causes the symmetry.
Fig. 3 schematically shows a weighting factor for mixing the mixed linear and non-linear contrast enhanced signal with the input image in accordance with an embodiment. The factor k2 is shown as function of the level of the input image signal FI which is represented by 8 bit words. In this example, the factor k2 is 1 in the range of input values from the threshold value THl to the threshold value TH2. The threshold value THl may 25 and the threshold value of TH2 may be 230. However, these threshold values THl and TH2 are arbitrary and may be selected at a different distance from the border values 0 and 255. The distance of the threshold value THl from 0 and the distance of the threshold value TH2 from 255 may differ. Below the threshold value THl and above the threshold value TH2, the factor k2 depends on the amount of local detail FHF which dependence is indicated by the doted crosses. An embodiment of this dependence has already been discussed with respect to Fig. 2.
Fig. 4 schematically shows an embodiment of a circuit for controlling the mixing of the mixed linear and non- linear contrast enhanced signal with the input image. The comparator 60 compares the input image signal FI with the threshold levels THl and TH2 and outputs a window signal WI indicating whether the level of the input image signal FI is or is not within the range from the threshold level THl to the threshold level TH2. For example, for an 8 bit representation of the input image signal FI, the window signal WI is one from zero to the threshold level THl, zero from the threshold level THl to the threshold level TH2, and one again from the threshold level TH2 to 255. The multiplier 61 multiplies the local detail FHF with the constant C to obtain a factor k2'. The multiplier 62 multiplies the window signal WI with the factor k2' to obtain a factor k2" which is equal to the factor k2' if the level of the input image signal FI is below the threshold level THl or above the threshold level TH2. The factor k2" is zero in the range from the threshold level THl to the threshold level TH2. The unit 63 replaces the zero value of the factor k2" in the range from the threshold level THl to the threshold level TH2 by 1, and supplies the factor k2. It has to be noted that the factor k2 can be generated in many other ways. For example, the circuit 60 may directly output the level 1 in the range from the threshold level THl to the threshold level TH2, and receives the factor k2' to output this factor outside this range.
Fig. 5 schematically shows an embodiment of a portable apparatus comprising a display 10, a camera 7 and buttons 25. The image sensor of the camera 7 supplies an image sensor signal to a processing circuit (not shown) which supplies the input image signal FI. The portable device may be a PDA (Personal Digital Assistant), a mobile phone, a digital camera or the like. The camera 7 may alternatively be positioned at the other side of the portable apparatus than the display 10. In addition to or instead of the buttons 25 other user input means may be present. Fig. 6 schematically shows a block diagram of a processor system. The processor executes the method of contrast enhancement in accordance with the present invention. The processor PR receives the input image signal FI and supplies the output image signal FO. The processor PR executes a program stored in the storage medium SM which, for example, is a memory or an optical storage medium. The storage medium SM may be present in the processor PR. Alternatively, the processor PR may process the input image signal FI to supply the mixed image signal FMA.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.
In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:
1. A method of image enhancement comprising: linear contrast enhancing (1) an input image signal (FI) to obtain a first enhanced signal (FLC), non-linear contrast enhancing (2) the input image signal (FI) to obtain a second enhanced signal (FGA), determining (3) an average pixel value (FLF) for pixels of the input image signal (FI), and mixing (4) the first enhanced signal (FLC) and the second enhanced signal (FGA) in response to the average pixel value (FLF) to obtain a mixed signal (FMA), wherein a contribution of the first enhanced signal (FLC) to the mixed signal (FMA) is lower for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF), and wherein a contribution of the second enhanced signal (FGA) to the mixed signal (FMA) is higher for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF).
2. A method as claimed in claim 1, wherein the linear contrast enhancing (1) is a local linear contrast enhancing, the non-linear contrast enhancing (2) is a local non-linear contrast enhancing, and - the average pixel value (FLF) is a local average luminance.
3. A method as claimed in claim 1, wherein the determining the average pixel value (FLF) comprises low-pass filtering for supplying a low-pass filtered input image signal as the average pixel value (FLF).
4. A method as claimed in claim 1, wherein the linear contrast enhancing comprises local histogram stretching.
5. A method as claimed in claim 4, wherein the first enhanced signal (FLC) is defined by FLC = FLF + g (FI - FLF), wherein FLC is the first enhanced signal, FLF is the average pixel value, and the factor g > 1.
6. A method as claimed in claim 1, wherein the non- linear contrast enhancing comprises local gamma correction.
7. A method as claimed in claim 5, wherein the second enhanced signal is defined by
FGA =
Figure imgf000013_0001
and wherein M = N/2-1, and N is the highest value in the range of valid values, FI is the input image signal, and FLF is the average pixel value.
8. A method as claimed in claim 1, wherein the mixing (4) determines the mixed signal (FMA) as a weighted linear combination of the first enhanced signal (FLC) and the second enhanced signal (FGA), a first weighting factor of the first enhanced signal is smaller for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF), and a second weighting factor of the second enhanced signal is larger for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF).
9. A method as claimed in claim 8, wherein the weighted linear combination is defined by
FMA = kl * FGA + (1 - kl) * FLC wherein FMA is the mixed signal, 1 - kl is the first weighting factor, kl is the second weighting factor, FGA is the second enhanced signal and FLC is the first enhanced signal.
10. A method as claimed in claim 8 or 9, wherein both the first and second weighting factors are a function of the average luminance (FLF) being symmetrical around the mid value of a valid range of the average luminance (FLF).
11. A method as claimed in claim 1 , further comprising further mixing (6) of the input image signal (FI) and the mixed signal (FMA) in response to a factor (k2) to obtain an output signal (FO), determining (5) local detail (FHF) in the input image signal (FI), generating the factor (k2) in response to the local detail (FHF) and the input image signal (FI) to obtain:
(i) for values of the average pixel value (FLF) either below a first threshold value
(THl) or above a second threshold value (TH2), a contribution of the mixed signal (FMA) to the output signal (FO) increasing for increasing local detail (FHF), and a contribution of the input image signal (FI) to the output signal (FO) decreasing for increasing local detail (FLF), and
(ii) for values of the average pixel value (FLF) in between the first threshold value
(THl) and the second threshold value (TH2), only a contribution of the mixed signal (FMA) to the output signal (FO).
12. A method as claimed in claim 11, wherein the output signal (FO) is defined by:
FO = k2 * FMA + (1 - k2) * FI for FI < THl or FI > TH2
FO = FMA for THK FK TH2 wherein FO is the output signal, FMA is the mixed signal, FI is the input image signal, THl is the first threshold value, TH2 is the second threshold value, and wherein the factor k2 = C * FHF, wherein C is a constant and FHF is the value of the local detail (FHF).
13. An image enhancement apparatus comprising: a linear contrast enhancer (1) for applying a linear contrast enhancement on an input image signal (FI) to obtain a first enhanced signal (FLC), a non-linear contrast enhancer (2) for applying a non-linear contrast enhancement on the input image signal (FI) to obtain a second enhanced signal (FGA), an average pixel value determiner (3) for determining an average pixel value (FLF) for pixels of the input image signal (FI), and - a mixer (4) for mixing the first enhanced signal (FLC) and the second enhanced signal (FGA) in response to the average pixel value (FLF) to obtain a mixed signal (FMA), wherein a contribution of the first enhanced signal (FLC) to the mixed signal (FMA) is lower for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF), and wherein a contribution of the second enhanced signal (FGA) to the mixed signal (FMA) is higher for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF).
14. A display apparatus comprising the image enhancement apparatus as claimed in claim 13, and a display for displaying the mixed signal (FMA).
15. A camera comprising the image enhancement apparatus as claimed in claim 13, and an image sensor for supplying the input image signal (FI).
16. A portable device comprising the display apparatus as claimed in claim 14.
17. A computer program product comprising a processor readable code to enable a processor to execute the method of claim 1, the processor readable code comprising: code for linear contrast enhancing (1) an input image signal (FI) to obtain a first enhanced signal (FLC), code for non-linear contrast enhancing (2) the input image signal (FI) to obtain a second enhanced signal (FGA), code for determining (3) an average pixel value (FLF) for pixels of the input image signal (FI), and - code for mixing (4) the first enhanced signal (FLC) and the second enhanced signal (FGA) in response to the average pixel value (FLF) to obtain a mixed signal (FMA), wherein a contribution of the first enhanced signal (FLC) to the mixed signal (FMA) is lower for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF), and wherein a contribution of the second enhanced signal (FGA) to the mixed signal (FMA) is higher for low and high values of the average pixel value (FLF) than for mid values of the average pixel value (FLF).
18. A computer program product as claimed in claim 17, wherein the computer program product is a software plug-in in an image processing application.
PCT/IB2008/051975 2007-05-21 2008-05-20 Image enhancement WO2008142641A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP07108524 2007-05-21
EP07108524.5 2007-05-21

Publications (2)

Publication Number Publication Date
WO2008142641A2 true WO2008142641A2 (en) 2008-11-27
WO2008142641A3 WO2008142641A3 (en) 2009-05-07

Family

ID=40032247

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2008/051975 WO2008142641A2 (en) 2007-05-21 2008-05-20 Image enhancement

Country Status (1)

Country Link
WO (1) WO2008142641A2 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6766064B1 (en) * 2000-03-10 2004-07-20 General Electric Company Method and apparatus for performing a contrast based dynamic range management algorithm
US20070092136A1 (en) * 2005-10-20 2007-04-26 Sharp Laboratories Of America, Inc. Methods and systems for automatic digital image enhancement

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6766064B1 (en) * 2000-03-10 2004-07-20 General Electric Company Method and apparatus for performing a contrast based dynamic range management algorithm
US20070092136A1 (en) * 2005-10-20 2007-04-26 Sharp Laboratories Of America, Inc. Methods and systems for automatic digital image enhancement

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BURT P J ET AL: "Enhanced image capture through fusion" COMPUTER VISION, 1993. PROCEEDINGS., FOURTH INTERNATIONAL CONFERENCE O N BERLIN, GERMANY 11-14 MAY 1993, LOS ALAMITOS, CA, USA,IEEE COMPUTER SOCI, 11 May 1993 (1993-05-11), pages 173-182, XP010128631 ISBN: 978-0-8186-3870-1 *
CAPRA A ET AL: "Dynamic Range Optimization by Local Contrast Correction and Histogram Image Analysis" CONSUMER ELECTRONICS, 2006. ICCE '06. 2006 DIGEST OF TECHNICAL PAPERS. INTERNATIONAL CONFERENCE ON LAS VEGAS, NV, USA 07-11 JAN. 2006, PISCATAWAY, NJ, USA,IEEE, 7 January 2006 (2006-01-07), pages 309-310, XP010896628 ISBN: 978-0-7803-9459-9 *

Also Published As

Publication number Publication date
WO2008142641A3 (en) 2009-05-07

Similar Documents

Publication Publication Date Title
US8131103B2 (en) Image detail enhancement
US7020332B2 (en) Method and apparatus for enhancing a digital image by applying an inverse histogram-based pixel mapping function to pixels of the digital image
US7433514B2 (en) Tone mapping of high dynamic range images
US8860744B2 (en) System for image enhancement
US9691139B2 (en) Adaptive contrast in image processing and display
Lee et al. A space-variant luminance map based color image enhancement
CN102063703B (en) System for enhancing input image, image display system and method for enhancing image
US8761539B2 (en) System for high ambient image enhancement
US8103120B2 (en) Method and apparatus of local contrast enhancement
KR102317613B1 (en) Systems and methods for localized contrast enhancement
US8284271B2 (en) Chroma noise reduction for cameras
US9214015B2 (en) System for image enhancement
EP2059902B1 (en) Method and apparatus for image enhancement
CN111738966B (en) Image processing method and device, storage medium and terminal
CN112019762B (en) Video processing method and device, storage medium and electronic equipment
JP2004326082A (en) Display controller and display device
JP2004326082A5 (en)
US20040212739A1 (en) Display control device and display device
US20060039622A1 (en) Sharpness enhancement
US8280183B2 (en) Noise robust contrast-enhancement engine employing IIR filters
EP1183408A2 (en) Picture signal enhancement
WO2008142641A2 (en) Image enhancement
JP2006519447A (en) Variable contrast mapping of digital images
Wang et al. Tone-preserving contrast enhancement in images using rational tone mapping and constrained optimization
EP2166509A1 (en) Method and apparatus of local contrast enhancement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08751257

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 08751257

Country of ref document: EP

Kind code of ref document: A2