WO2010007584A1

WO2010007584A1 - Mobile electronic device and method of controlling a display in a mobile electronic device

Info

Publication number: WO2010007584A1
Application number: PCT/IB2009/053053
Authority: WO
Inventors: Peter De Kruif
Original assignee: Nxp B.V.
Priority date: 2008-07-15
Filing date: 2009-07-14
Publication date: 2010-01-21

Abstract

A mobile electronic device is provided which comprises a display unit (LCD) for displaying image data and/or video data; a backlight unit (BL) for backlighting the display unit (LCD); a histogram unit (HU) for receiving YUV data and for generating at least one first Y-histogram, at least one first U-histogram, at least one first V-histogram or a combination UV histogram for the received YUV data, wherein each histogram having at least one bin for each data value and a data value indicates a bin number; a histogram data translation unit (HDU) for translating the first U-histogram and the first V-histogram into a second U-histogram and a second V-histogram; a histogram calculation unit (HCU) for calculating a third histogram based on the first Y-histogram and the second U-histogram and the second V-histogram, a display control unit (DPC) for controlling the operation of the display in dependence on the third histogram; and a backlight control unit (BLC) for controlling the operation of the backlight (BL) in dependence on the third histogram.

Description

Mobile electronic device and method of controlling a display in a mobile electronic device

FIELD OF THE INVENTION

The present invention relates to a mobile electronic device and to a method of controlling a display in a mobile electronic device.

BACKGROUND OF THE INVENTION

Mobile electronic devices typically comprise a display which can for example be implemented as a liquid crystal display LCD. Such a liquid crystal display may comprise a thin layer of liquid crystal LC and a backlight arranged behind the thin layer of liquid crystal LC. The layer of liquid crystal LC can be controlled by means of a control voltage such that the light emitted by the backlight can pass or can be partly or completely blocked. Typically, the layer of liquid crystal LC is divided into several small cells each representing an individual pixel of an image. Each cell can be controlled individually. To create color images, a white backlight can be used while the liquid crystal cells can be covered with red, green or blue color filters. However, it should be noted that the backlight in the liquid crystal display

LCD is typically operated continuously on its maximum brightness, even if the image to be displayed on the liquid crystal display LCD is predominantly dark (thus blocking a great amount of light of the backlight). If the backlight is operated at its maximum brightness, a substantial amount of power will be consumed during the operation. Such a continuous operation is clearly not desirable for mobile or portable electronic devices. One solution to overcome this problem is to modulate the backlight to reduce the overall backlight level to a maximum level occurring in an image. To compensate for the reduced backlight level, the images which are displayed on the liquid crystal display LCD can be adapted in order to increase the transmissiveness of the liquid crystal LC such that the overall light level which is perceivable by the viewer remains the same. Accordingly, both the backlight level and the video gain can be adjusted in such a way that they compensate each other such that the perceived light for the viewer remains the same. Next, by not adjusting the video gain to exactly match the backlight, a contrast improvement can be achieved. It should be noted that the backlight level can even be reduced below the maximum level occurring in the image. However, this can result in a so-called clipping effect for those pixels which have a higher brightness level than the backlight level. Such a clipping effect will reduce the image quality while reducing the power consumption. This may be acceptable for some applications but not for all applications.

Accordingly, to obtain a good trade-off between the image quality and the power reduction, the image content must be analyzed with respect to the level and amount of pixels of bright areas for each color, namely for the red, green and blue cells. Typically, this is performed based on a histogram having several bins, wherein each bin relates to a number of pixels having a specific brightness level with respect to red, green and blue. The required backlight for a maximum allowable amount of clipped pixels can be determined by adding the number of pixels in the histogram bins from the maximum input level downwards until the given maximum number is reached. The bin number found in this way may be named "high threshold". A bin number corresponding to the high threshold will correspond to the minimum level of backlight. In a situation where the maximum number of clipped pixels should not be exceeded, the following higher histogram bin number must be selected as high threshold. Furthermore, a translation from the high threshold to the actual backlight brightness may be required, wherein the non-linear characteristics of the display device must be accounted for. Hence, both the backlight level and the video gain are results from the histogram analysis.

If such a histogram of an image is available, the related information may be used to further enhance the contrast impressions of the image. The liquid crystal control levels of the display are shifted downwards until the minimum display level corresponds to an input signal level which is above zero, i.e. displayed black level. However, it must be ensured that no or just a few pixels contain such a low brightness value. Accordingly, dark pixels with a signal level above the displayed black level can be displayed at darker levels than according to their original properties. Moreover, pixels with signal levels below the displayed black level can be clipped to black. This can be advantageous as a larger difference between bright and dark areas in the image can be created, thus the impression for the viewer may have an enhanced contrast.

However, the contrast enhancement and the number of clipped pixels must be evaluated carefully. The displayed black level can be determined from the histogram by adding the number of pixels in the histogram bins from the minimum input level upwards until the maximum allowed number is reached for a given allowable maximum number of clipped pixels. This bin number, which may be named "low threshold", will correspond to the maximum signal level displayed as black. It may be required that the maximum number of clipped pixels is not exceeded. This can be achieved by selecting the next lower histogram bin number as low threshold. It may be required that a translation from the low threshold to the actual backlight brightness is performed, wherein non-linear characteristics of the display device must be considered. In practice, the high and low thresholds must be limited to avoid extreme values for backlight level and video gain when the input image is e.g. a solid color (without limitation, the result for a black image would be infinite gain with backlight off). It should be noted that the above is relevant for the RGB color domain. However, if the YUV format is used in the electronic device, this format is not directly suitable for histogram analysis to perform a modulation of the backlight. For example: The Y signal is created as a weighted combination of the red, green and blue information with weighting factors of approx. 0,3, 0,59 and 0,11. Accordingly, any histogram which is based on the Y signal does not contain all relevant information with respect to high values of the primary colors red, green and blue. For example, if a histogram has 100% of pixels in a bin at a level of 0, 11, such a situation may correspond to an image contents of solid blue at its maximum level or that an image is completely grey at 11% of the maximum brightness. In the first case, a maximum backlight level is required such that no power reduction can be performed. In the second case, a backlight reduction to 11% can be performed such that a large power reduction is possible.

The image data could be transformed from the YUV domain into the RGB domain such that each pixel is analyzed in the above described way. However, a full 3x3 matrix operation for these pixels would be required. This full 3x3 matrix must be performed by a processor which would require additional processor cycles per pixel which would lead to an additional increase of power consumption. Therefore, the transformation of the image data from the YUV domain to the RGB domain is not suitable for any mobile or portable devices.

SUMMARY OF THE INVENTION

It is therefore an object of the invention to provide a mobile electronic device as well as a method for controlling a display of a mobile electronic device.

This object is solved by a mobile electronic device according to claim 1 and a method according to claim 4.

Therefore, a mobile electronic device is provided which comprises a display unit for displaying image data and/or video data, a display control unit for controlling the operation of the display unit, a backlight unit for backlighting the display unit and a backlight control unit for controlling the operation of the backlight. The mobile electronic device furthermore comprises a histogram unit for receiving YUV data and for generating a first Y- histogram, a first U-histogram and a first V-histogram for the received YUV data. Each histogram comprises at least one bin for each data value and the data value indicates a bin number. The mobile electronic device furthermore comprises a histogram data translation unit for translating the first U- and V-histogram into a second U- and a second V-histogram. Furthermore, a histogram calculation unit is provided for calculating a third histogram based on the first Y-histogram and the second U- and V-histogram. A display control unit is adapted for controlling the operation of the display unit in dependence on the third histogram. A backlight control unit is adapted for controlling the operation of the backlight in dependence on the third histogram.

According to an aspect of the invention, the histogram data translation unit is adapted to translate the bin numbers of the first U- and V-histogram into bin numbers for the second U- and V-histogram and then to divide the content of each source bin over two destination bins.

According to a further aspect of the invention, the histogram calculation unit is adapted to calculate a third bin histogram by selecting bins from the first Y-histogram until a first bin number and by selecting bins from the first Y-histogram and the second U and V histogram from the first bin number onwards.

The invention also relates to a method for controlling a display in a mobile electronic device having a display unit for displaying image data and/or video data and a backlight unit for backlighting the display unit: YUV data is received and at least one first Y- histogram, at least one first U-histogram, at least one first V-histogram or a combination UV histogram is generated from the received YUV data. Each histogram comprises at least one bin for each data value and a data value indicates a bin number. The first U-histogram and the first V-histogram is translated into a second U-histogram and a second V-histogram. A third histogram is calculated based on the first Y-histogram and a second U- and V-histogram. The operation of the display is controlled in dependence on the third histogram. The operation of the backlight is controlled in dependence on the third histogram.

The invention relates to the idea that YUV data is used for the analysis instead of RGB data. Firstly, a first U- and V histogram is translated into a second U- and V histogram. A first Y histogram and second U- and V histograms are translated into a final histogram from which the control values for the video gain and backlight level can be derived in the same way as for the RGB histogram Furthermore, the invention relates to the idea to find high levels of primary colors in an image at low processing power. A histogram analysis for highly saturated colors can be performed for image data in the YUV domain.

Further aspects of the invention are defined in the dependent claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments and advantages of the present invention will now be described with reference to the Figures.

Fig. 1 shows a block diagram of a mobile device according to a first embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 shows a block diagram of a mobile device according to a first embodiment. The mobile device comprises a liquid crystal display LCD, a display control unit DPC for controlling the display LCD, a backlight BL for backlighting the display LCD and a backlight control unit BLC for controlling the backlight BL. The mobile device furthermore comprises a histogram unit HU; a histogram data translation unit HDU, a pixel counter PCU and a histogram calculation unit HCU.

According to the embodiment of the invention, the mobile unit may receive image data based on the YUV domain. Typically, the UV signals are signed and are within a normalized range of -0,5 to +0,5. They can be represented e.g. by an 8-bit binary number: One bit is used for the sign and seven bits are used to cover the range -127 to +127. This range can be used to represent the normalized range of -0,5 to +0,5. If an 8-bit binary number in unsigned format is used, a binary value of 128 can be added to the signed value. With such an 8-bit unsigned format, having a range of 0 to 255, wherein a level 128 represents zero, values lower than 128 correspond to negative values while values higher than 128 correspond to positive values. If the color saturation of a pixel is to be determined, the U and V values can be combined. This can for example be performed by a square root of the sum of squares of U and V. It should be noted that highly saturated single colors (full red, full green or full blue) can have a relative low value of luminescence (Y-signal) but a high saturation value. Accordingly, if a pixel has the combination of a low luminescence value with a high saturation value, this may indicate a high single color brightness value for that pixel. Moreover, the absolute values of U and V may also serve as an indication of the saturation of pixels. The sign of the U and V values is related to the hue of the color but having no impact on the saturation. It should be noted that the analysis of the luminescence and saturation based on the U and V values is more power efficient than a RGB analysis but is less accurate than the RGB analysis.

In the above, an 8-bit resolution is used for the UV signals but also other resolutions are possible.

The histogram unit HU is used to receive YUV data as input data IN and to create a histogram for the Y, U and V data, respectively. In each histogram, a bin is used for each data value, i.e. for 8-bit image data. The histogram has 256 bins with an index ranging from 0 to 255. The value of the Y, U and V data, in unsigned format as described before, can be directly used as an index for the bin number in the histogram. It should be noted that the analysis can also be applied to any sub-set of the image pixels in order to further reduce the power usage. Using a sub-set of image pixels is known as sub-sampling. Each pixel that is used for the analysis will lead to an increment in the bins of the respective histogram relating to the Y, U and V value of the pixels. The pixel counter PCU is used to count the total number of pixels which are used during the analysis. This number of pixels can be referred to as variable pixel count. During the analysis, only four increments on mutually independent variables are required for each pixel. The mutual independent variables relate to the separate histograms for the Y, U and V data and the pixel count. These simple increments can be advantageously performed very process cycle efficient in an electronic device which enables a parallel execution of instructions.

Although in the above, the histogram has been described with 256 bins, it should be noted that the histogram may also comprise less bins. The pixel counter PCU will contain the number of pixels used in the analysis. Alternatively, at the end of a frame, this value can be determined by adding all contents of the histogram bins. In that way, only three increments per pixel are required and only 255 additions will be needed to obtain an accumulation of the bin contents which is the same as the total pixel count.

The histogram data translation unit HDU is used to perform a translation of the histograms which are determined by the histogram unit HU. The histogram data translation unit HDU transforms the Y-, U- and V-histograms. A new histogram is created by the histogram calculation unit. The new histogram comprises 256 bins which represent values in the range of 0 to 255. The new histogram can be used as basis for the modulation of the backlight and the image data. Hence, based on the new histogram, the contrast impression of an image can be enhanced and the backlight power usage can be reduced. In the following, the operation of the histogram data translation unit HDU is described in more detail. It should be noted that for the U and V histogram, bin numbers 128 to 255 represent values between 0 and 0,5 in the normalized range and bin numbers 0 to 128 represent negative values from -0.5 to 0 in the normalized range. The bin numbers 128 to 255 can be translated to the new histogram using the next formula and the content of each bin is thus divided over two new bins, namely one bin corresponding to the index of the same signal level and a next higher bin. newUVhisto[i] = 0.5 * oldUVhisto[(256 + i)/2], with 0 <= i <= 255 and (256 + i)/2 is truncated to integer value. The bin numbers 0 to 128 can be translated into bin numbers of the new histogram using a different formula and the contents of each bin is again divided over two bins in the new histogram: newUVhisto[i] = 0.5 * oldUVhisto[(255 - i)/2], with 0 <= i <= 255 and (255 - i)/2 is truncated to integer value. Here, it should be noted that the range of the bin index is different from the previous formula in order to avoid that the index number 128 is used twice and to correctly include the source bin with the index 0.

The histogram calculation unit HCU is used to calculate a final histogram. The bins in the translated U and V histograms with a lower index can comprise relative high numbers as typically video images comprise a relative high number of pixels with low saturated colors. It should be noted that these bins are not suitable for analyzing the image data to perform a contrast enhancement. On the other hand, the Y-histogram can be ambiguous but the lower part of the Y-histogram may still be used to control the contrast enhancement in an effective and satisfactory way. Accordingly, the translated versions of the U and V histograms may be used to identify highly saturated primary colors in an image.

Hence, a final histogram is obtained from the original Y-histogram and the translated U- and V-histogram. The lower bins from the Y-histogram are used for the lower bins of the final histogram and the maximum values of the Y-, U- and V-histograms are used for the higher bins. The point when to use the Y-histogram and the translated U- and V- histogram can be referred to as switching point binSwitch. This switching point can be adjusted in order to optimize the results. The final histogram can be calculated based on the following equation:

FinalHistogram[i] = Y-histogram[i], for 0 <= i <= binSwitch, and

FinalHistogram[i] = MAX{Y-histogram[i], newUhisto[i], newVhisto[i]}, for (binSwitch+1) <= i <= 255.

It should be noted that the sum of the contents of all bins of the final histogram will not correspond to the number of pixels used for the analysis. This is due to the fact that the relation between the Y, U and V values of a single pixel is lost in the histogram. In order to determine thresholds of the final histogram, the original number of pixels used for the analysis can be used. Alternatively, a new sum of content of all bins can be calculated separately which requires an iteration over the number of histogram bins. This is however advantageous as the pixel count during the analysis can then be skipped.

It should be noted that the final histogram has an undefined format as it contains data of two different signal types, namely Y representing the luminescence and UV representing color saturation.

It should be noted that the low threshold can be derived as described above, based on user-defined numbers representing the amount of pixels allowed to be clipped, with the limitation that the data of the Y-histogram does not reliably represent low brightness properties of an image but the result is still very effective and acceptable.

The display control unit DPC serves for controlling the operation of the display in dependence on the output of the histogram calculation unit HCU. The backlight control unit BLC serves for controlling the operation of the backlight in dependence on the output of the histogram calculation unit HCU.

The high threshold can be derived as described above. This can be performed based on user-defined numbers representing the amount of pixels allowed to be clipped. This can be expressed as a fraction of the total amount of pixels. The above-described scheme is advantageous as both an increase of image contrast and a power reduction can be obtained by using only a very limited amount of processing power. For a given data format, i.e. 8 bits, every histogram will hold a maximum of 256 bins. If an image is in QVGA format, the number of pixels is 360 x 240 = 86400. If five processor cycles are used to process each histogram bin, a total of 1280 cycles per frame is required while only one extra cycle per pixel in the QVGA image will result in 86400 cycles per frame.

The above-described scheme can be used in an electronic device which comprises a display and a backlight. However, the above-described scheme is advantageous for mobile or portable media devices having a display. Such devices may include mobile phones, laptop computers, PDA, portable media recorders and players, photo/video cameras, etc.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Furthermore, any reference signs in the claims shall not be constrained as limiting the scope of the claims.

Claims

CLAIMS:

1. Mobile electronic device, comprising: a display unit (LCD) for displaying image data and/or video data; a backlight unit (BL) for backlighting the display unit (LCD); a histogram unit (HU) for receiving YUV data and for generating at least one first Y-histogram, at least one first U-histogram, at least one first V-histogram or a combination UV histogram for the received YUV data, wherein each histogram having at least one bin for each data value and a data value indicates a bin number; a histogram data translation unit (HDU) for translating the first U-histogram and the first V-histogram into a second U-histogram and a second V-histogram; - a histogram calculation unit (HCU) for calculating a third histogram based on the first Y-histogram and the second U-histogram and the second V-histogram, a display control unit (DPC) for controlling the operation of the display in dependence on the third histogram; and a backlight control unit (BLC) for controlling the operation of the backlight (BL) in dependence on the third histogram.

2. Mobile electronic device according to claim 1, wherein the histogram data translation unit (HDU) is adapted to translate the bin numbers of the first U- and V-histogram into bin numbers for the second U- and V-histogram and to divide the content of each bin over two new bins.

3. Mobile electronic device according to claim 1 or 2, wherein the histogram calculation unit (HCU) is adapted to calculate the third histogram by selecting bins from the first Y-histogram until a first bin number and by selecting bins from the first Y-histogram and the second U- and V-histogram from the first bin number onwards.

4. Method for controlling a display in a mobile electronic device having a display unit for displaying image data and/or video data and a backlight unit for backlighting the display unit (LCD), comprising the steps of: receiving YUV data and generating at least one first Y-histogram, at least one first U-histogram, at least one first V-histogram or a combination UV histogram from the received YUV data, wherein each histogram comprises at least one bin for each data value and a data value indicates a bin number, - translating the first U-histogram and the first V-histogram into a second U- histogram and a second V-histogram, calculating a third histogram based on the first Y-histogram and a second U- and V-histogram; controlling the operation of the display in dependence on the third histogram; and controlling the operation of the backlight in dependence on the third histogram.