WO2010020949A1

WO2010020949A1 - Video processing with spatially-based backlight control

Info

Publication number: WO2010020949A1
Application number: PCT/IB2009/053649
Authority: WO
Inventors: Erwin Ben Bellers
Original assignee: Nxp B.V.
Priority date: 2008-08-19
Filing date: 2009-08-19
Publication date: 2010-02-25

Abstract

Video data is processed for a display having a plurality of segments, where the video data for each segment has a corresponding backlight drive level for controlling backlight for the display. The backlight drive level of a target is set based upon video data for the target segment, the backlight drive level of a neighboring one of the segments, and a spatially-based intensity characteristic of video data for the neighboring one of the segments.

Description

VIDEO PROCESSING WITH SPATIALLY-BASED BACKLIGHT CONTROL

FIELD OF INVENTION

This patent document relates to processing video data for a display having a plurality of segments, and more particularly, to determining the backlight drive levels for segments of video data that are to be displayed on the segmented display.

BACKGROUND Video displays are used in a variety of applications including television, computers, video games, hand- held devices and others. As video technology has improved, so has the demand for high-definition video and video displays, for presenting images having a variety of different characteristics. For displaying video, this has often involved the use and display of images that include relatively fast-moving objects, such as for sporting events, video games or action movies. For a variety of video displays, one or more illumination sources are used to illuminate pixels or other regions of an image to be displayed. When displaying an image (or series of images), respective pixels or regions of the display are illuminated according to a brightness of the portion of an image to be displayed at the pixel or region. In this context, neighboring pixels or regions of a particular image may have relatively different brightness levels, depending upon the nature of the image being displayed.

In many applications, displaying images having different levels of brightness or contrast has been challenging. When moving objects are displayed, adjacent image portions exhibiting differing levels of brightness are often susceptible to blurring, flicker or artefacts. For instance, liquid-crystal displays (LCDs) use backlights to display an image representative of video data. Modulating the backlight to generate appropriate illumination for fast-moving objects in subsequently-displayed images can be challenging, as otherwise spatio-temporal inconsistencies become visible in the form of flicker, loss of contrast, clipping, and other undesirable characteristics. These and other issues have continued to present challenges to displaying images, and particularly to displaying images of varying brightness for fast-moving scenes. SUMMARY

The present invention is directed to overcoming the above-mentioned challenges and others related to the types of applications discussed above and in other applications. These and other aspects of the present invention are exemplified in a number of illustrated implementations and applications, some of which are shown in the figures and characterized in the claims section that follows.

According to an example embodiment of the present invention, video data is processed for a display having a plurality of segments, where video data for each segment has a corresponding backlight drive level for controlling backlight for the display. The backlight drive level of one of the segments is set using video data for the one of the segments, the backlight drive level of a neighboring one of the segments, and a spatially-based intensity characteristic of video data for the neighboring one of the segments. This method and/or system-based approach can be implemented with a variety of video displays, such as with a controller for an LCD monitor or television. According to another example embodiment of the present invention, a method is provided for processing video data for a display. The video data is divided into a plurality of segments. The method includes storing a spatial position of a brightest region in each of the segments and calculating a backlight drive level for each of the segments. The method further includes adjusting the backlight drive level for each of the segments based on a weighted mix of the backlight drive levels of neighboring ones of the segments, the backlight drive level of each of the neighboring segments being weighted based on the stored spatial position of the brightest region in the neighboring segment, relative to the spatial position of the segment for which the calculated backlight drive level is adjusted. The above summary is not intended to describe each illustrated embodiment or every implementation of the present invention. The figures and detailed description that follow more particularly exemplify these embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

The invention may be more completely understood in consideration of the following detailed description of various embodiments of the invention in connection with the accompanying drawings, in which: FIG. 1 shows a process for calculating the settings for backlight drive levels, according to an example embodiment of the present invention;

FIG. 2 shows a flow chart of a process for calculating backlight drive levels, according to another example embodiment of the present invention;

FIG. 3 shows a segmented display and an example of weighting given to the backlight drive levels, according to an example embodiment of the present invention; and

FIG. 4 shows a block diagram of a system for processing video data for a display, according to an example embodiment of the present invention.

While the invention is amenable to various modifications and alternative forms, specifics thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit the invention to the particular embodiments described. On the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the scope of the invention, including that defined by the claims.

DETAILED DESCRIPTION

The present invention is applicable to a variety of video devices, circuits and methods of processing video data, and is particularly applicable to controlling light for displaying video. While the present invention is not necessarily limited to such applications, an appreciation of various aspects of the invention may be gained through a discussion of examples in such an environment.

According to an example embodiment of the present invention, video data is processed for displaying an image having a plurality of segments, each segment having a corresponding backlight drive level. The backlight drive level of each of the segments is controlled using video data for that segment, the backlight drive level of neighboring ones of the segments, and a spatial-based intensity characteristic of video data in the neighboring segments.

In some implementations, the backlight drive level is a drive level that is used to drive a backlight for a video display, such as a backlight for a liquid-crystal display (LCD). In one implementation, the backlight drive levels are histogram-based. One or more light sources may be used to illuminate one or more segments of a particular image, and corresponding video-series of images, according to the determined backlight drive levels. In one particular implementation, the display is a LCD panel having a segmented light-emitting diode (LED) backlight, operated using the above backlight-based approach to control the contrast ratio of an image displayed on the LCD panel. For example, in completely dark image parts (e.g., segments), the backlight can be switched off locally, whereas in image parts with highlights, the local backlight drive value can be boosted. The segmented LED generates light that contributes to many pixels within the segment, and it also contributes to many pixels (or most) of neighboring segments as well. As a result, a change in the local backlight drive value for a segment affects every pixel within the picture region affected by the backlight.

The drive values for the segmented LED backlight are set according to the drive levels of neighboring regions, and further according to the spatial position of a brightest portion (e.g., pixel) of the respective neighboring regions. For instance, when computing a drive level for a particular image region, a neighboring region having a brightest portion thereof immediately adjacent to the particular region is given higher weight, relative to a neighboring region having a brightest portion thereof further away from the particular region.

Various embodiments of the present invention are directed to controlling backlight drive values to compensate for undesirable image characteristics when moving objects are displayed. For example, a moving object with a bright-level different from its neighborhood will affect the drive level for the segment in which it is located, and to lesser extent the drive levels for neighboring segments, which can result in a flickering behavior that is noticeable in certain sequences. In certain embodiments, the backlight drive level of the segments is adjusted, based on characteristics of neighboring segments of the display, to compensate for these effects. The backlight drive level of each segment of the display is stored, as well as information concerning the spatial location of the brightest pixel and its intensity in each segment of the display. The backlight drive level of a current (or target) segment is adjusted by applying a weighted filtering of the backlight drive values of neighboring segments of the display. The closer the brightest pixel of a neighboring segment is to the current segment, the larger the contribution of the backlight drive level of that segment is towards the drive level of the current segment. Using this approach, resulting video data can be produced in a manner that mitigates flicker.

In some applications, the adjustment of backlight drive levels is carried out to affect a perceived intensity in adjacent or nearby segments. For instance, where a target image segment is much brighter than all immediately adjacent segments, the drive level of the target image segment can be reduced so as not to adversely affect perceived brightness in neighboring segments. Similarly, where a target image segment is desirably brighter than can be achieved with a particular backlight, the intensities of one or more neighboring segments may be increased to affect a perceived higher or brighter intensity in the target segment. Other approaches are used to increase or decrease perceived brightness, by controlling the brightness of a particular image segment in a manner that affects a perceived brightness in neighboring segments.

Turning now to the figures, Figure 1 illustrates a process for calculating the settings for the backlight drive levels for segments of video data that are to be displayed on a display having a corresponding segmented backlight, according to another example embodiment of the present invention. Video data 102 for the display is received from a video source, such as a video playback device, or a satellite, cable or broadcast source. A histogram for each segment of the video data 102 is built in block 104, and collects data (e.g., on MAX(R, G, B)) for each segment. The initial histogram-based backlight drive level for each of the segments is then calculated in block 106. The spatial position and intensity of the brightest or most relevant pixel (or area) within each segment of the video data 102 is stored in block 108. The histogram-based backlight drive levels for each of the segments are then determined in block 110. The histogram-based backlight drive level of a target segment is adapted based on a weighted mix of the histogram- based backlight drive levels of the neighboring segments of the video data 102. The weight of each of the histogram-based backlight drive levels of the neighboring segments is determined based on the distance of the stored spatial location of a given neighboring segment with respect to the current target segment.

In one implementation, the backlight drive levels for the segments are determined using an iterative process. The backlight drive levels determined for the segment are used as part of the weighted mix of the backlight drive levels when calculating the backlight drive level of the current target segment. The backlight drive level determined for the current target segment is then used as part of the weighted mix of the backlight drive levels when calculating the backlight drive level of the next target segment (and, as appropriate, additional target segments adjacent to the aforesaid current target segment). In many applications, the process of determining the backlight drive level for each of the segments is repeated at least twice to account for the adjustments made to backlight drive levels in different segments. Crosstalk compensation is applied in block 112 to eliminate crosstalk caused by variations in the video data 102 used to drive the segments of the display. The determined histogram-based backlight drive levels 114 are then used to drive the backlight segments of the display, resulting in the video data 102 being displayed on the display.

Figure 2 illustrates a process for calculating the backlight drive levels for segments of a display, according to another example embodiment of the present invention. The display is divided into a plurality of segments that are each driven by a corresponding backlight segment, with the backlight drive level for each of the segments being controlled based upon the backlight drive levels of neighboring ones of the segments, and further upon the distance of a brightest portion of the neighboring ones of the segments.

A target segment of the display is selected in step 202, and the backlight drive level of the target segment is adjusted based on the backlight drive levels of segments that neighbor the target segment. The segments that neighbor the target segment are determined in step 204, and the backlight drive levels of the neighboring segments are weighted based on the spatial position and intensity of the brightest or most relevant pixel (or area) within each of the neighboring segments relative to the target segment. In step 206, one of the identified neighboring segments is selected, and the backlight drive level of the selected neighboring segment is then weighted responsive to the location and intensity of the brightest pixel or region in the selected neighboring segment with respect to the target segment in step 208. It is next determined in step 210 if all the backlight drive levels of the neighboring segments of the target segment have been weighted. If all of the backlight drive levels of the neighboring segments have not been weighted, then one of the remaining neighboring segments is selected in step 206 and its weighted backlight drive level is determined in step 208. This selection and weighting of neighboring segments thus continues until all neighboring segments have been weighted.

When all of the backlight drive levels of the neighboring segments of the target segment have been weighted, the process proceeds to step 212, in which the backlight drive level of the target segment is calculated. The backlight drive level of the target segment is determined by adjusting the backlight drive level of the target segment based on the weighted backlight drive levels of the neighboring segments that were determined in step 208. The process then proceeds to step 214 where it is determined whether the backlight drive level for each segment of the display has been calculated. If the backlight drive levels for all segments of the display have not been determined, then the process returns to step 202 where one of the remaining segments is selected as the target segment. Steps 204-212 are then repeated for the new target segment in order to determine the backlight drive levels for this segment. If the previous target segment is a neighboring segment of the new target segment, then the determined backlight drive level for previous target segment is used as part of the weighted mix of the backlight drive levels for determining the backlight drive level of the new target segment. In other words, the backlight drive level that is determined in step 212 is used to adjust the backlight drive levels of other segments of the display. When the backlight drive levels of all segments of the display have been determined, the process proceeds to step 216 where it is determined whether the process should be repeated. If the backlight drive levels are to be further adjusted based upon the newly determined backlight drive levels, then the process returns to step 202 where a target segment is selected. The backlight drive level of each segment is then further adjusted via steps 202-214 as discussed above. When the process has been repeated a desired number of times, the determined backlight drive levels for the segments of the display are provided in step 218, and can be used in processing and/or presenting video for display, with corresponding backlight drive levels.

In one implementation, the backlight drive level of the target segment is increased in step 212 based on the weighted backlight drive levels of the neighboring segments. For example, the backlight drive level of the target segment could be increased in order to increase the brightness of that region of the display to compensate for the inability of a neighboring segment to display the desired brightness.

In another implementation, the backlight drive level of the target segment is decreased in step 212 based on the weighted backlight drive levels of the neighboring segments. For example, the backlight drive level of the target segment could be decreased in order to increase the contrast between the target segment and a neighboring segment when the neighboring segment is capable of displaying the desired brightness.

Figure 3 illustrates a segmented display and an example of weighting given to the backlight drive levels of neighboring segments of the display, according to an example embodiment of the present invention. The following discussion is exemplary both in terms of Figure 3 and the selected example values to be assigned as weighting values. The display 300 is divided into a plurality of segments 302-318 and the display has a corresponding segmented backlight that is provided with backlight drive levels for each of the segments. The backlight drive level of each of the segments is adjusted based on a weighted mix of the backlight drive levels of neighboring ones of the segments. The backlight drive levels of the neighboring segments are weighted based on the spatial position and intensity of the brightest pixel within each of the neighboring segments relative to the target segment.

As an example, segment 310 is selected as the target segment, with segments 302-308 and 312-318 neighboring segment 310. If a different one of the segments 302- 318 is selected as the target segment, its respective neighboring segments are different. For example, if segment 312 is selected as the target segment, then the neighboring segments would be segments 304, 306, 310, 316 and 318.

The backlight drive levels of neighboring segments 302-308 and 312-318 are weighted responsive to the location of the brightest pixel within each of segments 302- 308 and 312-318 with respect to the target segment 310. The closer the location of the brightest pixel to the target segment 310, the more weight that is given to the drive level of the neighboring segment. As shown in Figure 3, the neighboring segments 302-308 and 312-318 each contain a dot that represents the brightest pixel or area in that neighboring segment. The larger the dot in the neighboring segment, the greater the intensity of the brightest pixel relative to the brightest pixels in other neighboring segments. The backlight drive level of neighboring segment 308 is weighted with a value of 0.50 to provide an example of how the difference in location relative to the target segment 310 in a given neighboring segment affects the weight given to its backlight drive level.

The brightest pixel of neighboring segment 312 has about the same intensity as the brightest pixel of neighboring segment 308; however, the brightest pixel of neighboring segment 312 is located farther away from the target segment 310 than the brightest pixel of neighboring segment 308. As such, the backlight drive level of neighboring segment 312 is weighted with a value of 0.40, which is less than the weight given to the backlight drive level of neighboring segment 308. In this regard, while the segments 312 and 308 have similar brightest-pixel intensities, their respective contribution to the target segment 310 is weighted higher for segment 308 because the corresponding "brightest" portion of that segment is closer to the target segment. In another example, the brightest pixel of neighboring segment 304 is located the same distance from the target segment 310 as the brightest pixel of neighboring segment 308, but the intensity of the brightest pixel of neighboring segment 304 is greater than the intensity of the brightest pixel of neighboring segment 308. Thus, the backlight drive level of neighboring segment 304 affects the target segment 310 more than the backlight drive level of neighboring segment 308 due to differences in intensity, though weighting is the same for each. The backlight drive levels of the remaining neighboring segments 302, 306, 314, 316, 318 are weighted accordingly. For example, the backlight drive level of neighboring segment 306 will have greater effect upon the target segment 310, relative to the effect of neighboring segment 308, even though the brightest pixel of segment 306 is located farther from the target segment 310 than that of neighboring segment 308.

Figure 4 illustrates a block diagram of a system 400 for processing video data for a display 402 having a plurality of segments, according to an example embodiment of the present invention. The system 400 includes a processor 416 that processes video for the display 402 and may, for example, be included with the display itself. In general, the video data for each segment of the display 402 has a corresponding backlight drive level that is used for controlling a backlight in the display, and the processor 416 facilitates this backlight level control. A processor-readable device 404 is configured with software modules 406, 408,

410, 412 and 414, to facilitate backlight control. Execution of the instructions of software modules 406, 408, 410, 412 and 414 by processor 416 causes the processor to determine the backlight drive levels 418 for the segments of display 402. In one implementation, the backlight drive levels 418 are stored on the processor readable device 404.

Execution of the instructions of software module 406 causes the processor 416 to determine and store the initial backlight drive levels 420 for the video data that is to be displayed on display 402. The initial backlight drive levels 420 can be stored on the processor readable device 404. Execution of the instructions of software module 408 causes the processor 416 to determine and store the location and intensity of the brightest (or most relevant) pixel (or area) of each segment of the video data. The location and intensity of the brightest pixel data 422 can also be stored on the processor readable device 404. Execution of the instructions of software module 410 causes the processor 416 to determine the weighting given to the backlight drive levels of neighboring segments of the display 402 relative to a target segment of the display 402. The backlight drive levels of the neighboring segments are weighted using the location and intensity of the brightest pixel data 422 of the neighboring segments. Execution of the instructions of software module 412 causes the processor 416 to determine the backlight drive level of the target segment based on the weighted backlight drive levels of the neighboring segments.

The initial backlight drive levels 420 are used as the backlight drive levels for the neighboring segments by the processor 416 and software module 410 until a new backlight drive level is determined for a given target segment. The new backlight drive level is then used in the determination of the backlight drive level for each subsequent target segment in order to account for the change in the backlight drive level of the given target segment. The processor 416 and software modules 410 and 412 determine the backlight drive levels for each segment of the display 402. Execution of the instructions of software module 414 causes processor 416 to output the backlight drive levels 418 to the display 402, which displays the video data.

In addition to the above, the various processing approaches described herein can be implemented using a variety of devices and methods including general purpose processors implementing specialized software, digital signal processors, programmable logic arrays, discrete logic components and fully-programmable and semi-programmable circuits such as PLAs (programmable logic arrays). For example, the above algorithms are executed on a microcomputer (a.k.a. microprocessor) in connection with certain embodiments, and may be implemented as part of one or more of the devices shown in the figures.

The various embodiments described above and shown in the figures are provided by way of illustration only and should not be construed to limit the invention. Based on the above discussion and illustrations, those skilled in the art will readily recognize that various modifications and changes may be made to the present invention without strictly following the exemplary embodiments and applications illustrated and described herein. For example, various types of video data from various sources can be processed as described. This processing may involve outputting information that is directly used to control backlight drive levels for a display (e.g., with a circuit in the display carrying out respective functions ), or outputting information that is used later as part of video playback. In addition, drive level values may be calculated using two or more iterative steps to arrive at a final drive level for each segment. Furthermore, the video segments as described may include one or more pixels, or one or more regions of an image to be displayed. Such modifications and changes do not depart from the true scope of the present invention.

Claims

What is Claimed is:

1. A method for processing video data for a display having a plurality of segments, the video data for each segment having a corresponding backlight drive level for controlling backlight for the display, the method comprising: setting the backlight drive level of a target segment based upon video data for the target segment, the backlight drive level of a neighboring one of the segments, and a spatially-based intensity characteristic of video data for the neighboring one of the segments.

2. The method of claim 1, wherein the backlight drive level of the neighboring segment is weighted based on a spatial position of a brightest region in the neighboring segment relative to the target segment.

3. The method of claim 1, further comprising setting the backlight drive level of the target segment based upon the backlight drive levels of all neighboring ones of the segments adjacent the target segment.

4. The method of claim 3, wherein the backlight drive level of each of the neighboring segments is weighted based on a spatial position of a brightest portion of a given one of the neighboring segments relative to the target segment.

5. The method of claim 1, further comprising setting the backlight drive level of each of the segments using, for each segment, video data for the segment, the backlight drive level of neighboring ones of the segments, and a spatially-based intensity characteristic of video data for the neighboring ones of the segments.

6. The method of claim 1, further comprising setting the backlight drive level of another target segment based upon video data for the other target segment, the set backlight drive level of the target segment, and a spatially-based intensity characteristic of video data for the target segment.

7. The method of claim 1, wherein setting the backlight drive level of the target segment includes adjusting an initial backlight drive level of the target segment.

8. The method of claim 1, wherein setting the backlight drive level of the target segment includes decreasing previously-set backlight drive level of the target segment in response to neighboring ones of the segments having a relatively high backlight drive level.

9. The method of claim 1, wherein setting the backlight drive level of the target segment includes increasing a backlight drive level of the target segment.

10. The method of claim 1, wherein setting the backlight drive level of the target segment includes adjusting a previously-set backlight drive level of the target segment to effect a perceived brightness level of a neighboring one of the segments.

11. The method of claim 1 , wherein the backlight drive levels for the segments are histogram-based backlight drive levels.

12. A method for processing video data for a display, the video data including data for each of a plurality of segments of the display, the method comprising: storing a spatial position of a brightest region in each of the segments; calculating a backlight drive level for each of the segments; adjusting the calculated backlight drive level for each of the segments based on a weighted mix of the backlight drive levels of neighboring ones of the segments, the backlight drive level for each of the neighboring segments being weighted based on the stored spatial position of the brightest region in the neighboring segment, relative to the spatial position of the segment for which the calculated backlight drive level is adjusted.

13. The method of claim 12, wherein adjusting the calculated backlight drive level for each of the segments includes iteratively adjusting the backlight drive level for a target one of the segments based on previously adjusted backlight drive levels for neighboring ones of the segments.

14. The method of claim 12, further comprising adjusting the calculated backlight drive level for a target one of the segments based upon video data for the target segment, a previously adjusted backlight drive level for another one of the segments, and the stored spatial position of the brightest region in the other segment.

15. The method of claim 12, wherein adjusting the calculated backlight drive level includes setting a histogram-based backlight drive level.

16. The method of claim 12, wherein storing the spatial position of the brightest region in each of the segments includes storing a spatial position of a brightest pixel in each of the segments, and the backlight drive level for each of the neighboring segments is weighted based on the stored spatial position of the brightest pixel in the neighboring segment, relative to the spatial position of the segment for which the calculated backlight drive level is adjusted.

17. The method of claim 12, wherein adjusting the calculated backlight drive level for each of the segments includes adjusting the calculated backlight drive level of at least one of the segments to effect a perceived brightness of a neighboring one of the segments.

18. The method of claim 12, wherein adjusting the calculated backlight drive level for each of the segments includes adjusting the calculated backlight drive level of at least one of the segments to a drive level that is different than a target drive level for the at least one of the segments, in response to the calculated drive level of neighboring ones of the segments that contribute to a perceived brightness of the at least one of the segments.

19. A system for processing video data, the system comprising: a video data processing circuit for providing video to a display having a plurality of segments, the video data for each segment having a corresponding backlight drive level for controlling backlight for the display, the data processing circuit being programmed to set the backlight drive level of a target segment based upon video data for the target segment, the backlight drive level of a neighboring one of the segments, and a spatially-based intensity characteristic of video data for the neighboring one of the segments.

20. The system of claim 19, further including a video display having a segmented backlight and a backlight controller to use the set backlight drive level to control the intensity of each segment in the segmented backlight.