WO2007113740A1

WO2007113740A1 - Ambient lighting filter control

Info

Publication number: WO2007113740A1
Application number: PCT/IB2007/051077
Authority: WO
Inventors: Mauro Barbieri; Srinivas Gutta
Original assignee: Koninklijke Philips Electronics, N.V.; U.S. Philips Corporation
Priority date: 2006-03-31
Filing date: 2007-03-27
Publication date: 2007-10-11

Abstract

In controlling an ambient lighting effect, an abrupt transition in rendered content is detected, a low-pass filter operation is applied to modify an ambient lighting signal associated with the rendered content, and the modified ambient lighting signal is rendered to avoid an abrupt transition in an ambient lighting effect. In one embodiment, detecting may include detecting a mode of operation of a host device to determine if the mode will produce the abrupt transition and/or analyzing the rendered content to identify the abrupt transition.

Description

AMBIENT LIGHTING FILTER CONTROL

This application claims the benefit of U.S. Provisional Patent Application No. 60/788,470, filed March 31, 2006.

The present system relates to ambient lighting effects that accompany events of a host system.

Koninklijke Philips Electronics N.V. (Philips) and other companies have disclosed means for changing ambient or peripheral lighting to enhance video content for typical home or business applications. Ambient lighting modulated by video content that is provided together with a video display or television has been shown to reduce viewer fatigue and improve realism and depth of experience. Currently, Philips has a line of televisions, including flat panel televisions with ambient lighting, where a frame around the television includes ambient light sources that project ambient light on the back wall that supports or is near the television. Further, light sources separate from the television may also be modulated relative to the video content to produce remote ambient lighting that may be similarly controlled.

In a case of a single color light source, modulation of the light source may only be a modulation of the brightness of the light source. A light source capable of producing multi-color light provides an opportunity to modulate many aspects of the multi-color light source based on rendered video content including a wide selectable color range per light source. Ambient lighting control data for controlling ambient lighting effects may be produced based on real-time analysis of content that is or will be rendered or may be

provided as data similar to the content that is being rendered. For example, ambient lighting data may be provided as a track stored on a Digital Versatile Disk (DVD) or it may be provided from a remote source, such as a server accessible over a connection (e.g., an Internet connection). Regardless of how the ambient lighting data is provided, it typically is intended to be rendered during rendering of corresponding content to in effect, complement the rendering of the content. However, during rapid transitions in the rendered content, present ambient lighting solutions may produce a visually disturbing change in the rendered ambient lighting in an attempt to track the rapid changes in the content .

It is an object of the present system to overcome disadvantages in the prior art and/or to provide an ambient lighting effect that is less visually disturbing during rapid transitions in corresponding rendered content.

The present system provides a method, program and device for controlling an ambient lighting effect. In accordance with an embodiment, an abrupt transition in rendered content is detected, a low-pass filter operation is applied to modify an ambient lighting signal associated with the rendered content, and the modified ambient lighting signal is rendered to avoid an abrupt transition in an ambient lighting effect. In one embodiment, detecting may include detecting a mode of operation of a host device to determine if the mode will produce the abrupt transition and/or analyzing the rendered content to identify the abrupt transition. The low-pass filter operation may be applied if a given transition in the rendered content and/or in the ambient lighting signal

exceeds a threshold, such as a threshold in a number of transitions within a period of time.

The present system is explained in further detail, and by way of example, with reference to the accompanying drawings wherein:

FIG. 1 shows an a flow diagram in accordance with an embodiment of the present system;

FIG. 2 shows an a flow diagram in accordance with an embodiment of the present system; and

FIG. 3 shows a device in accordance with an embodiment of the present system. The following are descriptions of illustrative embodiments that when taken in conjunction with the following drawings will demonstrate the above noted features and advantages, as well as further ones. In the following description, for purposes of explanation rather than limitation, specific details are set forth such as the particular architecture, interfaces, techniques, etc., for illustration. However, it will be apparent to those of ordinary skill in the art that other embodiments that depart from these specific details would still be understood to be within the scope of the appended claims. Moreover, for the purpose of clarity, detailed descriptions of well-known devices, circuits, and methods are omitted so as not to obscure the description of the present system.

It should be expressly understood that the drawings are included for illustrative purposes and do not represent the scope of the present system.

The present system operates to smooth out a response of an ambient lighting effect that may occur as a result of an

abrupt transition in a quality of rendered content without a need to switch off the ambient lighting effect during these times. As utilized herein, an abrupt transition may be an immediate transition of a content characteristic from one temporal portion to another temporal portion. For example, an abrupt transition in a rendered video sequence, such as caused by a scene change within the sequence, may result in a smooth ambient lighting effect in accordance with the present system. The term abrupt transition as utilized herein is also intended to include a series of transitions within a relatively small period of time. Other types of content may be similarly rendered together with rendering of the ambient lighting data for modulating one or more ambient lighting devices, including audio content, audio/visual content, image content (e.g., such as browsing photos), and other content that may be rendered and consumed. The term rendering as utilized herein refers to providing the content such that it may be perceived by at least one of the five senses, including a sense of sight and a sense of hearing. Receipt by the user of the rendered content is termed consumption. For example, consumption may include a user watching a movie, listening to music, browsing photos, etc.

FIG. 1 shows a flow diagram 100 in accordance with an embodiment of the present system. During act 110, the process begins which may occur as a result of activation of a content rendering device, herein also termed a host device, in that the host device hosts content for rendering. As content is rendered, the present system provides an ambient lighting signal for rendering an ambient lighting effect that corresponds to the rendered content as would be readily

appreciated by a person of ordinary skill in the art. The ambient lighting signal may be produced by analyzing the content directly, such as in real-time as the content is rendered, or in advance of the rendering of the content. In alternate embodiments, the ambient lighting signal may be stored together with the content or may be simply associated with the content. For example, the ambient lighting signal may be received by the system from a source other than the source of the content. In any event, the ambient lighting signal is utilized to drive ambient lighting elements.

During act 120, the present system makes a determination whether a mode of rendering content is selected wherein abrupt transitions in the content rendering are likely. These modes may be predetermined in advance or may be determined based on an analysis of particular content. For example, when content transport controls are operated during trick-play, such as fast forward or fast reverse, it is likely that abrupt changes in the content will occur which in previous systems, may have resulted in abrupt changes in ambient lighting effects produced by ambient lighting signals that correspond to the content . The abrupt changes in the resultant ambient lighting effect may be singularly abrupt, as in a large change in an ambient lighting characteristic and/or may be temporally abrupt, as in a large number of transitions of an ambient lighting effect within a relatively short period of time. Other modes of operation may also result in an abrupt change in the rendered content including switching between different channels of content, selection of a new content item, jumping from one portion of the content to another (e.g., jumping between chapter breaks), rendering

discrete content portions, such as a slide show of discrete image content, and other operations as may be readily appreciated by a person of ordinary skill in the art. However, in accordance with the present system, operations that are likely to result in an abrupt change are detected during act 120. In accordance with an embodiment, the detection of a mode of operation that is likely to result in abrupt transitions in the content may be sufficient for applying a low-pass filtering operation to a corresponding ambient lighting signal (see, Act 130) without actually detecting an abrupt change in the content. Thereafter, a low- pass filtering operation may be performed on the ambient lighting signals during act 130 to smooth rapid transitions in a resultant ambient lighting effect. For example, the low-pass filter may be applied in time to smooth-out a color change rate and/or be applied in a color space to reduce a degree of abruptness (e.g., color change) . In one embodiment wherein a fast-forward or fast-reverse mode of operation is selected, a rate of change of corresponding ambient lighting effects may be reduced by sub-sampling the ambient lighting signal and rendering the ambient lighting effect based on the sub-sampled ambient lighting signal.

In accordance with a further embodiment, the present system may monitor ambient lighting signals in addition or in place of monitoring a mode of operation of the host, and the low-pass filter may pass ambient lighting signals that result in a low frequency and/or magnitude of change of the corresponding ambient lighting effect, but may attenuate changes that are above an acceptable threshold of change.

For example, in one embodiment, the low-pass filter may filter a rate of transitions (e.g., number of transitions per unit of time) in the resultant ambient lighting effect. In another embodiment, temporally contiguous samples of the ambient lighting signal may be compared to determine whether a magnitude of change exceeds an acceptable threshold.

In one embodiment, an abrupt transition may be relative to a "normal" transition for TV content, which may typically have an average duration of a shot that may vary between 3 and 5-6 seconds. For certain content types, e.g. commercial advertisements, music video clips, action sequences, the average shot duration may go below 3 seconds . An ambient lighting system may smooth small color and luminance variations within a video shot without small variations being operated on as an abrupt transition. Transitions with a period smaller than 3 seconds may be viewed as an abrupt change by the present system, for example, transitions less than 2.5 seconds or for less than 2 seconds . In one embodiment, a threshold may be dynamically determined based on a type of content being monitored for abrupt transitions. For example, for commercial content, the threshold for an embodiment of the present system may be determined to be less than 2.2 seconds. There are studies that show that the human visual system processes changes in luminance faster than changes in chrominance such as described for example in R.W. Bowen, "LATENCIES FOR CHROMATIC AND ACHROMATIC VISUAL MECHANISMS", provided in VISION RESEARCH, 21 ( 10 ): 1457-1466 , 1981, incorporated herein as if set out in entirety. Accordingly, the threshold may be relatively higher for transitions related to chrominance as opposed to luminance since the human visual system is more likely to track changes in luminance characteristics without the transitions seeming abrupt .

By passing the ambient lighting signal through a low- pass filtering operation during applications that result in abrupt changes of the resultant ambient lighting effect or are likely to result in abrupt changes, the abrupt changes may be avoided, resulting in a more pleasing ambient lighting effect during act 140.

In accordance with an embodiment, an abrupt transition may be determined by identifying an abrupt transition in the content since the ambient lighting signal and corresponding ambient lighting effect generally correspond to the content. For example, in an embodiment wherein the content is analyzed directly to determine the ambient lighting signal, an abrupt transition provided in the content or resulting from an operation of the host (e.g., chapter jump), may be determined directly by comparing contiguous temporal portions of the content (e.g., frames of video content) to identify the abrupt transition in the content.

For example, a simple pixel-wise frame difference may be used to detect a transition (e.g., a video shot cut) . In another embodiment, a difference between color histograms of successive frames may be calculated. The difference between two histograms of consecutive content may be calculated using histogram intersection or simple Ll distance as may be readily appreciated. Systems for detection of scene changes are described in U.S. Patent No. 6,996,805, filed February 7, 2006, and entitled "Method And System For Scene Change Detection"; U.S. Patent No. 7,110,454, filed December 21, 1999, and entitled "Integrated Method For Scene Change Detection"; R. Lienhart, "Comparison of Automatic Shot Boundary Detection Algorithms", Proceedings of Storage and Retrieval for Image and Video Databases VII, January 1999, San Jose, USA, pages 290-301, Vol. 3656; and U.Gargi, R.Kasturi and S. Strayer, "Performance Characterization of Video-Shot-Change Detection Methods," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 10, No. 1, February 2000, pages 1-13, each incorporated herein as if set out in entirety. These systems, portions thereof, and others may be suitably applied in accordance with the present system. For example, in one embodiment, a system for detecting changes in the content such as scene changes is employed and a number of changes in a given period of time is identified to determine if an abrupt change in the content has occurred. For example, the identified number of transitions may be compared to a threshold and if the identified number exceeds the threshold, the low-pass filtering operation may be employed. For content comprising individual content portions, such as a slide show of image content, each individual transition between images may be treated as an abrupt change. An ambient lighting system that modulates an ambient lighting element as a result of the image content may produce a similarly abrupt change in the rendered ambient lighting effect. In this embodiment, the low-pass filter may operate to produce a fade of color, intensity, and/or one or more other characteristics of the ambient lighting effect to bridge a difference in the ambient lighting effect that is created by transitions from one content portion (e.g., an image) to a next content portion. In one embodiment, the low-pass filter of the present system

may produce a predetermined number of transitions between a first ambient lighting effect and a next ambient lighting effect resulting from an abrupt change in the content. In an alternate embodiment, the number of transitions of the ambient lighting effect may be determined dynamically based on how large the difference in one or more characteristics of the content is from just prior to just after the abrupt change. In this embodiment, the larger the change in one or more content characteristics, the larger the number of transitions provided in the ambient lighting effect.

In accordance with an embodiment, slide show transitions may only produce a filtering operation of resultant ambient lighting signals when a large change in a characteristic, such as a large color change, is detected between successive photos. For example, two or more successive photos may only produce transitions in the ambient lighting effect that are below a threshold of change. As long as the transitions of ambient lighting effects are below the threshold of change, the filter in accordance with the present system will pass ambient lighting signals to one or more ambient lighting elements that produce the ambient lighting transitions without alteration. However, when a transition in the ambient lighting signal is detected that would result in a change in the ambient lighting effect above the threshold of change, the filter in accordance with the present system may produce intermediate transitions that are below the threshold of change. These intermediate transitions may be provided between the signals that produced the above threshold change such that the resultant ambient lighting signal produces an ambient lighting effect that transitions in accordance with an embodiment of the present system. For example, the rendered ambient lighting signal may, in effect, produce a fade from a state prior to the detected abrupt change, to the state after the abrupt change, however in rendering the intermediate transitions, avoid the abruptness of the transition.

In an alternate embodiment, if an abrupt sequence of transitions is detected, the present system may simply retain an ambient lighting effect determined from prior to the abrupt sequence of transitions. The ambient lighting effect may be retained until it is determined that the abrupt sequence of transitions has terminated. Thereafter, the ambient lighting effect may again be determined by a then current content portion. The transition to the then current content portion may be immediate, or in one embodiment, may be provided by a sequence of changes if an abrupt transition would occur in transitioning to the then current content portion. Similarly, the retained ambient lighting effect may simply be retained until the system determines that a trick- play mode is exited, such as when a fast-forward, rewind, etc., operation has ended.

In some cases, transitions between differing content portions are managed by a content rendering device to create a particular type of transition effect for the content. For example, with image content, fades, image swipes, decompositions, etc. may be introduced to transition from one image to another. For audio content, an amplitude of the audio level may diminish on a previous audio content portion and then increase on the subsequent (different) content portion. In one embodiment, the present system may provide a pre-defined transition that is associated with the type of

transition effect of the content as opposed to being associated with the content. In this embodiment, when a transition type is determined, an ambient lighting signal associated with the transition type may be rendered during the transition in the content. The ambient lighting signal, for example, may be encoded in an ambient lighting script as may be readily appreciated by a person of ordinary skill in the art. After the transition in the content, an ambient lighting signal associated with the after transition content is rendered. The predefined transition may produce an ambient lighting effect with predefined ambient lighting characteristics such as a flashing lighting effect, pulsating lighting effect, etc. In one embodiment, only a portion of the ambient lighting characteristics of the predefined transition are predefined. For example, during a fast- forward trick play, a predefined transition may produce a pulsating lighting effect, however, other characteristics, such as a color, intensity, or other characteristic (or characteristics for that matter) of the ambient lighting effect during the transition may be otherwise determined from the content. Further, predefined transitions may be selected by a user for association with given light scripts. In accordance with the present system, a mode of operation of the host device may be detected to determine whether an abrupt change will occur and thereby, activate the low-pass feature. In a same or further embodiment, transitions in the content may be directly monitored to identify abrupt transitions. In any event, after the ambient lighting signal is provided to the ambient lighting elements during act 140, the process repeats with a waiting state during act 120.

FIG. 2 shows a device 200 in accordance with an embodiment of the present system. The device 200 may have a processor 210 operationally coupled to a memory 220, one or more ambient lighting elements 260, an input/output (I/O) 240 and a user input device 270. The device 200 may be standalone, such as an ambient lighting element, wherein it is operationally coupled to a host, or the device 200 may be partially or wholly incorporated into the host. The device 200 may have a rendering portion 230, such as a display, for interacting within a user interface paradigm supporting operation of the present system as described herein. In an embodiment wherein the device 200 is incorporated into the host, the rendering portion 230 may be a display that has additional or separate functionality to support features of the host.

The memory 220 may be any type of device for storing application data as well as other data, such as ambient lighting data, light scripts, host data (e.g., in an integrated application) , content, such as video and audio content, etc. In one embodiment, the content may be stored in the memory 220 to enable detection of abrupt transitions as previously described. The application data and other data are received by the processor 210 for configuring the processor 210 to perform operation acts in accordance with the present system. The operation acts include rendering an ambient lighting signal to control one or more of the ambient lighting elements 260 to display ambient lighting effects in accordance with the present system. The user input 270 may include a keyboard, mouse, or other devices, including touch sensitive displays, which may be stand alone or be a part of

a system, such as part of a personal computer, personal digital assistant, or display device such as a television, for communicating with the processor 210 via any type of link, such as a wired or wireless link. For example, the user input 270 may be utilized for activating the features described. Clearly the processor 210, memory 220, rendering portion 230, ambient lighting element 260 and/or user input 270 may all or partly be a portion of a television platform, such as a stand-alone television, may be a portion of another host device, or may be standalone devices.

The methods of the present system are particularly suited to be carried out by a computer software program, such computer software program preferably containing modules corresponding to the individual steps or acts of the methods. Such software may of course be embodied in a computer- readable medium, such as an integrated chip, a peripheral device or memory, such as the memory 220 or other memory coupled to the processor 210.

The computer-readable medium and/or memory 220 may be any recordable medium (e.g., RAM, ROM, removable memory, CD- ROM, hard drives, DVD, floppy disks or memory cards) or may be a transmission medium (e.g., a network comprising fiber- optics, the world-wide web, cables, or a wireless channel using time-division multiple access, code-division multiple access, or other radio-frequency channel) . Any medium known or developed that can provide information suitable for use with a computer system may be used as the computer-readable medium and/or memory 220.

The memory 220 configures processor 210 to implement the methods, operational acts, and functions disclosed herein. The memory may be distributed or local and the processor 210, where additional processors may be provided, may also be distributed, as for example based within the ambient lighting elements, or may be singular. The memories may be implemented as electrical, magnetic or optical memory, or any combination of these or other types of storage devices. Moreover, the term "memory" should be construed broadly enough to encompass any information able to be read from or written to an address in an addressable space accessible by a processor. With this definition, information on a network is still within memory 220, for instance, because the processor 210 may retrieve the information from the network for operation in accordance with the present system.

The processor 210 is capable of providing control signals and/or performing operations in response to input signals from the user input 270 and executing instructions stored in the memory 220. The processor 210 may be an application-specific or general-use integrated circuit (s). Further, the processor 210 may be a dedicated processor for performing in accordance with the present system or may be a general-purpose processor wherein only one of many functions operates for performing in accordance with the present system. The processor 210 may operate utilizing a program portion, multiple program segments, or may be a hardware device utilizing a dedicated or multi-purpose integrated circuit. The I/O 240 may be utilized for requesting and/or receiving one or more light scripts, and/or for other operations as described above.

Of course, it is to be appreciated that any one of the above embodiments or processes may be combined with one or more other embodiments or processes or be separated in accordance with the present system.

Finally, the above-discussion is intended to be merely illustrative of the present system and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, while the present system has been described with reference to exemplary embodiments, it should also be appreciated that numerous modifications and alternative embodiments may be devised by those having ordinary skill in the art without departing from the broader and intended spirit and scope of the present system as set forth in the claims that follow. Accordingly, the specification and drawings are to be regarded in an illustrative manner and are not intended to limit the scope of the appended claims.

In interpreting the appended claims, it should be understood that: a) the word "comprising" does not exclude the presence of other elements or acts than those listed in a given claim; b) the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements; c) any reference signs in the claims do not limit their scope; d) several "means" may be represented by the same item or hardware or software implemented structure or function; e) any of the disclosed elements may be comprised of hardware portions (e.g., including discrete and integrated electronic circuitry), software portions (e.g., computer programming), and any combination thereof; f) hardware portions may be comprised of one or both of analog and digital portions; g) any of the disclosed devices or portions thereof

may be combined together or separated into further portions unless specifically stated otherwise; and h) no specific sequence of acts or steps is intended to be required unless specifically indicated.

Claims

CLAIMS :

1. A method of controlling an ambient lighting element, the method comprising acts of: detecting an abrupt transition in rendered content; applying a low-pass filter operation to modify an ambient lighting signal associated with the rendered content; and rendering the modified ambient lighting signal to avoid an abrupt transition in an ambient lighting effect.

2. The method of Claim 1, wherein the act of detecting comprises an act of detecting a mode of operation of a host device to determine if the mode will produce the abrupt transition.

3. The method of Claim 1, wherein the act of detecting comprises an act of analyzing the rendered content to identify the abrupt transition.

4. The method of Claim 3, wherein the act of analyzing comprises an act of applying the low-pass filter operation only if a given transition in the rendered content exceeds a threshold.

5. The method of Claim 4, wherein the threshold is expressed in a number of transitions within a period of time.

6. The method of Claim 1, wherein the act of detecting comprises an act of analyzing an ambient lighting signal to identify the abrupt transition.

7. The method of Claim 6, wherein the act of analyzing comprises an act of applying the low-pass filter operation only if a given transition in the ambient lighting signal exceeds a threshold.

8. The method of Claim 7, wherein the threshold is expressed in a number of transitions within a period of time.

9. The method of Claim 1, wherein the modified ambient lighting signal is arranged to produce at least one addition transition between an ambient lighting effect determined from prior to the abrupt transition and an ambient lighting effect determined from after the abrupt transition.

10. The method of Claim 9, wherein how many additional transitions to provide is determined by analyzing characteristics of the abrupt transition.

11. An application embodied on a computer readable medium configured to control an ambient lighting element, the application comprising: a portion configured to detect an abrupt transition in rendered content; a portion configured to apply a low-pass filter operation to modify an ambient lighting signal associated with the rendered content; and a portion configured to render the modified ambient lighting signal to avoid an abrupt transition in an ambient lighting effect.

12. The application of Claim 11, wherein the portion configured to detect comprises a portion configured to detect a mode of operation of a host device to determine if the mode will produce the abrupt transition in the ambient lighting effect .

13. The application of Claim 11, wherein the portion configured to apply the low-pass filter is configured to apply the low-pass filter operation only if transitions in the rendered content exceed a given number of transitions within a period of time.

14. The application of Claim 11, wherein the portion configured to apply the low-pass filter is configured to apply the low-pass filter operation only if transitions in the ambient lighting signal exceed a given number of transitions within a period of time.

15. The application of Claim 11, wherein the modified ambient lighting signal is arranged to produce a reduced number of transitions in the ambient lighting effect.

16. A device for controlling an ambient lighting element, the device comprising: a memory 220; and a processor 210 operationally coupled to the memory 220, wherein the processor 210 is configured to: detect an abrupt transition in rendered content; apply a low-pass filter operation to modify an ambient lighting signal associated with the rendered content; and render the modified ambient lighting signal to avoid an abrupt transition in an ambient lighting effect.

17. The device of Claim 16, wherein the processor 210 is configured to detect a mode of operation of a host device to determine if the mode will produce the abrupt transition in the ambient lighting effect.

18. The device of Claim 16, wherein the processor 210 is configured to apply the low-pass filter operation only if transitions in the rendered content exceed a given number of transitions within a period of time.

19. The application of Claim 16, wherein the processor 210 is configured to apply the low-pass filter operation only if transitions in the ambient lighting signal exceed a given number of transitions within a period of time.

20. The application of Claim 19, wherein the processor 210 is arranged to determine the given number of transitions as a threshold that is adjusted based on a detected type of rendered content .