WO2020151993A1 - A controller for controlling a lighting device based on media content and a method thereof - Google Patents

A controller for controlling a lighting device based on media content and a method thereof Download PDF

Info

Publication number
WO2020151993A1
WO2020151993A1 PCT/EP2020/050759 EP2020050759W WO2020151993A1 WO 2020151993 A1 WO2020151993 A1 WO 2020151993A1 EP 2020050759 W EP2020050759 W EP 2020050759W WO 2020151993 A1 WO2020151993 A1 WO 2020151993A1
Authority
WO
WIPO (PCT)
Prior art keywords
media content
light
media
content
control mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2020/050759
Other languages
French (fr)
Inventor
Dzmitry Viktorovich Aliakseyeu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Signify Holding BV
Original Assignee
Signify Holding BV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Signify Holding BV filed Critical Signify Holding BV
Publication of WO2020151993A1 publication Critical patent/WO2020151993A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4131—Peripherals receiving signals from specially adapted client devices home appliance, e.g. lighting, air conditioning system, metering devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/654—Transmission by server directed to the client
    • H04N21/6547—Transmission by server directed to the client comprising parameters, e.g. for client setup
    • H—ELECTRICITY
    • H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
    • H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
    • H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
    • H05B47/10—Controlling the light source
    • H05B47/155—Coordinated control of two or more light sources

Definitions

  • the invention relates to a method of controlling a lighting device based on media content rendered on a media rendering device.
  • the invention further relates to a computer program product for executing the method.
  • the invention further relates to a controller for controlling a lighting device based on media content rendered on a media rendering device, and to a lighting system comprising the controller.
  • the invention further relates to a media rendering device comprising the controller.
  • the use of light effects that match with and support media content can significantly enhance the media content.
  • Such light effects are typically specified in a light script. If the media content (e.g. video, audio or a game) is being streamed, the light script may be streamed together with or separately from the media content.
  • the inventors have realized that when a predefined light script is being streamed from a remote server (e.g. via the internet), it may occur that the stream becomes unavailable or that it is not (yet) available while rendering the media content. This may be due to bandwidth limitations (e.g. in the local network or in the remote network), due to (temporary) unavailability of the script, due to inaccessibility of the remote server, etc. It is therefore an object of the present invention to provide a controller for controlling a lighting device when a streamed light script is unavailable.
  • the object is achieved by a method of controlling one or more lighting devices based on media content rendered on a media rendering device, the method comprising:
  • streaming a light script associated with the media content from the remote server controlling, in a first control mode, the one or more lighting devices according to the light script when the light script is available from the remote server,
  • the one or more lighting devices are controlled according to a (predefined) light script
  • the second control mode wherein the one or more lighting devices are controlled according to light settings extracted from the media content
  • the one or more lighting devices are still controlled based on the media content. This is beneficial, because when the stream of the light script is interrupted, the user experience provided by the one or more lighting devices of a (home) entertainment lighting system is uninterrupted.
  • the method may further comprise: switching back to the first control mode upon determining that the stream of the light script has become available.
  • Switching back to the (predefined) light script is beneficial, because it removes the need for analyzing the media content to generate one or more light settings based on the media content.
  • the light script may comprise light settings that may not be retrievable from the media content. Such light settings (light effects) may further improve the user experience of the media content.
  • the method may further comprise: synchronizing the stream of the light script with the media content. After the stream of the light script has become available again, it is beneficial to synchronize the stream of the light script with the media content. This step may occur before switching back to the first control mode upon determining that the stream of the light script has become available.
  • the method may further comprise the step of: analyzing the one or more upcoming parts of the media content to identify a transition from a first type of upcoming parts to a second type of upcoming parts.
  • the step of switching back to the first control mode may be executed when the second type of upcoming parts are rendered on the media rendering device. This enables switching back to the first control mode when the type of content rendered on the media rendering device changes substantially. This is beneficial, because the transition from the second control mode back to the first control mode may be smooth and less noticeable or unnoticeable for a user.
  • the step of analyzing the media content to generate one or more light settings based on the media content may be executed while controlling the one or more lighting devices in the first control mode.
  • the media content may be analyzed while simultaneously controlling the one or more lighting devices in the first control mode.
  • the method may further comprise: analyzing one or more upcoming parts of the media content and analyzing the one or more upcoming parts to generate the one or more light settings based on the media content.
  • the upcoming parts may be buffered (i.e. temporarily stored) in a memory. This may further result in an improved, seamless user experience.
  • the step of analyzing the media content to generate one or more light settings based on the media content may be executed upon switching to the second control mode. This may require less local storage and computing resources, because the media content is (only) analyzed when the light script becomes interrupted/unavailable.
  • the media content may be video content
  • the step of analyzing the media content to generate one or more light settings based on the media content may comprise: generating the one or more light settings by extracting color information, saturation information and/or brightness information from the video content. Images of the video content may be analyzed to extract color, saturation and/or brightness levels from the images, and the one or more lighting devices may be controlled based on these levels.
  • the media content may be audio content
  • the step of analyzing the media content to generate one or more light settings based on the media content may comprise: generating the one or more light settings based on one or more audio characteristics of the audio content.
  • the audio characteristics may, for example, comprise one or more of: beat, timbre, pitch, intensity, rhythm, major and minor key.
  • the media content may be game content
  • the step of analyzing the media content to generate one or more light settings based on the media content comprises:
  • generating the one or more light settings by extracting color information, saturation information and/or brightness information form the game content, and/or generating the one or more light settings based on one or more game events of the game content.
  • Images of the game content may be analyzed to extract color, saturation and/or brightness levels from the images, and the one or more lighting devices may be controlled based on these levels.
  • (upcoming/predefmed) game events may be analyzed to determine color, saturation and/or brightness levels, and the one or more lighting devices may be controlled based on these levels.
  • the media content may be streamed from the remote server or from a second remote server to the media rendering device.
  • the object is achieved by a computer program product for a computing device, the computer program product comprising computer program code to perform any one of the above-mentioned methods when the computer program product is run on a processing unit of the computing device.
  • the object is achieved by a controller for controlling one or more lighting devices based on media content rendered on a media rendering device, the controller comprising:
  • a processor configured to control, in a first control mode, the one or more lighting devices according to the light script when the light script is available from the remote server or the second remote server,
  • processor is further configured to switch to a second control mode upon determining that a stream of the light script has been interrupted, to analyze the media content to generate one or more light settings based on the media content, and to control, in the second mode, the one or more lighting devices according to the generated one or more light settings.
  • the object is achieved by a lighting system comprising the controller, and the one or more lighting devices configured to be controlled by the controller.
  • the object is achieved by a media rendering device comprising the controller, wherein the media rendering device is configured to stream the media content to the media rendering device.
  • Fig. 1 shows schematically an embodiment of a system and a remote server, wherein the system comprises a media rendering device, a lighting device and a controller for controlling the lighting device;
  • Fig. 2a shows schematically an alternative embodiment of the system of Fig.
  • controller is comprised in the media rendering device
  • Fig. 2b shows schematically an alternative embodiment of the system of Fig. 2a, wherein lighting devices are comprised in the media rendering device;
  • Fig. 3a shows schematically a timeline of playback of media content, and control of one or more lighting devices by switching between control modes
  • Fig. 3b shows schematically a timeline of playback of media content, and control of one or more lighting devices by switching between control modes with a transitional period;
  • Fig. 3c shows schematically a timeline of playback of media content, wherein control of one or more lighting devices starts in a second control mode
  • Fig. 4 shows schematically a method of controlling one or more lighting devices based on media content rendered on a media rendering device.
  • Fig. 1 shows a system 100 and a remote server 130.
  • the system comprises a media rendering device 120, a lighting device 110 and a controller 102.
  • Fig 2a illustrates an alternative configuration of the system 200a, wherein the controller 102 is comprised in the media rendering device 120.
  • Fig 2b illustrates an alternative configuration of the system 200b, wherein lighting devices 110 are comprised in the media rendering device 120.
  • the controller 102 is configured to control the lighting device 110 based on media content rendered on the media rendering device 120.
  • the media content e.g. video content, game content or audio content
  • the media content may be stored locally, for instance on the media rendering device 120, on an auxiliary/peripheral device such as a local hard drive, a Blu- ray/DVD disk/player, a game console, a media center, a smartphone, etc.
  • the media content may be streamed from the remote server 130 (or another remote server) to the media rendering device 120.
  • the controller 102 comprises a receiver 104 for streaming a light script associated with the media content from the remote server 13o.
  • the controller 102 further comprises a processor 106 (e.g.
  • a microchip, circuitry, a microcontroller, etc. configured to control, in a first control mode, the lighting device 110 according to the light script when a stream of the light script is available from the remote server 130.
  • the processor 106 is further configured to switch to a second control mode upon determining that the stream of the light script has been interrupted.
  • the processor 106 is further configured to analyze the media content to generate one or more light settings based on the media content, and to control, in the second mode, the lighting device 110 according to the generated one or more light settings.
  • the controller 102 may be comprised in any type of device for controlling the lighting device 110.
  • the controller 102 may, for example, be comprised in a personal device such as a smartphone, a smartwatch, a tablet pc.
  • the controller 102 may be comprised in a central control device such as a central come control system, a bridge device, etc.
  • the controller 102 may be comprised in the media rendering device (see Figs. 2a and 2b), or in the lighting device 110.
  • the receiver 104 is configured to receive an incoming stream of the light script associated with the media content.
  • the light script may comprise predefined light settings (e.g. color, saturation, brightness, beam width/size etc.) according to which the lighting device 110 is to be controlled while the media content is being rendered on the media rendering device 120.
  • the light script may be streamed in a similar manner as media content may be streamed.
  • the light script may be divided in multiple parts, and upcoming parts (i.e. parts that correspond to upcoming media content that has not yet been rendered) may be buffered in a memory (not shown). This prevents disruptions if there are momentary delays in transmission of the light script from the remote server.
  • the stream of the light script is not (yet) available or that the stream is interrupted while the media content is being rendered. This may be due to bandwidth limitations (e.g. in the local network or in the remote network/server 130), due to (temporary) unavailability of the script, due to inaccessibility of the remote server 130, etc.
  • Another reason for the unavailability of the light script may be that the user forwards to a part of the (streamed) media content (and therewith to a part of the light script) that has not been buffered yet.
  • the processor 106 may switch to the second control mode wherein the processor 106 is configured to generate light settings by analyzing the media content.
  • the processor 106 is configured to operate in two control modes.
  • the processor 106 is configured to control the lighting device 110 according to the light script when the light script is available.
  • the processor 106 may be configured to communicate light settings of the light script to the lighting device 110.
  • the processor 106 may be further configured to synchronize the media content with the light script stream, such that the lighting device 110 is controlled at the correct moment in time while the media content is being rendered by the media rendering device 120. Synchronization of light scripts and media content is known in the art, and will therefore not be discussed in detail.
  • the processor 106 is further configured determine/detect that the stream of the light script is interrupted and to switch to the second control mode. In the second control mode, the lighting device 110 is no longer controlled based on the streamed light script. Instead, the processor 106 controls the lighting device 110 according to the generated one or more light settings, which one or more light settings are based on the media content.
  • the processor 106 may be configured to analyze the media content while controlling the lighting device 110 in the first control mode. In other words, the processor 106 may analyze the media content while controlling the lighting device 110 in the first control mode. This enables the processor 106 to immediately switch to the second control mode when the stream of the light script suddenly becomes unavailable, or if it is interrupted. The processor 106 may then continue to analyze the media content to generate light settings for the lighting device 110 and to control the lighting device 110 based on the media content (which may be streamed to the media rendering device 120). This provides the benefit that the switch from the first control mode to the second control mode (and the switch from the first way of lighting control to the second way of lighting control) is seamless.
  • the processor 106 may be configured to temporarily store the generated one or more light settings in a memory, and the processor 106 may be configured to remove the stored light settings from the memory after the part of the media content associated with those light settings has been rendered by the media rendering device 120.
  • the processor 106 may be configured to start analyzing the media content to generate one or more light settings upon switching to the second control mode. If, for example, upcoming parts of the streamed light script have already been buffered, the processor 106 may control the lighting device 110 according to the buffered parts, start analyzing the media content while still controlling the lighting device according to light settings of the light script, and switch to the second control mode when or before the last upcoming part of the light script has been used. This may require less local storage and computing resources, because the media content is (only) analyzed when the light script is interrupted/unavailable.
  • the processor 106 may be further configured to analyze one or more upcoming parts of the media content (e.g. upcoming video scenes or images, upcoming game events, upcoming audio/songs, etc.) and analyze the one or more upcoming parts to generate the one or more light settings based on the media content.
  • the upcoming parts may be buffered (i.e. temporarily stored) in a memory (not shown) when the media content is streamed.
  • the processor 106 may control the lighting device 110 when the upcoming parts are rendered on the image rendering device 120.
  • the processor 106 may be configured to temporally store the generated one or more light settings in the memory, and the processor 106 may be configured to remove the stored light settings from the memory after the associated part of the media content has been rendered by the media rendering device 120.
  • the media content may be video content (e.g. a movie, a video clip, a news broadcast, etc.).
  • the video content may be streamed from a video streaming platform.
  • the processor 106 may be configured to analyze the video content and generate the one or more light settings by extracting color information, saturation information and/or brightness information from the video content.
  • the processor 106 may be configured to analyze images/scenes of the video content to extract color, saturation and/or brightness levels from the images, and control the lighting device 110 based on these levels. Techniques for extracting such information from video content is known in the art, and will therefore not be discussed in detail.
  • the media content may be audio content (e.g. a song, an audiobook, etc.).
  • the audio content may be streamed from a music streaming platform.
  • the processor 106 may be configured to analyze the audio content and generate the one or more light settings based on one or more audio characteristics of the audio content.
  • the audio characteristics may, for example, comprise one or more of: beat, timbre, pitch, intensity, rhythm, major and minor key. Additionally, or alternatively the one or more audio characteristics may comprise audio features.
  • the audio features may comprise direct mood, valence and arousal/energy. Direct mood may be estimated using set of mood labels (which may be user-defined or
  • a combination of valence and arousal may be used for defining mood.
  • the media content may be video game content.
  • the video game content may be streamed from a video game streaming platform (e.g. a cloud gaming platform).
  • the processor 106 may be configured to analyze the video game content and generate the one or more light settings by extracting color information, saturation information and/or brightness information form the game content. Additionally or alternatively, the processor 106 may be configured to generate the one or more light settings based on one or more game events of the game content. Similar to the above-mentioned video analysis, images/scenes of the video game content may be analyzed to extract color, saturation and/or brightness levels from the images, and the one or more lighting devices may be controlled based on these levels.
  • (predefined) game events of the game content may be accessible, and these (predefined) game events (e.g. explosions, change of scenery, opening of a door, etc.) may be analyzed by the processor 106. Color, saturation and/or brightness levels may, for example, be extracted from the game events, and the processor 106 may control the lighting device 110 accordingly.
  • game events may be associated with predefined light settings, and the processor may control the lighting device 106 based on the predefined light settings.
  • the processor 106 may be further configured to switch back from the second control mode to the first control mode upon determining that the stream of the light script has become available again.
  • Figs. 3a shows schematically a timeline t of playback of media content.
  • the processor 106 may start in the first control mode, wherein the lighting device 110 is controlled based on the light script 5.
  • the processor 106 may switch to the second control mode, wherein the processor 106 analyzes the media content to generate the light settings based thereon, and wherein the processor 106 controls the lighting device 110 based on the light settings a automatically generated based on the media content.
  • the processor 106 may switch back from the second control mode to the first control mode, wherein the lighting device 110 is controlled again based on the light script 5.
  • the processor 106 may be further configured to transition from the first mode to the second mode over time.
  • Fig. 3b illustrates a transitional period tp after the stream of the light script has become available again.
  • the processor 106 may therefore be configured to buffer a plurality of upcoming parts of the media content, and to analyze the one or more upcoming parts of the media content to identify a transition from a first type of upcoming parts to a second type of upcoming parts.
  • the processor 106 may be configured to switch back to the first control mode when the second type of upcoming parts are rendered on the media rendering device. This enables switching back to the first when the type of content rendered on the media rendering device 130 changes substantially.
  • the switching back from the second mode to the first mode may occur when such an event occurs (i.e. after the transitional period tp).
  • the processor 106 may be further configured to start in the second control mode when the media content is being rendered on the media rendering device 120.
  • the stream of the media content may be available from the remote server 130, whereas the stream of the light script may not yet be available from the remote server 130.
  • Fig. 3c illustrates an example wherein the processor 106 starts (at tO ) in the second control mode, wherein the processor 106 controls the lighting device 110 based on the light settings a automatically generated based on the media content.
  • the processor 106 may switch to the first control mode, wherein the lighting device 110 is controlled based on the light script.
  • the processor 106 may again switch to the second control mode.
  • the processor 106 may switch back to the first control mode.
  • the controller 102 may comprise a communication unit 108 configured to (wirelessly) communicate lighting control commands to the lighting device 110.
  • the lighting control commands may comprise lighting control instructions for the lighting device 110 to control its light output according to the light settings of the light script, or according to the light settings generated based on the media content.
  • the communication unit 108 may comprise hardware for transmitting the lighting control commands via any wired or wireless communication protocol.
  • wired and wireless communication protocols may be used, for example Ethernet, DMX, DALI, USB, Bluetooth, Wi-Fi, Li-Fi, 3G, 4G, 5G or ZigBee.
  • the communication unit 108 may be an output of the controller 102 for directly controlling the lighting device 110.
  • the receiver 104 may be further configured to receive an incoming stream of the media content.
  • the processor 106 may use this stream for analyzing the media content. Additionally, the processor 106 may be configured to stream the media content to the media rendering device 120, for instance via the communication unit 108 or via a different communication unit. In embodiments wherein the controller 102 is comprised in the media rendering device 120 (see Figs. 2a and 2b), the processor 106 may be configured to receive the incoming stream of the media content directly from the media rendering device 120, or from an auxiliary device connected to the media rendering device 120.
  • the media rendering device 120 may be any type of device configured to communicate the media content to the user.
  • the media rendering device 120 may, for example, be a television, a tablet pc, a smartphone, a (home) audio system, a pc, etc.
  • the processor 106 of the controller 102 may be a processor of the media rendering device 120.
  • the processor 106 may be configured to stream the media content to the media rendering device 120.
  • the media rendering device 120 may be configured to receive the stream of the media content from the remote server 130, and the processor 106 of the separate controller 102 may be configured to receive the same stream, or at least information about the streamed media content (e.g. timing information, color information, audio characteristic information, etc.), and the processor 106 may analyze the media content and generate the light settings based thereon.
  • the processor 106 of the separate controller 102 may be configured to receive the same stream, or at least information about the streamed media content (e.g. timing information, color information, audio characteristic information, etc.), and the processor 106 may analyze the media content and generate the light settings based thereon.
  • the lighting device 110 comprises one or more light sources (e.g. LED light sources) and a control unit, such as a microcontroller (not shown), for controlling the light output generated by the light sources based on lighting control commands received from the processor 106 of the controller 102.
  • a lighting control command may comprise lighting control settings for controlling the light output.
  • the lighting control settings may relate to light properties such as the color, intensity, saturation, beam size, beam shape, etc. according to which the lighting device 110 is to be controlled.
  • the lighting control settings may be indicative of a dynamic light scene, wherein one or more light properties change over time.
  • the remote server 130 may be part of an online media content streaming platform (e.g. a video streaming platform, a
  • music/audiobook streaming platform a video game streaming platform, etc.
  • the remote server 130 may be accessible via a network, for instance via the internet.
  • a first remote server may be configured for streaming the light script to the controller 102
  • the second remote server may be configured for streaming the media content (to the controller 102 and/or to the media rendering device 120).
  • the remote server 130 may be a light script streaming platform for streaming light settings to the controller 102, whereupon the processor 106 of the controller 102 may control the lighting device 110 based on the light settings.
  • the processor 106 may be configured to synchronize the light script streamed from the second server with the media (streamed) content. Synchronization of light scripts and media content from different sources is known in the art, and will therefore not be discussed in detail.
  • Fig. 4 shows schematically a method 400 of controlling one or more lighting devices 110 based on media content rendered on a media rendering device 120.
  • the method comprises: streaming 404 a light script associated with the media content from the remote server, controlling 406, in a first control mode, the one or more lighting devices according to the light script when the light script is available from the remote server, switching 408 to a second control mode upon determining that a stream of the light script has been interrupted, analyzing 410 the media content to generate one or more light settings based on the media content, and controlling 412, in the second mode, the one or more lighting devices according to the generated one or more light settings.
  • the step of analyzing 510 may be performed before the switching step 408, for instance during, before or after steps 404 and 406.
  • the method 400 may further comprise the step of streaming 402 the media content from the remote server to the media rendering device.
  • the method 400 may further comprise the step of switching 414 back to the first control mode upon determining that the stream of the light script has become available.
  • the method 400 may be executed by computer program code of a computer program product when the computer program product is run on a processing unit of a computing device, such as the processor 106 of the controller 102.
  • any reference signs placed between parentheses shall not be construed as limiting the claim.
  • Use of the verb "comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim.
  • the article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements.
  • the invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer or processing unit. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
  • aspects of the invention may be implemented in a computer program product, which may be a collection of computer program instructions stored on a computer readable storage device which may be executed by a computer.
  • the instructions of the present invention may be in any interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs) or Java classes.
  • the instructions can be provided as complete executable programs, partial executable programs, as modifications to existing programs (e.g. updates) or extensions for existing programs (e.g. plugins).
  • parts of the processing of the present invention may be distributed over multiple computers or processors or even the‘cloud’.
  • Storage media suitable for storing computer program instructions include all forms of nonvolatile memory, including but not limited to EPROM, EEPROM and flash memory devices, magnetic disks such as the internal and external hard disk drives, removable disks and CD-ROM disks.
  • the computer program product may be distributed on such a storage medium, or may be offered for download through HTTP, FTP, email or through a server connected to a network such as the Internet.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Automation & Control Theory (AREA)
  • Circuit Arrangement For Electric Light Sources In General (AREA)

Abstract

A method (400) of controlling one or more lighting devices based on media content rendered on a media rendering device is disclosed. The method (400) comprises: streaming (404) a light script associated with the media content from a remote server, controlling (406), in a first control mode, the one or more lighting devices according to the light script when the light script is available from the remote server, switching (408) to a second control mode upon determining that a stream of the light script has been interrupted, analyzing (410) the media content to generate one or more light settings based on the media content, and controlling (412), in the second mode, the one or more lighting devices according to the generated one or more light settings.

Description

A controller for controlling a lighting device based on media content and a method thereof
FIELD OF THE INVENTION
The invention relates to a method of controlling a lighting device based on media content rendered on a media rendering device. The invention further relates to a computer program product for executing the method. The invention further relates to a controller for controlling a lighting device based on media content rendered on a media rendering device, and to a lighting system comprising the controller. The invention further relates to a media rendering device comprising the controller.
BACKGROUND
Home entertainment lighting systems have proven to add a great deal to the experience of games, movies and music. For example, the use of light effects that match with and support media content can significantly enhance the media content. Such light effects are typically specified in a light script. If the media content (e.g. video, audio or a game) is being streamed, the light script may be streamed together with or separately from the media content.
SUMMARY OF THE INVENTION
The inventors have realized that when a predefined light script is being streamed from a remote server (e.g. via the internet), it may occur that the stream becomes unavailable or that it is not (yet) available while rendering the media content. This may be due to bandwidth limitations (e.g. in the local network or in the remote network), due to (temporary) unavailability of the script, due to inaccessibility of the remote server, etc. It is therefore an object of the present invention to provide a controller for controlling a lighting device when a streamed light script is unavailable.
According to a first aspect of the present invention, the object is achieved by a method of controlling one or more lighting devices based on media content rendered on a media rendering device, the method comprising:
streaming a light script associated with the media content from the remote server, controlling, in a first control mode, the one or more lighting devices according to the light script when the light script is available from the remote server,
switching to a second control mode upon determining that a stream of the light script has been interrupted,
analyzing the media content to generate one or more light settings based on the media content, and
controlling, in the second mode, the one or more lighting devices according to the generated one or more light settings.
By switching from the first control mode (wherein the one or more lighting devices are controlled according to a (predefined) light script) to the second control mode (wherein the one or more lighting devices are controlled according to light settings extracted from the media content) when the light script is interrupted/unavailable, the one or more lighting devices are still controlled based on the media content. This is beneficial, because when the stream of the light script is interrupted, the user experience provided by the one or more lighting devices of a (home) entertainment lighting system is uninterrupted.
The method may further comprise: switching back to the first control mode upon determining that the stream of the light script has become available. Switching back to the (predefined) light script is beneficial, because it removes the need for analyzing the media content to generate one or more light settings based on the media content. Also, the light script may comprise light settings that may not be retrievable from the media content. Such light settings (light effects) may further improve the user experience of the media content.
The method may further comprise: synchronizing the stream of the light script with the media content. After the stream of the light script has become available again, it is beneficial to synchronize the stream of the light script with the media content. This step may occur before switching back to the first control mode upon determining that the stream of the light script has become available.
The method may further comprise the step of: analyzing the one or more upcoming parts of the media content to identify a transition from a first type of upcoming parts to a second type of upcoming parts. The step of switching back to the first control mode may be executed when the second type of upcoming parts are rendered on the media rendering device. This enables switching back to the first control mode when the type of content rendered on the media rendering device changes substantially. This is beneficial, because the transition from the second control mode back to the first control mode may be smooth and less noticeable or unnoticeable for a user. The step of analyzing the media content to generate one or more light settings based on the media content may be executed while controlling the one or more lighting devices in the first control mode. In other words, the media content may be analyzed while simultaneously controlling the one or more lighting devices in the first control mode. This is beneficial, because if the light script suddenly becomes unavailable, or if it is interrupted, an immediate switch to the second control mode may be executed, to create an improved, seamless user experience. The method may further comprise: analyzing one or more upcoming parts of the media content and analyzing the one or more upcoming parts to generate the one or more light settings based on the media content. The upcoming parts may be buffered (i.e. temporarily stored) in a memory. This may further result in an improved, seamless user experience.
Alternatively, the step of analyzing the media content to generate one or more light settings based on the media content may be executed upon switching to the second control mode. This may require less local storage and computing resources, because the media content is (only) analyzed when the light script becomes interrupted/unavailable.
The media content may be video content, and the step of analyzing the media content to generate one or more light settings based on the media content may comprise: generating the one or more light settings by extracting color information, saturation information and/or brightness information from the video content. Images of the video content may be analyzed to extract color, saturation and/or brightness levels from the images, and the one or more lighting devices may be controlled based on these levels.
The media content may be audio content, and the step of analyzing the media content to generate one or more light settings based on the media content may comprise: generating the one or more light settings based on one or more audio characteristics of the audio content. The audio characteristics may, for example, comprise one or more of: beat, timbre, pitch, intensity, rhythm, major and minor key.
The media content may be game content, and the step of analyzing the media content to generate one or more light settings based on the media content comprises:
generating the one or more light settings by extracting color information, saturation information and/or brightness information form the game content, and/or generating the one or more light settings based on one or more game events of the game content. Images of the game content may be analyzed to extract color, saturation and/or brightness levels from the images, and the one or more lighting devices may be controlled based on these levels.
Additionally or alternatively, (upcoming/predefmed) game events (such as explosions) may be analyzed to determine color, saturation and/or brightness levels, and the one or more lighting devices may be controlled based on these levels.
The media content may be streamed from the remote server or from a second remote server to the media rendering device.
According to a second aspect of the present invention, the object is achieved by a computer program product for a computing device, the computer program product comprising computer program code to perform any one of the above-mentioned methods when the computer program product is run on a processing unit of the computing device.
According to a third aspect of the present invention, the object is achieved by a controller for controlling one or more lighting devices based on media content rendered on a media rendering device, the controller comprising:
a receiver for streaming a light script associated with the media content from the remote server,
a processor configured to control, in a first control mode, the one or more lighting devices according to the light script when the light script is available from the remote server or the second remote server,
wherein the processor is further configured to switch to a second control mode upon determining that a stream of the light script has been interrupted, to analyze the media content to generate one or more light settings based on the media content, and to control, in the second mode, the one or more lighting devices according to the generated one or more light settings.
According to a fourth aspect of the present invention, the object is achieved by a lighting system comprising the controller, and the one or more lighting devices configured to be controlled by the controller.
According to a fifth aspect of the present invention, the object is achieved by a media rendering device comprising the controller, wherein the media rendering device is configured to stream the media content to the media rendering device.
It should be understood that the computer program product, the controller, the lighting system and/or the media rendering device may have similar and/or identical embodiments and advantages as the above-mentioned methods.
BRIEF DESCRIPTION OF THE DRAWINGS
The above, as well as additional objects, features and advantages of the disclosed systems, devices and methods will be better understood through the following illustrative and non-limiting detailed description of embodiments of devices and methods, with reference to the appended drawings, in which:
Fig. 1 shows schematically an embodiment of a system and a remote server, wherein the system comprises a media rendering device, a lighting device and a controller for controlling the lighting device;
Fig. 2a shows schematically an alternative embodiment of the system of Fig.
1, wherein the controller is comprised in the media rendering device;
Fig. 2b shows schematically an alternative embodiment of the system of Fig. 2a, wherein lighting devices are comprised in the media rendering device;
Fig. 3a shows schematically a timeline of playback of media content, and control of one or more lighting devices by switching between control modes;
Fig. 3b shows schematically a timeline of playback of media content, and control of one or more lighting devices by switching between control modes with a transitional period;
Fig. 3c shows schematically a timeline of playback of media content, wherein control of one or more lighting devices starts in a second control mode; and
Fig. 4 shows schematically a method of controlling one or more lighting devices based on media content rendered on a media rendering device.
All the figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested.
DETAILED DESCRIPTION OF EMBODIMENTS
Fig. 1 shows a system 100 and a remote server 130. The system comprises a media rendering device 120, a lighting device 110 and a controller 102. Fig 2a illustrates an alternative configuration of the system 200a, wherein the controller 102 is comprised in the media rendering device 120. Fig 2b illustrates an alternative configuration of the system 200b, wherein lighting devices 110 are comprised in the media rendering device 120.
The controller 102 is configured to control the lighting device 110 based on media content rendered on the media rendering device 120. The media content (e.g. video content, game content or audio content) may be stored locally, for instance on the media rendering device 120, on an auxiliary/peripheral device such as a local hard drive, a Blu- ray/DVD disk/player, a game console, a media center, a smartphone, etc. Alternatively, the media content may be streamed from the remote server 130 (or another remote server) to the media rendering device 120. The controller 102 comprises a receiver 104 for streaming a light script associated with the media content from the remote server 13o. The controller 102 further comprises a processor 106 (e.g. a microchip, circuitry, a microcontroller, etc.) configured to control, in a first control mode, the lighting device 110 according to the light script when a stream of the light script is available from the remote server 130. The processor 106 is further configured to switch to a second control mode upon determining that the stream of the light script has been interrupted. The processor 106 is further configured to analyze the media content to generate one or more light settings based on the media content, and to control, in the second mode, the lighting device 110 according to the generated one or more light settings.
The controller 102 may be comprised in any type of device for controlling the lighting device 110. The controller 102 may, for example, be comprised in a personal device such as a smartphone, a smartwatch, a tablet pc. In embodiments, the controller 102 may be comprised in a central control device such as a central come control system, a bridge device, etc. In embodiments, the controller 102 may be comprised in the media rendering device (see Figs. 2a and 2b), or in the lighting device 110.
The receiver 104 is configured to receive an incoming stream of the light script associated with the media content. The light script may comprise predefined light settings (e.g. color, saturation, brightness, beam width/size etc.) according to which the lighting device 110 is to be controlled while the media content is being rendered on the media rendering device 120. The light script may be streamed in a similar manner as media content may be streamed. The light script may be divided in multiple parts, and upcoming parts (i.e. parts that correspond to upcoming media content that has not yet been rendered) may be buffered in a memory (not shown). This prevents disruptions if there are momentary delays in transmission of the light script from the remote server. It may, however, occur that the stream of the light script is not (yet) available or that the stream is interrupted while the media content is being rendered. This may be due to bandwidth limitations (e.g. in the local network or in the remote network/server 130), due to (temporary) unavailability of the script, due to inaccessibility of the remote server 130, etc. Another reason for the unavailability of the light script may be that the user forwards to a part of the (streamed) media content (and therewith to a part of the light script) that has not been buffered yet. To solve this problem, the processor 106 may switch to the second control mode wherein the processor 106 is configured to generate light settings by analyzing the media content. The processor 106 is configured to operate in two control modes. In the first control mode, the processor 106 is configured to control the lighting device 110 according to the light script when the light script is available. The processor 106 may be configured to communicate light settings of the light script to the lighting device 110. The processor 106 may be further configured to synchronize the media content with the light script stream, such that the lighting device 110 is controlled at the correct moment in time while the media content is being rendered by the media rendering device 120. Synchronization of light scripts and media content is known in the art, and will therefore not be discussed in detail.
The processor 106 is further configured determine/detect that the stream of the light script is interrupted and to switch to the second control mode. In the second control mode, the lighting device 110 is no longer controlled based on the streamed light script. Instead, the processor 106 controls the lighting device 110 according to the generated one or more light settings, which one or more light settings are based on the media content.
The processor 106 may be configured to analyze the media content while controlling the lighting device 110 in the first control mode. In other words, the processor 106 may analyze the media content while controlling the lighting device 110 in the first control mode. This enables the processor 106 to immediately switch to the second control mode when the stream of the light script suddenly becomes unavailable, or if it is interrupted. The processor 106 may then continue to analyze the media content to generate light settings for the lighting device 110 and to control the lighting device 110 based on the media content (which may be streamed to the media rendering device 120). This provides the benefit that the switch from the first control mode to the second control mode (and the switch from the first way of lighting control to the second way of lighting control) is seamless. The processor 106 may be configured to temporarily store the generated one or more light settings in a memory, and the processor 106 may be configured to remove the stored light settings from the memory after the part of the media content associated with those light settings has been rendered by the media rendering device 120.
Alternatively, the processor 106 may be configured to start analyzing the media content to generate one or more light settings upon switching to the second control mode. If, for example, upcoming parts of the streamed light script have already been buffered, the processor 106 may control the lighting device 110 according to the buffered parts, start analyzing the media content while still controlling the lighting device according to light settings of the light script, and switch to the second control mode when or before the last upcoming part of the light script has been used. This may require less local storage and computing resources, because the media content is (only) analyzed when the light script is interrupted/unavailable.
The processor 106 may be further configured to analyze one or more upcoming parts of the media content (e.g. upcoming video scenes or images, upcoming game events, upcoming audio/songs, etc.) and analyze the one or more upcoming parts to generate the one or more light settings based on the media content. The upcoming parts may be buffered (i.e. temporarily stored) in a memory (not shown) when the media content is streamed. The processor 106 may control the lighting device 110 when the upcoming parts are rendered on the image rendering device 120. The processor 106 may be configured to temporally store the generated one or more light settings in the memory, and the processor 106 may be configured to remove the stored light settings from the memory after the associated part of the media content has been rendered by the media rendering device 120.
The media content may be video content (e.g. a movie, a video clip, a news broadcast, etc.). The video content may be streamed from a video streaming platform. The processor 106 may be configured to analyze the video content and generate the one or more light settings by extracting color information, saturation information and/or brightness information from the video content. The processor 106 may be configured to analyze images/scenes of the video content to extract color, saturation and/or brightness levels from the images, and control the lighting device 110 based on these levels. Techniques for extracting such information from video content is known in the art, and will therefore not be discussed in detail.
The media content may be audio content (e.g. a song, an audiobook, etc.). The audio content may be streamed from a music streaming platform. The processor 106 may be configured to analyze the audio content and generate the one or more light settings based on one or more audio characteristics of the audio content. The audio characteristics may, for example, comprise one or more of: beat, timbre, pitch, intensity, rhythm, major and minor key. Additionally, or alternatively the one or more audio characteristics may comprise audio features. The audio features may comprise direct mood, valence and arousal/energy. Direct mood may be estimated using set of mood labels (which may be user-defined or
automatically generated). A combination of valence and arousal may be used for defining mood.
The media content may be video game content. The video game content may be streamed from a video game streaming platform (e.g. a cloud gaming platform). The processor 106 may be configured to analyze the video game content and generate the one or more light settings by extracting color information, saturation information and/or brightness information form the game content. Additionally or alternatively, the processor 106 may be configured to generate the one or more light settings based on one or more game events of the game content. Similar to the above-mentioned video analysis, images/scenes of the video game content may be analyzed to extract color, saturation and/or brightness levels from the images, and the one or more lighting devices may be controlled based on these levels.
Additionally or alternatively, (predefined) game events of the game content may be accessible, and these (predefined) game events (e.g. explosions, change of scenery, opening of a door, etc.) may be analyzed by the processor 106. Color, saturation and/or brightness levels may, for example, be extracted from the game events, and the processor 106 may control the lighting device 110 accordingly. Alternatively, game events may be associated with predefined light settings, and the processor may control the lighting device 106 based on the predefined light settings.
The processor 106 may be further configured to switch back from the second control mode to the first control mode upon determining that the stream of the light script has become available again. Figs. 3a shows schematically a timeline t of playback of media content. At tO , the processor 106 may start in the first control mode, wherein the lighting device 110 is controlled based on the light script 5. Upon determining that the light script is interrupted/unavailable (at tl ), the processor 106 may switch to the second control mode, wherein the processor 106 analyzes the media content to generate the light settings based thereon, and wherein the processor 106 controls the lighting device 110 based on the light settings a automatically generated based on the media content. When the stream of the light script has become available again (at /2), the processor 106 may switch back from the second control mode to the first control mode, wherein the lighting device 110 is controlled again based on the light script 5.
The processor 106 may be further configured to transition from the first mode to the second mode over time. Fig. 3b illustrates a transitional period tp after the stream of the light script has become available again. The processor 106 may therefore be configured to buffer a plurality of upcoming parts of the media content, and to analyze the one or more upcoming parts of the media content to identify a transition from a first type of upcoming parts to a second type of upcoming parts. The processor 106 may be configured to switch back to the first control mode when the second type of upcoming parts are rendered on the media rendering device. This enables switching back to the first when the type of content rendered on the media rendering device 130 changes substantially. If, for example, a cutscene event in a video game or a fade-to-black event in a video is upcoming, the switching back from the second mode to the first mode may occur when such an event occurs (i.e. after the transitional period tp).
The processor 106 may be further configured to start in the second control mode when the media content is being rendered on the media rendering device 120. In embodiments wherein the media content is streamed, the stream of the media content may be available from the remote server 130, whereas the stream of the light script may not yet be available from the remote server 130. In such cases, it may be beneficial if the processor 106 starts in (or switches to) the second control mode when the light script is not (yet) available. Fig. 3c illustrates an example wherein the processor 106 starts (at tO ) in the second control mode, wherein the processor 106 controls the lighting device 110 based on the light settings a automatically generated based on the media content. When the stream of the light script becomes available (at tl ), the processor 106 may switch to the first control mode, wherein the lighting device 110 is controlled based on the light script. When the stream of the light script is later interrupted (at 12), the processor 106 may again switch to the second control mode. When the stream of the light script becomes available again (at t3), the processor 106 may switch back to the first control mode.
The controller 102 may comprise a communication unit 108 configured to (wirelessly) communicate lighting control commands to the lighting device 110. The lighting control commands may comprise lighting control instructions for the lighting device 110 to control its light output according to the light settings of the light script, or according to the light settings generated based on the media content. The communication unit 108 may comprise hardware for transmitting the lighting control commands via any wired or wireless communication protocol. Various wired and wireless communication protocols may be used, for example Ethernet, DMX, DALI, USB, Bluetooth, Wi-Fi, Li-Fi, 3G, 4G, 5G or ZigBee. In embodiments wherein the controller 102 is comprised in the lighting device 110, or wherein both the controller 102 and the lighting device 110 are comprised in the media rendering device 130 (as illustrated in Fig. 3b), the communication unit 108 may be an output of the controller 102 for directly controlling the lighting device 110.
The receiver 104 may be further configured to receive an incoming stream of the media content. The processor 106 may use this stream for analyzing the media content. Additionally, the processor 106 may be configured to stream the media content to the media rendering device 120, for instance via the communication unit 108 or via a different communication unit. In embodiments wherein the controller 102 is comprised in the media rendering device 120 (see Figs. 2a and 2b), the processor 106 may be configured to receive the incoming stream of the media content directly from the media rendering device 120, or from an auxiliary device connected to the media rendering device 120.
The media rendering device 120 may be any type of device configured to communicate the media content to the user. The media rendering device 120 may, for example, be a television, a tablet pc, a smartphone, a (home) audio system, a pc, etc. In embodiments wherein the controller 102 is comprised in the media rendering device 120, the processor 106 of the controller 102 may be a processor of the media rendering device 120. In embodiments wherein the controller 102 is comprised in a separate device (such as a smartphone, a bridge device, a pc, a set-top box, etc.), the processor 106 may be configured to stream the media content to the media rendering device 120. In embodiments, the media rendering device 120 may be configured to receive the stream of the media content from the remote server 130, and the processor 106 of the separate controller 102 may be configured to receive the same stream, or at least information about the streamed media content (e.g. timing information, color information, audio characteristic information, etc.), and the processor 106 may analyze the media content and generate the light settings based thereon.
The lighting device 110 comprises one or more light sources (e.g. LED light sources) and a control unit, such as a microcontroller (not shown), for controlling the light output generated by the light sources based on lighting control commands received from the processor 106 of the controller 102. A lighting control command may comprise lighting control settings for controlling the light output. The lighting control settings may relate to light properties such as the color, intensity, saturation, beam size, beam shape, etc. according to which the lighting device 110 is to be controlled. The lighting control settings may be indicative of a dynamic light scene, wherein one or more light properties change over time.
The remote server 130 (and optionally the second remote server) may be part of an online media content streaming platform (e.g. a video streaming platform, a
music/audiobook streaming platform, a video game streaming platform, etc.).
The remote server 130 (and optionally the second remote server) may be accessible via a network, for instance via the internet. In embodiments, a first remote server may be configured for streaming the light script to the controller 102, and the second remote server may be configured for streaming the media content (to the controller 102 and/or to the media rendering device 120). The remote server 130 may be a light script streaming platform for streaming light settings to the controller 102, whereupon the processor 106 of the controller 102 may control the lighting device 110 based on the light settings. The processor 106 may be configured to synchronize the light script streamed from the second server with the media (streamed) content. Synchronization of light scripts and media content from different sources is known in the art, and will therefore not be discussed in detail.
Fig. 4 shows schematically a method 400 of controlling one or more lighting devices 110 based on media content rendered on a media rendering device 120. The method comprises: streaming 404 a light script associated with the media content from the remote server, controlling 406, in a first control mode, the one or more lighting devices according to the light script when the light script is available from the remote server, switching 408 to a second control mode upon determining that a stream of the light script has been interrupted, analyzing 410 the media content to generate one or more light settings based on the media content, and controlling 412, in the second mode, the one or more lighting devices according to the generated one or more light settings. The step of analyzing 510 may be performed before the switching step 408, for instance during, before or after steps 404 and 406. The method 400 may further comprise the step of streaming 402 the media content from the remote server to the media rendering device.
The method 400 may further comprise the step of switching 414 back to the first control mode upon determining that the stream of the light script has become available.
The method 400 may be executed by computer program code of a computer program product when the computer program product is run on a processing unit of a computing device, such as the processor 106 of the controller 102.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.
In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer or processing unit. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Aspects of the invention may be implemented in a computer program product, which may be a collection of computer program instructions stored on a computer readable storage device which may be executed by a computer. The instructions of the present invention may be in any interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs) or Java classes. The instructions can be provided as complete executable programs, partial executable programs, as modifications to existing programs (e.g. updates) or extensions for existing programs (e.g. plugins). Moreover, parts of the processing of the present invention may be distributed over multiple computers or processors or even the‘cloud’.
Storage media suitable for storing computer program instructions include all forms of nonvolatile memory, including but not limited to EPROM, EEPROM and flash memory devices, magnetic disks such as the internal and external hard disk drives, removable disks and CD-ROM disks. The computer program product may be distributed on such a storage medium, or may be offered for download through HTTP, FTP, email or through a server connected to a network such as the Internet.

Claims

CLAIMS:
1. A method (400) of controlling one or more lighting devices based on media content rendered on a media rendering device, the method (400) comprising:
streaming (404) a light script associated with the media content from a remote server,
controlling (406), in a first control mode, the one or more lighting devices according to the light script when the light script is available from the remote server,
switching (408) to a second control mode upon determining that a stream of the light script has been interrupted,
analyzing (410) the media content to generate one or more light settings based on the media content, and
controlling (412), in the second mode, the one or more lighting devices according to the generated one or more light settings.
2. The method (400) of claim 1, further comprising:
switching (414) back to the first control mode upon determining that the stream of the light script has become available.
3. The method (400) of claim 2, further comprising:
synchronizing the stream of the light script with the media content.
4. The method (400) of claim 3, wherein the step of switching back to the first control mode is executed after the stream of the light script has been synchronized with the media content.
5. The method (400) of claim 2 or 3, further comprising the step of:
analyzing the one or more upcoming parts of the media content to identify a transition from a first type of upcoming parts to a second type of upcoming parts, and wherein the step of switching back to the first control mode is executed when the second type of upcoming parts are rendered on the media rendering device.
6. The method (400) of any preceding claim, wherein the step of analyzing the media content to generate one or more light settings based on the media content is executed while controlling the one or more lighting devices in the first control mode.
7. The method (400) of any of claims 1 to 5, wherein the step of analyzing the media content to generate one or more light settings based on the media content is executed upon switching to the second control mode.
8. The method (400) of any preceding claim, wherein the media content is video content, and wherein the step of analyzing the media content to generate one or more light settings based on the media content comprises: generating the one or more light settings by extracting color information, saturation information and/or brightness information from the video content.
9. The method (400) of any preceding claim, wherein the media content is audio content, and wherein the step of analyzing the media content to generate one or more light settings based on the media content comprises: generating the one or more light settings based on one or more audio characteristics of the audio content.
10. The method (400) of any preceding claim, wherein the media content is game content, and wherein the step of analyzing the media content to generate one or more light settings based on the media content comprises: generating the one or more light settings by extracting color information, saturation information and/or brightness information form the game content, and/or generating the one or more light settings based on one or more game events of the game content.
11. The method (400) of any preceding claim, wherein the media content is streamed from the remote server or from a second remote server to the media rendering device (120).
12. A computer program product for a computing device, the computer program product comprising computer program code to perform the method (400) of any one of claims 1-10 when the computer program product is run on a processing unit of the computing device.
13. A controller (102) for controlling one or more lighting devices (110) based on media content rendered on a media rendering device (120), the controller (102) comprising:
a receiver (104) for streaming a light script associated with the media content from a remote server (130),
a processor (106) configured to control, in a first control mode, the one or more lighting devices (110) according to the light script when the light script is available from the remote server (130),
wherein the processor (106) is further configured to switch to a second control mode upon determining that a stream of the light script has been interrupted, to analyze the media content to generate one or more light settings based on the media content, and to control, in the second mode, the one or more lighting devices (110) according to the generated one or more light settings.
14. A lighting system comprising the controller (102) of claim 13, and the one or more lighting devices (110) configured to be controlled by the controller.
15. A media rendering device (120) comprising the controller (102) of claim 13, wherein the media rendering device (120) is configured to render the media content on the media rendering device (120).
PCT/EP2020/050759 2019-01-21 2020-01-14 A controller for controlling a lighting device based on media content and a method thereof Ceased WO2020151993A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP19152685 2019-01-21
EP19152685.4 2019-01-21

Publications (1)

Publication Number Publication Date
WO2020151993A1 true WO2020151993A1 (en) 2020-07-30

Family

ID=65138838

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2020/050759 Ceased WO2020151993A1 (en) 2019-01-21 2020-01-14 A controller for controlling a lighting device based on media content and a method thereof

Country Status (1)

Country Link
WO (1) WO2020151993A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023012154A1 (en) * 2021-08-06 2023-02-09 Signify Holding B.V. A controller and a method for controlling lighting units over time based on media content

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030057884A1 (en) * 1997-12-17 2003-03-27 Dowling Kevin J. Systems and methods for digital entertainment
WO2007113738A1 (en) * 2006-03-31 2007-10-11 Koninklijke Philips Electronics, N.V. Combined video and audio based ambient lighting control
WO2007144813A2 (en) * 2006-06-13 2007-12-21 Koninklijke Philips Electronics N.V. Fingerprint, apparatus, method for identifying and synchronizing video

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030057884A1 (en) * 1997-12-17 2003-03-27 Dowling Kevin J. Systems and methods for digital entertainment
WO2007113738A1 (en) * 2006-03-31 2007-10-11 Koninklijke Philips Electronics, N.V. Combined video and audio based ambient lighting control
WO2007144813A2 (en) * 2006-06-13 2007-12-21 Koninklijke Philips Electronics N.V. Fingerprint, apparatus, method for identifying and synchronizing video

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023012154A1 (en) * 2021-08-06 2023-02-09 Signify Holding B.V. A controller and a method for controlling lighting units over time based on media content

Similar Documents

Publication Publication Date Title
EP1967005B1 (en) Script synchronization using fingerprints determined from a content stream
CN106257930B (en) Generate the dynamic time version of content
US20140288686A1 (en) Methods, systems, devices and computer program products for managing playback of digital media content
US11510300B2 (en) Determinning light effects based on video and audio information in dependence on video and audio weights
US20050226601A1 (en) Device, system and method for synchronizing an effect to a media presentation
EP3804471B1 (en) Selecting one or more light effects in dependence on a variation in delay
WO2017132832A1 (en) Method, electronic device, and system for immersive experience
KR20090019830A (en) Fingerprints, apparatus and methods for identifying and synchronizing video
CN111096078B (en) Method and system for creating light script of video
WO2020089150A1 (en) Selecting a method for extracting a color for a light effect from video content
CN111727608B (en) Storage medium, content playing method and content playing system
US20170249971A1 (en) Method for generating image in which sound source is inserted and apparatus therefor
US20240323482A1 (en) A controller and a method for controlling lighting units over time based on media content
GB2579488A (en) Platform for control in synchronization with music and control method therefor
EP3288344A1 (en) A method of controlling lighting sources, corresponding system and computer program product
US20220295135A1 (en) Video providing system and program
WO2023274700A1 (en) A controller for controlling a plurality of lighting devices based on media content and a method thereof
KR101811366B1 (en) Media playe program
JP7736433B2 (en) Systems and methods for facilitating selective interaction presentation
HK40031785B (en) Storage medium, content reproduction method, and content reproduction system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20700594

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20700594

Country of ref document: EP

Kind code of ref document: A1