WO2023198548A1

WO2023198548A1 - Subtitle based lighting control

Info

Publication number: WO2023198548A1
Application number: PCT/EP2023/058944
Authority: WO
Inventors: Nam Chin Cho; Tarun Dass MATHUR; Kaitlin Burke; Parth JOSHI
Original assignee: Signify Holding B.V.
Priority date: 2022-04-12
Filing date: 2023-04-05
Publication date: 2023-10-19

Abstract

A method of controlling lighting includes receiving, by a media device, media content that includes video content and subtitle data. The method further includes providing, by the media device, the video content to a display device. The method also includes providing, by the media device, lighting command information to a lighting system that comprises one or more light fixtures. The one or more light fixtures are controllable based on the lighting command information. The lighting command information is obtained or generated from the subtitle data.

Description

Subtitle based lighting control

TECHNICAL FIELD

The present disclosure relates generally to lighting solutions, and more particularly to using subtitle data of media content for lighting control.

BACKGROUND

In some cases, movie viewing experiences are based on a two-dimensional display. Lighting provided by lighting fixtures in a video viewing area is unrelated to events in a video. In general, a video viewing area, such as a theater, is kept dark during the movie watching experience. Even when a video watching area is not kept dark, such as during watching a video on a television, the lighting provided by light fixtures in the viewing area (e.g., a living room) is often not coordinated with the movie being watched. In some cases, residential lighting may be controlled by analyzing TV signals to determine colors in scenes in a transmitted video and to control residential light fixtures to emulate the colors in the scenes. However, the lighting provided by the light fixtures is limited to colors in the scenes of the video and may not adequately represent the lighting including colors at a location shown in the video. Thus, a solution that enhances video viewing experience may be desired.

SUMMARY

The present disclosure relates generally to lighting solutions, and more particularly to using subtitle data of media content for lighting control. In an example embodiment, a method of controlling lighting includes receiving, by a media device, media content that includes video content and subtitle data. The method further includes providing, by the media device, the video content to a display device. The method also includes providing, by the media device, lighting command information to a lighting system that comprises one or more light fixtures. The one or more light fixtures are controllable based on the lighting command information. The lighting command information is obtained or generated from the subtitle data.

In another example embodiment, a non-transitory tangible computer-readable storage medium includes instructions that are executable by a processor to control lighting. The instructions include receiving media content that includes video content and subtitle data. The instructions further include providing the video content to a display device and providing lighting command information to a lighting system that includes one or more light fixtures. The lighting command information is obtained or generated from the subtitle data, and the one or more light fixtures are controllable based on the lighting command information.

These and other aspects, objects, features, and embodiments will be apparent from the following description and the appended claims.

BRIEF DESCRIPTION OF THE FIGURES

Reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:

FIG. 1 illustrates a system for controlling light fixtures based on subtitle data embedded in media content according to an example embodiment;

FIG. 2 illustrates media content received by the media device of FIG. 1 according to an example embodiment;

FIG. 3 illustrates details of the format of the subtitle data of FIG. 2 according to an example embodiment;

FIG. 4 illustrates subtitle data according to an example embodiment;

FIG. 5 illustrates the media device of FIG. 1 according to an example embodiment;

FIG. 6 illustrates the lighting controller of FIG. 1 according to an example embodiment;

FIG. 7 illustrates a method of controlling lighting based on subtitle data contained in received media content according to an example embodiment;

FIG. 8 illustrates a system for controlling light fixtures based on a lighting sequence provided by a cloud server according to an example embodiment; and

FIG. 9 illustrates a method of controlling lighting based on a lighting sequence received from a cloud server according to an example embodiment.

The drawings illustrate only example embodiments and are therefore not to be considered limiting in scope. The elements and features shown in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the example embodiments. Additionally, certain dimensions or placements may be exaggerated to help visually convey such principles. In the drawings, the same reference numerals used in different drawings may designate like or corresponding but not necessarily identical elements.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

In the following paragraphs, example embodiments will be described in further detail with reference to the figures. In the description, well known components, methods, and/or processing techniques are omitted or briefly described. Furthermore, reference to various feature(s) of the embodiments is not to suggest that all embodiments must include the referenced feature(s).

FIG. 1 illustrates a system 100 for controlling light fixtures based on subtitle data embedded in media content according to an example embodiment. In some example embodiments, the system 100 may include a media content consumer area 102 and a media content provider area 104. For example, the consumer area 102 may be a residential or other space where a person may typically view video content received from a remote location, and the provider area 104 may be a live event space such as a sport arena, a concert area, etc. Media content may be transmitted from the provider area 104 to the consumer area 102 via internet 106. For example, a game, a concert, and other events may be transmitted from the provider area 104 to the consumer area 102 as media content. To illustrate, the media content may be streamed to the consumer area 102 or may otherwise be transmitted as broadcast media context, for example, via satellite or cable.

In some example embodiments, a network device 108 at the consumer area 102 may be communicably coupled to the internet 106 and may receive media content from the provider area 104 through the internet 106. For example, the network device 108 may include a gateway device and/or a router device. A media device 110 may be located at the consumer area 102 and may receive the media content from the provider area 104 through the network device 108. An audio-video output device 112 may be connected to the media device 110 to display video and to output sound contained in the received media content. For example, the audio-video output device 112 may include one or more devices including a television or another display device and speakers.

In some example embodiments, a lighting controller 114 and light fixtures 118-126 may be at the consumer area 102. For example, the lighting controller 114 may control the operation of the light fixtures 118-126. The light fixtures 118-126 may each be a downlight, a track lighting fixture, a recessed light fixture, light strips, and/or another type of light fixture. The lighting controller 114 may transmit lighting control commands to the light fixtures 118-126 individually, to a subset of the light fixtures 118-126, or to all of the light fixtures 118-126. For example, the light fixtures 118-126 may each be turned on or off based on a respective command from the lighting controller 114. The lights provided by the light fixtures 118-126 may also be adjusted based on commands from the lighting controller 114. For example, the intensity, the color, the color temperature, etc. of the light provided by each one of the light fixtures 118-126 may be adjusted in response to respective one or more lighting control commands from the lighting controller 114.

In some example embodiments, the network device 108, the lighting controller 114, a user device 116, and the light fixtures 118-126 may communicate wirelessly using wireless signals such as wireless signals 136 that are complaint with one or more communication standards such as Wi-Fi, ZigBee, Bluetooth, etc. For example, the media device 110 and the lighting controller 114 may communicate with or through the network device 108 using Wi-Fi wireless signals. To illustrate, the media device 110 may communicate with the lighting controller 114 wirelessly through the network device 108. The lighting controller 114 may communicate (e.g., transmit lighting control commands) to the light fixtures 118-126 wirelessly. For example, the lighting controller 114 may communicate with the light fixtures 118-126 via the network device 108 using Wi-Fi wireless signals or may communicate directly with the light fixtures 118-126 using ZigBee wireless signals as can be readily understood by those of ordinary skill in the art. Alternatively or in addition, the lighting controller 114 may communicate with one or more of the light fixtures 118-126 via a wired connection (e.g., CAT 5 cable, CAT 5e cable, CAT 6 cable).

In some example embodiments, each individual light fixture 118-126 may be identifiable based on the location of the light fixture 118-126. For example, the light fixture 118 may be a front right light fixture, the light fixture 120 may be a front left light fixture, the light fixture 122 may be a rear left light fixture, the light fixture 124 may be a rear center light fixture, and the light fixture 126 may be a rear right light fixture. Alternatively, the lights provided by the light fixtures 118-126 may be referenced based on the locations of the light fixtures 118-126. For example, the light provided by the light fixture 118 may be a front right light, the light provided by the light fixture 120 may be a front left light, the light provided by the light fixture 122 may be a rear left light, the light provided by the light fixture 124 may be a rear center light, and the light provided by the light fixture 126 may be a rear right light.

In some example embodiments, subsets of the light fixtures 118-126 may be identifiable in groups. For example, the light fixtures 118, 120 may be in a front light fixture group, the light fixtures 122-126 may be in a rear light fixture group, the light fixtures 120, 122 may also be in a left light fixture group, the light fixtures 118, 126 may also be in a right light fixture group, and the light fixture 124 may be in a center light fixture group. The light(s) provided by one or more light fixtures in a particular group may also be referenced in a similar manner.

In some example embodiments, the lighting controller 114 may control the light fixtures 118-126 based on programmed settings as can be readily understood by those of ordinary skill in the art. For example, the light controller 114 may be HUE BRIDGE or another lighting controller. The lighting controller 114 may also control the light fixtures 118-126 based on user inputs. For example, the lighting controller 114 may control the light fixtures 118-126 based on lighting command information from the user device 116. For example, the user device 116 may be a mobile phone that executes a lighting control software application. Alternatively, the user device 116 may be a smart speaker or another device that can wirelessly communicate with the lighting controller 114, for example, directly or through the network device 108. The user device 116 may send the lighting command information to the lighting controller 114, and the lighting controller 114 may control the light fixtures 118- 126 based on the received lighting command information as can be readily understood by those of ordinary skill in the art. For example, the lighting controller 114 may generate, from the received lighting command information, lighting control commands that are compatible with the light fixtures 118-126 and may send the lighting control commands to the light fixtures 118-126 to control the lighting in the consumer area 102.

In some example embodiments, the lighting controller 114 in the consumer area 102 may also control the light fixtures 118-126 based on lighting control information contained in the media content received from the media content provider area 104. For example, the media content provided from the provider area 104 may include subtitle data containing lighting control information that can be used to control the light fixtures 118-126. To illustrate, an audio-video system 128 in the provider area 104 may capture audio and video of events in the provider area 104. For example, the audio-video system 128 may include one or more camcorders that capture audio and video of live events (e.g., sport games, concerts, etc.) in the provider area 104. The audio-video system 128 may provide the captured audio and video (i.e., audio-video content) to a transmitter 134 at the provider area 104. The transmitter 134 may include the captured audio-video content in the media content and transmit the media content to the consumer area 102. In some example embodiments, the audio-video system 128 may provide separate audio content and video content to the transmitter 134, and the transmitter 134 may include the audio content and/or video content in the media content and transmit the media content to the consumer area 102.

In some example embodiments, luminaires (i.e., light fixtures) 132 may provide lighting for the provider area 104. A lighting controller 130 in the provider area 104 may control the luminaires 132 to control the lighting in the provider area 104. To illustrate, the lighting controller 130 may provide lighting control commands to the luminaires 132 to turn on and off lights provided by the luminaires 132 and to adjust intensity, color, color temperature, etc. of the lights provided by the luminaires 132. For example, the lighting controller 130 may be a DMX lighting controller or another type of lighting controller. The transmitter 134 may receive the lighting control commands provided to the luminaires 132 by the lighting controller 130. The transmitter 134 may encode the lighting control commands as subtitle data and include the subtitle data in the media content along with the audio-video content, the audio content, or the video content, and transmit the media content to the consumer area 102.

In some example embodiments, the subtitle data generated by the transmitter 134 may include lighting control information that can be used to control the lights provided by the light fixtures 118-126 such that the lighting at the consumer area 102 is similar to the lighting at the provider location (i.e., at the provider area 104). As another example, the subtitle data may include lighting control information that can be used to control the lights provided by the light fixtures 118-126 to have particular characteristics (e.g., intensity, color, color temperature, etc.) based on the lighting control commands provided to the luminaires 132 by the lighting controller 130. The transmitter 134, which may include a processor to perform operations described herein, may encode the lighting control commands provided to the luminaires 132 as subtitle data with reference to generic light fixtures at the consumer area 102 as explained below in more detail with respect to FIGS. 2-4. The lighting control information may include timing information that indicates the relationship between the lighting control commands to the luminaires 132 and events in the provider area 104 as captured in the audio content, the video content, or the audio-video content.

In some alternative embodiments, instead of providing the lighting control commands to the transmitter 134, the lighting controller 130 may provide to the transmitter 134 information corresponding to the lighting control commands, and the transmitter 134 may encode the information from the lighting controller 130 as subtitle data and may transmit to the consumer area 102 the media content that includes the subtitle data as well as the audio-video content, the audio content, or the video content from the audio-video system 128. The information from the lighting controller 130 may refer to generic light fixtures at the consumer area 102. Alternatively, the transmitter 134 may encode the information from the lighting controller 130 as the subtitle data with reference to generic light fixtures at the consumer area 102.

In some example embodiments, the media device 110 at the consumer area 102 may receive, for example via the network device 108, the media content transmitted by the transmitter 134 and may provide the media content to the audio-video output device 112. Alternatively, the media device 110 may extract (i.e., obtain) audio-video content from the media content and provide the audio-video content to the audio-video output device 112. Alternatively, the media device 110 may extract video content from the media content and provide the video content to the audio-video output device 112.

In some example embodiments, the media device 110 may extract (i.e., obtain) the subtitle data from the received media content. For example, the media device 110 may process the lighting control information contained in the subtitle data to generate and send to the lighting controller 114 lighting command information. The lighting control information in the subtitle data may include timing information, light fixture identifier information, one or more lighting control commands, and/or other information that may be used to control the light fixtures 118-126. To illustrate, the lighting command information generated by the media device 110 may be similar to the lighting command information sent to the lighting controller 114 by the user device 116. The media device 110 may execute or otherwise use one or more application programming interfaces (APIs) and/or software modules that translate the lighting control information contained in the subtitle data into lighting command information that are usable by the lighting controller 114 to control the light fixtures 118-126. The lighting controller 114 may receive the lighting command information from the media device 110 and control the light fixtures 118-126 accordingly. For example, the lighting controller 114 may generate, from the received lighting command information, lighting control commands and send the lighting control commands to the light fixtures 118-126 to turn on and off lights provided by the light fixtures 118-126 and to adjust intensity, color, color temperature, and/or other characteristics of the lights.

In some alternative embodiments, the media device 110 may send to the lighting controller 114 the lighting control information contained in the subtitle data without processing the lighting control information to generate lighting command information. The lighting controller 114 may receive the lighting control information from the media device 110 and may control the light fixtures 118-126 accordingly. For example, the lighting controller 114 may translate the received lighting control information into lighting control commands that are sent to the light fixtures 118-126 to control the lights provided by the light fixtures 118-126.

In some example embodiments, the media device 110 may use timing information contained in the received subtitle data to control the timing of sending the video content to the audio-video output device 112 and the timing of sending the lighting command information to the lighting controller 114. For example, the media device 110 may control the timing such that the lighting providing by the light fixtures 118-126 based on the lighting command information is coordinated with the video content displayed by the audio-video output device 112. For example, the timing information contained in the received subtitle data may be referenced to the beginning of the video content, to events in the video content (e.g., indicated by markers), etc.

By using the subtitle data contained in the media content received from the provider area 104, the system 100 may enable lighting similar to the lighting associated with events (e.g., live events) at the provider area 104 to be provided at the consumer area 102 in coordination with video of the events. By coordinating the timing of the display of the video content on the audio-video output device 112 with the timing of the lighting provided by the light fixtures 118-126, a consumer in the consumer area 102 may have an immersive viewing experience.

In some alternative embodiments, more or fewer light fixtures may be located at the consumer area 102. In some alternative embodiments, the system 100 may include devices other than those located at the areas 102, 104 without departing from the scope of this disclosure. In some alternative embodiments, some of the devices at the consumer area 102 may be integrated in a single device. For example, the media device 110 and the audiovideo output device 112 may be integrated in a single device. In some alternative embodiments, some of the devices (e.g., the network device 108, the lighting controller 114, and the light fixtures 118-126) at the consumer area 102 may be connected to other devices in a different configuration than shown in FIG. 1 without departing from the scope of this disclosure. In some alternative embodiments, some of the devices at the provider area 104 may be connected to other devices in a different configuration or manner (e.g., wired or wirelessly) than shown in FIG. 1 without departing from the scope of this disclosure. In some alternative embodiments, some of the devices at the provider area 104 may be integrated in a single device. For example, the transmitter 134 and the lighting controller 130 may be integrated in a single device. In some alternative embodiments, the transmitter 134 may send the media content to the consumer area 102 via a communication path other than the internet 106 without departing from the scope of this disclosure. In some alternative embodiments, the media content transmitted by the transmitter 134 may be previously recorded media content that is generated in the same manner as described above.

FIG. 2 illustrates media content 200 including subtitle data 204 received by the media device 110 of FIG. 1 according to an example embodiment, and FIG. 3 illustrates details of the format of the subtitle data 204 of FIG. 2 according to an example embodiment. Referring to FIGS. 1-3, in some example embodiments, the media content 200 may be transmitted to the media device 110 by the transmitter 134 at the provider area 104. The media content 200 may include audio-video content 202 and subtitle data 204. As described above, the audio-video content 202 may include audio and video of an event (e.g., a live event) at the provider area 104 as captured by the audio-video system 128 and provided to the transmitter 134, and the transmitter 134 may encode the lighting control commands provided to the luminaires 132 as the subtitle data 204.

In some example embodiments, the subtitle data 204 includes lighting control information that may be used to control the light fixtures 118-126 at the consumer area 102 as described above. For example, the subtitle data 204 may include timing information 206, light fixture identifier information 208, and lighting action information 210. The timing information 206 may include time values that indicate a start-time and an end-time for a light adjustment or operation indicated by the lighting action information 210 to be performed. The timing information 206 may include, for example, time values with reference to the beginning of the video content included in the audio-video content 202. Alternatively or in addition, the timing information 206 may include time values with reference to events in the video content. The media device 110 may interpret the timing information 206 and control the timing of the transmission of the media content 200, the audio-video content 202, or the video content contained in the media content 200 to the audio-video output device 112 and the timing of the transmission of lighting command information generated from the subtitle data 204 to the lighting controller 114.

In some example embodiments, the light fixture identifier information 208 may generically identify light fixtures provided by light fixtures. For example, the light fixture identifier information 208 may generically refer to light fixtures based on the locations of the light fixtures in a generic room or space. To illustrate, the light fixture identifier information 208 may include an identifier such as Left, Right, Center, Rear, Front Left, Front Right, Rear Left, Rear Right, etc., where one or more light fixtures may be referred to using a particular identifier. For example, Left may refer to one or more light fixtures that are on the left side of a space or a room such as the consumer area 102. The light fixture identifier information 208 may alternatively generically refer to the light(s) provided by the light fixtures 118-126 to refer to the light fixtures 118-126. For example, the light fixture identifier information 208 may include an identifier such as Left Lights, Right Lights, etc. The media device 110 that receives the media content 200 including the subtitle data 204 may be configured to interpret the light fixture identifier information 208 as referring to particular ones of the light fixtures 118-126 in the consumer area 102. To illustrate, when the light fixture identifier information 208 is Left, the media device 110 may interpret Left as referring to the light fixtures 120, 122. As another example, the media device 110 may interpret Right Lights as referring to the lights provided by the light fixtures 118, 126.

In some example embodiments, the lighting action information 210 may specify one or more light actions or operations. For example, the lighting action information 210 may specify light actions or operations such as Transition Color, Transition Intensity, Transition Color Temperature, etc., where each action/operation is followed by one or more parameters.

In some alternative embodiments, the subtitle data 204 may include additional information without departing from the scope of this disclosure. In some alternative embodiments, the subtitle data 204 may include less information than shown without departing from the scope of this disclosure. In some alternative embodiments, the audiovideo content 202 may include video content only or audio content only without departing from the scope of this disclosure. In some alternative embodiments, the information and content included in the media content 200 may be arranged in a different format than shown without departing from the scope of this disclosure.

FIG. 4 illustrates the subtitle data 204 of FIG. 2 according to an example embodiment. Referring to FIGS. 1-4, in some example embodiments, the subtitle data 204 includes multiple lighting control sequence lines such as sequence lines 402, 404. For example, the sequence line 402 may include elements such as a start-time 406 and an endtime 408 that, for example, correspond to the timing information 206 shown in FIG. 2. To illustrate, the start-time 406, going left to right, may include 00 that indicates hour, 12 that indicates minutes, 11 that indicates seconds, and 020 that indicates milliseconds, all with respect to the beginning of the video content in the audio-video content 202 of FIG. 2. The end-time 408 has a similar format as the start-time 406 and may also indicate time with respect to the beginning of the video content in the audio-video content 202 of FIG. 2. In some example embodiments, the start-time 406 and the end-time 408 are associated with a light fixture identifier 410 (Center Lights), a lighting action 412 (Transition Color), and a lighting action 418 (Transition Intensity) included in the sequence line 402 of the subtitle data 204. For example, the media device 110 may interpret Center Lights as referring to the light fixture 124 in FIG. 1. To illustrate, the light fixture identifier information 208 shown in FIG. 2 may include the identifier 410. The light action information 210 shown in FIG 2 may include the lighting actions 412, 418 together with their respective parameters 414, 416 and 420, 422.

In some example embodiments, the start-time 406 may indicate a time at which the lighting fixture identified by the light fixture identifier 410 (Center Lights) should start changing the color of the light provided by the particular light fixture from the color specified by the parameter 414 to the color specified by the parameter 416. The start-time 406 may also indicate the time at which the lighting fixture should start changing the intensity level of the light from a level (e.g., 0) specified by a parameter 420 to a level (e.g., 150) specified in a parameter 422. The end-time 408 may indicate the time at which the lighting fixture is to finish changing the color of the light to the color specified by the parameter 416. The end-time 408 may also indicate the time at which the lighting fixture is to finish changing the intensity level of the light to the level specified in the parameter 422. The parameters 414, 416 for the lighting action 412 (Transition Color) may be specified as R, G, B levels ranging from 0 to 255, chromaticity (X, Y), hue and saturation or other color space values. The parameters 420, 422 for the lighting action 418 (Transition Intensity) may be specified, for example, using values ranging from 0 to 255.

In some example embodiments, the sequence line 404 and other sequence lines of the subtitle data 204 may be similar to the sequence line 402. For example, the sequence line 404 may apply to the same light fixture indicated by the identifier 410 of the sequence line 402. The sequence line 404 may also include start-time and end-time that specify different time values from the time values specified by the start-time 406 and the endtime 408 of the sequence line 402.

In some example embodiments, each sequence line of the subtitle data 204 shown in FIG. 4 may include more or fewer lighting actions. For example, the lighting action information 210 that includes the line actions may include one or more other lighting actions such as Transition On, Transition Off, Transition CCT (correlated color temperature), etc. In some alternative embodiments, the start-time and the end-time of the timing information 206 may specify the same time value, for example, to indicate immediate action/operation. In some alternative embodiments, the start-time and the end-time may be replaced by a single time. In some example embodiments, the lighting action information 210 may have a single parameter or no parameter without departing from the scope of this disclosure. In some alternative embodiments, the subtitle data 204 may have different formats, elements, actions, order, parameters, etc. than shown in FIGS. 2-4 without departing from the scope of this disclosure.

FIG. 5 illustrates the media device 110 of FIG. 1 according to an example embodiment. Referring to FIGS. 1-5, in some example embodiments, the media device 110 may include a controller 502, a memory device 504, a communication interface unit 506, and an audio-video interface unit 508. The memory device 504 may include one or more memory devices such as an EEPROMs, static random access memory device, a flash memory, and/or another type of memory device. For example, software code and data may be stored in the memory device 504. The controller 502 may execute the software code to perform operations described herein with respect to the media device 110 such as extracting and/or processing the subtitle data 204. For example, the controller 502 may include one or more microprocessors and/or microcontrollers that can execute the software code. The controller 502 may also use data stored in the memory device 504 during the execution of the software code stored and may store generated or received data in the memory device 504. During the operations of the media device 110, the controller 502 may also store in the memory device 504 the media content 200 received from the provider area 104 or portions of the media content 200.

In some example embodiments, the controller 502 may use the communication interface unit 506 to communicate, wirelessly and/or via one or more wired connections, with the other devices in the consumer area 102. To illustrate, the communication interface unit 506 may include one or more transmitters and/or receivers that use signals compatible with one or more wireless network standards such as Bluetooth, Wi-Fi, and/or ZigBee and/or wired communication standard such as Ethernet. For example, the controller 502 may receive the media content 200 from the transmitter 134 through the network device 108 via the communication interface unit 506. As another example, the controller 502 may communicate with the lighting controller 114 using the communication interface unit 506. To illustrate, the controller 502 may transmit to the lighting controller 114, via the communication interface unit 506, the lighting control information obtained or generated by the controller 502 from the subtitle data 204 contained in the media content 200. The controller 502 may also perform other operations described herein with respect to the media device 110. In some example embodiments, the media device 110 may communicate with the audio-video output device 112 via the audio-video interface unit 508. For example, the audio-video interface unit 508 may be connected to the audio-video output device 112 using a cable such as an HDMI cable. To illustrate, the controller 502 may use the audio-video interface unit 508 to provide the audio-video content 202, video content, or audio content received in the media content 200 to the audio-video output device 112.

In some alternative embodiments, the media device 110 may include other or different components than shown in FIG. 5 without departing from the scope of this disclosure. In general, the memory device 504 may be a non-transitory tangible computer readable storage medium. In some alternative embodiments, some of the components of the media device 110 may be integrated into a single component. In some alternative embodiments, the components of the media device 110 may be connected in a different configuration than shown in FIG. 5 without departing from the scope of this disclosure. In some example embodiments, the media device 110 may be a set-top box device such as a ROKU device or another type of device.

FIG. 6 illustrates the lighting controller 114 of FIG. 1 according to an example embodiment. Referring to FIGS. 1-4 and 6, in some example embodiments, the lighting controller 114 may include a controller 602, a memory device 604, a communication interface unit 606, and a lighting control interface unit 608. The memory device 604 may include one or more memory devices such as an EEPROMs, static random access memory device, a flash memory, and/or another type of memory device. For example, software code and data may be stored in the memory device 604. The controller 602 may execute the software code to perform operations described herein with respect to the lighting controller 114. For example, the controller 602 may include one or more microprocessors and/or microcontrollers that can execute the software code. The memory device 604 may also use data stored in the memory device 604 during the execution of the software code stored and may store generated or received data in the memory device 604. The controller 602 may also store data received from the media device 110 in the memory device 604.

In some example embodiments, the controller 602 may use the communication interface unit 606 to communicate, wirelessly and/or via one or more wired connections, with the other devices in the consumer area 102. To illustrate, the communication interface unit 606 may include one or more transmitters and/or receivers that use signals compatible with one or more wireless network standards such as Bluetooth, Wi-Fi, and/or ZigBee and/or wired communication standard such as Ethernet. For example, the controller 602 of the lighting controller 114 may receive, via the communication interface unit 606, the lighting control information from the media device 110 through the network device 108. As another example, the controller 602 may communicate with one or more of the light fixtures 118-126 via the communication interface unit 606. The controller 602 may process the lighting control information received from the media device 110 and perform other operations described herein with respect to the lighting controller 114.

In some example embodiments, the lighting controller 114 may communicate with the light fixtures 118-126 via the lighting control interface unit 608. For example, the lighting control interface unit 608 may use signals compatible with one or more wireless network standards such as Bluetooth, Wi-Fi, and/or ZigBee and/or wired communication standard such as Ethernet. To illustrate, the controller 602 may send lighting control commands to the light fixtures 118-126 via the lighting control interface unit 608.

In some alternative embodiments, the lighting controller 114 may include other or different components than shown in FIG. 6 without departing from the scope of this disclosure. In general, the memory device 604 may be a non-transitory tangible computer readable storage medium. In some alternative embodiments, some of the components of the lighting controller 114 may be integrated into a single component. In some alternative embodiments, the components of the lighting controller 114 may be connected in a different configuration than shown in FIG. 6 without departing from the scope of this disclosure.

FIG. 7 illustrates a method 700 of controlling lighting based on the subtitle data 204 contained in the media content 200 according to an example embodiment. Referring FIGS. 1-7, in some example embodiments, the method 700 includes, at step 702, receiving, by the media device 110, the media content 200 that includes the video content (e.g., video content included in the audio-video content 202) and the subtitle data 204. For example, the media content 200 may be streamed media content of a live event and/or recorded event at the provider area 104. At step 704, the method 700 may include extracting, by the media device 110, the subtitle data 204 from the media content 200. For example, the media device 110 may extract the subtitle data 204 to process the lighting control information, such as the timing information 206, the light fixture identifier information 208, and the lighting action information 210 contained in the subtitle data 204 and generate lighting command information from the lighting control information. At step 706, the method 700 may include providing, by the media device 110, the video content to a display device (e.g., the audiovideo output device 112). In some example embodiments, at step 708, the method 700 includes providing, by the media device 110, lighting command information to a lighting system that includes one or more light fixtures (e.g., a lighting system that includes the lighting controller 114 and the light fixtures 118-126). For example, the light fixtures 118-126 may be controllable based on the lighting command information. The lighting command information may be obtained or generated from the subtitle data 204 as described above. To illustrate, the media device 110 may obtain or generate the lighting command information from the lighting control information such as the timing information 206, the light fixture identifier information 208, and the lighting action information 210 contained in the subtitle data 204. The method 700 may also include transmitting, by the lighting controller 114, one or more lighting control commands to the one or more light fixtures (e.g., the light fixtures 118-126), where the one or more lighting control commands may include or may be generated from the lighting command information received from the media device 110.

In some alternative embodiments, the method 700 may include more or fewer steps than shown without departing from the scope of this disclosure. In some alternative embodiments, the steps of the method 700 may be performed in a different order than shown without departing from the scope of this disclosure.

FIG. 8 illustrates a system 800 for controlling light fixtures based on a lighting sequence provided by a cloud server 806 according to an example embodiment. In some example embodiments, the system 800 includes the media content consumer area 102 and a media content provider area 802. As described above with respect to FIG. 1, the consumer area 102 may be a residential or other space where a person may typically view video content received from a remote location, and the provider area 802 may be a live event space such as a sport arena, a concert area, etc. Media content that includes video content and/or audio content may be transmitted from the provider area 802 to the consumer area 102 via the internet 106. For example, a game, a concert, and other events may be transmitted from the provider area 802 to the consumer area 102 as media content. To illustrate, the media content may be streamed to the consumer area 102 or may otherwise be transmitted as broadcast media context, for example, via satellite or cable.

In some example embodiments, luminaires (i.e., light fixtures) 812 may provide lighting for the provider area 802. A lighting controller 814 in the provider area 802 may control the luminaires 812 to control the lighting in the provider area 802. To illustrate, the lighting controller 814 may provide lighting control commands to the luminaires 812 to turn on and off lights provided by the luminaires 812 and to adjust intensity, color, color temperature, etc. of the lights provided by the luminaires 812. For example, the lighting controller 814 may be a DMX lighting controller or another type of lighting controller.

In some example embodiments, an audio-video system 816 in the provider area 802 may capture audio and video of events in the provider area 802. For example, the audio-video system 816 may include one or more camcorders that capture audio and video of live events (e.g., sport games, concerts, etc.) in the provider area 802. The audio-video system 816 may provide the captured audio and video (i.e., audio-video content) to a transmitter 810 at the provider area 802. The transmitter 810 may transmit to the consumer area 102 the audio-video content as media content. At the consumer area 102, the media device 110 may receive the media content, for example through the internet 106 and the network device 108 and may provide the media content to the audio-video output device 112. In some example embodiments, the transmitter 810 may transmit the audio content or video content in the media content.

In some example embodiments, an event manager device 804 at the provider area 802 may provide a trigger event indicator to the transmitter 810 that transmits to the cloud server 806 the trigger event indicator or another message generated from the trigger event indicator. For example, a trigger event indicated by a trigger event indicator may be a celebration event such as a basket during a basketball game, a goal during a soccer game, a touchdown during a football game, etc. The event manager device 804 may be manually operated by a person that provides an input to the event manager device 804 to indicate a trigger event. As a non-limiting example, the event manager device 804 may have a user interface, where a person can press different buttons to indicate different trigger events. In some alternative embodiments, a person may provide the trigger event indicator to the transmitter 810 that transmits the trigger event indicator to the cloud server 806.

In some example embodiments, the cloud server 806 may send to the lighting controller 114 at the consumer area 102 a particular lighting sequence corresponding to a trigger event indicated by the trigger event indicator or by a message received from the transmitter 810. For example, the cloud server 806 may store lighting sequences 808 associated with a particular subscriber account. As a non-limiting example, the lighting sequence shown in FIG. 4 as subtitle data 204 may be a lighting sequence of the lighting sequences 808. For a particular subscriber account, individual lighting sequences of the lighting sequences 808 may be associated with respective trigger events that may be indicated by the event manager device 804. For a particular subscriber account, sets of lighting sequences of the lighting sequences 808 may be associated with respective activities (e.g., basketball games, football games, music concerts, etc.), and a trigger event indicator may be associated with different activities. The cloud server 806 may store lighting sequences 808 in a memory device, such as a flash memory device, etc.

In some example embodiments, the lighting controller 114 may control the light fixtures 118-126 based on a lighting sequence received from the cloud server 806. For example, the lighting controller 114 may translate or otherwise generate from the lighting sequence one or more lighting control commands that can be used to control the light fixtures 118-126. To illustrate, the controller 602 of the lighting controller 114 shown in FIG. 6 may translate or otherwise generate from the lighting sequence one or more lighting control commands. The controller 602 may send the one or more commands to the light fixtures 118- 126, for example, via the lighting control interface unit 608 of the lighting controller 114. As described above, the controller 602 may execute the software code stored in the memory device 604 to perform operations described herein with respect to the lighting controller 114.

In the absence of a trigger event indicator or message from the transmitter 810, the cloud server 806 may send a default lighting sequence to the lighting controller 114. For example, the default lighting sequence may be specific to the activity at the provider area 804 that is streamed or otherwise broadcast to the media device 110 and shown on the audiovideo output device 112. In some alternative embodiments, instead of sending a default lighting sequence, the cloud server 806 may indicate to the lighting controller 114 to control the light fixtures 118-126 based on a default light sequence stored on the lighting controller 114. Alternatively, the lighting controller 114 may control the light fixtures 118-126 based on default settings, other user inputs, etc. if no lighting sequence is indicated by or received from the cloud server 806.

In some example embodiments, a subscriber (i.e., a person) may sign in to the cloud server 806 and indicate a particular activity (e.g., basketball games, football games, music concerts, etc. at the provider area 802 or another location), for example, using the user device 116. For example, the user device 116 may send an activity indicator to the cloud server 806 based on a user input to the user device 116 selecting or indicating an activity that is shown or will be shown on the audio-video output device 112. In response, the cloud server 806 may communicate with the event manager device 804 or the transmitter 810 at the provider area 802 so that the event manager device 804 or the transmitter 810 provides trigger event indicators or messages to the cloud server 806 as trigger events occur. The lighting controller 114 may be associated with the subscriber, for example, during a subscriber signing up process such that the cloud server 806 is configured to send lighting sequences associated with the particular subscriber to the lighting controller 114. For example, the subscriber signing up process may be performed through the lighting controller 114, and the lighting controller 114 may be automatically associated with the particular subscriber.

Because the lighting sequences from the cloud server 806 are associated with trigger events at the provider area 802 and because the trigger events are events in an activity (e.g., a game at the provider area 802) that is shown on the audio-video output device 112 as provided from the provider area 802, characteristics (e.g., intensity, color, CCT, etc.) of the light at the consumer area 102 may by reasonably synchronized with the trigger events. By controlling the lighting at the consumer area 102 based of the lighting sequences from the cloud server 806, a person at the consumer area 102 may have a more immersive viewing experience.

In some example embodiments, the lighting sequences 808 may reflect the lighting at the provider area 802 as provided by the luminaires 812. Alternatively, the lighting sequences 808 may result in the light fixtures 118-126 providing a lighting at the consumer area 102 that is different from or independent of the lighting provided by the luminaires 812. In some alternative embodiments, more or fewer light fixtures may be located at the consumer area 102. In some alternative embodiments, the system 800 may include devices other than those located at the areas 102, 802 without departing from the scope of this disclosure. In some alternative embodiments, some of the devices at the consumer area 102 may be integrated in a single device. For example, the media device 110 and the audio-video output device 112 may be integrated in a single device. In some alternative embodiments, some of the devices (e.g., the network device 108, the lighting controller 114, and the light fixtures 118-126) at the consumer area 102 may be connected to other devices in a different configuration than shown in FIG. 8 without departing from the scope of this disclosure. In some alternative embodiments, some of the devices at the provider area 802 may be connected to other devices in a different configuration or manner (e.g., wired or wirelessly) than shown in FIG. 8 without departing from the scope of this disclosure. In some alternative embodiments, some of the devices at the provider area 802 may be integrated in a single device. For example, the transmitter 810 and the lighting controller 814 may be integrated in a single device. In some alternative embodiments, the transmitter 810 may send the media content to the consumer area 102 via a communication path other than the internet 106 without departing from the scope of this disclosure. In some alternative embodiments, the media content transmitted by the transmitter 810 may be a previously recorded media content that is generated in the same manner as described above.

FIG. 9 illustrates a method 900 of controlling lighting based on a lighting sequence received from a cloud server 806 according to an example embodiment. Referring FIGS. 8 and 9, in some example embodiments, the method 900 includes, at step 902, sending, by the lighting controller 114 at the consumer area 102, an activity identifier indicating a live event at a remote location (e.g., the provider area 802). For example, the consumer may already be watching or may be planning to watch the particular activity on the audio-video output device 112. The consumer may use the user device 116 to sign in to the cloud server 806 through the lighting controller 114 and to indicate an activity (e.g., a game, a concert, etc.) at the provider area 802. Alternatively, the consumer may use the user device 116 to send the activity identifier to the cloud server 806 without going through the lighting controller 114.

In some example embodiments, after receiving the activity indicator, the cloud server 806 may indicate the selected activity to the event manager device 804. Upon occurrences of trigger events during the activity at the provider area 802, the event manager device 804 may send one or more trigger event indicators to the cloud server 806 via the transmitter 810, and the cloud server 806 may send to the lighting controller 114 one or more lighting sequences of the lighting sequences 808 associated with the subscriber and the trigger events indicated by the trigger event indicators. At step 904, the method 900 may include receiving, by the lighting controller 114, the one or more lighting sequences from the cloud server 806. At step 906, the method 900 may include controlling, by the lighting controller 114, the one or more light fixtures (e.g., the light fixtures 118-126) at the consumer area 102 based on the one or more lighting sequences.

In some example embodiments, the cloud server 806 may send to the lighting controller 114 one or more default lighting sequences that are not associated with trigger events indicated by the event manager device 804. For example, when no trigger event indicators are received by the cloud server 806 from the event manager device 804, the cloud server 806 may send to the lighting controller 114 one or more default lighting sequences associated with the particular subscriber and/or activity at the provider area 802.

In some alternative embodiments, the method 900 may include more or fewer steps than shown without departing from the scope of this disclosure. In some alternative embodiments, the steps of the method 900 may be performed in a different order than shown without departing from the scope of this disclosure. Although particular embodiments have been described herein in detail, the descriptions are by way of example. The features of the example embodiments described herein are representative and, in alternative embodiments, certain features, elements, and/or steps may be added or omitted. Additionally, modifications to aspects of the example embodiments described herein may be made by those skilled in the art without departing from the scope of the following claims, the scope of which are to be accorded the broadest interpretation so as to encompass modifications and equivalent structures.

Claims

CLAIMS:

1. A method (700) of controlling lighting, the method comprising: receiving (702), by a media device (110), media content (200), the media content comprising: video content (202); and subtitle data (204), wherein subtitle data includes lighting command information as subtitle data; providing (706), by the media device, the video content to a display device (112); and providing (708), by the media device, the lighting command information of the subtitle data to a lighting system that comprises one or more light fixtures (118-126), wherein the lighting command information is obtained from the subtitle data (204) and wherein the one or more light fixtures (118-126) are controllable based on the lighting command information obtained from the subtitle data.

2. The method of Claim 1, wherein the lighting command information of the subtitle data (204) includes one or more lighting control commands comparable to lighting effects performed by one or more luminaires (132) at a live event area (104) that is different from an area (102) of location of the one or more light fixtures (118-126), wherein the one or more light fixtures (118-126) generate one or more lighting actions based on the one or more lighting commands that are comparable to one or more light effects at the live event area.

3. The method of Claim 1, wherein the subtitle data (204) comprises timing information (206) for controlling the one or more light fixtures (118-126), wherein the timing information include at least one time value that indicate a start-time for a lighting action.

4. The method of Claim 3, wherein the subtitle data (204) comprises lighting fixture identifier information (208) for identifying the one or more lighting fixtures (118-

5. The method of Claim 3, wherein the timing information (206) indicates one or more time values in reference to a beginning of the video content.

6. The method of Claim 1, further comprising extracting (704), by the media device (110), the subtitle data (204) from the media content (200) to retrieve the lighting command information contained in the subtitle data.

7. The method of Claim 1, further comprising transmitting, by a lighting controller (114), one or more lighting control commands to the one or more light fixtures (118-126) and wherein the one or more lighting control commands include or are generated from the lighting command information.

8. The method of Claim 1, wherein the media content (200) is streamed media content.

9. A non-transitory tangible computer-readable storage medium (504) comprising instructions that are executable by a processor control lighting, the instructions comprising: receiving media content (200), the media content comprising: video content (202); and subtitle data (204), wherein subtitle data includes lighting command information as subtitle data; providing the video content (202) to a display device (112); and providing the lighting command information of the subtitle data to a lighting system that comprises one or more light fixtures (118-126), wherein the lighting command information is obtained from the subtitle data (204) and wherein the one or more light fixtures (118-126) are controllable based on the lighting command information obtained from the subtitle data.

10. The storage medium of Claim 9, wherein the subtitle data (204) comprises timing information (206) for controlling the one or more light fixtures (118-126) based on the one or more lighting control commands, wherein the timing information include at least one time value that indicate a start-time for a lighting action.

11. The storage medium of Claim 10, wherein the subtitle data (204) comprises lighting fixture identifier information (208) identifying the one or more lighting fixtures.

12. The storage medium of Claim 10, wherein the timing information (206) indicates one or more time values in reference to a beginning of the video content.

13. The storage medium of Claim 9, wherein the lighting command information of the subtitle data (204) includes one or more lighting commands comparable to one or more lighting control commands provided to one or more luminaires (132) at a live event area (104) that is different from an area (102) of location of the one or more light fixtures (118-

126), wherein the one or more light fixtures (118-126) generate one or more lighting actions based on the one or more lighting commands that are comparable to one or more light effects at the live event area.