CN109699102B - System for synchronizing light emission control signal and light emission pattern of light emission effect device - Google Patents
System for synchronizing light emission control signal and light emission pattern of light emission effect device Download PDFInfo
- Publication number
- CN109699102B CN109699102B CN201811193945.4A CN201811193945A CN109699102B CN 109699102 B CN109699102 B CN 109699102B CN 201811193945 A CN201811193945 A CN 201811193945A CN 109699102 B CN109699102 B CN 109699102B
- Authority
- CN
- China
- Prior art keywords
- color control
- color
- lighting
- lighting effect
- interactive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- H05B45/00—Circuit arrangements for operating light-emitting diodes [LED]
- H05B45/10—Controlling the intensity of the light
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- H05B45/00—Circuit arrangements for operating light-emitting diodes [LED]
- H05B45/20—Controlling the colour of the light
Landscapes
- Circuit Arrangement For Electric Light Sources In General (AREA)
Abstract
An interactive lighting effect control system for synchronizing lighting effect patterns of a remote interactive lighting effect device is disclosed. The synchronous luminous effect can be remotely generated while watching the luminous effect performance by using other interactive luminous effect devices which emit light according to the original ground of the live playground. The remotely obtained synchronized lighting effect can generate corresponding virtual analog perception when watching the real-time streaming video, as if the user were present in the same concert venue. Due to the color control signal generation method and the use of the color control pattern mixing module for generating a mixed video frame comprising color control patterns, the delay of the remote generation of a variation of a lighting effect with respect to the lighting effect observed in live video at a concert venue is low, enabling a remote generation of a highly efficient lighting effect pattern.
Description
Technical Field
The invention relates to a system for synchronizing luminous effect patterns and luminous effect control signals of an interactive luminous effect device; in particular, to a method and system for synchronizing a lighting effect pattern and a lighting effect control signal of a remote interactive lighting effect device while watching a streaming video live broadcast of a live performance event at a first location to generate a synchronized dynamic interactive lighting effect and to achieve virtual simulation perception as if attending the same concert venue.
Background
Interactive light-emitting effect devices, such as LED light-emitting wristbands, are very common portable electronic devices that are commonly used to achieve large-scale interactive light-emitting effects in various venues, such as concerts, sporting events, mass gatherings, educational activities, political gatherings, educational institution gatherings, and the like, that produce large-scale continuous light-emitting visual effects when operated in a coordinated light-emitting pattern. The interactive lighting effect devices (such as LED lighting wristbands and handheld LED lighting sticks) can be remotely controlled wirelessly by radio frequency signals broadcast by a wireless radio frequency transmitter, a lighting controller and proprietary control software arranged in a notebook computer or a personal computer. In order to allow more people who also wear LED light wristbands but are located at other different (remote) locations to also (virtually) achieve a simulated perception or experience of the same light effects performance as those participants who also wear LED light wristbands but are located at live event venues (i.e. actual concert participants wear LED light wristbands and sit in seats at the concert venue), various conventional interactive light effects control systems have been developed in the marketplace. Therefore, people who cannot attend the concert field in person can still watch the live streaming video through the live streaming video of the concert, and enjoy the same synchronous light-emitting effect experience through the LED light-emitting wrists worn by the people. For example, in U.S. patent application publication No. 20140184386 (published: 2014/07/03), Jason Charles Regler et al discloses a conventional method for synchronizing a light effect pattern at a remote viewing location with a live concert venue, which explains that in some embodiments, the circuitry within the LED light emitting wristbands may include: a timer/clock coupled to the controller and the timer/clock may be used to synchronize lighting effects such that the programmed clock may be used to synchronize lighting output from the plurality of LED lighting wristbands. Also, a battery isolation tag may be pulled out when entering the venue, and the controller of the LED light bracelet may provide a preset flashing sequence to inform the user that the LED light bracelet on the user's hand is functional. Thus, the above-mentioned patent publication describes a time-based trigger within an LED light-emitting wristband and a suitable crystal for allowing a user to view a concert through a television or computer relay and experience a perceived association (i.e., a user viewing video on a remotely located television screen can experience a perception of a virtual presence in the venue of the concert). However, the above-mentioned conventional method for synchronizing the remote viewing position with the luminous effect pattern of the live concert venue has several disadvantages: (1) once unexpected time delay occurs at the beginning stage of a concert field, the clock of each LED light-emitting wrist band cannot adjust the time delay by itself, which may cause the light-emitting effect to be triggered at wrong time/interval; since a time delay at the concert site is expected or predicted, additional time is required to calibrate or adjust the clock of each LED illuminated wristband in advance before the concert begins, which takes some time to do; (2) although the user may be notified to pull the battery isolation label to start the timer for synchronizing the lighting effects at the beginning of the concert, since the actual running time of each part of the light effect script of the concert sometimes changes dynamically, it is almost impossible to perform completely according to the predetermined schedule of each part of the light effect script of the concert during all the actual running times, so once the dynamic changes are introduced, the lighting effects generated on all the LED lighting wristbands will no longer be performed in perfect chronological order; (3) any time delay that occurs in the actual playing of the video, such as: either a broadcast interruption or a video stream buffering delay can cause a significant time difference between the clock on the LED light emitting wristband and the original default clock of the controller at the concert event.
There is therefore a need in the art for a more efficient and effective method and system for dynamically synchronizing lighting effect patterns and lighting effect control signals to control interactive lighting effect devices located at different locations to enable lighting effect performances across multiple zones to produce synchronized lighting effects across zones/zones without the audience at remote locations experiencing significant time delays as compared to the audience at the live arena.
Disclosure of Invention
The invention provides an interactive lighting effect control system, which is suitable for synchronizing a lighting effect pattern and a lighting effect control signal of an interactive lighting effect device at a remote position with a lighting effect pattern and a lighting effect control signal of an interactive lighting effect device at a field activity site at a first position.
The total number of the interactive lighting effect devices of the present invention may be one or more, and the interactive lighting effect control system may efficiently and effectively manage and process a large number of interactive lighting effect devices over a million.
The interactive light effect device of the present invention has at least one light emitting source, and the controller is configured to selectively illuminate the at least one light emitting source of the interactive light effect device in accordance with the received illumination color sequence data to provide for providing a continuous, automatic, coordinated lighting effect.
The received light-emitting color sequence data of the invention comprises a plurality of light-emitting diodes (LEDs) with preset light-emitting intensity and nested hierarchical partition codes required by the red, green and blue LEDs, and is arranged in the RF data pulse train transmitted according to the time sequence.
The invention provides a plurality of Light Emitting Diodes (LEDs) which at least have red, green and blue light emitting colors and are positioned in each portable light emitting device with interactive light emitting effect, wherein the light emitting intensity of the red, green and blue Light Emitting Diodes (LEDs) is respectively formed according to red, green and blue color codes (R, G, B) from 0 to 255.
The luminous state of the luminous source provided by the invention comprises on, off or flashing, and the luminous source is arranged in the portable luminous device with an interactive luminous effect.
The present invention provides a plurality of interactive light effects devices capable of receiving the same set of emission color and region assignment data to enable a plurality of light emitting sources to selectively produce light effects in accordance with the same emission color and region assignment data.
The present invention provides a zone code for assigning to a seat position within a seating area.
The set of emission color data provided by the present invention is comprised of the emission intensities of the red, green, and blue LEDs corresponding to the partition code.
The present invention provides a variety of interactive light effects devices implemented in a variety of different device configurations and configurations, such as a smartphone, an LED light emitting bracelet, an LED light emitting necklace, an LED light emitting bracelet, a light emitting headband, a pair of light emitting glasses, a set of LED gloves, or a handheld LED light emitting wand.
The present invention provides an interactive lighting effect control system suitable for use with a plurality of interactive lighting effect devices. The interactive lighting effect control system comprises a camera serving as a video recorder, a wireless transmitter, a storage, a processing unit and a lighting controller. In other embodiments, the interactive lighting effect control system may include a personal electronic device, such as:
PCs, laptop computers, wireless electronic devices, etc., which are classified as video recorders, color control pattern mixing modules, lighting controllers, wireless transmitters, and/or storage.
The light emitting controller generates a color control signal, wherein the color control signal includes a light emitting color and region allocation data. The lighting controller is a memory coupled to the wireless transmitter, the lighting controller transmits the color control signal to the wireless transmitter, and the wireless transmitter is an RF transmitter configured to broadcast the color control signal stored in a plurality of sequentially transmitted second RF data bursts. Upon verifying its authenticity, the at least one wireless receiver is configured to intercept and respond to the plurality of RF data bursts broadcast from the wireless transmitter.
The wireless transmitter of the present invention broadcasts redundant RF data bursts to the interactive light effects device in sequence at time tn and at time tn +1, which contains the same redundant partition code signal in successive RF data bursts to ensure data transmission integrity.
The method provided by the present invention comprises the step of using an interactive lighting effect control system to dynamically configure the lighting effect pattern of an interactive lighting effect device, wherein the interactive lighting effect device can be located at a venue for performance to produce a large scale of continuously dynamic lighting effects.
The method of the present invention is suitable for dynamically wirelessly configuring a partition code for each interactive lighting effect device via an RF data burst when authentication of the interactive lighting effect device is complete.
The method and system for synchronizing the luminous effect patterns and the luminous effect control signals of a plurality of interactive luminous effect devices at a plurality of different positions provided by the invention have the following advantages: achieving a low delay between lighting effect changes produced at a remote location relative to lighting effect changes in video frames of a current live streaming video at a concert venue; in other words, the lighting effect changes generated at the remote location can be successfully synchronized with the corresponding lighting effect changes seen at the current live streaming video at the concert venue; (II) because the existing equipment can easily realize the video recorder and the color control pattern mixing module for mixing the video images, the luminous effect pattern and the luminous effect control signal can be synchronized without additional equipment; and (III) the synchronous expression of the whole luminous effect can be ensured or promoted no matter how far the actual distance between the remote positions of the concert field is.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below. It is to be noted that the components in the attached drawings are merely schematic and are not shown in actual scale.
Drawings
FIG. 1 is a block diagram of a conventional interactive lighting effect control system that uses color control signal RF data bursts to synchronize lighting effect control signals and lighting effect patterns of multiple interactive lighting effect devices.
Fig. 2 is an illustrative example of a layout showing a partition code allocation arrangement for seating areas in an event floor according to a conventional method.
Fig. 3 is an illustrative example showing a partition code listed with respect to a specific seating area in the prior art.
FIG. 4 is a block diagram of an interactive lighting effect control system, which is adapted to synchronize the lighting effect patterns and lighting effect control signals of a plurality of interactive lighting effect devices at a second location with the lighting effect patterns and lighting effect control signals of a plurality of interactive lighting effect devices at a first location, respectively, according to an embodiment of the present invention.
FIG. 5 is a flowchart of a method for generating a color control pattern and a composite video frame to control the lighting state of an interactive lighting effect device according to the color control signal of the lighting controller according to an embodiment of the present invention.
FIG. 6 is a detailed flow chart of a method for synchronizing the lighting effect patterns and lighting effect control signals of a plurality of interactive lighting effect devices at a second location with the lighting effect patterns and lighting effect control signals of a plurality of interactive lighting effect devices at a first location, respectively, according to an embodiment of the invention, at different points in time.
FIG. 7 is a flow chart of a method for generating lighting effect pattern variations and lighting effect patterns for controlling an interactive lighting effect device at a remote location based on a color control signal vector, in accordance with one embodiment of the present invention.
Fig. 8A, 8B are a number of illustrative examples of color control patterns, according to embodiments of the invention.
Fig. 9 is a flowchart of a method of extracting a color control signal including color values of a color region from a color control pattern using an algorithm according to an embodiment of the present invention.
Fig. 10A, 10B are schematic diagrams of processing time schemes (including delay times/time lags) of a method for synchronizing a light effect pattern and a light effect control signal according to an embodiment of the present invention.
FIG. 11 is a flowchart illustrating a process for controlling the lighting state of the interactive lighting effects device at a remote location using a color control pattern extraction module to extract color control signals from color control patterns in a composite video frame.
FIG. 12 is a block diagram of an interactive lighting effect control system of the present invention that is adapted to synchronize the lighting effect pattern and the lighting effect control signal of an interactive lighting effect device at a remote location with the lighting effect pattern and the lighting effect control signal of an interactive lighting effect device at a venue location.
List of references to the drawings
100 interactive lighting effect control system
10 Interactive light effect device
LED1 red LED
LED2 green LED
LED3 blue light emitting diode
R, R' Red color value
G. G' Green color value
B. B' blue color value
ZC zone code
15 radio transmitter
18 luminous controller
20 first position
25 color control pattern mixing module
35 color control pattern extraction module
40 second position
45 wireless control module
50 video recorder (Camera)
tn, tn +1 time point
time points t1, t 2', t3
time points of t4, t5 ', t6 ', t7 ' and t8
Time difference of Δ t 1' and Δ t2
Time differences of Δ t3 ', Δ t4 ', Δ t5 ' and Δ t6
S1, S2, S3 and S4 steps
S100, S105, S110, S115, S120, S125, S130, S135, S140
S160, S165, S170, S175 and S180
S200, S205, S210, S215, S220
Detailed Description
To achieve the above objects and advantages, the present invention provides a technical means and a structure, which will be described in detail with respect to the preferred embodiments of the present invention as follows for a complete understanding.
In the following text, several terms or expressions will be used throughout the invention, which have the following definitions, for example:
the synchronized lighting effect control signal has the following definition: when sequentially generating a first-row lighting effect control signal according to a concert script to wirelessly control interactive lighting effect devices held by viewers at the local location of the live concert a, viewers at remote locations and also holding interactive lighting effect devices can also perceive, by watching a streaming video of the live concert a, that a lighting effect control change of the interactive lighting effect devices in their hands occurs at approximately the same time as a lighting effect change of the interactive lighting effect devices held by viewers at the local location of the live concert a in the streaming video.
The definition of the color control pattern is as follows: is based on an image pattern generated by a color control pattern generating unit. The color control pattern is for implementing a synchronous lighting effect control signal for controlling a remotely located interactive lighting effect device. The color control signal data may be mapped onto the color control pattern as color values of R, G, B. The color control pattern is then blended with the original video image displayed on the video frame of the live streaming video.
Reference is made to the disclosure of US patent publication No. 20170048951 (referred to herein as US-Pub20170048951), published as 2017, 16, which discloses an interactive lighting-effect portable lighting device (also referred to herein as an interactive lighting-effect device), an interactive lighting-effect control system, a plurality of LED lighting sources operatively responsive to changing lighting states, and wireless data transmission in the form of RF data bursts from the interactive lighting-effect control system, including lighting color and area assignment data, while the lighting color and area assignment sequence data file includes preset lighting intensities and nested hierarchical partition codes required for a plurality of red, green, and blue light-emitting diodes (LEDs), disposed in the RF data bursts transmitted in chronological order. A plurality of Light Emitting Diodes (LEDs) having at least red, green, and blue light emitting colors of light emitting sources, while various other teachings and techniques may be fully incorporated into embodiments of the present invention so as to be effectively adapted for use in various embodiments of the present invention.
The luminous state of the luminous source provided by the invention comprises on, off or flashing, and the luminous source is arranged in the portable luminous device with an interactive luminous effect.
The present invention provides a variety of interactive light effects devices implemented in a variety of different device configurations and arrangements, such as an LED light bracelet, an LED light necklace, or a hand-held LED light wand.
Based on the disclosure of U.S. patent publication No. 20170048951, the present invention provides an interactive lighting effects control system that is suitable for use with a plurality of interactive lighting effects devices. The interactive lighting effect control system includes a wireless transmitter having a memory, at least one lighting controller, and at least one wireless receiver. Wherein the at least one lighting controller generates a color control signal comprising a lighting color and a region allocation data, the lighting controller being coupled to the memory of the wireless transmitter, the color control signal being transmitted to the wireless transmitter, the wireless transmitter being an RF transmitter arranged to broadcast the color control signal in a plurality of sequentially transmitted RF data bursts. Upon verifying its authenticity, the at least one wireless receiver is configured to intercept and respond to the RF data burst broadcast from the wireless transmitter. Each RF data burst includes emission color and region assignment data for at least one wireless receiver having a memory storing at least one region indexed by one or more nested hierarchical partition codes. The wireless transmitter starts to send out a continuous broadcast signal at time point tn, that is, to send out a color control signal of one RF data pulse train to the interactive lighting effect device, and starts to send out a continuous broadcast signal of another color control signal of another RF data pulse train to the interactive lighting effect device at time point tn +1, after the interactive lighting effect device receives the color control signal in the RF data pulse train, the controller performs the identification confirmation of the identifier according to the lighting color and area distribution data contained in the RF data pulse train of the received color control signal, and checks whether the identifier of the calculated color control signal is correct, and then provides the interactive lighting effect device to generate a plurality of LED lighting color changes according to the color control signal, wherein the plurality of lighting colors are according to a plurality of lighting colors of a plurality of lighting sources in the interactive lighting effect device, the color codes comprising red, green, and blue (R, G, B) are defined lighting colors.
The wireless receiver provided by the invention is arranged in the interactive luminous effect device.
The present invention provides for broadcasting redundant RF data bursts to the interactive light-emitting effect device in sequence at time point tn and at time point tn +1, which comprise identical redundant color control signals in successive RF data bursts.
In an embodiment of the invention, the following features or resources are provided: the lighting status of the light emitting source includes on, off, or flashing, in other words, any number of red, green, and blue light emitting diodes LEDs (LEDs 1, LEDs 2, LEDs 3) may be turned on/on, turned off/off, or flashed on and off in a repeating pattern according to a specified frequency (e.g., twice per second). Open means that at least one of the color codes of R, G, B is not zero; off means R, G, B is set to zero in color code, or the dimmer is set to zero in color code; the color of the blinking on/off may be set through the corresponding color code data of R, G, B. Also, the flicker rate may be set by a default flicker rate value (e.g., two flashes per second) of a controller of the interactive light effect device, or a random number may be generated by the controller. The wireless data transmission from the wireless transmitter to the wireless receiver may be implemented by RF data bursts or by wireless data bursts under Wi-Fi, Bluetooth (Bluetooth), or ZigBee transmission technologies. The interactive light-emitting device may be an LED light-emitting wrist band, an LED light-emitting necklace, or a handheld LED light-emitting stick, but is not limited thereto, and may also be other types of light-emitting devices with wireless communication capability.
As shown in fig. 4, an interactive lighting effect control system 100 according to an embodiment of the present invention is adapted to synchronize lighting effect patterns and lighting effect control signals of a plurality of interactive lighting effect devices 10 at a second position 40 with lighting effect patterns and lighting effect control signals of a plurality of interactive lighting effect devices 10 at a first position 20, respectively, the interactive lighting effect control system 100 comprising: a video recorder 50, a lighting controller 18, a wireless transmitter 15, a color control pattern mixing module 25, a color control pattern extraction module 35, and a wireless control module 45, wherein the video recorder 50 includes a camera. Referring also to FIG. 1 (explained in more detail in US-Pub20170048951), in the above-described interactive lighting-effect control system 100, each interactive lighting-effect device 10 includes a storage, a wireless receiver, a controller, and at least one light source (LED1, LED2, LED3) disposed therein. The memory is an area for storing an index by a partition code within the color control signal RF data burst. The wireless receiver is used to acquire the RF data burst from the wireless transmitter. A controller is responsive to the RF data burst and is coupled to the memory. At least one light emitting source (LED1, LED2, LED3) is operatively responsive to the controller for changing its light state, wherein the at least one light emitting source includes a plurality of light emitting diodes (LED1, LED2, LED3), the LEDs 1, LED2, LED3 have at least red, green, and blue light emission colors, and the light emission intensities of the red, green, and blue LEDs are defined according to red, green, and blue codes R, G, B from 0 to 255, respectively. In the illustrated embodiment, the color control signal is generated and the current video frame is obtained at the first location 20, wherein the first location 20 may be a live action venue, the color control signal is obtained from the lighting change script of the live action venue and the current video frame, and the current video frame comprises an image of the plurality of lighted interactive lighting effect devices 10 at the first location 20. The color control signal is transmitted to the color control pattern mixing module 25 at the location 20 through a direct wire connection or through a wireless transmission of an RF data burst broadcast, and the color control pattern mixing module 25 is used to mix the color control pattern with the current video frame of the live action venue to generate a mixed video frame and transmit the mixed video frame to a live streaming server through the internet, and then transmit the mixed video frame to the color control pattern extraction module 35 at the second location 40, and the color control pattern extraction module 35 sequentially extracts the color control pattern and the color control signal from the mixed video frame to control the lighting state of the interactive lighting effect device 10 at the second location 40 to form a lighting effect pattern. As shown in fig. 1, the color control signal includes R, G, B data of a color code and a partition code. As shown in the examples of fig. 2 and 3, the zone code stored in the interactive light effects device 10 matches each zone (or seating location zone) in a venue activity. In the embodiment of fig. 4, the second location 40 may be one or more of the remote locations 40 broadcasting the mixed video frames, and the color control pattern extraction module 35 may be coupled to a wireless control module 45 at the second location 40 to wirelessly transmit the color control signal to the interactive light effects device 10 at the remote location 40. The lighting states of the light emitting sources in the interactive light effects device 10 include on, off, or blinking, respectively, which when illuminated together produce a change in lighting effect. In other embodiments (not shown), the Radio Frequency (RF) data burst may be replaced by a wireless data burst under Wi-Fi, Bluetooth (Bluetooth), or ZigBee transmission technology. In the embodiment of fig. 1 and 4, if the memory of two or more interactive light effects devices 10 are pre-configured with the same partition code, they are able to receive the same set of pattern related data for the light effects to cause the light emitting sources to selectively emit light according to the same light color data. The interactive lighting effect device 10 may be an LED lighting bracelet, an LED lighting necklace, an LED lighting bracelet, a lighting headband, a pair of lighting glasses, a set of LED gloves, or a hand-held LED lighting wand. Further, the interactive light effects device 10 may include a drone configured with LED lighting. Furthermore, the interactive light effects device 10 may also be a combination of various different types of devices, such as: a set of LED glow sticks and a set of LED drones may be located at one location 20 and/or the second location 40, respectively.
As shown in fig. 4, the video recorder 50 may be a conventional digital camera, such as: sony HDR is a digital camera, or a mobile device with a camera module. The color control pattern mixing module 25 includes the following components: (1) a color control pattern generation unit, which may be a desktop computer, a laptop computer or a mobile device for executing a pattern generation program; (2) a video image blending unit, for example: a Sony MCX-500 visual mixer, or a desktop, laptop or mobile device for executing an image blending program; (3) a reservoir (not shown); (4) an upload unit for compressing and uploading video to the internet may be implemented by a desktop computer, a laptop computer, or a mobile device. In other words, the color control pattern mixing module 25 is an electronic device or module adapted to generate a color control pattern according to a color control signal, mix the color control pattern with a video frame, and upload to the internet after the video mixing is completed. The color control pattern mixing module 25 includes a color control pattern generation unit, a video image mixing unit, and an upload unit, and is implemented by installing a set of computer programs in a desktop computer, a laptop computer, or a mobile device, and/or by adding a configuration algorithm to a driver source file of an embedded system, and/or by adding compiled code/programs to a microprocessor.
As shown in fig. 4, the wireless transmitter 15 may be the same as the wireless transmitter disclosed in US-Pub 20170048951. The color control pattern extraction module 35 includes the following components: (1) a download unit (not shown) which may be a desktop computer, a laptop computer or a mobile device for downloading video from the internet; (2) a video image extracting unit (not shown) may be a desktop computer, a laptop computer or a mobile device for executing the decompressed video program, and is used for executing the program for extracting the color values of the color control pattern. In other words, the color control pattern extraction module 35 is an electronic module or device adapted to download video image frames from the internet and extract color control patterns and color control signals to control the lighting state of the interactive lighting effect device 10 at the remote location 40. The color control pattern extraction module 35 includes a download unit and a video image extraction unit, and can be implemented by installing a set of computer programs on a desktop computer, laptop computer, or mobile device, and/or by adding a configuration algorithm to the source file of the driver for the embedded system, and/or by adding compiled code/program to the microprocessor. In the illustrated embodiment, the wireless control module 45 includes the following components: (1) a lighting controller (not shown), which may be a desktop computer, laptop computer, or mobile device, configured to receive color values extracted from the color control pattern and copy them into a data structure as a remotely located color control signal vector to control the interactive lighting effects device; and (2) a wireless transmitter (not shown) which may be the same as that disclosed in US-Pub 20170048951. An upload unit (not shown) of the color control pattern mixing module 25 is used to upload the mixed video frame to an online video streaming platform, for example: YouTube (www.youtube.com), Twitch (https:// www.twitch.tv /), etc., for a user to host a live concert at a remote location. To conserve bandwidth, these video frames are typically compressed prior to uploading to the online video streaming platform.
As shown in fig. 9, fig. 9 is a flowchart of a method for directly extracting color values from color control patterns in a current video frame (finding color values of each region by comparing colors of the whole scanned image to differences to find a color value of a center point of each color region, wherein the color control patterns have a uniform shape/structure, such as a four-region square shown in fig. 8A and 8B), according to an embodiment of the present invention, the method includes the following steps: in step S200, a full-screen image (PxQ pixels) of the current video frame is acquired by a video recorder 50 and stored in memory using software programs. In step S205, image pixels from (0,0) to (xn, yn) are scanned to check color values. For example, yn is set to zero (0) and xn is set from 0 to P for scanning. Then, yn is set to 1, and xn is set from 0 to P for scanning. During each scan, yn is incremented by 1 each time, and then the same procedure is repeated a number of times until after yn reaches Q, xn is set to increment from 0 to P for the scan. In step S210, when M/2 pixel colors are found to be substantially the same color in the scanning process (i.e., substantially the same color is defined as the color difference of the M/2 pixel colors being smaller than a threshold value d), the last image position of (xn, yn) can be found and recorded, i.e., the points (M/2,0), (M/2,1) … (M/2, N), (M,0), … (M, N) are recorded, respectively. In step S215, since the upper left point of the color control pattern is (0,0), the center point of the color control pattern falls within (M/2, N/2). In step S220, color values of the color regions are extracted by obtaining image position data located at the center of each color region, for example, in the embodiments of fig. 5 and 8A, the center image positions of the color regions are (M/4, N/4), (3M/4, N/4), (M/4,3N/4), (3M/4,3N/4), respectively.
In the illustrated embodiment, the extraction of color values for each color region by finding the center point of each color region is merely an example for the convenience of calculation, however, conventional techniques using pattern recognition algorithms can also be used to process color control patterns having irregular or uneven shapes, such as: a pattern recognition algorithm may be used to find the center of the color control pattern on the scanned video image. Since the color control pattern is pre-designed, the known relative position of the center point of the color control pattern and each color area of the color control pattern can be calculated or determined based on simple geometric relational equations according to the known size/dimension of the color control pattern. Then, the color data of each color region of the color control pattern is extracted. For the pattern recognition algorithm described above, the conventional open source algorithm described in the "feature detection and description part" of OpenCV (http:// docs. OpenCV. org/3.0-beta/doc/py _ components/py _ feature2d/py _ table _ of _ contents _ feature2d/py _ table _ of _ contents _ feature2d. html) can refer to the features of the object that is looking for an image to find the corresponding center point. These algorithms include SIFT, SURF, BRIEF, etc.
Referring to fig. 6, fig. 6 is a detailed flowchart of a method for synchronizing a light-emitting effect pattern and a light-emitting effect control signal according to an embodiment of the present invention, which includes the following steps. In step S100, a color control signal is generated by a lighting controller of a live action field located at a first position, wherein the color control signal is sequentially obtained from a lighting pattern change script of the live action field. In step S105, a current video frame is acquired at the live action venue located at the first location, wherein the current video frame contains an image of the live action venue having a plurality of lighted interactive light effect devices. In step S110, the color control signal is wirelessly broadcast as a plurality of repeated RF data bursts via a wireless transmitter, the color control signal is received by a color control pattern mixing module located at a first position, and a mixed video frame is generated by mixing the color control pattern with the current video frame of the live action venue using the color control pattern mixing module. In step S115, the mixed video frame is transmitted to a live streaming server through the internet. In step S120, the video player is used to download, decompress, play the mixed video frame, and determine the position of the color control pattern within the mixed video frame. In step S125, a color value is extracted from the color control pattern of the blended video frame and transmitted to the wireless control module located at the second position. In step S130, the color values are copied to a data structure to form a color control signal vector, and the color control signal vector is sent to the wireless control module at the second location. In step S135, the color control signal vector is broadcast via a wireless transmitter as a plurality of repeated RF data bursts from the wireless control module located at the second location 40. In step S140, the color control signal vector is received by the interactive light-emitting device at the second position, and the light-emitting variation is activated according to the color control signal vector to form a light-emitting effect pattern. In another embodiment, the step S110 can be replaced by the following steps: the color control signal is directly transmitted to a color control pattern mixing module located at a first location through a direct wire connection, and a mixed video frame is generated using the color control pattern mixing module, the mixed video frame including a color control pattern after mixing and a current video frame of the live event venue. In an embodiment of the invention, the color control pattern comprises at least one area within the image of the live action arena, and the color control pattern comprises a color data comprising color values of red (R), green (G), blue (B), wherein the color value of each color area of the color control pattern is in the range of 0 to 255, respectively.
Referring to fig. 10A-10B, at time t1, a color control signal is generated and a current video frame is obtained at a live action site at a first location 20, wherein the color control signal is sequentially obtained from a light pattern change script of the live action site, the current video frame includes an image of the live action site, and the live action site includes a plurality of light-emitting interactive light effect devices 10. In the illustrated embodiment, the color control signal is broadcast via the wireless transmitter 15 as a plurality of repeated RF data bursts to the color control pattern mixing module 25 at a location 20, and at time t 2', the color control pattern mixing module 25 is used to generate a blended video frame that includes a blended color control pattern, blank image, and current video frame of the live action venue (for further details regarding how the blended video frame and color control pattern are generated, please refer to fig. 5 as well). Then, at time t 3', the mixed video frame is uploaded to a live streaming server through the internet; at time point t 4', using the video player to download, decompress, play the mixed video frame, and determine the location of the color control pattern within the mixed video frame; at a point of time t 5', extracting color values from the color control pattern of the hybrid video frame and transmitting the color values to the wireless control module 45 located at the second location 40; at point in time t 6', copying the color values to a data structure to form a color control signal vector and sending the color control signal vector to the wireless control module 45 located at the second location 40; at time t 7', broadcasting the color control signal vector through the wireless transmitter as a plurality of repeated bursts of RF data from the wireless control module 45 at the second location 40; at the time point t 8', the color control signal vector is received by the interactive light effect device 10 at the second position 40, and the light emission variation is activated according to the color control signal vector to form the light emission effect pattern. According to the experimental results, a delay time (time lag) is defined as the sum of a time difference from a time point t4 'to a time point t 5' (Δ t3 '), a time difference from a time point t 5' to a time point t6 '(Δ t 4'), a time difference from a time point t6 'to a time point t 7' (Δ t5 '), and a time difference from a time point t 7' to a time point t8 '(Δ t 6'). In the method of synchronizing a light effect pattern and a light effect control signal of the present invention, determining or evaluating the delay time includes the following: the delay time from time point t4 'to time point t 5' is less than 3 seconds for a first lighting change of the color control signal vector, but less than 1 millisecond for a subsequent lighting change of the color control signal vector; the delay time from the time point t5 'to the time point t 6' is less than 1 millisecond, and the delay time from the time point t6 'to the time point t 7' is less than 1 millisecond for all subsequent lighting changes of the color control signal vector; the delay time from time point t7 'to time point t 8' is less than 5 milliseconds for all subsequent lighting changes of the color control signal vector. Further, the sum of the delay time from the time point t4 'to the time point t 5', the delay time from the time point t5 'to the time point t 6', the delay time from the time point t6 'to the time point t 7', and the delay time from the time point t7 'to the time point t 8' is less than 8 msec. However, the above example is only one embodiment, and the present invention is not limited thereto, and other embodiments with improved delay results also fall within the scope of the present invention. In addition, since the time point t4 ' with respect to the time point t3 ' may be flexibly selected according to the user's preference, the delay time is not defined or limited by the time points t1, t2 ' and/or t3 '. In other words, the user is free to decide when to watch the video at the remote location again, because it does not have to occur sequentially in the chronological order shown in fig. 10A.
Referring to fig. 5, fig. 5 is a flowchart illustrating a method for generating a color control pattern and a composite video frame to control the lighting state of an interactive lighting effect device according to the color control signal of the lighting controller, according to an embodiment of the present invention, which includes the following steps: in step S1, a color control signal is obtained from a light controller. In step S2, a color control pattern is generated (i.e., R is 0, G is 0, and B is 0). In step S3, each color region is filled with an individual color value (i.e., R, G, B) according to data from the color control signal (see step S3 in fig. 5 for more details). In step S4, the color control pattern is mixed with the background-removed blank image, and the mixed video frame is output to the video image mixing unit (see step S4 in fig. 5 for more details).
Referring to fig. 7, fig. 7 is a flowchart illustrating a method for generating a light effect pattern and a light effect control signal at a remote location according to an embodiment of the present invention, including the following steps S160-S180. In step S160, the light emission controller receives the color value (R, G, B value) of each color region from the color control pattern extraction module (compare to step S125 in fig. 6). In step S165, the data (i.e., R, G, B values and partition codes) is copied to a data structure as a color control signal vector at a remote location to control a plurality of interactive light effects devices (compare to step S130 in fig. 6). In step S170, the color control signal vector is transmitted to the wireless transmitter of the wireless control module at the remote location (compare to step S130 in fig. 6). In step S175, the color control signal vector is broadcast via RF data bursts in sequence from the wireless transmitter of the remotely located wireless control module (compare to step S135 in fig. 6). In step S180, the color control signal vector is received by the interactive lighting effect device at the remote location, and lighting variation is initiated according to the color control signal vector to form a lighting effect pattern (compare to step S140 in fig. 6).
Referring to fig. 8B (third example), fig. 8B is an illustrative example of a method of acquiring a color control pattern according to an embodiment of the invention, in which four color regions include a square shape, respectively including the following color values:
first color region R is 255, G is 255, and B is 255;
[ second color region ] R ═ 0, G ═ 0, and B ═ 0;
third color region R-128, G-128, B-128;
[ fourth color region ] R is 30, G is 30, and B is 30.
The first square color region sets R of the color control signal to 255, the second square color region sets G of the color control signal to 0, the third square color region sets B of the color control signal to 128, and the fourth square color region sets a partition code (ZC) of the color control signal to 30. Further, fig. 8A contains example 1 and example 2, showing other examples for obtaining a color control pattern.
It should be understood, however, that the description and drawings are only for the purpose of illustrating various embodiments of the present invention and are not to be construed as limiting the scope of the present invention.
Claims (11)
1. A method for synchronizing a light emitting effect pattern and a light emitting effect control signal, comprising:
step S1: generating a color control signal using a lighting controller of a field event venue located at a first location, wherein the color control signal is obtained from a lighting pattern variation script of the field event venue in a sequence;
step S2: obtaining a current video frame at the live action site at the first location, wherein the current video frame comprises an image of the live action site, and the current video frame comprises an image composed of a plurality of luminous interactive luminous effect devices of the live action site at the first location;
step S3: wirelessly broadcasting the color control signal as a plurality of repeating RF data bursts via a wireless transmitter,
receiving the color control signal through a color control pattern mixing module at the first location, and generating a mixed video frame using the color control pattern mixing module, the mixed video frame including a mixed color control pattern and a current video frame of the live action venue, wherein the color control pattern is generated by a color control pattern generating unit based on image patterns, and the color control pattern is used to achieve synchronization of the lighting effect control signal;
step S4: transmitting the mixed video frame to a video streaming server via the internet;
step S5: downloading, decompressing and playing the mixed video frame by using a video player, and confirming a position of the color control pattern in the mixed video frame;
step S6: extracting a color value from the color control pattern of the hybrid video frame and transmitting the color value to a wireless control module at a second location;
step S7: copying the color value into a data structure to form a color control signal vector, and transmitting the color control signal vector to the wireless control module at the second location;
step S8: wirelessly broadcasting the color control signal vector as a plurality of repeating RF data bursts from the wireless control module at the second location via a wireless transmitter; and
step S9: the color control signal vector is received through an interactive luminous effect device located at the second position, and a luminous change is started according to the color control signal vector to form a luminous effect light pattern.
2. The method of claim 1, wherein the step S3 is replaced by the following steps:
the color control signal is directly transmitted to a color control pattern mixing module located at the first position through a direct wire connection, and a mixed video frame is generated by using the color control pattern mixing module, wherein the mixed video frame comprises a color control pattern after mixing and a current video frame of the live event site.
3. The method of claim 1, wherein the color control pattern comprises at least one area within the image of the field event venue.
4. The method of claim 1, wherein the color control pattern comprises a color data comprising color values of red (R), green (G), and blue (B), wherein the color value of each color region of the color control pattern is in a range of 0 to 255, respectively.
5. An interactive lighting effect control system adapted to synchronize lighting effect patterns and lighting effect control signals of a plurality of interactive lighting effect devices at a second location with lighting effect patterns and lighting effect control signals of a plurality of interactive lighting effect devices at a first location, respectively, the interactive lighting effect control system comprising:
a video recorder comprising a camera;
a light emission controller;
a wireless transmitter and a color control pattern mixing module;
a color control pattern extraction module;
a wireless control module; and
each interactive light effect device comprises:
a storage for storing a region indexed by a zone code;
a wireless receiver for acquiring a repeating RF data burst from the wireless transmitter;
a controller responsive to the repeating RF data burst and coupled to the memory; and
at least one light source, operatively responsive to the controller, for changing a lighting state, wherein the at least one light source comprises a plurality of Light Emitting Diodes (LEDs) having at least red, green, and blue lighting colors, the lighting intensities of the red, green, and blue LEDs being defined according to red, green, and blue codes R, G, B from 0 to 255, respectively;
generating a color control signal at the first location and obtaining a current video frame, wherein the first location is a live event venue, the color control signal is obtained from a lighting pattern variation script of the live event venue, and the current video frame comprises an image including a plurality of lighted interactive lighting effect devices located at the first location;
transmitting the color control signal to the color control pattern mixing module at the first location via a direct wire connection or via wireless transmission of the repeating RF data burst broadcast;
generating a mixed video frame using the color control pattern mixing module, the mixed video frame including a color control pattern and the current video frame of the live action field after mixing, wherein the color control pattern is based on an image pattern generated by a color control pattern generating unit, and the color control pattern is used for synchronizing the light effect control signal;
transmitting the mixed video frame to a video streaming server via the internet, and transmitting the mixed video frame to a color control pattern extraction module located at the second location; and
and extracting the color control pattern and the color control signal from the mixed video frame to control the light-emitting state of the interactive light-emitting effect devices at the second position to form a light-emitting effect pattern.
6. The interactive lighting effect control system of claim 5, wherein the color control signal comprises the R, G, B color codes and the partition code.
7. The interactive lighting effect control system of claim 5, wherein the second location is a remote location from where the mixed video frame is broadcast, the color control pattern extraction module is coupled to the wireless control module at the second location for wirelessly transmitting the color control signal to the interactive lighting effect devices at the remote location.
8. The interactive lighting effect control system of claim 5, wherein the lighting state of the at least one lighting source comprises on, off, or flashing.
9. The interactive lighting effect control system of claim 5, wherein the repeating RF data bursts may be replaced by wireless data bursts under Wi-Fi, Bluetooth (Bluetooth), or ZigBee transmission technologies.
10. The interactive lighting effect control system of claim 5, wherein two or more interactive lighting effect devices, each pre-configured to have the same partition code in the storage, are capable of receiving a same set of pattern-related data associated with a lighting effect, such that the at least one lighting source selectively generates the lighting effect according to the same set of pattern-related data.
11. The interactive lighting effect control system of claim 5, wherein the interactive lighting effect device is an LED lighting necklace, an LED lighting bracelet, a lighting headband, a pair of LED glasses, a set of LED gloves, or a hand-held LED light wand.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/791,350 US10003485B2 (en) | 2016-01-25 | 2017-10-23 | Systems of synchronizing lighting effect control signals and patterns for controlling interactive lighting effect devices |
| US15/791,350 | 2017-10-23 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109699102A CN109699102A (en) | 2019-04-30 |
| CN109699102B true CN109699102B (en) | 2021-01-15 |
Family
ID=66230063
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811193945.4A Active CN109699102B (en) | 2017-10-23 | 2018-10-15 | System for synchronizing light emission control signal and light emission pattern of light emission effect device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109699102B (en) |
| TW (1) | TWI679616B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113973414A (en) * | 2020-07-23 | 2022-01-25 | 青岛海尔智能家电科技有限公司 | Method, device and equipment for color synchronous display |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102027807A (en) * | 2008-03-17 | 2011-04-20 | 安东尼·杜布莱 | Interactive system and control method for lighting and/or image diffusion |
| CN102577125A (en) * | 2009-10-10 | 2012-07-11 | 株式会社奥巴尔 | Field equipment photoelectric sensing sensitivity adjustment |
| CN102726124A (en) * | 2010-01-29 | 2012-10-10 | 皇家飞利浦电子股份有限公司 | Interactive lighting control system and method |
| CN105050273A (en) * | 2015-07-31 | 2015-11-11 | 浙江大丰实业股份有限公司 | Pre-programmed wireless control system for stage lighting |
| CN105080774A (en) * | 2015-07-06 | 2015-11-25 | 浙江大学 | Expandable fountain controlling system and method for synchronizing music and water type |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7697925B1 (en) * | 2007-01-23 | 2010-04-13 | Sprint Communications Company L.P. | Synchronized light shows on cellular handsets of users at a gathering |
| GB201113805D0 (en) * | 2011-08-11 | 2011-09-21 | Rb Concepts Ltd | Interactive lighting effect and wristband |
| US9141330B2 (en) * | 2012-10-16 | 2015-09-22 | Production Resource Group Llc | Mobile interface to a show |
| US9215018B2 (en) * | 2013-03-15 | 2015-12-15 | Central Technology, Inc. | Light display production strategy and device control |
| TWM504296U (en) * | 2014-12-31 | 2015-07-01 | Litenet Corp | Synchronized lighting control system |
| US9763311B2 (en) * | 2015-08-11 | 2017-09-12 | Lumic Technology Inc. | Interactive lighting effect portable light illuminating devices and system thereof |
| CN204929340U (en) * | 2015-08-19 | 2015-12-30 | 湖北华远科泰照明装饰工程有限公司 | Interactive light control system of intelligence sight |
-
2018
- 2018-10-09 TW TW107135493A patent/TWI679616B/en active
- 2018-10-15 CN CN201811193945.4A patent/CN109699102B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102027807A (en) * | 2008-03-17 | 2011-04-20 | 安东尼·杜布莱 | Interactive system and control method for lighting and/or image diffusion |
| CN102577125A (en) * | 2009-10-10 | 2012-07-11 | 株式会社奥巴尔 | Field equipment photoelectric sensing sensitivity adjustment |
| CN102726124A (en) * | 2010-01-29 | 2012-10-10 | 皇家飞利浦电子股份有限公司 | Interactive lighting control system and method |
| CN105080774A (en) * | 2015-07-06 | 2015-11-25 | 浙江大学 | Expandable fountain controlling system and method for synchronizing music and water type |
| CN105050273A (en) * | 2015-07-31 | 2015-11-11 | 浙江大丰实业股份有限公司 | Pre-programmed wireless control system for stage lighting |
Non-Patent Citations (2)
| Title |
|---|
| GMF4iTV: new method for interactivity with moving objects in digital television;Stoll, G.; Probst, M.;《Fernseh- und Kino-Technik》;20061231;全文 * |
| 视频与灯光系统交互融合的控制技术;彭妙颜;《电视技术》;20100731;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| TW201917702A (en) | 2019-05-01 |
| CN109699102A (en) | 2019-04-30 |
| TWI679616B (en) | 2019-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10003485B2 (en) | Systems of synchronizing lighting effect control signals and patterns for controlling interactive lighting effect devices | |
| JP7483985B2 (en) | Method for controlling multiple cheering light-emitting devices at a performance venue | |
| US9763311B2 (en) | Interactive lighting effect portable light illuminating devices and system thereof | |
| US10893597B2 (en) | Group-performance control method using light-emitting devices | |
| KR102247264B1 (en) | Performance directing system | |
| US11297253B2 (en) | Control system for controlling plurality of user terminals | |
| CN109699102B (en) | System for synchronizing light emission control signal and light emission pattern of light emission effect device | |
| JP7372991B2 (en) | Performance production system and its control method | |
| KR102437858B1 (en) | Crowd control system for controlling plurality of user devices | |
| KR102220750B1 (en) | Crowd control system for controlling plurality of user devices | |
| KR102494134B1 (en) | Performance directing system and method therefor | |
| WO2022009444A1 (en) | Viewing system and method | |
| KR20210049753A (en) | Performance directing system | |
| KR20220123501A (en) | Crowd control system for controlling plurality of user devices | |
| JP2023182937A (en) | Reaction management device and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |