CN114422846A - Lighting effect control method, system, device, electronic device and storage medium - Google Patents

Abstract

The present application discloses a lighting effect control method, system, device, electronic device and storage medium. By acquiring an audio stream in a video stream to be tested, the audio stream is framed to obtain an audio frame sequence; the audio frames are sequentially acquired. One of the audio frames that is not traversed in the sequence is used as the detected audio frame; the audio frame in the detected audio frame sequence is subjected to audio recognition processing to obtain the recognition result corresponding to the detected audio frame, and the lighting effect of the lighting equipment is controlled according to the recognition result. The stream is divided into frames, and the audio recognition processing is performed in units of audio frames. The calculation is simple and convenient, and the lighting effects of the lighting equipment are controlled in real time based on the recognition results of the audio frames, which enriches the control methods of the lighting effects.

Description

Lighting effect control method, system, device, electronic device and storage medium

technical field

The present application relates to the technical field of lighting equipment control, and more particularly, to a lighting effect control method, system, device, electronic device and storage medium.

Background technique

In the related art, the rhythm point of the audio is usually detected to control the lighting effect according to the change of the audio rhythm point. In order to obtain the rhythm point of the audio, a large number of complex operations are required, and only the change information such as the amplitude and spectrum of the audio is concerned, and the lighting effect control is single.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention proposes a lighting effect control method, system, device, electronic device and storage medium to improve the above problems.

In a first aspect, an embodiment of the present application provides a lighting effect control method. The method includes: acquiring an audio stream in a video stream to be tested; performing frame segmentation processing on the audio stream to obtain an audio frame sequence; and sequentially acquiring an audio frame sequence One of the audio frames that has not been traversed is used as the detected audio frame; the audio recognition processing is performed on the detected audio frame to obtain the recognition result corresponding to the detected audio frame; the lighting effect of the lighting equipment is controlled according to the recognition result, and the execution is returned to obtain the sequence of audio frames in sequence One of the audio frames not traversed in is used as the step of detecting audio frames.

Optionally, performing audio identification processing on the detected audio frame to obtain the identification result corresponding to the detected audio frame includes: determining the probability that the audio frame belongs to a preset category; wherein, the number of preset categories is multiple; determining the preset category with the highest probability; Set the category as the music category corresponding to the audio frame. Controlling the lighting effects of the lighting equipment according to the identification results includes: determining the control lighting effects corresponding to the music categories, and controlling the lighting equipment to display the control lighting effects.

Optionally, controlling the lighting device to display the control lighting effect includes: if the probability of the music category is higher than a preset category threshold, controlling the lighting device to display the control lighting effect.

Optionally, determining the control lighting effect corresponding to the music category includes: if the music category belongs to another category, not controlling the lighting device; otherwise, determining the control lighting effect corresponding to the music category.

Optionally, controlling the lighting device to display the controlled lighting effect includes: determining the current state of the lighting device; if the current state is the lighting effect display state, not controlling the lighting device; otherwise, controlling the lighting device to display the controlled lighting effect.

Optionally, controlling the lighting device to display the control lighting effect includes: controlling the lighting device to repeatedly display the control lighting effect within a preset filter time; wherein, the display time of a single control lighting effect is less than the preset filter time .

Optionally, sequentially acquiring one of the audio frames that are not traversed in the audio frame sequence as the detection audio frame, comprising: sequentially acquiring one of the audio frames that are not traversed in the audio frame sequence at a preset sampling interval as the detection audio frame; wherein , the display time of a single control lighting effect is greater than the preset sampling interval.

Optionally, before acquiring the video stream to be tested, the method further includes: acquiring an original video stream; generating at least two video streams to be tested based on the original video stream; controlling a display device to display based on one of the video streams to be tested; The audio stream in the video stream; including: obtaining the audio stream in the other video stream to be tested.

Optionally, the method further includes: in response to a control operation on the display device, controlling the display device to enter a display state corresponding to the control operation, and simultaneously controlling the lighting device to enter a lighting effect state corresponding to the control operation.

Optionally, the lighting device includes one or more lighting units.

In a second aspect, an embodiment of the present application further provides a lighting effect control system, including: a control device and a lighting device connected to the control device. Wherein, the control device is configured to execute the lighting effect control method as described in the first aspect.

In a third aspect, an embodiment of the present application further provides a lighting effect control device, including: an audio stream acquisition module, an audio frame sequence acquisition module, a detection audio frame determination module, an identification module, and a lighting effect control module. Among them, the audio stream acquisition module is used to acquire the audio stream in the video stream to be tested; the audio frame sequence acquisition module is used to perform frame processing on the audio stream to obtain the audio frame sequence; the detection audio frame determination module is used to sequentially acquire the audio frames One of the audio frames in the sequence that is not traversed as the detected audio frame. The recognition module is used for performing audio recognition processing on the detected audio frames to obtain recognition results corresponding to the detected audio frames; the lighting effect control module is used for controlling the lighting effects of the lighting equipment according to the recognition results.

In a fourth aspect, an embodiment of the present application further provides an electronic device, including: one or more processors; a memory; and one or more application programs. One or more application programs are stored in the memory and configured to be executed by one or more processors, and the one or more application programs are configured to execute the light effect control method as described in the first aspect above.

In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the lighting effect described in the first aspect above Control Method.

The technical scheme provided by the present invention, by acquiring the audio stream in the video stream to be tested, the audio stream is divided into frames to obtain the audio frame sequence; one of the audio frames that is not traversed in the audio frame sequence is sequentially acquired as the detection audio frame; Perform audio recognition processing on the detected audio frames to obtain the recognition results corresponding to the detected audio frames, control the lighting effect of the lighting equipment according to the recognition results, and perform audio recognition processing in units of audio frames by dividing the audio stream into frames. And based on the recognition result of the audio frame, the lighting effect of the lighting equipment is controlled in real time, and the control method of the lighting effect is enriched.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. not all examples. Based on the embodiments of the present application, all other embodiments and drawings obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

FIG. 1 shows a schematic diagram of an application environment involved in an embodiment of the present application.

FIG. 2 shows a schematic diagram of another application environment involved in an embodiment of the present application.

FIG. 3 shows a schematic flowchart of a lighting effect control method proposed by an embodiment of the present application.

FIG. 4 shows a schematic diagram of a time domain diagram of an audio stream in an embodiment of the present application.

FIG. 5 shows a schematic flowchart of generating a video stream to be tested according to an embodiment of the present application.

FIG. 6 shows a structural block diagram of a lighting effect control device proposed by an embodiment of the present application.

FIG. 7 shows a structural block diagram of an electronic device proposed by an embodiment of the present application.

FIG. 8 shows a structural block diagram of a computer-readable storage medium proposed by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

In order to make those skilled in the art better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application.

In the related art, the rhythm point of the audio is usually detected to control the lighting effect according to the change of the audio rhythm point. In order to obtain the rhythm point of the audio, it is necessary to calculate the time domain and the frequency domain of the audio at the same time, and to obtain the rhythm point by combining the calculation results of the time domain and the frequency domain, the calculation amount is large and complicated. In addition, when the audio energy fluctuation is small, the rhythm point cannot be accurately obtained, which affects the lighting effect control. And the lighting effect is controlled through the rhythm point, only focusing on the change information such as the amplitude and frequency spectrum of the audio, and the lighting effect control is single.

In order to improve the above problems, the inventor proposes a lighting effect control method, system, device, electronic device and storage medium provided by the present application. By acquiring the audio stream in the video stream to be tested, the audio stream is divided into frames to obtain audio frames. Sequence; sequentially obtain one of the audio frames that are not traversed in the audio frame sequence as the detection audio frame; perform audio recognition processing on the detected audio frame to obtain the recognition result corresponding to the detected audio frame, and control the lighting effect of the lighting equipment according to the recognition result. The audio stream is divided into frames, and the audio recognition processing is performed in units of audio frames. The calculation is simple and convenient, and the lighting effects of the lighting equipment are controlled in real time based on the recognition results of the audio frames, which enriches the control methods of the lighting effects.

The following describes the application environment of the light effect control method provided by the implementation of the present invention.

Referring to FIG. 1 , the lighting effect control method provided by the implementation of the present invention can be applied to a lighting effect control system A, and the lighting effect control system A includes a control device B and a lighting device C. It should be noted that FIG. 1 is an exemplary application environment, and the methods provided in the embodiments of the present application may also be run in other application environments, which are not limited thereto.

In some embodiments, the control device B can be used to obtain the video stream to be tested. The control device B may include a hardware interface for acquiring the video stream to be tested.

In some embodiments, the control device B can be connected to devices such as XBOX (home video game console), PS (PlayStation, electronic game console) through a hardware interface to obtain the video stream to be tested.

In order to enable the control device B to be compatible with video streams to be tested in multiple formats, in the embodiments of the present application, the control device B may further include a video interface conversion module, which can be used to perform format conversion on the received video stream to be tested. , such as converting the video stream to be tested into I2S (Inter-IC Sound, integrated circuit built-in audio bus), MIPI (MobileIndustry Processor Interface, mobile industry processor interface), LVDS (Low VoltageDifferential Signaling, low voltage differential signal), BT656, The format required by the control device B such as BT1120 can be specifically set according to actual use needs, which is not limited in this application.

In some embodiments, the control device B may further include a control unit, and the control unit may be configured to process the video stream to be tested, for example, perform frame division processing on the audio stream to obtain a sequence of audio frames, and perform processing on the audio frames in the sequence of audio frames. The audio recognition process obtains the recognition result corresponding to the audio frame, and controls the lighting effect of the lighting device C according to the recognition result. Exemplarily, the control unit may adopt SOC (System on Chip, system level chip), CPU (central processing unit, central processing unit), MPU (Microcontroller Unit, micro control unit), etc. There are no restrictions on the application.

Optionally, the control unit may be loaded with a pre-trained audio recognition model for performing audio recognition processing on the audio frame.

Optionally, the control unit may not set the audio recognition model, the control unit may send the audio frame to the server loaded with the pre-trained audio recognition model, the server performs audio recognition processing on the audio frame, and sends the recognition result to the control unit. device B.

Wherein, the lighting device C can be connected with the control device B in a wired or wireless manner. When the wireless connection is adopted, the lighting device C can communicate with the control device B through the network. The Internet is usually the Internet, but can be any network, including but not limited to Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless Any combination of network, private network, or virtual private network. In some embodiments, data exchanged over a network is represented using technologies and/or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), and the like. In addition, you can also use methods such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. Conventional encryption techniques to encrypt all or some of the links. In other embodiments, custom and/or dedicated data communication techniques may also be used in place of or in addition to the data communication techniques described above.

In some embodiments, the lighting device C may be a strip light, a strip light, a light column, a flat light (containing a plurality of light beads), or a space-arranged light.

In some embodiments, the lighting device C may include a control unit and a lighting unit, and the control unit may be configured to control the lighting effect of the lighting unit according to the control device B.

In some embodiments, the lighting device C may include one or more lighting units, so as to make corresponding combinations according to actual use requirements.

In some embodiments, each lighting unit can include one or more lamp beads, and the control unit can control the brightness, color, etc. of the lamp beads of the lighting unit, and can also control the number of lights, so as to realize different lights of the lighting equipment. effect.

In some embodiments, the position and number of lighting devices can be set according to actual use requirements, and the length, height, and lighting effects on area can also be controlled according to the positions of the lighting devices.

Referring to FIG. 2 , in some embodiments, the lighting control system A may further include a display device D, and the display device D and the control device B are connected in a wired or wireless manner.

In some embodiments, the display device D can display according to the video stream to be tested, so that while displaying the video, the lighting device can display lighting effects corresponding to the video to create an immersive atmosphere and improve user experience.

Optionally, the display device D may be, for example, a smart TV, a tablet computer, a smart speaker with a display screen, a mobile phone, etc., which is not limited in this application.

In some embodiments, the lighting control system A may further include a video splitting unit for dividing the original video stream into at least two video streams to be tested, so that one of the video streams to be tested can be displayed by the display device D, The other video stream to be tested can be used by the control device B to control the lighting effect of the lighting device C. Exemplarily, the video splitting unit may adopt a video one-to-two interface chip, a video distributor chip, etc., which is not limited in this application.

In the embodiments of the present application, the control device B may also execute the lighting control methods involved in the following embodiments, which will be described below with reference to specific embodiments.

Referring to FIG. 3, the present application provides a lighting effect control method according to an embodiment, which can be applied to the above-mentioned lighting effect control system. In the method provided by the embodiment of the present application, the execution subject of the method may be the above-mentioned lighting effect control system. control device in . Specifically, the lighting effect control method provided by the embodiment of the present application includes steps S110 to S140.

In step S110, the audio stream in the video stream to be tested is acquired.

In the embodiment of the present application, the video stream to be tested includes a picture frame for display and an audio stream for playback. When the picture frame and the audio are played at the same time, a sound picture viewed by the user is formed. The audio stream in this article includes but is not limited to background music. In order to control the lighting device according to the background music corresponding to the video, the control device needs to obtain the audio stream in the video stream to be tested. Specifically, the required audio stream can be obtained by decoding and extracting the video stream to be tested. Exemplarily, the audio stream may be acquired by means of FFMpeg (Fast Forward Mpeg, Fast Forward Moving Picture Experts Group), Premiere (video editing tool), etc., which is not limited in this application.

In step S120, the audio stream is framed to obtain an audio frame sequence.

In the embodiment of the present application, the control device performs frame division processing on the audio data stream to obtain multiple audio frames, and the multiple audio frames constitute an audio frame sequence.

As shown in FIG. 4 , FIG. 4 shows a schematic diagram of a time domain diagram of an audio stream in an embodiment of the present application, wherein, by dividing the audio stream into frames, the audio stream can be divided into multiple audio frames a. In the embodiment of the present application, wlen is the frame length, and inc is the frame shift. The next audio frame can be obtained by shifting the frame shift inc, thereby dividing the audio stream into multiple audio frames to form an audio frame sequence.

Optionally, to avoid missing the content of the audio stream, the length of the frame shift inc can be set to be less than or equal to the frame length of a single audio frame. It can be understood that, for an audio stream of the same length, the smaller the frame shift length is, the more audio frames are obtained, so that subtle changes in audio frames can be more sensitively reflected, but at the same time, the amount of calculation will increase. The length of the frame shift can be set according to the actual needs of sensitivity and calculation amount, which is not limited in this application.

By dividing the audio stream into a plurality of audio frames, and performing audio recognition processing on the audio frames, the calculation is simple and the calculation amount can be effectively reduced.

In step S130, one of the audio frames not traversed in the audio frame sequence is sequentially acquired as the detected audio frame.

In the embodiments of the present application, the control device may sequentially acquire one of the audio frames that are not traversed in the audio frame sequence as the detection audio frame in chronological order, so as to control the current lighting effect of the lighting device according to the detected audio frame.

Exemplarily, if the audio frame sequence includes a first audio frame, a second audio frame and a third audio frame. The time of the first audio frame is earlier than the time of the second audio frame, and the time of the second audio frame is earlier than the time of the third audio frame. The control device has controlled the lighting effect of the lighting device according to the first audio frame, then the audio frames that are not traversed in the audio frame sequence are the second audio frame and the third audio frame, because the time of the second audio frame is earlier than that of the third audio frame. frame time, so the second audio frame can be acquired as the detection audio frame.

In some embodiments, one of the audio frames not traversed in the audio frame sequence may be sequentially acquired at a preset sampling interval as the detection audio frame.

Optionally, the presentation time of the lighting effect of a single control is greater than the preset sampling interval, so that repeated triggering of the lighting device can be reduced.

In step S140, audio recognition processing is performed on the detected audio frames to obtain recognition results corresponding to the detected audio frames.

In the embodiments of the present application, the control device may perform audio recognition processing on the detected audio frames to obtain recognition results corresponding to the detected audio frames. In other embodiments, the control device may also send the detected audio frame to the server, and the server performs audio recognition processing, and returns the recognition result corresponding to the detected audio frame.

In some embodiments, a pre-trained neural network model may be used to perform audio recognition processing on the detected audio frames to obtain recognition results corresponding to the detected audio frames. The identification result may be, for example, detecting the category corresponding to the audio frame, so as to perform lighting effect control according to the corresponding category.

Step S140 will be described in detail below with a specific implementation manner. As shown in FIG. 6 , step S140 may include the following steps.

(1) Determine the probability that the detected audio frame is a preset category.

Wherein, the number of preset categories is multiple.

In the embodiment of the present application, the audio features in the audio frame can be extracted and detected by an audio extraction algorithm. The audio transform algorithm can be, for example, FFT (fast Fourier transform, high-speed Fourier transform) spectral transform, MFCC (Mel-scale Frequency Cepstral Coefficients, Mel cepstral coefficients) feature transform, etc., which can be set according to actual needs. There are no restrictions on the application.

In the embodiment of the present application, a pre-trained neural network model may be used to perform audio recognition processing on the extracted audio features, so as to determine the probability that the audio frame is a preset category.

Exemplarily, the number of preset categories can be 20, and the last layer of the neural network model can be normalized by using softmax (normalized exponential function) to obtain the probability of detecting audio frames corresponding to each preset category. , also known as confidence.

In some embodiments, to determine the detected audio frame as a preset category, deep learning convolution classification such as Resnet (residual neural network), VGG (Visual Geometry Group) convolutional neural network, and Mobilenet (lightweight convolutional network) can also be used algorithm, which is not limited in this application.

Exemplarily, the preset categories may include but are not limited to: flowing water sound category, popping sound category, wind sound category, rain sound category, etc., which may be set according to actual usage needs, which is not limited in this application.

In some embodiments, other categories may also be set in the preset categories. For example, the preset categories include: the first category, the second category, the third category, and other categories. If the audio frame recognition result is that it does not belong to any one of the first category, the second category, and the third category, the The categories of audio frames are grouped into other categories.

(2) Determine the preset category with the highest probability as the music category corresponding to the detected audio frame.

In the embodiment of the present application, after obtaining the probability that the detected audio frame corresponds to each preset category through the above steps, the preset category with the highest probability is used as the music category corresponding to the detected audio frame.

In step S150, the lighting effect of the lighting device is controlled according to the recognition result. and returning to the step of sequentially acquiring one of the audio frames that are not traversed in the audio frame sequence as a detected audio frame.

In the embodiment of the present application, the control device controls the lighting effect of the lighting device according to the recognition result of the detected audio frame, so as to present the lighting effect corresponding to the detected audio frame.

In the embodiment of the present application, the control device may control the lighting effect of the lighting device according to the music category.

In some embodiments, after determining the music category corresponding to the detected audio frame, a further judgment may be made. If the probability of detecting the music category corresponding to the audio frame is higher than the preset category threshold, the lights of the lighting equipment are controlled according to the music category. effect. Otherwise, even if the detected audio frame belongs to the music category, the lighting effect of the lighting device will not be controlled according to the music category, that is, the detected audio frame is invalid and needs to be discarded, so the lighting effect of the lighting device will not be controlled.

It can be understood that the lower the preset category threshold is set, the higher the probability that the audio frame belongs to the corresponding music category, and the more sensitive the triggering of the control of the lighting equipment. Therefore, the specific value of the preset category threshold can be set according to actual use requirements, which is not limited in this application.

In some embodiments, each music category is preset with a corresponding control lighting effect. After the music category corresponding to the audio frame is determined, the control lighting effect corresponding to the music category can be determined, and then the lighting equipment can be controlled to display the corresponding control lighting effect, so as to present the corresponding lighting effect according to the category of the audio frame, so as to construct an immersive lighting atmosphere to enhance the user experience.

In some embodiments, if the music category belongs to other categories, the lighting device is not controlled, otherwise, the control lighting effect corresponding to the music category is determined. That is, when the audio frame belongs to other categories, the audio frame needs to be discarded, that is, the lighting device is not controlled. Subsequent processing takes place only when the music category does not belong to other categories, thus ensuring precise control over the lighting effects of the lighting equipment.

In some embodiments, when the lighting effect of the last detected audio frame has not been fully displayed, if the control lighting effect corresponding to the currently detected audio frame has been determined at this time, the lighting effect of the currently detected audio frame is controlled to be displayed according to the lighting effect of the currently detected frame. The lighting equipment of the lighting effect will cause the lighting effect of the last detected audio frame to be incompletely displayed. In order to ensure the effect of the lighting effect display, the current state of the lighting equipment must be determined before the lighting effect control. If the current state of the lighting device is the lighting effect display state, that is, when the lighting device is still displaying the lighting effect of the audio frame detected before, the lighting device will not be controlled. Otherwise, control the lighting device to display the control lighting effect corresponding to the currently detected audio frame.

In some embodiments, after determining the control lighting effect for controlling the lighting device, the lighting device may also be controlled to exhibit one or more control lighting effects. That is, it is assumed that the time required to control a complete single control lighting effect is 2s, and the time required to control two complete control lighting effects requires 4s. Specifically, a preset number of times that the lighting device needs to display the lighting effect can be preset. The number of preset times can be set according to actual usage needs, which is not limited in this application.

In some embodiments, after determining the control lighting effect for controlling the lighting device, the lighting device may also be controlled to repeatedly execute the control lighting effect within a preset filtering time.

Optionally, the presentation time of a single control lighting effect is shorter than the preset filtering time, so that the complete control lighting effect is displayed at least once within the preset filtering time. For example, the preset filter time is 6s, and the display time of a single control light effect is 2s, then the corresponding control light effect is repeatedly displayed 3 times within the preset filter time.

Optionally, the presentation time of a single controlled lighting effect may also be greater than the preset filtering time, so that only part of the controlled lighting effects are displayed within the preset filtering time. The specific settings can be adjusted according to actual usage needs, which are not limited in this application.

In the embodiment of the present application, the corresponding lighting effect may be set according to the corresponding music category. The lighting effects include but are not limited to: the position, quantity, brightness, color, flickering frequency, etc. of the lighting equipment, so that lighting effects matching the music category can be combined according to actual needs. For example, when the music category is the popping sound category, the corresponding light effect can be the current color gradient for 1s, change to red, and flash with a flickering cycle of 0.05s. After flashing for 1s, the color changes from red to the original color for 1s. . Specifically, any combination can be made according to actual use requirements, which is not limited in this application.

In some embodiments, before step S110, the lighting effect control method of the embodiment of the present application further includes the following steps.

(1) Get the original video stream.

(2) generate at least two video streams to be tested based on the original video stream;

(3) Sending one of the video streams to be tested to the display device, so that the display device displays according to one of the video streams to be tested.

The lighting effect control method according to the embodiment of the present application can also be applied to a scenario where video display is performed simultaneously and lighting effect control is performed by using a video stream.

Specifically, the original video stream may be obtained through a video source, and the video source may be, for example, a TV program, a game screen, a network video, or the like.

Further, two channels of video streams to be tested are generated from the original video stream. As shown in Figure 6, the control device realizes the access of the video stream through the hardware interface, and then divides the original video stream into two to generate at least two video streams to be tested, one of which is used to control the display device to display the specific video stream. The image content, the other channel is used to convert the video stream type that can be received by the multimedia processing chip through the video interface, and then the video stream to be tested is processed by the multimedia processing chip into frames to obtain the video frame image sequence, so as to control the lights of the lighting equipment. Effectively transform. Wherein, the hardware interface can be, for example, VGA (Video Graphics Array, video graphics array), DP (DisplayPort, display interface), HDMI (High Definition Multimedia Interface, high-definition multimedia interface), DVI (Digital Visual Interface, digital video interface), etc.; video The interface can be, for example, MIPI (Mobile Industry Processor Interface, mobile industry processor interface), LVDS (Low-Voltage Differential Signaling, low-voltage differential signaling interface), etc.; the multimedia processing chip can be, for example, SoC (System on Chip, system-on-chip) chip. Further, looping out the original video stream can be realized by calling the video capture card.

It can be understood that the content of the two video streams to be tested is the same, one of the video streams to be tested is used for lighting effect control, and the other video stream to be tested is used to control the display device for display, thereby ensuring the consistency of lighting effect control and video display.

In some embodiments, the control device may also control the lighting device in combination with the image frames and audio frames in the video stream. For example, if there is a monster in the image frame, and the music category corresponding to the audio frame is the horror category, you can adjust the flickering frequency and brightness change of the lighting effect of the horror category according to the number and distance of the monsters, so as to present a richer lighting effect control. .

In some embodiments, in order to ensure the immersive atmosphere of the lighting effect, the lighting effect control and the video display need to be synchronized. Since the display device and the lighting device are controlled by two video streams to be tested, after the user operates the display device, the Changing the display state of the display device, and if the lighting device still controls the lighting effect, will cause the video display to be out of sync with the lighting effect control, affecting the user's experience.

In order to solve the synchronization problem, in the embodiment of the present application, the lighting effect control method may further include: when the display device enters a display state corresponding to the control operation in response to the control operation, controlling the lighting device to enter the lighting effect corresponding to the control operation state.

Optionally, the control operation may include, but is not limited to: a suspend operation, a shutdown operation, and a restart operation. The user can send control instructions to the display device through the terminal device to control the display device.

Optionally, when the display device receives the control command, it can send a lighting effect control command corresponding to the control command to the control device, so that the control device can control the lighting device synchronously.

Optionally, when sending the control command, the user sends two control commands at the same time, one of which is sent to the display device to control the display device, and the other control command is sent to the control device to correspond to the control device. lighting effect control.

In this embodiment, by acquiring the audio stream in the video stream to be tested, the audio stream is framed to obtain an audio frame sequence; one of the audio frames not traversed in the audio frame sequence is sequentially acquired as the detection audio frame; Detect the audio frame and perform audio recognition processing to obtain the recognition result corresponding to the detected audio frame, control the lighting effect of the lighting equipment according to the recognition result, and perform audio recognition processing in units of audio frames by dividing the audio stream into frames, the calculation is simple and convenient, and Real-time control of lighting effects of lighting equipment based on the recognition results of audio frames, enriching the control methods of lighting effects.

Please refer to FIG. 6, which shows a lighting effect control device 200 provided by an embodiment of the present invention. The lighting effect control device 200 includes: an audio stream acquisition module 210, an audio frame sequence acquisition module 220, a detection audio frame determination module 230, The identification module 240 and the lighting effect control module 250 .

Wherein, the audio stream acquisition module 210 is configured to acquire the audio stream in the video stream to be tested.

The audio frame sequence acquisition module 220 is configured to perform frame segmentation processing on the audio stream to obtain the audio frame sequence.

The detected audio frame determination module 230 is configured to sequentially acquire one audio frame that is not traversed in the audio frame sequence as the detected audio frame.

The identification module 240 is configured to perform audio identification processing on the audio frames in the audio frame sequence to obtain identification results corresponding to the audio frames.

The lighting effect control module 240 is used to control the lighting effect of the lighting device according to the recognition result.

It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same and similar parts among the various embodiments, refer to each other Can. As for the apparatus type embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant part, please refer to the partial description of the method embodiment. Any processing manners described in the method embodiments may be implemented by corresponding processing modules in the apparatus embodiments, and will not be repeated in the apparatus embodiments.

In several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be electrical, mechanical or otherwise.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically alone, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules.

Referring to FIG. 7 , based on the above-mentioned lighting effect control method, another electronic device 300 including a processor that can execute the aforementioned lighting effect control method is further provided by the embodiments of the present application, and the electronic device 300 further includes one or more processors 310, memory 320, and one or more applications. Wherein, the memory 320 stores a program that can execute the content in the foregoing embodiments, and the processor 310 can execute the program stored in the memory.

The processor 310 may include one or more cores for processing data and a message matrix unit. The processor 310 uses various interfaces and lines to connect various parts in the entire electronic device, and executes the electronic device 300 by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. various functions and processing data. Optionally, the processor 310 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in hardware. The processor 310 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may also not be integrated into the processor, and be implemented by a communication chip alone.

The memory 320 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 420 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing various method embodiments described below, and the like. The storage data area can also store the terminal created in use.

Please refer to FIG. 8 , which shows a structural block diagram of a computer-readable storage medium 400 provided by an embodiment of the present application. The computer-readable storage medium 400 stores program code 410, and the program code 410 can be invoked by the processor to execute the methods described in the above method embodiments.

The computer-readable storage medium 400 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 400 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 400 has storage space for program code to perform any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products. The program code may, for example, be compressed in a suitable form.

Embodiments of the present application also provide a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the lighting effect control methods described in the various optional embodiments above.

In summary, the present application provides a lighting effect control method, system, device, electronic device and storage medium, by acquiring the audio stream in the video stream to be tested, and performing frame segmentation processing on the audio stream to obtain an audio frame sequence; Perform audio recognition processing on the audio frames in the audio frame sequence to obtain the recognition results corresponding to the audio frames, control the lighting effect of the lighting equipment according to the recognition results, and perform audio recognition processing in units of audio frames by dividing the audio stream into frames. Calculate Simple and convenient, and real-time control of the lighting effect of the lighting equipment based on the recognition result of the audio frame, enriching the control methods of the lighting effect.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (14)

1. A lighting effect control method, wherein the method comprises: Get the audio stream in the video stream to be tested; Framing the audio stream to obtain a sequence of audio frames; Sequentially obtain one of the audio frames that are not traversed in the audio frame sequence as a detection audio frame; Performing audio recognition processing on the detected audio frame to obtain a recognition result corresponding to the detected audio frame; Control the lighting effect of the lighting device according to the recognition result, and return to the step of sequentially acquiring one of the audio frames that is not traversed in the audio frame sequence as a detection audio frame. 2. The method according to claim 1, wherein, performing audio recognition processing on the detected audio frame to obtain a recognition result corresponding to the detected audio frame, comprising: Determine the probability that the detected audio frame belongs to a preset category; wherein, the number of the preset category is multiple; determining the preset category with the highest probability as the music category corresponding to the detected audio frame; The controlling the lighting effect of the lighting device according to the recognition result includes: determining the control lighting effect corresponding to the music category; The lighting device is controlled to exhibit the controlled lighting effect. 3. The method according to claim 2, wherein the controlling the lighting device to exhibit the controlled lighting effect comprises: If the probability of the music category is higher than a preset category threshold, the lighting device is controlled to exhibit a controlled lighting effect. 4. The method according to claim 2, wherein the determining the control lighting effect corresponding to the music category comprises: If the music category belongs to other categories, do not control the lighting equipment; Otherwise, determine the control lighting effect corresponding to the music category. 5. The method according to claim 2, wherein the controlling the lighting device to exhibit the controlled lighting effect comprises: determining the current state of the lighting device; If the current state is a lighting effect display state, the lighting equipment is not controlled; Otherwise, the lighting device is controlled to exhibit the controlled lighting effect. 6. The method according to claim 2, wherein the controlling the lighting device to exhibit the controlled lighting effect comprises: The lighting device is controlled to repeatedly display the control lighting effect within a preset filter time; wherein, the display time of the single control lighting effect is less than the preset filter time. 7. The method according to claim 2, wherein the sequentially acquiring one of the audio frames that are not traversed in the audio frame sequence as a detection audio frame, comprising: Acquiring one of the audio frames that are not traversed in the audio frame sequence according to the preset sampling interval in sequence as the detection audio frame; Wherein, the display time of the single control lighting effect is greater than the preset sampling interval. 8. The method according to any one of claims 1 to 7, wherein before the acquiring the video stream to be tested, the method further comprises: Get the original video stream; Generate at least two video streams to be tested based on the original video stream; sending one of the video streams to be tested to the display device, so that the display device displays according to the one of the video streams to be tested; Described acquiring the audio stream in the video stream to be tested; including: Acquire the audio stream in the other video stream to be tested. 9. The method according to claim 8, wherein the method further comprises: When the display device enters a display state corresponding to the control operation in response to the control operation, the lighting device is controlled to enter a lighting effect state corresponding to the control operation. 10. The method of any one of claims 1 to 8, wherein the lighting device comprises one or more lighting units. 11. A lighting effect control system, comprising: a control device and a lighting device connected to the control device; Wherein, the control device is used to execute the lighting effect control method according to any one of claims 1 to 10. 12. A lighting effect control device, comprising: The audio stream acquisition module is used to acquire the audio stream in the video stream to be tested; an audio frame sequence acquisition module, for performing frame segmentation processing on the audio stream to obtain an audio frame sequence; The detection audio frame determination module is used to sequentially obtain one of the audio frames that are not traversed in the audio frame sequence as the detection audio frame; an identification module, configured to perform audio identification processing on the detected audio frame to obtain a corresponding identification result of the detected audio frame; The lighting effect control module is used to control the lighting effect of the lighting equipment according to the recognition result. 13. An electronic device, characterized in that, comprising: one or more processors; memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more application programs are configured to execute The light effect control method according to any one of claims 1-10. 14. A computer-readable storage medium, wherein a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute any one of claims 1-10 lighting effect control method.

Images