CN114422846A - Lamp effect control method, system, device, electronic equipment and storage medium - Google Patents
Lamp effect control method, system, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114422846A CN114422846A CN202111666958.0A CN202111666958A CN114422846A CN 114422846 A CN114422846 A CN 114422846A CN 202111666958 A CN202111666958 A CN 202111666958A CN 114422846 A CN114422846 A CN 114422846A
- Authority
- CN
- China
- Prior art keywords
- audio
- audio frame
- control
- light effect
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 230000000694 effects Effects 0.000 title claims description 35
- 230000001795 light effect Effects 0.000 claims abstract description 113
- 238000012545 processing Methods 0.000 claims abstract description 48
- 238000001514 detection method Methods 0.000 claims abstract description 19
- 238000009432 framing Methods 0.000 claims abstract description 18
- 238000001914 filtration Methods 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 10
- 230000033764 rhythmic process Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 230000037433 frameshift Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 239000011324 bead Substances 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Circuit Arrangement For Electric Light Sources In General (AREA)
Abstract
The application discloses a light effect control method, a system, a device, electronic equipment and a storage medium, wherein an audio stream in a video stream to be detected is obtained, and the audio stream is subjected to framing processing to obtain an audio frame sequence; sequentially acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame; the method comprises the steps of carrying out audio identification processing on audio frames in a detected audio frame sequence to obtain an identification result corresponding to the detected audio frames, controlling the light effect of the lighting equipment according to the identification result, framing the audio stream, carrying out audio identification processing by taking the audio frames as units, and carrying out real-time control on the light effect of the lighting equipment by using the identification result of the audio frames, thereby enriching the control mode of the light effect.
Description
Technical Field
The present disclosure relates to the field of lighting device control technologies, and in particular, to a method, a system, a device, an electronic device, and a storage medium for controlling light efficiency.
Background
In the related art, usually, a rhythm point of an audio is detected to perform light effect control according to a change situation of the rhythm point of the audio. In order to obtain the rhythm point of the audio, a large amount of complex operation is required, only the change information of the amplitude, the frequency spectrum and the like of the audio is concerned, and the light effect control is single.
Disclosure of Invention
In view of the foregoing, the present invention provides a method, a system, a device, an electronic apparatus and a storage medium for controlling lamp efficiency to improve the foregoing problems.
In a first aspect, an embodiment of the present application provides a lamp efficiency control method, where the method includes: acquiring an audio stream in a video stream to be detected; performing framing processing on the audio stream to obtain a sequence of audio frames; sequentially acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame; carrying out audio identification processing on the detected audio frame to obtain an identification result corresponding to the detected audio frame; and controlling the light effect of the lighting equipment according to the identification result, and returning to execute the step of sequentially acquiring one audio frame which is not traversed in the audio frame sequence as the detected audio frame.
Optionally, performing audio recognition processing on the detected audio frame to obtain a recognition result corresponding to the detected audio frame, including: determining the probability that the audio frame belongs to a preset category; the number of the preset categories is multiple; and determining the preset category with the highest probability as the music category corresponding to the audio frame. Controlling the lamp effect of the lighting device according to the recognition result includes: and determining a control light effect corresponding to the music category, and controlling the lighting equipment to display the control light effect.
Optionally, controlling the lighting device to exhibit the control light effect comprises: and if the probability of the music category is higher than a preset category threshold value, controlling the lighting equipment to display the control light effect.
Optionally, determining a control light effect corresponding to the music category includes: if the music category belongs to other categories, the lighting equipment is not controlled; otherwise, determining the control light effect corresponding to the music category.
Optionally, controlling the lighting device to exhibit a control light effect, comprising: determining the current state of the lighting equipment; if the current state is a lamp effect showing state, the lighting equipment is not controlled; otherwise, controlling the lighting equipment to display the control light effect.
Optionally, controlling the lighting device to exhibit the control light effect comprises: controlling the lighting equipment to repeatedly show the control light effect within a preset filtering time; and the display time of the single control lamp effect is less than the preset filtering time.
Optionally, sequentially acquiring one of the audio frames in the sequence of audio frames that is not traversed as a detection audio frame, including: acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame according to a preset sampling interval in sequence; wherein, the display time of the single control lamp effect is longer than the preset sampling interval.
Optionally, before the video stream to be tested is acquired, the method further includes: acquiring an original video stream; generating at least two paths of video streams to be detected based on the original video streams; displaying based on one path of video stream to be detected to control the display equipment; acquiring an audio stream in a video stream to be detected; the method comprises the following steps: and acquiring the audio stream in the other path of video stream to be tested.
Optionally, the method further comprises: and responding to the control operation of the display equipment, controlling the display equipment to enter a display state corresponding to the control operation, and simultaneously controlling the lighting equipment to enter a lamp effect state corresponding to the control operation.
Optionally, the lighting device comprises one or more lighting units.
In a second aspect, an embodiment of the present application further provides a lamp efficiency control system, including: the control device and the lighting equipment who is connected with the control device. Wherein the control device is adapted to perform the lamp effect control method as described in the first aspect above.
In a third aspect, an embodiment of the present application further provides a lamp effect control device, including: the device comprises an audio stream acquisition module, an audio frame sequence acquisition module, a detection audio frame determination module, an identification module and a light effect control module. The audio stream acquisition module is used for acquiring an audio stream in a video stream to be detected; the audio frame sequence acquisition module is used for framing the audio stream to obtain an audio frame sequence; the detection audio frame determining module is used for sequentially acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame. The identification module is used for carrying out audio identification processing on the detected audio frame to obtain an identification result corresponding to the detected audio frame; the light effect control module is used for controlling the light effect of the lighting equipment according to the identification result.
In a fourth aspect, an embodiment of the present application further provides an electronic device, including: one or more processors; a memory; one or more applications. Wherein one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more application programs being configured to perform the light effect control method as described above in relation to the first aspect.
In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code can be called by a processor to execute the light effect control method according to the first aspect.
According to the technical scheme provided by the invention, the audio stream in the video stream to be detected is obtained, and the audio stream is subjected to framing processing to obtain an audio frame sequence; sequentially acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame; the method comprises the steps of carrying out audio identification processing on a detected audio frame to obtain an identification result corresponding to the detected audio frame, controlling the light effect of the lighting equipment according to the identification result, framing the audio stream, carrying out audio identification processing by taking the audio frame as a unit, and carrying out real-time control on the light effect of the lighting equipment by using the identification result of the audio frame, so that the control mode of the light effect is enriched.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below, and it is apparent that the drawings in the following description are only some embodiments, not all embodiments, of the present application. All other embodiments and drawings obtained by a person skilled in the art based on the embodiments of the present application without any inventive step are within the scope of the present invention.
Fig. 1 is a schematic diagram illustrating an application environment according to an embodiment of the present application.
Fig. 2 is a schematic diagram illustrating another application environment according to an embodiment of the present application.
Fig. 3 is a schematic flow chart illustrating a lamp efficiency control method according to an embodiment of the present application.
Fig. 4 shows a schematic diagram of a time domain diagram of an audio stream in an embodiment of the present application.
Fig. 5 is a schematic flowchart illustrating a process of generating a video stream to be tested according to an embodiment of the present application.
Fig. 6 shows a block diagram of a lamp effect control device according to an embodiment of the present application.
Fig. 7 shows a block diagram of an electronic device according to an embodiment of the present application.
Fig. 8 shows a block diagram of a computer-readable storage medium according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
In the related art, usually, a rhythm point of an audio is detected to perform light effect control according to a change situation of the rhythm point of the audio. In order to obtain the rhythm point of the audio, the time domain and the frequency domain of the audio need to be calculated at the same time, and the rhythm point is obtained by combining the calculation results of the time domain and the frequency domain, so that the calculation amount is large and complex. In addition, the rhythm point cannot be accurately obtained under the condition of small audio energy fluctuation, and the lamp effect control is influenced. And the light effect control is carried out through the rhythm point, only the change information such as the amplitude, the frequency spectrum and the like of the audio is concerned, and the light effect control is single.
In order to improve the above problem, the inventor proposes a light effect control method, a system, an apparatus, an electronic device and a storage medium provided by the present application, which perform framing processing on an audio stream by acquiring the audio stream in a video stream to be detected to obtain an audio frame sequence; sequentially acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame; the method comprises the steps of carrying out audio identification processing on a detected audio frame to obtain an identification result corresponding to the detected audio frame, controlling the light effect of the lighting equipment according to the identification result, framing the audio stream, carrying out audio identification processing by taking the audio frame as a unit, and carrying out real-time control on the light effect of the lighting equipment by using the identification result of the audio frame, so that the control mode of the light effect is enriched.
The following is a description of an application environment of the lamp efficiency control method provided by the embodiment of the invention.
Referring to fig. 1, the lamp effect control method provided by the embodiment of the present invention can be applied to a lamp effect control system a, where the lamp effect control system a includes a control device B and a lighting device C. It should be noted that fig. 1 is an exemplary application environment, and the method provided in the embodiment of the present application may also be executed in other application environments, which is not limited in this respect.
In some embodiments, control device B may be used to obtain a video stream under test. The control device B may comprise a hardware interface for acquiring the video stream to be tested.
In some embodiments, the control device B may be connected to an XBOX (home television game), a PS (PlayStation, electronic game), or the like through a hardware interface to acquire a video stream to be tested.
In order to make the control device B compatible with the video stream to be tested in multiple formats, in an embodiment of the present application, the control device B may further include a video Interface conversion module, which may be configured to convert the format of the received video stream to be tested, for example, convert the video stream to be tested into a format required by the control device B, such as I2S (Inter-IC Sound, audio bus built in an integrated circuit), MIPI (Mobile Industry Processor Interface), LVDS (Low Voltage Differential Signaling), BT656, BT1120, and the like, and may be specifically set according to actual use needs, which is not limited in this application.
In some embodiments, the control device B may further include a control unit, and the control unit may be configured to process the video stream to be tested, for example, perform framing processing on the audio stream to obtain an audio frame sequence, perform audio identification processing on audio frames in the audio frame sequence to obtain an identification result corresponding to the audio frames, control the light effect of the lighting device C according to the identification result, and the like. For example, the control Unit may adopt an SOC (System on Chip), a CPU (central processing Unit), an MPU (micro control Unit), and the like, and may be specifically set according to actual use needs, which is not limited in this application.
Alternatively, the control unit may be loaded with a pre-trained audio recognition model for performing audio recognition processing on the audio frames.
Optionally, the control unit may not set the audio recognition model, and the control unit may send the audio frame to a server loaded with a pre-trained audio recognition model, where the server performs audio recognition processing on the audio frame and sends a recognition result to the control device B.
The lighting equipment C can be connected with the control equipment B in a wired or wireless mode. When wireless connection is adopted, the lighting equipment C and the control equipment B can realize mutual communication through a network. The internet is typically the internet, but can be any Network including, but not limited to, a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), any combination of mobile, wireline or wireless networks, private or virtual private networks. In some embodiments, data exchanged over a network is represented using techniques and/or formats including Hypertext Mark-up Language (HTML), Extensible Markup Language (XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec). In other embodiments, custom and/or dedicated data communication techniques may also be used in place of, or in addition to, the data communication techniques described above.
In some embodiments, the lighting device C may be a strip, a bar, a pole, a flat light (containing multiple beads), or a spatially arranged light.
In some embodiments, the light device C may comprise a control unit and a light unit, and the control unit may be configured to control the light effect of the light unit according to the control device B.
In some embodiments, the light device C may include one or more light units to perform corresponding combinations according to actual use needs.
In some embodiments, each lighting unit may include one or more lamp beads, and the control unit may control brightness, color, and the like of the lamp beads of the lighting unit, and may also control the number of lights, thereby implementing different light effects of the lighting device.
In some embodiments, the positions and the number of the lighting devices can be set according to actual use requirements, and the light effect on the length, the height and the area can be controlled according to the positions of the lighting devices.
Referring to fig. 2, in some embodiments, the light control system a may further include a display device D, and the display device D is connected to the control device B in a wired or wireless manner.
In some embodiments, the display device D may display according to a video stream to be detected, so that the light effect corresponding to the video may be displayed through the lighting device while the video is displayed, thereby creating an immersive atmosphere and enhancing the user experience.
Optionally, the display device D may be, for example, a smart television, a tablet computer, a smart speaker with a display screen, a mobile phone, and the like, which is not limited in this application.
In some embodiments, the lighting control system a may further include a video splitting unit, configured to split the original video stream into at least two video streams to be detected, so that one of the video streams to be detected may be used by the display device D for display, and the other video stream to be detected may be used by the control device B for controlling the light effect of the lighting device C. For example, the video stream splitting unit may employ a video-to-two interface chip, a video distributor chip, and the like, which is not limited in this application.
In the embodiment of the present application, the control device B may further execute a light control method referred to in the following embodiments, which will be described below with reference to specific embodiments.
Referring to fig. 3, an embodiment of the present application provides a lamp efficiency control method, which can be applied to the lamp efficiency control system. Specifically, the lamp efficiency control method provided by the embodiment of the present application includes: step S110 to step S140.
In step S110, an audio stream in the video stream to be tested is acquired.
In the embodiment of the application, the video stream to be tested comprises picture frames for display and an audio stream for playing, and when the picture frames and the audio are played simultaneously, a sound picture seen by a user is formed. The audio stream in this document includes, but is not limited to, background music, and in order to control the lighting device according to the background music corresponding to the video, the control device needs to acquire the audio stream in the video stream to be tested. Specifically, the required audio stream may be obtained by decoding and extracting the video stream to be detected. Illustratively, the audio stream may be obtained by means of FFMpeg (Fast Forward Mpeg), Premiere (video editing tool), and the like, which is not limited in this application.
In step S120, the audio stream is subjected to framing processing to obtain a sequence of audio frames.
In an embodiment of the application, the control device frames an audio data stream to obtain a plurality of audio frames, the plurality of audio frames constituting a sequence of audio frames.
As shown in fig. 4, fig. 4 is a schematic diagram illustrating a time domain diagram of an audio stream in an embodiment of the present application, wherein the audio stream can be divided into a plurality of audio frames a by framing the audio stream. In an embodiment of the present application, wlen is the frame length, inc is the frame shift, and the next audio frame may be obtained by translating the frame shift inc, thereby dividing the audio stream into a plurality of audio frames, forming the sequence of audio frames.
Alternatively, to avoid missing the content of the audio stream, the length of the frame shift inc may be set to be less than or equal to the frame length of a single frame of audio frame. It is understood that, for an audio stream of the same length, the smaller the frame shift length is, the greater the number of audio frames obtained, so that subtle changes of the audio frames can be reflected more sensitively, but at the same time, the calculation amount is increased. The length of the frame shift can be set according to the actual requirements on the sensitivity and the calculation amount, which is not limited in this application.
And the audio stream is divided into a plurality of audio frames, and the audio frames are used for audio identification processing, so that the calculation is simple, and the calculation amount can be effectively reduced.
In step S130, one of the audio frames in the sequence of audio frames that is not traversed is sequentially obtained as a detection audio frame.
In the embodiment of the application, the control device may sequentially obtain one of the audio frames in the sequence of audio frames that is not traversed as the detection audio frame according to the time sequence, so as to control the current light effect of the lighting device according to the detection audio frame.
Illustratively, if the audio frame sequence includes a first audio frame, a second audio frame and a third audio frame. The time of the first audio frame is earlier than that of the second audio frame, and the time of the second audio frame is earlier than that of the third audio frame. The control device controls the light effect of the lighting device according to the first audio frame, and the audio frames which are not traversed in the audio frame sequence are the second audio frame and the third audio frame.
In some embodiments, one of the audio frames that is not traversed in the sequence of audio frames may be sequentially acquired as the detection audio frame at a preset sampling interval.
Optionally, the presentation time of the single control light effect is longer than the preset sampling interval, so that repeated triggering of the lighting equipment can be reduced.
In step S140, an audio recognition process is performed on the detected audio frame to obtain a recognition result corresponding to the detected audio frame.
In the embodiment of the application, the control device may perform audio identification processing on the detected audio frame to obtain an identification result corresponding to the detected audio frame. In other embodiments, the control device may also send the detected audio frame to the server, where the server performs audio recognition processing and returns a recognition result corresponding to the detected audio frame.
In some embodiments, a pre-trained neural network model may be used to perform audio recognition processing on the detected audio frames to obtain recognition results corresponding to the detected audio frames. The recognition result may be, for example, detecting a category corresponding to the audio frame, so as to perform light effect control according to the corresponding category.
Step S140 will be described in detail in a specific embodiment, and as shown in fig. 6, step S140 may include the following steps.
(1) And determining the probability of detecting the audio frame as a preset category.
Wherein the number of the preset categories is multiple.
In embodiments of the present application, audio features in detected audio frames may be extracted by an audio extraction algorithm. The audio transform algorithm may be, for example, FFT (fast Fourier transform), MFCC (Mel-scale Frequency spectrum Coefficients), and the like, and may be specifically set according to actual use requirements, which is not limited in this application.
In the embodiment of the application, a pre-trained neural network model can be adopted to perform audio recognition processing on the extracted audio features, and the probability that the audio frame is in the preset category is determined.
For example, the number of the preset categories may be 20, and the last layer of the neural network model may be normalized by softmax (normalized exponential function), so as to obtain a probability, which may also be referred to as a confidence, that the detected audio frame corresponds to each preset category.
In some embodiments, deep learning convolution classification algorithms such as Resnet (residual neural network), vgg (visual Geometry group) convolutional neural network, mobilene (lightweight convolutional network), and the like may also be used to determine the detected audio frame as the preset category, which is not limited in this application.
Illustratively, the preset categories may include, but are not limited to: the type of the flowing water sound, the type of the explosion sound, the type of the wind sound, the type of the rain sound, and the like can be specifically set according to actual use requirements, and the present application does not limit this.
In some embodiments, other categories may also be set in the preset categories. For example, the preset categories include: and if the identification result of the audio frame does not belong to any one of the first category, the second category and the third category, classifying the category of the audio frame into the other category.
(2) And determining the preset category with the highest probability as the music category corresponding to the detected audio frame.
In the embodiment of the application, after the probability of each preset category corresponding to the detected audio frame is obtained through the above steps, the preset category with the highest probability is used as the music category corresponding to the detected audio frame.
In step S150, the light effect of the lighting device is controlled according to the recognition result. And returning to the step of sequentially acquiring one audio frame which is not traversed in the audio frame sequence as the detected audio frame.
In an embodiment of the application, the control device controls the light effect of the lighting device according to the recognition result of the detected audio frame to present the light effect corresponding to the detected audio frame.
In an embodiment of the application, the control device may control the light effect of the lighting device according to the music category.
In some embodiments, after determining the music category corresponding to the detected audio frame, a further determination may be made, and if the probability of detecting the music category corresponding to the audio frame is higher than a preset category threshold, the light effect of the lighting device may be controlled according to the music category. Otherwise, even if the detected audio frame belongs to the music category, the light effect of the lighting equipment cannot be controlled according to the music category, that is, the detected audio frame is invalid and needs to be discarded, so that the light effect of the lighting equipment is not controlled.
It will be appreciated that the lower the preset category threshold setting, the higher the probability that the audio frame belongs to the corresponding music category, the more sensitive the control of the lighting device is triggered. Therefore, specific numerical values of the preset category threshold can be set according to actual use requirements, and the method is not limited in the application.
In some embodiments, each music category is pre-set with a corresponding control light effect. After the music category corresponding to the audio frame is determined, the control light effect corresponding to the music category can be determined, and then the lighting equipment is controlled to display the corresponding control light effect, so that the corresponding light effect is displayed according to the category of the audio frame, an immersive lighting atmosphere is built, and the user experience is improved.
In some embodiments, if the music category belongs to other categories, the lighting equipment is not controlled, otherwise, the control light effect corresponding to the music category is determined. That is, when an audio frame belongs to another category, the audio frame needs to be discarded, i.e., no control is performed on the lighting device. When the music category does not belong to other categories, subsequent processing is carried out, so that the accurate control of the light effect of the lighting equipment is ensured.
In some embodiments, when the light effect of the last detected audio frame is not fully displayed, if the control light effect corresponding to the current detected audio frame is determined at this time, controlling the lighting device that is displaying the light effect according to the light effect of the current detected frame may result in incomplete display of the light effect of the last detected audio frame. In order to ensure the effect of the lamp effect display, the current state of the lighting equipment is determined before the lamp effect control is carried out. And if the current state of the lighting equipment is a light effect showing state, namely the lighting equipment detects the light effect of the audio frame before showing, not controlling the lighting equipment. Otherwise, controlling the lighting equipment to display the control light effect corresponding to the current detected audio frame.
In some embodiments, the lighting device may be controlled to exhibit one or more control light effects upon determining the control light effect for controlling the lighting device. That is, the time required for controlling the complete single control lamp effect is assumed to be 2s, and the time required for controlling the two complete control lamp effects is assumed to be 4 s. Specifically, a preset number of times that the lighting device is required to exhibit the lamp effect may be preset. The number of the preset times can be set according to actual use needs, and the application is not limited to this.
In some embodiments, after determining the control light effect for controlling the lighting device, the lighting device may be further controlled to repeatedly perform the control light effect within a preset filtering time.
Optionally, the display time of the single control light effect is less than the preset filtering time, so that the complete control light effect is displayed at least once within the preset filtering time. For example, if the preset filtering time is 6s and the single control light effect is 2s, the corresponding control light effect is repeatedly displayed 3 times within the preset filtering time.
Alternatively, the presentation time of the single control light effect may be longer than the preset filtering time, so that only part of the control light effects are presented within the preset filtering time. The specific setting can be adjusted according to actual use needs, and the application does not limit the setting.
In an embodiment of the present application, the corresponding light effect may be set according to the corresponding music category. Lamp effects include, but are not limited to: the position, the number, the brightness, the color, the flicker frequency and the like of lighting equipment, so that the light effects matched with the music categories are combined according to actual needs. For example, when the music category is a pop category, the corresponding light effect may be that the current color is changed from 1s to red, and flickers at a flickering cycle of 0.05s, and after flickering for 1s, the color is changed from 1s to the original color. The concrete can be matched according to the actual use requirement at will, and the application does not limit the method.
In some embodiments, before step S110, the lamp efficiency control method of the embodiment of the present application further includes the following steps.
(1) An original video stream is acquired.
(2) Generating at least two paths of video streams to be detected based on the original video streams;
(3) and sending the one path of video stream to be tested to the display equipment so that the display equipment displays the one path of video stream to be tested according to the display equipment.
The light effect control method of the embodiment of the application can also be applied to scenes for simultaneously performing video display and performing light effect control by using video streams.
Specifically, the original video stream may be obtained through a video source, which may be, for example, a television program, a game screen, a network video, and the like.
Further, the original video stream is generated into two paths of video streams to be detected. As shown in fig. 6, the control device accesses the video stream through the hardware interface, divides the original video stream into two parts after acquiring the original video stream to generate at least two paths of video streams to be detected, wherein one path is used for controlling the display device to display specific image content, the other path is used for converting the video stream into a video stream type that can be received by the multimedia processing chip through the video interface, and then frames the video stream to be detected through the multimedia processing chip to obtain a video frame image sequence, thereby controlling the light effect of the lighting device to be changed. The hardware Interface may be, for example, VGA (Video Graphics Array), DP (display Interface), HDMI (High Definition Multimedia Interface), DVI (Digital Visual Interface), etc.; the video Interface may be, for example, MIPI (Mobile Industry Processor Interface), LVDS (Low-Voltage Differential Signaling), or the like; the multimedia processing Chip may be, for example, an SoC (System on Chip) Chip. Furthermore, looping out the original video stream can be realized by calling a video capture card.
It can be understood that the contents of the two paths of video streams to be detected are the same, wherein one path of video stream to be detected is used for controlling the light effect, and the other path of video stream to be detected is used for controlling the display device to display, so that the consistency of the light effect control and the video display is ensured.
In some embodiments, the control device may also control the lighting device in conjunction with the image frames and audio frames in the video stream. For example, if a monster appears in the image frame and the music category corresponding to the audio frame is a horror category, the flash frequency, brightness change and the like of the lamp effect of the horror category can be adjusted according to the number and distance of the monsters, so that richer lamp effect control is presented.
In some embodiments, in order to ensure that the lamp effect immerses the atmosphere, the lamp effect control and the video display need to be synchronized, and since the display device and the lighting device are controlled by two paths of video streams to be detected, after the user operates the display device, the display state of the display device may be changed, and if the lighting device still performs the lamp effect control, the video display and the lamp effect control are not synchronized, which affects the user experience.
In order to solve the synchronization problem, in an embodiment of the present application, the lamp efficiency control method may further include: and when the display equipment responds to the control operation and enters a display state corresponding to the control operation, controlling the lighting equipment to enter a light effect state corresponding to the control operation.
Alternatively, the control operations may include, but are not limited to: pause operation, shutdown operation, and restart operation. The user can send a control instruction to the display device through the terminal device so as to control and operate the display device.
Optionally, when the display device receives the control instruction, the display device may send a light effect control instruction corresponding to the control instruction to the control device, so that the control device synchronously controls the lighting device.
Optionally, when the user sends the control instruction, the user sends two control instructions simultaneously, where one control instruction is sent to the display device to perform control operation on the display device, and the other control instruction is sent to the control device to perform corresponding light effect control on the control device.
In the embodiment, by acquiring an audio stream in a video stream to be detected, framing the audio stream to obtain an audio frame sequence; sequentially acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame; the method comprises the steps of carrying out audio identification processing on a detected audio frame to obtain an identification result corresponding to the detected audio frame, controlling the light effect of the lighting equipment according to the identification result, framing the audio stream, carrying out audio identification processing by taking the audio frame as a unit, and carrying out real-time control on the light effect of the lighting equipment by using the identification result of the audio frame, so that the control mode of the light effect is enriched.
Referring to fig. 6, a lamp effect control device 200 according to an embodiment of the present invention is shown, where the lamp effect control device 200 includes: an audio stream acquisition module 210, an audio frame sequence acquisition module 220, a detected audio frame determination module 230, an identification module 240, and a light effect control module 250.
The audio stream acquiring module 210 is configured to acquire an audio stream in a video stream to be tested.
The audio frame sequence acquiring module 220 is configured to perform framing processing on the audio stream to obtain an audio frame sequence.
The detected audio frame determining module 230 is configured to sequentially obtain one of the audio frames in the sequence of audio frames that is not traversed as a detected audio frame.
The identification module 240 is configured to perform audio identification processing on audio frames in the audio frame sequence to obtain an identification result corresponding to the audio frames.
The light effect control module 240 is configured to control the light effect of the lighting device according to the recognition result.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. For any processing manner described in the method embodiment, all the processing manners may be implemented by corresponding processing modules in the apparatus embodiment, and details in the apparatus embodiment are not described again.
In the several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or modules may be in an electrical, mechanical or other form.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
Referring to fig. 7, based on the light effect control method, another electronic device 300 including a processor capable of executing the light effect control method is provided in the embodiment of the present application, where the electronic device 300 further includes one or more processors 310, a memory 320, and one or more application programs. The memory 320 stores programs that can execute the content of the foregoing embodiments, and the processor 310 can execute the programs stored in the memory.
The Memory 320 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 420 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, and the like. The storage data area may also store data created by the terminal in use.
Referring to fig. 8, a block diagram of a computer-readable storage medium 400 according to an embodiment of the present disclosure is shown. The computer-readable storage medium 400 has stored therein a program code 410, the program code 410 being capable of being invoked by a processor to perform the method described in the above method embodiments.
The computer-readable storage medium 400 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable and programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 400 includes a non-volatile computer-readable storage medium. The computer readable storage medium 400 has storage space for program code for performing any of the method steps described above. The program code can be read from or written to one or more computer program products. The program code may be compressed, for example, in a suitable form.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the lamp effect control method described in the above-mentioned various alternative embodiments.
To sum up, the method, the system, the apparatus, the electronic device and the storage medium for controlling light effect provided by the present application perform framing processing on an audio stream to obtain an audio frame sequence by obtaining the audio stream in a video stream to be tested; the method comprises the steps of carrying out audio identification processing on audio frames in an audio frame sequence to obtain identification results corresponding to the audio frames, controlling the light effect of the lighting equipment according to the identification results, framing the audio stream, carrying out audio identification processing by taking the audio frames as units, and carrying out real-time control on the light effect of the lighting equipment by using the identification results of the audio frames, so that the control mode of the light effect is enriched.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (14)
1. A method of lamp efficiency control, the method comprising:
acquiring an audio stream in a video stream to be detected;
performing framing processing on the audio stream to obtain a sequence of audio frames;
sequentially acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame;
performing audio identification processing on the detected audio frame to obtain an identification result corresponding to the detected audio frame;
and controlling the light effect of the lighting equipment according to the identification result, and returning to execute the step of sequentially acquiring one audio frame which is not traversed in the audio frame sequence as the detected audio frame.
2. The method according to claim 1, wherein the performing audio recognition processing on the detected audio frame to obtain a recognition result corresponding to the detected audio frame comprises:
determining the probability that the detected audio frame belongs to a preset category; the number of the preset categories is multiple;
determining a preset category with the highest probability as a music category corresponding to the detected audio frame;
the lamp effect according to identification result control lighting equipment includes:
determining control light effects corresponding to the music categories;
and controlling the lighting equipment to display the control light effect.
3. The method of claim 2, wherein the controlling the lighting device to exhibit the control light effect comprises:
and if the probability of the music category is higher than a preset category threshold value, controlling the lighting equipment to display and control the light effect.
4. The method of claim 2, wherein the determining the control light effect corresponding to the music category comprises:
if the music category belongs to other categories, the lighting equipment is not controlled;
otherwise, determining the control light effect corresponding to the music category.
5. The method of claim 2, wherein the controlling the lighting device to exhibit the control light effect comprises:
determining the current state of the lighting equipment;
if the current state is a lamp effect showing state, the lighting equipment is not controlled;
otherwise, controlling the lighting equipment to display the control light effect.
6. The method of claim 2, wherein the controlling the lighting device to exhibit the control light effect comprises:
controlling the lighting equipment to repeatedly display the control light effect within a preset filtering time; and the single display time of the control light effect is less than the preset filtering time.
7. The method of claim 2, wherein the sequentially obtaining one of the audio frames in the sequence of audio frames that is not traversed as a detection audio frame comprises:
acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame according to a preset sampling interval in sequence;
and the single showing time of the control light effect is longer than the preset sampling interval.
8. The method according to any of claims 1 to 7, wherein before the obtaining the video stream to be tested, the method further comprises:
acquiring an original video stream;
generating at least two paths of video streams to be detected based on the original video streams;
sending one of the paths of video streams to be tested to display equipment so that the display equipment can display the video streams to be tested according to the one of the paths of video streams;
acquiring an audio stream in a video stream to be detected; the method comprises the following steps:
and acquiring the audio stream in the other path of the video stream to be tested.
9. The method of claim 8, further comprising:
and when the display equipment responds to the control operation and enters a display state corresponding to the control operation, controlling the lighting equipment to enter a light effect state corresponding to the control operation.
10. The method of any one of claims 1 to 8, wherein the lighting device comprises one or more lighting units.
11. A lamp efficiency control system, comprising: the lighting system comprises a control device and lighting equipment connected with the control device;
wherein the control device is configured to perform the lamp effect control method of any one of claims 1 to 10.
12. A lamp effect control device, comprising:
the audio stream acquisition module is used for acquiring an audio stream in the video stream to be detected;
an audio frame sequence obtaining module, configured to perform framing processing on the audio stream to obtain an audio frame sequence;
the detection audio frame determining module is used for sequentially acquiring one audio frame which is not traversed in the audio frame sequence as a detection audio frame;
the identification module is used for carrying out audio identification processing on the detected audio frame to obtain an identification result corresponding to the detected audio frame;
and the light effect control module is used for controlling the light effect of the lighting equipment according to the identification result.
13. An electronic device, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the light effect control method of any of claims 1-10.
14. A computer-readable storage medium having stored thereon program code that can be invoked by a processor to perform a lamp effect control method according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111666958.0A CN114422846A (en) | 2021-12-31 | 2021-12-31 | Lamp effect control method, system, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111666958.0A CN114422846A (en) | 2021-12-31 | 2021-12-31 | Lamp effect control method, system, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114422846A true CN114422846A (en) | 2022-04-29 |
Family
ID=81270724
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111666958.0A Pending CN114422846A (en) | 2021-12-31 | 2021-12-31 | Lamp effect control method, system, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114422846A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020038157A1 (en) * | 2000-06-21 | 2002-03-28 | Dowling Kevin J. | Method and apparatus for controlling a lighting system in response to an audio input |
| CN106851008A (en) * | 2017-03-01 | 2017-06-13 | 维沃移动通信有限公司 | A kind of lamp light control method and mobile terminal |
| CN107135578A (en) * | 2017-06-08 | 2017-09-05 | 复旦大学 | Intelligent music chord atmosphere lamp system based on TonaLighting regulation technologies |
| US20180061438A1 (en) * | 2016-03-11 | 2018-03-01 | Limbic Media Corporation | System and Method for Predictive Generation of Visual Sequences |
| WO2021074678A1 (en) * | 2019-10-17 | 2021-04-22 | Ghose Anirvan | A system to provide synchronized lighting and effects for cinema and home |
| CN113573442A (en) * | 2021-07-16 | 2021-10-29 | 广东和音元视电子科技有限公司 | Method, apparatus, and computer-readable storage medium for audio control of scenes |
| CN113613369A (en) * | 2021-08-11 | 2021-11-05 | 深圳市智岩科技有限公司 | Light effect control method, device, equipment and storage medium |
-
2021
- 2021-12-31 CN CN202111666958.0A patent/CN114422846A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020038157A1 (en) * | 2000-06-21 | 2002-03-28 | Dowling Kevin J. | Method and apparatus for controlling a lighting system in response to an audio input |
| US20180061438A1 (en) * | 2016-03-11 | 2018-03-01 | Limbic Media Corporation | System and Method for Predictive Generation of Visual Sequences |
| CN106851008A (en) * | 2017-03-01 | 2017-06-13 | 维沃移动通信有限公司 | A kind of lamp light control method and mobile terminal |
| CN107135578A (en) * | 2017-06-08 | 2017-09-05 | 复旦大学 | Intelligent music chord atmosphere lamp system based on TonaLighting regulation technologies |
| WO2021074678A1 (en) * | 2019-10-17 | 2021-04-22 | Ghose Anirvan | A system to provide synchronized lighting and effects for cinema and home |
| CN113573442A (en) * | 2021-07-16 | 2021-10-29 | 广东和音元视电子科技有限公司 | Method, apparatus, and computer-readable storage medium for audio control of scenes |
| CN113613369A (en) * | 2021-08-11 | 2021-11-05 | 深圳市智岩科技有限公司 | Light effect control method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109547819B (en) | Live list display method and device and electronic equipment | |
| CN107436921B (en) | Video data processing method, device, equipment and storage medium | |
| CN112669417B (en) | Virtual image generation method and device, storage medium and electronic equipment | |
| WO2022111110A1 (en) | Virtual video livestreaming processing method and apparatus, storage medium, and electronic device | |
| US11778263B2 (en) | Live streaming video interaction method and apparatus, and computer device | |
| CN114025219B (en) | Rendering method, device, medium and equipment for augmented reality special effects | |
| US11451858B2 (en) | Method and system of processing information flow and method of displaying comment information | |
| US20210406305A1 (en) | Image deformation control method and device and hardware device | |
| CN113542875B (en) | Video processing method, device, electronic equipment and storage medium | |
| US20220078530A1 (en) | Video labeling method and apparatus, device, and computer-readable storage medium | |
| US20230072759A1 (en) | Method and apparatus for obtaining virtual image, computer device, computer-readable storage medium, and computer program product | |
| CN112652041B (en) | Virtual image generation method and device, storage medium and electronic equipment | |
| CN110677718B (en) | Video identification method and device | |
| WO2019227429A1 (en) | Method, device, apparatus, terminal, server for generating multimedia content | |
| WO2023045635A1 (en) | Multimedia file subtitle processing method and apparatus, electronic device, computer-readable storage medium, and computer program product | |
| EP3660632A1 (en) | Display apparatus and control method thereof | |
| CN116168134B (en) | Digital person control method, digital person control device, electronic equipment and storage medium | |
| CN111405301A (en) | Screen recording interaction method and device for terminal, computer equipment and storage medium | |
| CN115089966B (en) | Video rendering method, system and related equipment applied to cloud game | |
| WO2023142434A1 (en) | Rendering engine testing method and apparatus, device, system, storage medium, computer program and computer program product | |
| CN114007064B (en) | Special effect synchronous evaluation method, device, equipment and storage medium | |
| CN114339069A (en) | Video processing method and device, electronic equipment and computer storage medium | |
| US11737192B2 (en) | Lighting effect control method, system, apparatus, electronic device, and storage medium | |
| CN114422846A (en) | Lamp effect control method, system, device, electronic equipment and storage medium | |
| CN115474088B (en) | Video processing method, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |