WO2019114582A1 - Procédé et appareil de traitement d'images vidéo, support de stockage informatique et terminal - Google Patents

Procédé et appareil de traitement d'images vidéo, support de stockage informatique et terminal Download PDF

Info

Publication number
WO2019114582A1
WO2019114582A1 PCT/CN2018/119266 CN2018119266W WO2019114582A1 WO 2019114582 A1 WO2019114582 A1 WO 2019114582A1 CN 2018119266 W CN2018119266 W CN 2018119266W WO 2019114582 A1 WO2019114582 A1 WO 2019114582A1
Authority
WO
WIPO (PCT)
Prior art keywords
music
beat point
video
beat
point
Prior art date
Application number
PCT/CN2018/119266
Other languages
English (en)
Chinese (zh)
Inventor
危文
袁少龙
周宇涛
丘智鉴
颜乐驹
魏启征
李敬
Original Assignee
广州市百果园信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201711353131.8A external-priority patent/CN108111909A/zh
Priority claimed from CN201711481160.2A external-priority patent/CN108259925A/zh
Priority claimed from CN201711476832.0A external-priority patent/CN108322802A/zh
Priority claimed from CN201711474216.1A external-priority patent/CN108259983A/zh
Priority claimed from CN201711481177.8A external-priority patent/CN108259984A/zh
Application filed by 广州市百果园信息技术有限公司 filed Critical 广州市百果园信息技术有限公司
Publication of WO2019114582A1 publication Critical patent/WO2019114582A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/57—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for processing of video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs

Definitions

  • the present invention relates to the field of image processing technologies, and in particular, to a video image processing method.
  • the music that can be selected can only be the music file in the terminal local or video application, and the video image in the playing video is often not related to the played music, and the music is simply added to the playing video.
  • the music played in the playing video is not sufficiently appealing in terms of hearing and visual, and the interactive way of simply selecting the music to play in the field of video live broadcasting cannot effectively improve the interaction enthusiasm between the viewer and the anchor, and is insufficient to satisfy the live video.
  • the demand which in turn affects the user's experience satisfaction.
  • the prior art texture application can usually simply add a user-selected texture to the video image, and the texture display mode is single, failing to fully mobilize the user.
  • the interaction with the application the user has less fun in the shooting process, and the user experience satisfaction is not high.
  • an embodiment of the present invention provides a video image processing method, including: acquiring a music signal of music to be played in a play video, detecting a beat point of the music according to the music signal; determining to play When playing music in the video, a beat point appears at the current play position; the special effect corresponding to the beat point is acquired; and the image in the play video is processed according to the special effect to obtain a video image including the special effect.
  • an embodiment of the present invention provides a video image processing method, including the steps of: identifying an audio feature of a sound in a play video; downloading, from a server, a music matching the audio feature and a beat point of the music The effect of playing the music in the play video, determining that a beat point occurs at a current play position of the music; processing the image in the play video according to the determined effect corresponding to the beat point, and obtaining the special effect Video image.
  • an embodiment of the present invention provides a video image processing method, including the steps of: acquiring a music signal of music to be played in a play video; determining whether a beat point and a music play for saving the music are pre-stored.
  • a beat point description file of the correspondence of the position if yes, acquiring the beat point description file; if not, detecting a beat point of the music according to the music signal, according to the detected beat point and the music play position Generating a beat point description file; determining, according to the beat point description file, that a beat point appears at a current play position when playing music in the play video; acquiring an effect corresponding to the beat point; and playing the video according to the special effect
  • the image is processed to obtain a video image containing the effect.
  • an embodiment of the present invention provides a music gift processing method in a live video broadcast, comprising: receiving a music gift sent by a viewer in a live video; the music gift includes music to be played in a live video; a music signal of the music, detecting a beat point of the music according to the music signal; determining to play a beat point in a current play position when playing the music in the music present in a live video; acquiring the beat point corresponding to The special effect; processing the image in the live video according to the special effect, and obtaining a live video of the video including the special effect.
  • an embodiment of the present invention provides a texture processing method for a video image, including the steps of: acquiring an image in a playback video, adding a texture to the image; and acquiring music of a music to be played in the played video. a signal, detecting a beat point of the music according to the music signal; determining that a beat point occurs at a current play position when playing music in the play video; acquiring a texture effect corresponding to the beat point; according to the map effect The texture in the image is processed to obtain a video image containing the texture effect.
  • an embodiment of the present invention provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the program is executed by the processor, the first aspect, the second aspect, and the first The method of the third aspect, the fourth aspect or the fifth aspect.
  • an embodiment of the present invention provides a terminal, including: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory And configured to be executed by the one or more processors, the one or more programs configured to: the method of the first aspect, the second aspect, the third aspect, the fourth aspect, or the fifth aspect.
  • the invention has the following beneficial effects:
  • the video image processing method provided by the present invention performs beat point detection on music to be played in a play video, and displays the beat point corresponding to the beat point in the video image according to the detected beat point during the music playing process.
  • Special effects so that the special effects displayed on the video image are closely related to the beat point of the selected video playing music, thereby improving the audio and the visual appeal of the music played in the video, increasing the interest of the video application and improving User experience satisfaction.
  • the beat point of the selected video playing music is quickly and accurately detected; and the present invention can select the type of the desired beat point according to the type of the special effect required by the video and the type of the music.
  • the method of detecting the beat point to achieve an accurate beat point by using an appropriate method, and can reduce the amount of calculation, shorten the detection time, and further ensure that the special effect displayed on the video image is close to the beat point of the selected video playing music. Relevance.
  • the video image processing method provided by the present invention can know the music that the user wants to play by recognizing the sound in the playing video, and the music played by the user in the playing video is no longer limited to the existing music file, and can be hummed.
  • the song segment quickly acquires the corresponding music, and then displays the corresponding special effect in the video image according to the beat point of the music during the music playing process, thereby realizing playing the music in the playing video and obtaining the special effect corresponding to the music beat point.
  • Video image expands the way of acquiring music, and can quickly and conveniently obtain the music to be played, the interaction between the user and the video application is strong, and the special effect displayed on the video image is closely related to the beat point of the music obtained through the sound input.
  • the music to be played in the live video can be confirmed by recognizing the humming sound of the anchor or the viewer, which can meet the needs of the live broadcast, highlight the live atmosphere, promote the interaction between the anchor and the viewer, and further increase the video. The fun of the app.
  • the video image processing method, the computer readable storage medium, and the terminal provided by the present invention obtain the beat point corresponding to the beat point description file by acquiring the beat point description file of the music or intelligently generating the beat point description file according to the music signal.
  • the special effect is to make the special effects displayed on the video image closely related to the beat point of the selected video playing music, thereby improving the visual and visual appeal of the music played in the video, and increasing the interest of the video application.
  • the music gift processing method provided by the present invention performs beat point detection on music in a music gift sent by the viewer to the anchor, and displays the beat point corresponding to the beat point in the video image according to the detected beat point during the playing process.
  • Special effects so that the special effects displayed on the video image are closely related to the beat points of the music selected by the viewer, thereby improving the auditory and visual appeal of the music played in the live video; and by playing the music with the anchor and the viewer
  • the combination of gift giving behavior enriches the interaction between the viewer and the anchor in the live video broadcast, significantly improves the interaction enthusiasm between the viewer and the anchor, effectively highlights the live broadcast atmosphere, meets the demand for live video, and increases the interest of the live video application.
  • the texture processing method of the video image provided by the present invention by adding a texture to the video image, performing beat point detection on the music to be played in the played video, and according to the detected beat point during the music playing process Displaying the texture effect corresponding to the beat point in the video image, so that the texture effect displayed on the video image is closely related to the beat point of the selected video playing music, and the music played in the playing video and the displayed texture are improved.
  • Hearing and visual appeal; and users can express their own personality through custom-set music, texture and texture effects and can edit and play personalized video in real time. This method satisfies the user's personalized video design needs. To increase the fun of video applications and the interactivity of apps and users.
  • FIG. 1 is a flowchart of a method for processing a video image according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a method for processing a video image according to an embodiment of the present invention
  • FIG. 3 is a flowchart of a method for processing a video image according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of a method for processing a music gift according to an embodiment of the present invention.
  • FIG. 5 is a flowchart of a method for mapping a video image according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
  • the embodiment of the invention provides a video image processing method. As shown in FIG. 1 , the method includes:
  • Step S101 Acquire a music signal of music to be played in the played video, and detect a beat point of the music according to the music signal.
  • the music to be played in the playing video may be the music selected from the pre-stored music of the video application, or the music loaded in the video application may be selected from the pre-stored music of the user terminal, and
  • the live music played by the other device is obtained by the microphone in the video application, and the source of the music is not limited in this embodiment.
  • the action of acquiring the music signal is performed after the user selects the music to be played in the playing video, and the action of selecting the music may be performed before the video is played, or may be performed in the playing video.
  • the music when applied to the field of video live broadcast, the music may be selected before the live broadcast of the video or in the live broadcast of the video, and the terminal may immediately acquire the music signal of the selected music, and detect the beat of the music according to the music signal. point.
  • the music is selected before the video is played, that is, before the video is recorded, and after the selected music, the terminal immediately acquires the music signal of the selected music, and according to the music signal. A beat point of the music is detected.
  • the beat point of the music detected according to the music signal may be implemented by using various beat point detection methods.
  • the beat point includes a strong beat point and a weak beat point; the method includes: filtering the music signal, filtering and performing short-time Fourier transform to obtain a spectrum; according to the spectrum, The energy change value of the detection point is determined; according to the energy change value, a strong beat point or a weak beat point is detected at the detection point.
  • the beat point includes a strong beat point and a weak beat point; the method includes: performing weighting processing on the music signal to obtain a weighted music signal; according to the weighted music The energy intensity value of the signal detects a strong beat point or a weak beat point at the detection point.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy intensity value of the music signal; and counting each adjacent two candidates according to each candidate beat point The time interval between the frames in which the beat points are located; according to the time interval, it is detected that the candidate beat points correspond to strong beat points corresponding to the detected points.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy variation difference of the music signal of the detected point; and according to the candidate beat point, each phase Two adjacent candidate beat points are taken as the signal starting point to intercept two pieces of music signals; according to the comparison result of the two pieces of music signals, it is detected that the candidate beat points corresponding to the detected points have strong beat points.
  • Step S102 When it is determined that the music is played in the playing video, a beat point appears at the current playing position.
  • each beat point detected in step S101 corresponds to a different play position of the music.
  • the music is played in the play video, if the current play position of the music corresponds to the beat point detected in step S101, it is determined. Whether the beat point is a strong beat point or a weak beat point.
  • Step S103 Acquire an effect corresponding to the beat point.
  • the special effect may be a material such as a fireworks, a love, a snowflake, etc., and the specific expression of the material is not limited in this embodiment.
  • Different shape feature parameters may be set for the same material, and the shape feature parameters include a size parameter, a color parameter, and the like, which are not limited in this embodiment.
  • the beat point in a piece of music includes a strong beat point and a weak beat point
  • the material in the special effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different.
  • Step S104 Processing an image in the played video according to the special effect to obtain a video image including the special effect.
  • the video in the effect is obtained by acquiring the material in the special effect and synthesizing the material and the image in the play video in a layer superposition manner to obtain a video image including the material in the special effect.
  • the video image including the special effect may be obtained by integrating data of the special effect with the image or modifying the image according to the shape characteristic parameter of the material in the special effect.
  • data of the effect data can be integrated with the image data to obtain a video image data packet, and the data packet is sent to the client, so that the client displays the video image containing the special effect.
  • the image may be scaled according to the parameter by obtaining the shape feature parameter of the material in the special effect to obtain a video image that can achieve the effect of the special effect.
  • the video image processing method provided by the embodiment performs beat point detection on the music to be played in the played video, and displays the special effect corresponding to the beat point in the video image according to the detected beat point during the music playing process.
  • the special effects displayed on the video image closely related to the beat point of the selected video playing music, thereby increasing the audio and visual appeal of the music played in the video, increasing the interest of the video application and improving the user experience. Satisfaction.
  • the video image processing method provided in this embodiment combines multiple beat point detection methods to quickly and accurately detect the beat point of the selected video playing music, further ensuring the special effects displayed on the video image and the selected video.
  • the beat points of playing music are closely related to further improve the satisfaction of the user experience.
  • Step S101 includes:
  • the candidate beat point is obtained according to the energy intensity value of the music signal, and the time interval between frames of each adjacent two candidate beat points is counted according to each candidate beat point, and the detection is performed according to the time interval. A strong beat point appears at the detection point corresponding to the candidate beat point;
  • the candidate beat point is obtained according to the energy variation difference of the music signal of the detection point, and according to the candidate beat point, two adjacent music beat signals are taken as the signal starting point of each adjacent two candidate beat points. According to the consistent result of the two pieces of music signals, it is detected that the candidate beat points correspond to strong beat points corresponding to the detected points;
  • the music signal is weighted to obtain a weighted music signal, and according to the energy intensity value of the weighted music signal, a strong beat point or a weak beat point is detected at the detection point;
  • the music signal is filtered, and after filtering, the short-time Fourier transform is performed to obtain a spectrum, and according to the spectrum, the energy change value of the detection point is determined, and according to the energy change value, the weak beat of the detection point is detected. Point or strong candidate beat points.
  • different beat point detection methods are corresponding for different types of required detection beat points and detection criteria.
  • the determining whether to detect a strong beat point or detecting a strong beat point and a weak beat point comprises:
  • Get the type of special effect required for the video judge whether to detect strong beat points according to the type of special effects required by the video, or detect strong beat points and weak beat points;
  • the intensity value detection or the change value detection is used, including:
  • the type of the music is acquired, and whether the intensity value or the change value is used is determined according to the type;
  • the intensity value or the change value is detected, including:
  • the type of the music is acquired, and whether the intensity value or the change value is detected is determined according to the type.
  • the type of the desired beat point is determined by the type of the effect required for the acquired video.
  • the effect type required for the video is the default effect type selected by the user or the video application. For example, the user wants to have endless effects in the playing video, so it is judged according to the type of special effect required by the video that both the strong beat point and the weak beat point are detected.
  • the detection criteria are judged by the type of music acquired.
  • the type of music acquired is rock, and the music signal of the music type tends to have a high intensity value, but the change value is not obvious, so the beat point of the music is detected by detecting the intensity value according to the type thereof.
  • the type of the desired beat point can be selected according to the type of the special effect required by the video, and then the method of detecting the desired beat point can be selected according to the type of the music, so as to obtain an accurate beat point by using an appropriate method. Moreover, the amount of calculation can be reduced, the detection time is shortened, and the special effect displayed on the video image is closely related to the beat point of the selected video playing music, thereby further improving the satisfaction of the user experience.
  • step S101 the method further includes: recording a correspondence between the music playing position and the beat point.
  • the beat point of the music is detected according to the music signal
  • the beat point is associated with the corresponding music play position, and the corresponding relationship is recorded.
  • the method for establishing the corresponding relationship is not limited in this embodiment, and may be a manner of adding label information.
  • the signal data of the music playing position corresponding to the beat point is added with tag information, and the tag information carries information indicating that the music playing position has a beat point and the beat point is a strong beat point or a weak beat point.
  • Step S102 includes: determining, according to the correspondence, that a beat point occurs at a current play position when playing music in the play video.
  • the correspondence record of the music play position and the beat point is acquired, and the corresponding relationship is extracted, and then according to the The correspondence determines that a beat point occurs at the current play position and determines that the beat point is a strong beat point or a weak beat point.
  • the embodiment by recording the correspondence between the music playing position and the beat point, it is possible to easily and quickly determine whether the current playing position has a beat point and whether the beat point is a strong beat point or a weak beat point.
  • the video is a live video, that is, the method in this embodiment is mainly applied to the field of video live broadcast.
  • the method further includes:
  • Step S103 includes: obtaining an effect corresponding to the beat point from the obtained special effects group.
  • the material of the special effects of the special effect group is the same, but the shape characteristic parameters of the material are different.
  • the specified effect group may be specified by the anchor or user of the live broadcast, and the specified effect group corresponds to the anchor live broadcast feature of the anchor, for example, the live broadcast of the anchor is performed by an animation.
  • the theme, and the anchor pre-specifies the use of the animation effect-themed effect group, so the special effects displayed in the live video of the live broadcast are obtained from the specified effect group.
  • the scenario corresponding to the live broadcast may be determined by extracting scene features corresponding to the location, weather, and landscape, such as a cafe, a sunny outdoor, a seaside, etc., and is not performed in this embodiment. limited.
  • the effect group matching the scene has a correlation with the scene feature of the scene.
  • the preset number is a value greater than or equal to 2, which may be a value of 3, 5, 6, etc., which is not limited in this embodiment.
  • the type of music includes rock, pop, and the like, and the type of the music is determined according to the acquired beat point of the music.
  • step S104 the method further includes:
  • the video image processing method provided by the present invention is applied to the field of live video, by selecting special effects according to the anchor intention, the type of music, and the will of the viewer, the requirements of the live broadcast can be met, the live broadcast atmosphere can be enhanced, and the interaction between the anchor and the audience can be better promoted. Improve user satisfaction.
  • the embodiment of the invention provides a video image processing method. As shown in FIG. 2, the method includes:
  • Step S201 Identify an audio feature of the sound in the played video.
  • Step S202 Download, from the server, the music that matches the audio feature and the special effect corresponding to the beat point of the music.
  • the manner in which the user acquires the music to be played in the played video is no longer limited to selecting the existing music file in the terminal or the video application, but by identifying the sound in the played video, and then determining the matching.
  • the music corresponding to the sound is downloaded from the server to realize the acquisition of the music to be played.
  • the terminal captures the sound in the played video and extracts the audio feature of the sound, and then sends the audio feature to the server, so that the server performs the audio feature with the music in the preset music library that it holds.
  • the matching is traversed to determine the music corresponding to the audio characteristics of the sound.
  • the sound may be a sound emitted by the user in the playing video, or may be a sound emitted by other terminal devices.
  • the terminal before performing an action of recognizing the audio feature of the sound in the played video, it is also necessary to determine whether the user transmits the audio feature recognition request. For example, when the user triggers the listening song function key of the video application interface, the terminal begins to perform the action of recognizing the audio feature of the sound in the playing video.
  • the special effect may be a material such as a fireworks, a love, a snowflake, etc., and the specific expression of the material is not limited in this embodiment.
  • Different shape feature parameters may be set for the same material, and the shape feature parameters include a size parameter, a color parameter, and the like, which are not limited in this embodiment.
  • the beat point in a piece of music includes a strong beat point and a weak beat point, and the material in the special effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different.
  • the special effect corresponding to the beat point of the music downloaded from the server in step 202 refers to the material of the different shape feature parameters corresponding to the strong beat point and the weak beat point of the music respectively, wherein the special effect may be the user advance Setting the special effect corresponding to the music may also be a default effect corresponding to the music by the server.
  • Step S203 playing the music in the play video, and determining that a beat point occurs at a current play position of the music.
  • whether the beat point occurs in the current play position of the music may be determined according to the corresponding relationship, and the beat is determined. Whether the point is a strong beat or a weak beat.
  • Step S204 Processing an image in the played video according to the determined special effect corresponding to the beat point to obtain a video image including the special effect.
  • the material in the special effect corresponding to the beat point downloaded from the server in step S202 is acquired, and the material in the play video is synthesized in a layer superposition manner to obtain A video image containing the material in the effect.
  • the obtaining of the video image including the special effect may also be implemented by integrating the special effect with the image data or modifying the image according to the shape characteristic parameter of the material in the special effect.
  • data of the effect data can be integrated with the image data to obtain a video image data packet, and the data packet is sent to the client, so that the client displays the video image containing the special effect.
  • the image may be scaled according to the parameter by obtaining the shape feature parameter of the material in the special effect to obtain a video image that can achieve the effect of the special effect.
  • the video image processing method provided by the present invention can know the music that the user wants to play by recognizing the sound in the playing video, and the music played by the user in the playing video is no longer limited to the existing music file, and can be quickly sung by the sing song piece.
  • the corresponding music is obtained, and then the corresponding special effect is displayed in the video image according to the beat point of the music during the music playing, thereby realizing playing the music in the playing video and obtaining a video image containing the special effect corresponding to the music beat point.
  • the method expands the way of acquiring music, and can quickly and conveniently obtain the music to be played, the interaction between the user and the video application is strong, and the special effect displayed on the video image is closely related to the beat point of the music obtained through the sound input. This improves the visual and visual appeal of the music played in the video, significantly increasing the interest of the video application and increasing user experience satisfaction.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the second embodiment, where
  • the method further includes:
  • the determining that a beat point occurs at a current play position of the music includes:
  • the music play position and the beat point recorded in the beat point description file are acquired.
  • Corresponding relationship determines that a beat point occurs at a current play position of the music, and may also determine whether the beat point is a strong beat point or a weak beat point.
  • the use of the beat point description file can achieve the corresponding relationship quickly and conveniently, and the video image containing the special effect corresponding to the music beat point can be further obtained after a very short file loading time, thereby further improving the user experience satisfaction. degree.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the second embodiment, where
  • the method further includes:
  • the determining that a beat point occurs at a current play position of the music includes:
  • the beat point of the music detected according to the music signal may be implemented by using various beat point detection methods.
  • the beat point includes a strong beat point and a weak beat point; the method includes: filtering the music signal, filtering and performing short-time Fourier transform to obtain a spectrum; according to the spectrum, The energy change value of the detection point is determined; according to the energy change value, a strong beat point or a weak beat point is detected at the detection point.
  • the beat point includes a strong beat point and a weak beat point; the method includes: performing weighting processing on the music signal to obtain a weighted music signal; and according to the weighted music The energy intensity value of the signal detects a strong beat point or a weak beat point at the detection point.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy intensity value of the music signal; and counting each adjacent two candidates according to each candidate beat point The time interval between the frames in which the beat points are located; according to the time interval, it is detected that the candidate beat points correspond to strong beat points corresponding to the detected points.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy variation difference of the music signal of the detected point; and according to the candidate beat point, each phase Two adjacent candidate beat points are taken as the signal starting point to intercept two pieces of music signals; according to the comparison result of the two pieces of music signals, it is detected that the candidate beat points corresponding to the detected points have strong beat points.
  • the correspondence between the music playing position and the beat point can be confirmed by detecting the beat point in real time, and the plurality of beat point detecting methods can quickly and accurately detect the beat point of the music corresponding to the sound in the played video, and then Obtaining a video image containing special effects corresponding to the music beat point can further improve user experience satisfaction.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the second embodiment, where
  • the strong beat point it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the candidate beat point is obtained according to the energy intensity value of the music signal, and each adjacent two candidate beats are counted according to each candidate beat point.
  • two adjacent music beat signals are taken as starting points of the two adjacent candidate beat points, and according to the comparison result of the two pieces of music signals, a strong beat point corresponding to the candidate beat point detection point is detected. ;
  • the strong beat point and the weak beat point are detected, it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the music signal is weighted to obtain the weighted music signal, according to the weighted music signal.
  • the energy intensity value is detected as a strong beat point or a weak beat point at the detection point; if the change value detection is used, the music signal is filtered, and then subjected to short-time Fourier transform to obtain a spectrum, and the detection is determined according to the spectrum.
  • the energy change value of the point is detected according to the energy change value, and a weak beat point or a strong candidate beat point is detected at the detection point;
  • different beat detection methods are corresponding for different types of required detection beat points and detection criteria.
  • the determining whether to detect a strong beat point or detecting a strong beat point and a weak beat point comprises:
  • the type of special effect required for the video judge whether to detect the strong beat point according to the type of special effect required by the video, or detect the strong beat point and the weak beat point; if the strong beat point is detected, it is judged whether the intensity value detection or the change value detection is used, including: Detecting a strong beat point, obtaining the type of the music, determining whether to use the intensity value detection or the change value detection according to the type; if detecting the strong beat point and the weak beat point, determining whether to use the intensity value detection or the change value detection, including: if detecting The strong beat point and the weak beat point acquire the type of the music, and determine whether to use the intensity value detection or the change value detection according to the type.
  • the type of the desired beat point can be determined by the type of the effect required for the acquired video.
  • the effect type required for the video is the default effect type selected by the user or the video application. For example, the user wants to have endless effects in the playing video, so it is judged according to the type of special effect required by the video that both the strong beat point and the weak beat point are detected.
  • the detection criteria can be determined by acquiring the type of music corresponding to the sound.
  • the type of music corresponding to the acquired sound is rock, and the music signal of the music type often has a high intensity value, but the change value is not obvious, so according to the type thereof, the beat of the music is detected by detecting the intensity value. point.
  • the type of the desired beat point and the method of detecting the beat point can be selected according to the type of the special effect required for the video and the type of the music, so as to obtain an accurate beat point by using an appropriate method, and the amount of calculation can be reduced. , shortening the detection time, further ensuring the close relationship between the special effects displayed on the video image and the music beat point, and further improving the satisfaction of the user experience.
  • the video is a live video, that is, the method in this embodiment is mainly applied to the field of video live broadcast.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the second embodiment, where
  • the step S201 includes:
  • the method further includes:
  • the interaction between the anchor and the viewer is added in the live video.
  • the viewer can send the audience to the live broadcast in addition to the common interactive behaviors such as speaking, praising and giving gifts.
  • Sing a song request After the anchor sings a song request by a certain viewer, the viewer client and the live terminal perform a connection, and the live broadcast end receives the singer voice of the viewer in the live video and extracts the audio feature of the voice, and then the The audio features are sent to the server to cause the server to traverse the audio features in the preset music library with which they are stored, thereby determining the music corresponding to the audio features of the viewer's humming sound.
  • the audience can obtain the opportunity for the audience to sing songs by paying.
  • the anchor can also give the sing-song opportunity as a gift to the audience in the live room.
  • the program can meet the needs of the live broadcast, highlight the live atmosphere, and significantly increase the anchor and The interaction between viewers further increases the fun of video applications.
  • the video is a live video, that is, the method in this embodiment is mainly applied to the field of video live broadcast.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the second embodiment, where
  • the step S201 includes:
  • the interactive song of the anchor and the viewer is added to the live video, and the anchor can quickly and easily acquire the music corresponding to the humming voice by humming after sending the humming download song command.
  • the live end receives the humming sound of the anchor in the live video and extracts the audio features of the sound, and then sends the audio feature to the server, so that the server matches the audio feature with the music in the preset music library A traversal match is made to determine the music corresponding to the audio characteristics of the anchor's humming sound.
  • the anchor can select the song with the highest audience voice during the live broadcast of the video according to the viewer's barrage information, so as to play the music in the live video and present the video image containing the special effect corresponding to the music beat point, the solution can satisfy the live broadcast.
  • Demand, highlighting the live broadcast atmosphere significantly increase the interaction between the anchor and the viewer, further increasing the fun of video applications.
  • the embodiment of the invention provides a video image processing method. As shown in FIG. 3, the method includes:
  • Step S301 Acquire a music signal of music to be played in the played video.
  • the music to be played in the playing video may be music selected from pre-stored music in the video application, or may be selected from the pre-stored music of the user terminal to be loaded into the video application, and may also be The live music played by the other device is obtained by the microphone in the video application, and the source of the music is not limited in this embodiment.
  • the action of acquiring the music signal is performed after the user selects the music to be played in the playing video, and the action of selecting the music may be performed before the video is played, or may be performed in the playing video.
  • the music when applied to the field of video live broadcast, the music may be selected before the live broadcast of the video or in the live broadcast of the video, and the terminal will immediately acquire the music signal of the selected music and perform subsequent preset steps.
  • the music is selected before the video is played, that is, before the video is recorded, and the terminal immediately acquires the music signal of the selected music after the selected music, and performs subsequent preset steps.
  • Step S302 determining whether a beat point description file for preserving a correspondence relationship between a beat point of the music and a music play position is prestored; if yes, acquiring the beat point description file; if not, detecting the sound according to the music signal The beat point of the music generates a beat point description file according to the corresponding relationship between the detected beat point and the music play position.
  • the present invention uses a beat point description file to determine the correspondence between the beat point of the music and the music play position.
  • determining whether a beat point description file for storing a correspondence relationship between a beat point of the music and a music play position is pre-stored; if yes, acquiring the beat point description file; comprising: determining whether a beat point is pre-stored in the local file a description file, if yes, obtaining the beat point description file from a local file; or determining whether the server pre-stores a beat point description file, and if so, downloading the beat point description file from the server; or determining whether the local file is pre-stored There is a beat point description file, and if so, the beat point description file is obtained from the local file; if not, it is determined whether the server pre-stores the beat point description file, and if so, the beat point description file is downloaded from the server.
  • the beat point description file may be obtained from a local file or may be obtained from a server, and the user may also try to obtain a beat point description file from the server after the local acquisition fails.
  • the existing beat point description file when the existing beat point description file is not obtained, it is necessary to intelligently generate a beat point description file according to the music signal, specifically, detecting a beat point of the music according to the music signal, according to the The corresponding relationship between the detected beat point and the music playback position generates a beat point description file.
  • the beat point description file After the beat point description file is intelligently generated, the beat point description file may also be uploaded to the server for each user to download and use.
  • the detecting a beat point of the music according to the music signal may be implemented by using a plurality of beat point detecting methods.
  • the beat point includes a strong beat point and a weak beat point; the method includes: filtering the music signal, filtering and performing short-time Fourier transform to obtain a spectrum; according to the spectrum, The energy change value of the detection point is determined; according to the energy change value, a strong beat point or a weak beat point is detected at the detection point.
  • the beat point includes a strong beat point and a weak beat point; the method includes: performing weighting processing on the music signal to obtain a weighted music signal; and according to the weighted music The energy intensity value of the signal detects a strong beat point or a weak beat point at the detection point.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy intensity value of the music signal; and counting each adjacent two candidates according to each candidate beat point The time interval between the frames in which the beat points are located; according to the time interval, it is detected that the candidate beat points correspond to strong beat points corresponding to the detected points.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy variation difference of the music signal of the detected point; and according to the candidate beat point, each phase Two adjacent candidate beat points are taken as the signal starting point to intercept two pieces of music signals; according to the comparison result of the two pieces of music signals, it is detected that the candidate beat points corresponding to the detected points have strong beat points.
  • Step S303 Determine, according to the beat point description file, that a beat point appears in the current play position when playing music in the play video.
  • the beat point description file records the correspondence between the beat point of the music and the music play position by using the information symbol that the terminal can read and understand, and when the music is played, by loading the beat point description file
  • the data is analyzed and the data is analyzed. According to the data analysis result, whether the beat point of the current playing position of the music is known or not, and whether the beat point is a strong beat point or a weak beat point is determined.
  • Step S304 Acquire an effect corresponding to the beat point; process the image in the play video according to the special effect to obtain a video image including the special effect.
  • the special effect may be a material such as a fireworks, a love, a snowflake, etc., and the specific expression of the material is not limited in this embodiment.
  • Different shape feature parameters may be set for the same material, and the shape feature parameters include a size parameter, a color parameter, and the like, which are not limited in this embodiment.
  • the beat point in a piece of music includes a strong beat point and a weak beat point
  • the material in the special effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different.
  • the video in the effect is obtained by acquiring the material in the special effect and synthesizing the material and the image in the play video in a layer superposition manner to obtain a video image including the material in the special effect.
  • the video image including the special effect may be obtained by integrating data of the special effect with the image or modifying the image according to the shape characteristic parameter of the material in the special effect.
  • data of the effect can be integrated with the image data to obtain a video image data packet, and the data packet is sent to the client, so that the client displays the video image containing the special effect.
  • the image may be scaled according to the parameter by obtaining the shape feature parameter of the material in the special effect to obtain a video image that can achieve the effect of the special effect.
  • the video image processing method provided by the present invention obtains a beat point description file of the music or intelligently generates a beat point description file according to the music signal, and obtains an effect corresponding to the beat point determined according to the beat point description file, so as to enable the special effect displayed on the video image. It is closely related to the beat point of the selected video playing music, thereby improving the visual and visual appeal of the music played in the video, increasing the interest of the video application and increasing the satisfaction of the user experience.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the third embodiment, where
  • the method further includes:
  • beat point description file is a beat point description file downloaded from the server, detecting a beat point of the music according to the music signal, and correcting the beat point description file according to the detected beat point, in the local file
  • the proof point description file after the proofreading is saved.
  • the beat point description file downloaded from the server stores the corresponding relationship between the beat point of the music and the music play position, but there may be a corresponding music in the beat point description file and the current music to be played is incomplete.
  • the user can intercept the selected music and choose to play the intercepted music piece in the playing video, while the beat point description file downloaded from the server corresponds to the entire music.
  • the correspondence between the beat point and the music playback position Therefore, it is necessary to proof the beat point description file according to the detected beat point, and save the proofed beat point description file in the local file for use by the end user.
  • the accuracy of the beat point description file can be ensured, thereby ensuring the accuracy of the correspondence between the beat point and the music playing position, and further ensuring the video image.
  • the associated effect of the displayed effect is related to the beat point of the selected video playback music.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the third embodiment, where
  • the detecting, in the step S302, the beat point of the music according to the music signal comprising:
  • the strong beat point it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the candidate beat point is obtained according to the energy intensity value of the music signal, and each adjacent two candidate beats are counted according to each candidate beat point.
  • two adjacent music beat signals are taken as starting points of the two adjacent candidate beat points, and according to the comparison result of the two pieces of music signals, a strong beat point corresponding to the candidate beat point detection point is detected. ;
  • the strong beat point and the weak beat point are detected, it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the music signal is weighted to obtain the weighted music signal, according to the weighted music signal.
  • the energy intensity value is detected as a strong beat point or a weak beat point at the detection point; if the change value detection is used, the music signal is filtered, and then subjected to short-time Fourier transform to obtain a spectrum, and the detection is determined according to the spectrum.
  • the energy change value of the point is detected according to the energy change value, and a weak beat point or a strong candidate beat point is detected at the detection point;
  • different beat detection methods are corresponding for different types of required detection beat points and detection criteria.
  • the determining whether to detect a strong beat point or detecting a strong beat point and a weak beat point comprises:
  • the type of special effect required for the video judge whether to detect the strong beat point according to the type of special effect required by the video, or detect the strong beat point and the weak beat point; if the strong beat point is detected, it is judged whether the intensity value detection or the change value detection is used, including: Detecting a strong beat point, obtaining the type of the music, determining whether to use the intensity value detection or the change value detection according to the type; if detecting the strong beat point and the weak beat point, determining whether to use the intensity value detection or the change value detection, including: if detecting The strong beat point and the weak beat point acquire the type of the music, and determine whether to use the intensity value detection or the change value detection according to the type.
  • the type of the desired beat point can be determined by the type of the effect required for the acquired video.
  • the effect type required for the video is the default effect type selected by the user or the video application. For example, the user wants to have endless effects in the playing video, so it is judged according to the type of special effect required by the video that both the strong beat point and the weak beat point are detected.
  • the detection criteria can be judged by the type of music acquired.
  • the type of music acquired is rock, and the music signal of the music type tends to have a high intensity value, but the change value is not obvious, so the beat point of the music is detected by detecting the intensity value according to the type thereof.
  • the type of the desired beat point and the method of detecting the beat point can be selected according to the type of the special effect required for the video and the type of the music, so as to obtain an accurate beat point by using an appropriate method, and the amount of calculation can be reduced.
  • the detection time is shortened, and the special effect displayed on the video image is closely related to the beat point of the selected video playing music, thereby further improving the satisfaction of the user experience.
  • the video is a live video, that is, the method in this embodiment is mainly applied to the field of video live broadcast.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the third embodiment, where
  • the method further includes:
  • step S304 Obtaining the special effect corresponding to the beat point in step S304, including:
  • the beat point description file not only stores the correspondence between the beat point and the music play position, but also stores the corresponding relationship between the two and the special effect information.
  • the special effect information carries the material information that is randomly selected or customized by the user, and the shape characteristic parameter set for the material.
  • the definition of the material and its shape characteristic parameter refer to the content of step S104 in the first embodiment, where not Let me repeat.
  • the special effect information may also be set by a user watching the live broadcast, and the user may send a special effect setting request to the live broadcast terminal to implement the special effect setting.
  • the video image processing method provided by the present invention is applied to the field of live broadcast, and the special effect corresponding to the beat point of the custom setting by the beat point description file can be realized, which can meet the requirement of the live broadcast and further increase the interest of the video application. .
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the third embodiment, where
  • the method further includes:
  • the anchor of the current live broadcast end interacts with the anchor of another live broadcast during the live broadcast of the live broadcast, and the user watching the current broadcast of the active broadcast can simultaneously view the live broadcast of the interactive anchor.
  • the live broadcast atmosphere can be effectively enhanced, and the popularity value of the live broadcasts of each interactive live end can be improved, which can meet the needs of the live broadcast and further increase the interest of the video application.
  • the music gift processing method provided by the invention is mainly applied to the field of video live broadcasting.
  • the anchor in order to activate the atmosphere and increase the interaction with the audience in the live broadcast, the anchor usually selects the music corresponding to the live broadcast theme of the live broadcast or the music with the highest audience in the live broadcast during the live broadcast. The audience will respond to the anchor by clicking, speaking or giving a gift.
  • the video image in the playing video is often not related to the played music.
  • the music is simply added to the playing video, and there is not enough appeal in the sense of hearing and visual, and the above interaction method is too single and cannot be effective. Increasing the enthusiasm of the audience and the anchor is not enough to meet the needs of live video.
  • the present invention combines the interactive behavior of the anchor playing music and the audience to give gifts, and provides a music gift processing method to achieve a significant improvement in the interaction between the viewer and the anchor and the satisfaction of the user experience.
  • the music gift processing method will be elaborated.
  • An embodiment of the present invention provides a music gift processing method. As shown in FIG. 4, the method includes:
  • Step S401 Receiving a music gift sent by a viewer in a live video broadcast; the music gift includes music to be played in a live video.
  • the viewer can interact with the anchor by sending a music gift to the anchor during the live video broadcast.
  • the music gift received by the live broadcast end carries music information, and the music corresponding to the music information is music selected by the viewer to be played by the anchor in the live video broadcast.
  • the music gift further includes special effect information corresponding to a beat point of the music.
  • the special effect information carries material information randomly set by the user terminal or user-defined, and shape characteristic parameters set for the material.
  • the special effect may be a material such as a fireworks, a love, a snowflake, etc., and the specific expression of the material is not limited in this embodiment.
  • Different shape feature parameters may be set for the same material, and the shape feature parameters include a size parameter, a color parameter, and the like, which are not limited in this embodiment.
  • the beat point in a piece of music includes a strong beat point and a weak beat point, and the material in the special effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different.
  • Step S402 Acquire a music signal of the music, and detect a beat point of the music according to the music signal.
  • the beat point is detected on the music, wherein the beat point of the music is detected according to the music signal. It can be implemented by a variety of beat point detection methods.
  • the beat point includes a strong beat point and a weak beat point; the method includes: filtering the music signal, filtering and performing short-time Fourier transform to obtain a spectrum; according to the spectrum, The energy change value of the detection point is determined; according to the energy change value, a strong beat point or a weak beat point is detected at the detection point.
  • the beat point includes a strong beat point and a weak beat point; the method includes: performing weighting processing on the music signal to obtain a weighted music signal; and according to the weighted music The energy intensity value of the signal detects a strong beat point or a weak beat point at the detection point.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy intensity value of the music signal; and counting each adjacent two candidates according to each candidate beat point The time interval between the frames in which the beat points are located; according to the time interval, it is detected that the candidate beat points correspond to strong beat points corresponding to the detected points.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy variation difference of the music signal of the detected point; and according to the candidate beat point, each phase Two adjacent candidate beat points are taken as the signal starting point to intercept two pieces of music signals; according to the comparison result of the two pieces of music signals, it is detected that the candidate beat points corresponding to the detected points have strong beat points.
  • Step S403 When it is determined that the music in the music gift is played in the live video, a beat point appears at the current play position.
  • each beat point detected in step S402 corresponds to a different play position of the music.
  • the music is played in the live video, if the current play position of the music corresponds to the beat point detected in step S402, then it is determined. Whether the beat point is a strong beat point or a weak beat point.
  • Step S404 Acquire an effect corresponding to the beat point.
  • Step S405 Process the image in the live video broadcast according to the special effect, and obtain a live video of the video including the special effect.
  • the material of the different shape feature parameters of the special effect corresponding to the special effect information is obtained from the music gift, and the material is combined with the image in the live video broadcast in a layer overlay manner to obtain an inclusion A live video image of the special effects.
  • the method further includes: sending the processed live video of the video to the client.
  • the video live image including the special effect may be obtained by integrating data of the special effect with the image or modifying the image according to the shape characteristic parameter of the material in the special effect.
  • the data of the special effect can be integrated with the image data to obtain a live video image data package, and the data packet is sent to the client, so that the client displays the live video image of the video containing the special effect.
  • the image is scaled according to the parameter, and after the video live image that can achieve the special effect is obtained, the image is sent to the viewer client, so that the client displays the inclusion.
  • a live video of the effect is obtained.
  • the material corresponding to the random setting or the anchor custom setting of the live broadcast may be obtained as the special effect corresponding to the beat point.
  • the music gift processing method provided by the present invention performs beat point detection on the music in the music gift sent by the viewer to the anchor, and displays the special effect corresponding to the beat point in the video image according to the detected beat point during the playing process.
  • the special effects displayed on the live video of the video closely related to the beat points of the music in the music gift, thereby improving the auditory and visual appeal of the music played in the live video broadcast; and sending the gift to the audience by playing the music with the anchor.
  • the combination of behaviors enriches the interaction between the viewer and the anchor in the live video broadcast, significantly improves the interaction enthusiasm between the viewer and the anchor, effectively highlights the live broadcast atmosphere, meets the demand for live video, and increases the interest of the live video application, improving the user. Satisfaction of experience.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the fourth embodiment, where
  • the method further includes: generating a beat point description file according to the detected correspondence between the beat point and the music playing position.
  • step S403 includes: determining, according to the beat point description file, that a beat point occurs at a current play position when the music in the music gift is played in a live video.
  • a beat point description file is used to determine the correspondence between the beat point of the music and the music play position.
  • the beat point description file records the correspondence between the beat point of the music and the music play position by using the information symbol that the terminal can read and understand, and when the music is played, the data in the file is described by loading the beat point and the data is Perform analysis, according to the data analysis result, whether the beat point of the current playing position of the music is known, and whether the beat point is a strong beat point or a weak beat point.
  • the correspondence between the beat point and the music playing position can be quickly and conveniently obtained through the beat point description file after a very short file loading time, thereby satisfying the demand of the live video, and further improving the user experience satisfaction.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the fourth embodiment, where
  • the method further includes: acquiring the special effect information, and saving the corresponding relationship between the beat point and the music playing position and the special effect information to the beat In the point description file; generate a beat point description file containing the effect information.
  • the step S404 includes: acquiring, according to the beat point description file containing the special effect information, the special effect corresponding to the beat point from the music gift.
  • the beat point description file not only stores the correspondence between the beat point and the music play position, but also stores the corresponding relationship between the two and the special effect information.
  • the special effect information carries the material information that is randomly selected or customized by the user, and the shape characteristic parameter that is set for the material. For the specific definition, refer to the content of step S401 in the first embodiment, and details are not described herein again.
  • the beat point description file can quickly and conveniently obtain the special effect corresponding to the beat point of the current music playing position, which satisfies the requirement of the live video, and can further improve the user experience satisfaction. degree.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the fourth embodiment, where
  • the method further includes:
  • Determining whether the anchor of the live end performs a preset action of opening the music gift if yes, continuing the step of acquiring the music signal of the music, detecting a beat point of the music according to the music signal; and transmitting the The audience of the music gift feedbacks the anchor to open the information of the music gift.
  • the anchor in a live video application scenario, there are usually multiple viewers simultaneously sending music presents to the anchor, and since the live broadcast time is limited, the anchor often cannot open all the music gifts, so the music beat point detection is performed. Before the subsequent preset steps, you need to determine if the anchor wants to open the currently received music gift.
  • the preset action performed by the anchor may be that the preset password is spoken in the live video, or the preset expression or gesture may be made in the live video, and the displayed on the display may also be clicked. Music gift logo.
  • the music gift appears in the form of a list.
  • the anchor says the password of “opening the eighth music gift”
  • the live terminal performs voice recognition on the password, and obtains the corresponding correspondence in the music gift list according to the recognition result.
  • the music gift appears in the form of a barrage.
  • the live broadcast terminal determines the anchor instruction to open the currently popped music gift according to the gesture recognition result of the live video image. And then get the music signal in the music gift and perform subsequent preset steps.
  • the music gift appears in the form of gift rain at a preset time.
  • the live terminal receives the instruction of the anchor click terminal display to trigger one of the music gifts, the live terminal immediately acquires the music gift. Music signals and perform subsequent preset steps.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the fourth embodiment, where
  • the step S402 includes:
  • the strong beat point it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the candidate beat point is obtained according to the energy intensity value of the music signal, and each adjacent two candidate beats are counted according to each candidate beat point.
  • two adjacent music beat signals are taken as starting points of the two adjacent candidate beat points, and according to the comparison result of the two pieces of music signals, a strong beat point corresponding to the candidate beat point detection point is detected. ;
  • the strong beat point and the weak beat point are detected, it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the music signal is weighted to obtain the weighted music signal, according to the weighted music signal.
  • the energy intensity value is detected as a strong beat point or a weak beat point at the detection point; if the change value detection is used, the music signal is filtered, and then subjected to short-time Fourier transform to obtain a spectrum, and the detection is determined according to the spectrum.
  • the energy change value of the point is detected according to the energy change value, and a weak beat point or a strong candidate beat point is detected at the detection point;
  • different beat detection methods are corresponding for different types of required detection beat points and detection criteria.
  • the determining whether to detect a strong beat point or detecting a strong beat point and a weak beat point comprises:
  • the special effect information in the music gift determining whether to detect a strong beat point or detecting a strong beat point and a weak beat point according to the special effect information; if detecting the strong beat point, determining whether to use the intensity value detection or the change value detection, including If the strong beat point is detected, the type of the music is obtained, and the intensity value detection or the change value detection is used according to the type judgment; if the strong beat point and the weak beat point are detected, whether the intensity value detection or the change value detection is used, including: If a strong beat point and a weak beat point are detected, the type of the music is acquired, and whether the intensity value detection or the change value detection is used is determined according to the type.
  • the type of the desired beat point can be determined by acquiring the special effect information in the music gift. For example, in the music gift sent to the live end, the viewer sets the shape feature parameters of the material for the strong beat point and the weak beat point respectively, and the user wants to see the corresponding beat point when the music gift is opened in the live video broadcast. Cool special effects, so according to the special effects information of the viewer's custom settings, it is judged that both the strong beat point and the weak beat point are detected.
  • the detection criteria can be judged by acquiring the type of music in the music gift. For example, the type of music in the acquired music gift is rock, and the music signal of the music type often has a high intensity value, but the change value is not obvious, so the music value is detected by detecting the intensity value according to the type selection. The beat point.
  • the type of the desired beat point and the method of detecting the beat point can be selected according to the special effect information and the type of the music in the music gift, so as to obtain an accurate beat point by using an appropriate method, and the operation can be reduced.
  • the quantity, shorten the detection time, further ensure the close relationship between the special effects displayed on the video image and the music beat point, and further improve the satisfaction of the user experience.
  • An embodiment of the present invention provides a mapping processing method for a video image. As shown in FIG. 5, the method includes:
  • Step S501 Acquire an image in the played video, and add a texture to the image.
  • the texture may be a two-dimensional model map based on AR augmented reality, or may be a three-dimensional model map based on AR augmented reality.
  • the AR is an augmented reality technology that combines and displays real world information and virtual world information. Through augmented reality technology, an image based on AR augmented reality is added to the image in the playing video, and the real world image and the computer virtual image in the playing video can be superimposed on the same screen to realize information integration of the real world and the virtual world. Interaction.
  • the texture may be a material such as a fireworks, a love, a snowflake, or the like used as a background-like texture, or may be a material such as a corner, a beard, or a glasses used as a face decoration type map, and the specific performance of the material.
  • the added map in the play video image can be user-defined settings, or it can be set by the video application by default. After determining the texture, the terminal performs scene recognition on the image in the played video, and then adds a texture to the corresponding position of the video image.
  • Step S502 Acquire a music signal of music to be played in the played video, and detect a beat point of the music according to the music signal.
  • the music to be played in the playing video may be music selected from pre-stored music in the video application, or may be selected from the pre-stored music of the user terminal to be loaded into the video application, and may also be The live music played by the other device is obtained by the microphone in the video application, and the source of the music is not limited in this embodiment.
  • the action of acquiring the music signal is performed after the user selects the music to be played in the playing video, and the action of selecting the music may be performed before the video is played, or may be performed in the playing video.
  • the music when applied to the field of video live broadcast, the music may be selected before the live broadcast of the video or in the live broadcast of the video, and the terminal may immediately acquire the music signal of the selected music, and detect the beat of the music according to the music signal. point.
  • the music is selected before the video is played, that is, before the video is recorded, and after the selected music, the terminal immediately acquires the music signal of the selected music, and according to the music signal. A beat point of the music is detected.
  • the beat point of the music detected according to the music signal may be implemented by using various beat point detection methods.
  • the beat point includes a strong beat point and a weak beat point; the method includes: filtering the music signal, filtering and performing short-time Fourier transform to obtain a spectrum; according to the spectrum, The energy change value of the detection point is determined; according to the energy change value, a strong beat point or a weak beat point is detected at the detection point.
  • the beat point includes a strong beat point and a weak beat point; the method includes: performing weighting processing on the music signal to obtain a weighted music signal; and according to the weighted music The energy intensity value of the signal detects a strong beat point or a weak beat point at the detection point.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy intensity value of the music signal; and counting each adjacent two candidates according to each candidate beat point The time interval between the frames in which the beat points are located; according to the time interval, it is detected that the candidate beat points correspond to strong beat points corresponding to the detected points.
  • the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy variation difference of the music signal of the detected point; and according to the candidate beat point, each phase Two adjacent candidate beat points are taken as the signal starting point to intercept two pieces of music signals; according to the comparison result of the two pieces of music signals, it is detected that the candidate beat points corresponding to the detected points have strong beat points.
  • Step S503 When it is determined that the music is played in the playing video, a beat point appears at the current playing position.
  • each beat point detected in step S502 corresponds to a different play position of the music.
  • the music is played in the live video, if the current play position of the music corresponds to the beat point detected in step S502, then it is determined. Whether the beat point is a strong beat point or a weak beat point.
  • Step S504 Acquire a texture effect corresponding to the beat point.
  • the texture effect refers to a texture state when the texture corresponds to different shape feature parameters.
  • the material in the map may be set with different shape feature parameters, and the shape feature parameters include a state parameter, a size parameter, a color parameter, and the like, which are not limited in this embodiment.
  • the beat point in a piece of music generally includes a strong beat point and a weak beat point, and the material in the texture effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different.
  • the texture selected by the user is a three-dimensional model of the bear, and different shape feature parameters are set for the strong beat point and the weak beat point for the texture.
  • the beat point is strong, the three-dimensional model of the bear dances in the video image has a large dance frequency. The dance frequency of dancing is lower at weak beats.
  • Step S505 Processing a texture in the image according to the texture effect to obtain a video image including the texture effect.
  • the material in which the shape parameter is set in the texture effect is obtained, and the material is combined with the image in the play video in a layer superposition manner to obtain the texture effect.
  • the video image including the texture effect may be obtained by integrating data of the texture effect with the image or modifying the image according to the shape characteristic parameter of the material in the texture effect.
  • data of the texture effect can be integrated with the live video image data to obtain a video image data packet, and the data packet is sent to the client, so that the client displays the video containing the texture effect. image.
  • the image may be scaled according to the shape feature parameter to obtain a video image that can achieve the effect of the special effect.
  • the method for mapping a video image adds a texture to a video image, performs beat point detection on the music to be played in the played video, and displays the video image according to the detected beat point during the music playing process.
  • the texture effect corresponding to the beat point is displayed, so that the texture effect displayed on the video image is closely related to the beat point of the selected video playing music, thereby improving the music played in the playing video and the displayed texture in the auditory and visual
  • This method satisfies the user's personalized play video design needs and increases the video.
  • the interest of the app and the interactivity of the app and the user significantly increase the satisfaction of the user experience.
  • the method for mapping a video image provided by the present invention can quickly and accurately detect the beat point of the selected video playing music, further ensuring the texture effect and selected on the video image.
  • the beat of the video playback music is closely related to further improve the satisfaction of the user experience.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps, where
  • the method further includes: recording a correspondence between the music playing position and the beat point.
  • the beat point of the music is detected according to the music signal
  • the beat point is associated with the corresponding music play position, and the corresponding relationship is recorded.
  • the method for establishing the corresponding relationship is not limited in this embodiment, and may be a manner of adding label information.
  • the signal data of the music playing position corresponding to the beat point is added with tag information, and the tag information carries information indicating that the music playing position has a beat point and the beat point is a strong beat point or a weak beat point.
  • the step S503 includes: determining, according to the correspondence, that a beat point occurs at a current play position when playing music in the play video.
  • the correspondence record of the music play position and the beat point is acquired, and the corresponding relationship is extracted, and then according to the The correspondence determines that a beat point occurs at the current play position and determines that the beat point is a strong beat point or a weak beat point.
  • the embodiment by recording the correspondence between the music playing position and the beat point, it is possible to easily and quickly determine whether the current playing position has a beat point and whether the beat point is a strong beat point or a weak beat point.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps, where
  • the step S501 includes:
  • the texture is mainly a face decoration type map, such as a corner, a beard, a glasses, and the like, and the terminal performs face recognition on the image in the played video, thereby adding a map to the corresponding position of the face in the video image.
  • the face area selected in the play video is added with virtual ear, beard and other textures, and the texture will move along with the user's face to realize the combination and interaction of virtual and reality.
  • the user can express his own personality by custom setting texture and texture effect and can edit and play the video of the personalized image design in real time, satisfying the user's personalized image design requirement and increasing the interest of the video application. And the interaction of the application with the user, significantly improving the satisfaction of the user experience.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps, where
  • the method further includes:
  • a texture switching instruction can be issued to the terminal to switch the texture to the texture material that he likes.
  • the manner of the indication map switching may be a voice indication, an action indication, and an input indication, which is not limited in this embodiment.
  • the terminal when the switching of the texture is indicated by voice, the terminal first acquires the sound in the played video, and recognizes the audio feature of the sound, and then switches the audio feature to the preset texture, such as “switching”, The audio features of the passwords such as "change” are matched. If the matching is consistent, it is confirmed that the texture switching instruction is received and the texture is switched.
  • the terminal when the switching of the map is instructed by the operation, the terminal first recognizes the person region in the image, and detects the motion of the person in the person region, and if a preset texture switching operation is detected in the person region, If the action of "hand waving” or “shaking the head” is performed, it is confirmed that the texture switching instruction is received and the texture is switched.
  • the user when inputting to indicate the switching of the texture, the user can switch the current texture by simply re-entering another image in the video application.
  • the user can switch the texture according to his or her own wishes conveniently and quickly, further increasing the interest of the video application and the interaction between the application and the user, and significantly improving the satisfaction of the user experience.
  • the video is a live video, that is, the method in this embodiment is mainly applied to the field of video live broadcast.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps, where
  • the method further includes:
  • the anchor can obtain the live video of the video including the map by performing a preset add map request during the live broadcast of the video.
  • the manner in which the request is added to the map may be a voice request, an action request, and an input request, and is not limited in this embodiment.
  • the terminal when requesting to add a texture by voice, the terminal first acquires the sound in the played video, and recognizes the audio feature of the sound, and then adds the audio feature to the preset added texture password, such as “map”, “ The audio features of the passwords are matched, and if the matches are consistent, the texture is added to the image of the played video.
  • the preset added texture password such as “map”, “ The audio features of the passwords are matched, and if the matches are consistent, the texture is added to the image of the played video.
  • the terminal when an action is requested to add a map, the terminal first recognizes a person region in the image, and detects a person motion in the person region, and if a preset texture switching action is detected in the person region, such as Adding a map to the image of the played video, such as "being heart” and "flying kiss”.
  • the texture may also be selected by a viewer in the live video room, or the anchor may select the hottest texture according to the wishes of the viewer in the live video broadcast.
  • the method can meet the requirements of the live broadcast, highlight the live broadcast atmosphere, and better promote the interaction between the anchor and the audience, and further improve the satisfaction of the user experience.
  • Another possible implementation manner of the embodiment of the present invention further includes the following steps, where
  • the step S502 includes:
  • the strong beat point it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the candidate beat point is obtained according to the energy intensity value of the music signal, and each adjacent two candidate beats are counted according to each candidate beat point.
  • two adjacent music beat signals are taken as starting points of the two adjacent candidate beat points, and according to the comparison result of the two pieces of music signals, a strong beat point corresponding to the candidate beat point detection point is detected. ;
  • the strong beat point and the weak beat point are detected, it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the music signal is weighted to obtain the weighted music signal, according to the weighted music signal.
  • the energy intensity value is detected as a strong beat point or a weak beat point at the detection point; if the change value detection is used, the music signal is filtered, and then subjected to short-time Fourier transform to obtain a spectrum, and the detection is determined according to the spectrum.
  • the energy change value of the point is detected according to the energy change value, and a weak beat point or a strong candidate beat point is detected at the detection point;
  • different beat detection methods are corresponding for different types of required detection beat points and detection criteria.
  • the determining whether to detect a strong beat point or detecting a strong beat point and a weak beat point comprises:
  • the type of texture effect required for the video judge whether to detect the strong beat point according to the type of texture effect required by the video, or detect the strong beat point and the weak beat point; if the strong beat point is detected, it is judged whether the intensity value detection or the change value detection is used, including If the strong beat point is detected, the type of the music is obtained, and the intensity value detection or the change value detection is used according to the type judgment; if the strong beat point and the weak beat point are detected, whether the intensity value detection or the change value detection is used, including: If a strong beat point and a weak beat point are detected, the type of the music is acquired, and whether the intensity value detection or the change value detection is used is determined according to the type.
  • the type of the desired beat point can be determined by the type of texture effect required for the acquired video.
  • the type of texture effect required for the video is the default effect type selected by the user or the video application. For example, the user wants to have an endless layer of texture effects in the playback video, so it is judged that the strong beat point is detected and the weak beat point is detected according to the type of texture effect required for the video.
  • the detection criteria are judged by the type of music acquired.
  • the type of music acquired is rock, and the music signal of the music type tends to have a high intensity value, but the change value is not obvious, so the beat point of the music is detected by detecting the intensity value according to the type thereof.
  • the type of the desired beat point and the method of detecting the beat point can be selected according to the type of the texture effect and the type of the music required by the video, so as to obtain an accurate beat point by using an appropriate method, and the operation can be reduced.
  • the amount, shorten the detection time, further ensure the close relationship between the texture effect displayed on the video image and the music beat point, and further improve the satisfaction of the user experience.
  • an embodiment of the present invention provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the program is executed by the processor, the video image processing method according to the first embodiment is implemented, or an embodiment The video image processing method according to the second embodiment, or the video image processing method according to the third embodiment, or the music gift processing method in the live video broadcast according to the fourth embodiment, or the texture processing method of the video image according to the fifth embodiment .
  • the computer readable storage medium includes, but is not limited to, any type of disk (including a floppy disk, a hard disk, an optical disk, a CD-ROM, and a magneto-optical disk), a ROM (Read-Only Memory), and a RAM (Random Access).
  • the storage device includes any medium that stores or transmits information in a readable form by a device (eg, a computer, a mobile phone), and may be a read only memory, a magnetic disk, an optical disk, or the like.
  • a computer readable storage medium provided by the embodiment of the present invention can perform beat point detection on music to be played in a play video, and display the video image in the music image according to the detected beat point.
  • the effect corresponding to the beat point, so that the special effect displayed on the video image is closely related to the beat point of the selected video playing music, thereby increasing the visual and visual appeal of the music played in the video, and increasing the video application.
  • the computer readable storage medium provided by the present invention can also implement a combination of multiple beat point detection methods to quickly and accurately detect the beat point of the selected video playing music; and can select the special effect type and music according to the video.
  • Type to select the type of beat point to be detected and the method of detecting the beat point, so as to achieve an accurate method to obtain accurate beat points, and reduce the amount of calculation, shorten the detection time, and further ensure the special effects and images displayed on the video image.
  • the beat points of the selected video playing music are closely related to further improve the satisfaction of the user experience.
  • the computer readable storage medium provided by the present invention is applied to the field of live video, it is also possible to select a special effect according to the anchor intention, the type of music, and the will of the viewer, thereby meeting the requirements of the live broadcast, highlighting the live broadcast atmosphere, and better promoting the anchor. Interact with the audience to further improve the satisfaction of the user experience.
  • the computer-readable storage medium provided by the embodiment of the present invention may implement the foregoing method embodiments.
  • the embodiment of the present invention further provides a terminal.
  • the terminal may include one or more processors 601, and further includes a memory 602, a WiFi (Wireless Fidelity) circuit 603, RF (Radio Frequency) circuit 604, audio circuit 605, sensor 606, output device 607, input device 604, power supply 609, and processor 601 are control centers of the terminals, and the above components are connected by various interfaces and lines.
  • WiFi Wireless Fidelity
  • RF Radio Frequency
  • audio circuit 605 sensor 606, output device 607, input device 604, power supply 609, and processor 601 are control centers of the terminals, and the above components are connected by various interfaces and lines.
  • the terminal structure shown in FIG. 6 does not constitute a limitation to the terminal, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements.
  • the WiFi circuit 603 can provide the user with wireless local area network or internet access; it can include an antenna, a WiFi module, and the like.
  • the RF circuitry 604 can transmit and receive information, or receive and transmit signals during a call; it can include an antenna, at least one amplifier, a tuner, one or more oscillators, a coupler, a duplexer, and the like.
  • the audio circuit 605 can convert the received audio data into an electrical signal for transmission to a speaker, and can also convert the sound signal collected by the microphone into audio data for processing by the processor 601; it can set a speaker, a microphone, a headphone interface, and the like.
  • the sensor 606 can be used to sense an external signal and send it to the processor 601 for processing; it can include a motion sensor, a light sensor, and the like.
  • the output device 607 can be used to display various signals; the display panel can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.
  • Input device 604 can be used to input information such as numbers and characters; it can be a physical button, a touch panel, or the like.
  • the power supply 609 can provide power to various portions of the terminal and is logically coupled to the processor 609 via a power management system; it can include one or more components such as a DC or AC power source, a charging system, a power status indicator, and the like.
  • the memory 602 can be used to store software programs and modules; it can be a computer readable storage medium, specifically a hard disk, a flash memory, or the like.
  • the processor is the control center of the terminal, and performs various functions of the terminal and processes the terminal data by running or executing software programs and/or modules stored in the memory 602, and calling data stored in the memory 602.
  • the terminal includes: one or more processors 601, a memory 602, one or more applications, wherein the one or more applications are stored in the memory 602 and configured to be configured by the one or
  • the plurality of processors 601 are configured to perform the video image processing method according to the first embodiment, or the video image processing method according to the second embodiment, or the video according to the third embodiment.
  • the terminal provided by the embodiment of the invention can perform the beat point detection on the music to be played in the played video, and display the beat point corresponding to the beat point in the video image according to the detected beat point during the music playing process.
  • Special effects so that the special effects displayed on the video image are closely related to the beat point of the selected video playing music, thereby improving the audio and the visual appeal of the music played in the video, increasing the interest of the video application and improving User experience satisfaction.
  • the terminal provided by the present invention can implement a combination of multiple beat point detection methods to quickly and accurately detect the beat point of the selected video playing music; and can select according to the type of special effect required by the video and the type of music.
  • the type of beat point and the method of detecting the beat point are detected to achieve an accurate beat point by using an appropriate method, and the amount of calculation can be reduced, the detection time is shortened, and the effect displayed on the video image and the selected video play are further ensured.
  • the beat points of the music are closely related to further improve the satisfaction of the user experience.
  • the terminal provided by the present invention when applied to the field of live video, it can realize the special effects according to the anchor intention, the type of music, and the will of the viewer, and can meet the requirements of the live broadcast, highlight the live broadcast atmosphere, and better promote the interaction between the anchor and the viewer. Further improve the satisfaction of the user experience.
  • the terminal provided by the embodiment of the present invention may implement the foregoing method embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Studio Circuits (AREA)

Abstract

L'invention concerne un procédé de traitement d'images vidéo, qui consiste : à obtenir un signal musical d'une musique que l'on souhaite lire dans une vidéo de lecture, et mesurer un point de battement de ladite musique selon ledit signal musical; à déterminer que, lorsque de la musique est lue dans la vidéo de lecture, un point de battement apparaît à une position de lecture courante; à obtenir un effet spécial correspondant audit point de battement; à traiter une image de la vidéo de lecture selon ledit effet spécial, et obtenir une image vidéo contenant l'effet spécial. Le procédé de traitement d'image vidéo de l'invention réalise une association étroite entre un effet spécial affiché sur une image vidéo et un point de battement d'une musique de lecture vidéo sélectionnée, et augmente ainsi l'attrait sonore et visuel de la musique lue dans la vidéo, ce qui augmente l'intérêt d'une application vidéo ainsi que le niveau de satisfaction de l'utilisateur de son expérience.
PCT/CN2018/119266 2017-12-15 2018-12-05 Procédé et appareil de traitement d'images vidéo, support de stockage informatique et terminal WO2019114582A1 (fr)

Applications Claiming Priority (10)

Application Number Priority Date Filing Date Title
CN201711353131.8A CN108111909A (zh) 2017-12-15 2017-12-15 视频图像处理方法及计算机存储介质、终端
CN201711353131.8 2017-12-15
CN201711481177.8 2017-12-29
CN201711481160.2A CN108259925A (zh) 2017-12-29 2017-12-29 视频直播中的音乐礼物处理方法、存储介质及终端
CN201711481160.2 2017-12-29
CN201711476832.0A CN108322802A (zh) 2017-12-29 2017-12-29 视频图像的贴图处理方法、计算机可读存储介质及终端
CN201711474216.1A CN108259983A (zh) 2017-12-29 2017-12-29 一种视频图像处理方法、计算机可读存储介质和终端
CN201711476832.0 2017-12-29
CN201711474216.1 2017-12-29
CN201711481177.8A CN108259984A (zh) 2017-12-29 2017-12-29 视频图像处理方法、计算机可读存储介质及终端

Publications (1)

Publication Number Publication Date
WO2019114582A1 true WO2019114582A1 (fr) 2019-06-20

Family

ID=66819948

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/119266 WO2019114582A1 (fr) 2017-12-15 2018-12-05 Procédé et appareil de traitement d'images vidéo, support de stockage informatique et terminal

Country Status (1)

Country Link
WO (1) WO2019114582A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110688496A (zh) * 2019-09-26 2020-01-14 联想(北京)有限公司 一种多媒体文件处理的方法及装置
CN112118482A (zh) * 2020-09-17 2020-12-22 广州酷狗计算机科技有限公司 音频文件的播放方法、装置、终端及存储介质

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101593541A (zh) * 2008-05-28 2009-12-02 深圳华为通信技术有限公司 一种与音频文件同步播放图像的方法及媒体播放器
US20100312559A1 (en) * 2007-12-21 2010-12-09 Koninklijke Philips Electronics N.V. Method and apparatus for playing pictures
CN104219570A (zh) * 2014-09-17 2014-12-17 百度在线网络技术（北京）有限公司 一种音频信号的播放方法及装置
CN105430494A (zh) * 2015-12-02 2016-03-23 百度在线网络技术（北京）有限公司 在播放视频的设备中识别视频中音频的方法和装置
CN105872838A (zh) * 2016-04-28 2016-08-17 徐文波 即时视频的媒体特效发送方法和装置
CN108111909A (zh) * 2017-12-15 2018-06-01 广州市百果园信息技术有限公司 视频图像处理方法及计算机存储介质、终端
CN108259984A (zh) * 2017-12-29 2018-07-06 广州市百果园信息技术有限公司 视频图像处理方法、计算机可读存储介质及终端
CN108259925A (zh) * 2017-12-29 2018-07-06 广州市百果园信息技术有限公司 视频直播中的音乐礼物处理方法、存储介质及终端
CN108259983A (zh) * 2017-12-29 2018-07-06 广州市百果园信息技术有限公司 一种视频图像处理方法、计算机可读存储介质和终端
CN108322802A (zh) * 2017-12-29 2018-07-24 广州市百果园信息技术有限公司 视频图像的贴图处理方法、计算机可读存储介质及终端

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100312559A1 (en) * 2007-12-21 2010-12-09 Koninklijke Philips Electronics N.V. Method and apparatus for playing pictures
CN101593541A (zh) * 2008-05-28 2009-12-02 深圳华为通信技术有限公司 一种与音频文件同步播放图像的方法及媒体播放器
CN104219570A (zh) * 2014-09-17 2014-12-17 百度在线网络技术（北京）有限公司 一种音频信号的播放方法及装置
CN105430494A (zh) * 2015-12-02 2016-03-23 百度在线网络技术（北京）有限公司 在播放视频的设备中识别视频中音频的方法和装置
CN105872838A (zh) * 2016-04-28 2016-08-17 徐文波 即时视频的媒体特效发送方法和装置
CN108111909A (zh) * 2017-12-15 2018-06-01 广州市百果园信息技术有限公司 视频图像处理方法及计算机存储介质、终端
CN108259984A (zh) * 2017-12-29 2018-07-06 广州市百果园信息技术有限公司 视频图像处理方法、计算机可读存储介质及终端
CN108259925A (zh) * 2017-12-29 2018-07-06 广州市百果园信息技术有限公司 视频直播中的音乐礼物处理方法、存储介质及终端
CN108259983A (zh) * 2017-12-29 2018-07-06 广州市百果园信息技术有限公司 一种视频图像处理方法、计算机可读存储介质和终端
CN108322802A (zh) * 2017-12-29 2018-07-24 广州市百果园信息技术有限公司 视频图像的贴图处理方法、计算机可读存储介质及终端

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110688496A (zh) * 2019-09-26 2020-01-14 联想(北京)有限公司 一种多媒体文件处理的方法及装置
CN112118482A (zh) * 2020-09-17 2020-12-22 广州酷狗计算机科技有限公司 音频文件的播放方法、装置、终端及存储介质

Similar Documents

Publication Publication Date Title
US11030987B2 (en) Method for selecting background music and capturing video, device, terminal apparatus, and medium
CN110267055B (zh) 推荐直播间的方法、装置和系统以及服务器、终端和介质
CN105159639B (zh) 音频封面显示方法及装置
CN108322802A (zh) 视频图像的贴图处理方法、计算机可读存储介质及终端
WO2019114514A1 (fr) Procédé et appareil permettant d'afficher des informations de hauteur tonale dans une salle de diffusion en direct, et support de stockage
TWI574256B (zh) 互動節拍特效系統及互動節拍特效處理方法
CN110740262A (zh) 背景音乐的添加方法、装置及电子设备
CN110267067A (zh) 直播间推荐的方法、装置、设备及存储介质
CN108259925A (zh) 视频直播中的音乐礼物处理方法、存储介质及终端
CN108259983A (zh) 一种视频图像处理方法、计算机可读存储介质和终端
CN105117102B (zh) 音频界面显示方法和装置
CN106531201B (zh) 歌曲录制的方法和装置
CN109302538A (zh) 音乐播放方法、装置、终端及存储介质
CN108111909A (zh) 视频图像处理方法及计算机存储介质、终端
CN110691633B (zh) 用于确定响应的反应时间以及使用户界面与正在渲染的内容同步的方法和系统
CN108259984A (zh) 视频图像处理方法、计算机可读存储介质及终端
CN109587549B (zh) 视频录制方法、装置、终端及存储介质
CN108090140A (zh) 一种歌曲播放方法及移动终端
CN109660855A (zh) 贴纸显示方法、装置、终端及存储介质
CN111711838B (zh) 视频切换方法、装置、终端、服务器及存储介质
CN108922506A (zh) 歌曲音频生成方法、装置和计算机可读存储介质
CN110087149A (zh) 一种视频图像分享方法、装置及移动终端
CN110209871A (zh) 歌曲评论发布方法及装置
CN108922562A (zh) 演唱评价结果显示方法及装置
CN110290392A (zh) 直播信息显示方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18889596

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18889596

Country of ref document: EP

Kind code of ref document: A1