WO2019114582A1

WO2019114582A1 - Video image processing method and computer storage medium and terminal

Info

Publication number: WO2019114582A1
Application number: PCT/CN2018/119266
Authority: WO
Inventors: 危文; 袁少龙; 周宇涛; 丘智鉴; 颜乐驹; 魏启征; 李敬
Original assignee: 广州市百果园信息技术有限公司
Priority date: 2017-12-15
Filing date: 2018-12-05
Publication date: 2019-06-20

Abstract

The present invention provides a video image processing method, said method comprising: obtaining a music signal of music which is desired to be played in a playback video, and measuring a beat point of said music according to said music signal; determining that when music is played in the playback video, a beat point appears at a current playback position; obtaining a special effect corresponding to said beat point; processing an image in the playback video according to said special effect, and obtaining a video image containing the special effect. The video image processing method provided by the present invention achieves a close association between a special effect displayed on a video image and a beat point of a selected video playing music, and thus increases the aural and visual appeal of the music played in the video, thereby increasing the interestingness of a video application and raising the user's level of satisfaction with the experience.

Description

Video image processing method and computer storage medium, terminal

Technical field

The present invention relates to the field of image processing technologies, and in particular, to a video image processing method.

Background technique

With the continuous development of Internet technology and audio and image processing technologies, in video applications such as short video and live video, music is usually played in the played video, and the played music can be selected by the user to give the user a video. The auditory enjoyment in the process, thereby improving the user experience satisfaction.

However, in the conventional technology, the music that can be selected can only be the music file in the terminal local or video application, and the video image in the playing video is often not related to the played music, and the music is simply added to the playing video. Among them, the music played in the playing video is not sufficiently appealing in terms of hearing and visual, and the interactive way of simply selecting the music to play in the field of video live broadcasting cannot effectively improve the interaction enthusiasm between the viewer and the anchor, and is insufficient to satisfy the live video. The demand, which in turn affects the user's experience satisfaction.

Corresponding to the shooting method of adding a texture to a video image during video shooting, the prior art texture application can usually simply add a user-selected texture to the video image, and the texture display mode is single, failing to fully mobilize the user. The interaction with the application, the user has less fun in the shooting process, and the user experience satisfaction is not high.

Summary of the invention

In order to overcome the above technical problems, in particular, the prior art cannot realize the problem that the video image in the playing video is closely related to the played music, and the following technical solutions are proposed:

In a first aspect, an embodiment of the present invention provides a video image processing method, including: acquiring a music signal of music to be played in a play video, detecting a beat point of the music according to the music signal; determining to play When playing music in the video, a beat point appears at the current play position; the special effect corresponding to the beat point is acquired; and the image in the play video is processed according to the special effect to obtain a video image including the special effect.

In a second aspect, an embodiment of the present invention provides a video image processing method, including the steps of: identifying an audio feature of a sound in a play video; downloading, from a server, a music matching the audio feature and a beat point of the music The effect of playing the music in the play video, determining that a beat point occurs at a current play position of the music; processing the image in the play video according to the determined effect corresponding to the beat point, and obtaining the special effect Video image.

In a third aspect, an embodiment of the present invention provides a video image processing method, including the steps of: acquiring a music signal of music to be played in a play video; determining whether a beat point and a music play for saving the music are pre-stored. a beat point description file of the correspondence of the position; if yes, acquiring the beat point description file; if not, detecting a beat point of the music according to the music signal, according to the detected beat point and the music play position Generating a beat point description file; determining, according to the beat point description file, that a beat point appears at a current play position when playing music in the play video; acquiring an effect corresponding to the beat point; and playing the video according to the special effect The image is processed to obtain a video image containing the effect.

In a fourth aspect, an embodiment of the present invention provides a music gift processing method in a live video broadcast, comprising: receiving a music gift sent by a viewer in a live video; the music gift includes music to be played in a live video; a music signal of the music, detecting a beat point of the music according to the music signal; determining to play a beat point in a current play position when playing the music in the music present in a live video; acquiring the beat point corresponding to The special effect; processing the image in the live video according to the special effect, and obtaining a live video of the video including the special effect.

In a fifth aspect, an embodiment of the present invention provides a texture processing method for a video image, including the steps of: acquiring an image in a playback video, adding a texture to the image; and acquiring music of a music to be played in the played video. a signal, detecting a beat point of the music according to the music signal; determining that a beat point occurs at a current play position when playing music in the play video; acquiring a texture effect corresponding to the beat point; according to the map effect The texture in the image is processed to obtain a video image containing the texture effect.

In a sixth aspect, an embodiment of the present invention provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the program is executed by the processor, the first aspect, the second aspect, and the first The method of the third aspect, the fourth aspect or the fifth aspect.

In a seventh aspect, an embodiment of the present invention provides a terminal, including: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory And configured to be executed by the one or more processors, the one or more programs configured to: the method of the first aspect, the second aspect, the third aspect, the fourth aspect, or the fifth aspect.

Compared with the prior art, the invention has the following beneficial effects:

(1) The video image processing method provided by the present invention performs beat point detection on music to be played in a play video, and displays the beat point corresponding to the beat point in the video image according to the detected beat point during the music playing process. Special effects, so that the special effects displayed on the video image are closely related to the beat point of the selected video playing music, thereby improving the audio and the visual appeal of the music played in the video, increasing the interest of the video application and improving User experience satisfaction. Further, in combination with a plurality of beat point detecting methods, the beat point of the selected video playing music is quickly and accurately detected; and the present invention can select the type of the desired beat point according to the type of the special effect required by the video and the type of the music. And the method of detecting the beat point to achieve an accurate beat point by using an appropriate method, and can reduce the amount of calculation, shorten the detection time, and further ensure that the special effect displayed on the video image is close to the beat point of the selected video playing music. Relevance.

(2) The video image processing method provided by the present invention can know the music that the user wants to play by recognizing the sound in the playing video, and the music played by the user in the playing video is no longer limited to the existing music file, and can be hummed. The song segment quickly acquires the corresponding music, and then displays the corresponding special effect in the video image according to the beat point of the music during the music playing process, thereby realizing playing the music in the playing video and obtaining the special effect corresponding to the music beat point. Video image. The method expands the way of acquiring music, and can quickly and conveniently obtain the music to be played, the interaction between the user and the video application is strong, and the special effect displayed on the video image is closely related to the beat point of the music obtained through the sound input. This improves the visual and visual appeal of the music played in the video, significantly increasing the interest of the video application and increasing user experience satisfaction. Further, when applied to the real-time field, the music to be played in the live video can be confirmed by recognizing the humming sound of the anchor or the viewer, which can meet the needs of the live broadcast, highlight the live atmosphere, promote the interaction between the anchor and the viewer, and further increase the video. The fun of the app.

(3) The video image processing method, the computer readable storage medium, and the terminal provided by the present invention obtain the beat point corresponding to the beat point description file by acquiring the beat point description file of the music or intelligently generating the beat point description file according to the music signal. The special effect is to make the special effects displayed on the video image closely related to the beat point of the selected video playing music, thereby improving the visual and visual appeal of the music played in the video, and increasing the interest of the video application.

(4) The music gift processing method provided by the present invention performs beat point detection on music in a music gift sent by the viewer to the anchor, and displays the beat point corresponding to the beat point in the video image according to the detected beat point during the playing process. Special effects, so that the special effects displayed on the video image are closely related to the beat points of the music selected by the viewer, thereby improving the auditory and visual appeal of the music played in the live video; and by playing the music with the anchor and the viewer The combination of gift giving behavior enriches the interaction between the viewer and the anchor in the live video broadcast, significantly improves the interaction enthusiasm between the viewer and the anchor, effectively highlights the live broadcast atmosphere, meets the demand for live video, and increases the interest of the live video application.

(5) The texture processing method of the video image provided by the present invention, by adding a texture to the video image, performing beat point detection on the music to be played in the played video, and according to the detected beat point during the music playing process Displaying the texture effect corresponding to the beat point in the video image, so that the texture effect displayed on the video image is closely related to the beat point of the selected video playing music, and the music played in the playing video and the displayed texture are improved. Hearing and visual appeal; and users can express their own personality through custom-set music, texture and texture effects and can edit and play personalized video in real time. This method satisfies the user's personalized video design needs. To increase the fun of video applications and the interactivity of apps and users.

The additional aspects and advantages of the invention will be set forth in part in the description which follows.

DRAWINGS

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from

1 is a flowchart of a method for processing a video image according to an embodiment of the present invention;

2 is a flowchart of a method for processing a video image according to an embodiment of the present invention;

3 is a flowchart of a method for processing a video image according to an embodiment of the present invention;

4 is a flowchart of a method for processing a music gift according to an embodiment of the present invention;

FIG. 5 is a flowchart of a method for mapping a video image according to an embodiment of the present invention; FIG.

FIG. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described in detail below, and the examples of the embodiments are illustrated in the drawings, wherein the same or similar reference numerals are used to refer to the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the drawings are intended to be illustrative of the invention and are not to be construed as limiting.

Embodiment 1

The embodiment of the invention provides a video image processing method. As shown in FIG. 1 , the method includes:

Step S101: Acquire a music signal of music to be played in the played video, and detect a beat point of the music according to the music signal.

For the embodiment, the music to be played in the playing video may be the music selected from the pre-stored music of the video application, or the music loaded in the video application may be selected from the pre-stored music of the user terminal, and The live music played by the other device is obtained by the microphone in the video application, and the source of the music is not limited in this embodiment.

For the embodiment, the action of acquiring the music signal is performed after the user selects the music to be played in the playing video, and the action of selecting the music may be performed before the video is played, or may be performed in the playing video.

For example, when applied to the field of video live broadcast, the music may be selected before the live broadcast of the video or in the live broadcast of the video, and the terminal may immediately acquire the music signal of the selected music, and detect the beat of the music according to the music signal. point.

For another example, when applied to recording of a short video, the music is selected before the video is played, that is, before the video is recorded, and after the selected music, the terminal immediately acquires the music signal of the selected music, and according to the music signal. A beat point of the music is detected.

For the embodiment, the beat point of the music detected according to the music signal may be implemented by using various beat point detection methods.

For example, for the beat detecting method, the beat point includes a strong beat point and a weak beat point; the method includes: filtering the music signal, filtering and performing short-time Fourier transform to obtain a spectrum; according to the spectrum, The energy change value of the detection point is determined; according to the energy change value, a strong beat point or a weak beat point is detected at the detection point.

For another example, for the beat point detection method, the beat point includes a strong beat point and a weak beat point; the method includes: performing weighting processing on the music signal to obtain a weighted music signal; according to the weighted music The energy intensity value of the signal detects a strong beat point or a weak beat point at the detection point.

For another example, for the beat detecting method, the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy intensity value of the music signal; and counting each adjacent two candidates according to each candidate beat point The time interval between the frames in which the beat points are located; according to the time interval, it is detected that the candidate beat points correspond to strong beat points corresponding to the detected points.

For another example, for the beat detecting method, the beat point includes a strong beat point; the method includes: obtaining a candidate beat point according to the energy variation difference of the music signal of the detected point; and according to the candidate beat point, each phase Two adjacent candidate beat points are taken as the signal starting point to intercept two pieces of music signals; according to the comparison result of the two pieces of music signals, it is detected that the candidate beat points corresponding to the detected points have strong beat points.

Step S102: When it is determined that the music is played in the playing video, a beat point appears at the current playing position.

For the present embodiment, each beat point detected in step S101 corresponds to a different play position of the music. When the music is played in the play video, if the current play position of the music corresponds to the beat point detected in step S101, it is determined. Whether the beat point is a strong beat point or a weak beat point.

Step S103: Acquire an effect corresponding to the beat point.

For the embodiment, the special effect may be a material such as a fireworks, a love, a snowflake, etc., and the specific expression of the material is not limited in this embodiment. Different shape feature parameters may be set for the same material, and the shape feature parameters include a size parameter, a color parameter, and the like, which are not limited in this embodiment.

For the embodiment, the beat point in a piece of music includes a strong beat point and a weak beat point, and the material in the special effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different.

Step S104: Processing an image in the played video according to the special effect to obtain a video image including the special effect.

For the embodiment, the video in the effect is obtained by acquiring the material in the special effect and synthesizing the material and the image in the play video in a layer superposition manner to obtain a video image including the material in the special effect. In other embodiments, the video image including the special effect may be obtained by integrating data of the special effect with the image or modifying the image according to the shape characteristic parameter of the material in the special effect.

For example, in the field of live video broadcasting, data of the effect data can be integrated with the image data to obtain a video image data packet, and the data packet is sent to the client, so that the client displays the video image containing the special effect.

For another example, the image may be scaled according to the parameter by obtaining the shape feature parameter of the material in the special effect to obtain a video image that can achieve the effect of the special effect.

The video image processing method provided by the embodiment performs beat point detection on the music to be played in the played video, and displays the special effect corresponding to the beat point in the video image according to the detected beat point during the music playing process. In order to make the special effects displayed on the video image closely related to the beat point of the selected video playing music, thereby increasing the audio and visual appeal of the music played in the video, increasing the interest of the video application and improving the user experience. Satisfaction.

In addition, the video image processing method provided in this embodiment combines multiple beat point detection methods to quickly and accurately detect the beat point of the selected video playing music, further ensuring the special effects displayed on the video image and the selected video. The beat points of playing music are closely related to further improve the satisfaction of the user experience.

Another possible implementation manner of the embodiment of the present invention, further comprising the following steps, where

Step S101 includes:

Obtaining a music signal of music to be played in the played video, determining whether to detect a strong beat point, or detecting a strong beat point and a weak beat point;

If the strong beat point is detected, it is judged whether the intensity value detection or the change value detection is used;

If the intensity value detection is used, the candidate beat point is obtained according to the energy intensity value of the music signal, and the time interval between frames of each adjacent two candidate beat points is counted according to each candidate beat point, and the detection is performed according to the time interval. A strong beat point appears at the detection point corresponding to the candidate beat point;

If the change value detection is used, the candidate beat point is obtained according to the energy variation difference of the music signal of the detection point, and according to the candidate beat point, two adjacent music beat signals are taken as the signal starting point of each adjacent two candidate beat points. According to the consistent result of the two pieces of music signals, it is detected that the candidate beat points correspond to strong beat points corresponding to the detected points;

If the strong beat point and the weak beat point are detected, it is judged whether the intensity value detection or the change value detection is used;

If the intensity value detection is used, the music signal is weighted to obtain a weighted music signal, and according to the energy intensity value of the weighted music signal, a strong beat point or a weak beat point is detected at the detection point;

If the change value detection is used, the music signal is filtered, and after filtering, the short-time Fourier transform is performed to obtain a spectrum, and according to the spectrum, the energy change value of the detection point is determined, and according to the energy change value, the weak beat of the detection point is detected. Point or strong candidate beat points.

For the present embodiment, different beat point detection methods are corresponding for different types of required detection beat points and detection criteria.

Wherein, the determining whether to detect a strong beat point or detecting a strong beat point and a weak beat point comprises:

Get the type of special effect required for the video, judge whether to detect strong beat points according to the type of special effects required by the video, or detect strong beat points and weak beat points;

If the strong beat point is detected, it is determined whether the intensity value detection or the change value detection is used, including:

If a strong beat point is detected, the type of the music is acquired, and whether the intensity value or the change value is used is determined according to the type;

If the strong beat point and the weak beat point are detected, it is determined whether the intensity value or the change value is detected, including:

If a strong beat point and a weak beat point are detected, the type of the music is acquired, and whether the intensity value or the change value is detected is determined according to the type.

For the present embodiment, the type of the desired beat point is determined by the type of the effect required for the acquired video. The effect type required for the video is the default effect type selected by the user or the video application. For example, the user wants to have endless effects in the playing video, so it is judged according to the type of special effect required by the video that both the strong beat point and the weak beat point are detected.

For the present embodiment, the detection criteria are judged by the type of music acquired. For example, the type of music acquired is rock, and the music signal of the music type tends to have a high intensity value, but the change value is not obvious, so the beat point of the music is detected by detecting the intensity value according to the type thereof.

For the present embodiment, the type of the desired beat point can be selected according to the type of the special effect required by the video, and then the method of detecting the desired beat point can be selected according to the type of the music, so as to obtain an accurate beat point by using an appropriate method. Moreover, the amount of calculation can be reduced, the detection time is shortened, and the special effect displayed on the video image is closely related to the beat point of the selected video playing music, thereby further improving the satisfaction of the user experience.

After step S101, the method further includes: recording a correspondence between the music playing position and the beat point.

For the embodiment, after the beat point of the music is detected according to the music signal, the beat point is associated with the corresponding music play position, and the corresponding relationship is recorded. The method for establishing the corresponding relationship is not limited in this embodiment, and may be a manner of adding label information.

For example, the signal data of the music playing position corresponding to the beat point is added with tag information, and the tag information carries information indicating that the music playing position has a beat point and the beat point is a strong beat point or a weak beat point.

Step S102 includes: determining, according to the correspondence, that a beat point occurs at a current play position when playing music in the play video.

For the embodiment, when there is a record corresponding to the beat point in the current play position when the music is played in the play video, the correspondence record of the music play position and the beat point is acquired, and the corresponding relationship is extracted, and then according to the The correspondence determines that a beat point occurs at the current play position and determines that the beat point is a strong beat point or a weak beat point.

For the embodiment, by recording the correspondence between the music playing position and the beat point, it is possible to easily and quickly determine whether the current playing position has a beat point and whether the beat point is a strong beat point or a weak beat point.

For the embodiment, the video is a live video, that is, the method in this embodiment is mainly applied to the field of video live broadcast.

Before step S103, the method further includes:

Determining whether the special effect group is specified in the live broadcast of the live video; if yes, obtaining the specified effect group; otherwise, obtaining the scene corresponding to the live broadcast, and obtaining the preset number of the hottest server in the current live server matching the scene from the live server Special effects group;

Determining whether the music type automatic adaptation special effect function is enabled in the live broadcast; if yes, identifying the type of the music, obtaining an effect group of the type adaptation from the preset number of special effect groups; otherwise, to the client Sending a request for selecting an effect group from the preset number of effect groups;

Determining whether the selection information received from the client according to the request is received; if yes, obtaining an effect group from the preset number of effect groups according to the selection information, otherwise, from the preset number of special effect groups Randomly get a special effects group.

Step S103 includes: obtaining an effect corresponding to the beat point from the obtained special effects group.

For the embodiment, the material of the special effects of the special effect group is the same, but the shape characteristic parameters of the material are different.

For the embodiment, the specified effect group may be specified by the anchor or user of the live broadcast, and the specified effect group corresponds to the anchor live broadcast feature of the anchor, for example, the live broadcast of the anchor is performed by an animation. The theme, and the anchor pre-specifies the use of the animation effect-themed effect group, so the special effects displayed in the live video of the live broadcast are obtained from the specified effect group.

For the embodiment, the scenario corresponding to the live broadcast may be determined by extracting scene features corresponding to the location, weather, and landscape, such as a cafe, a sunny outdoor, a seaside, etc., and is not performed in this embodiment. limited. The effect group matching the scene has a correlation with the scene feature of the scene.

For the present embodiment, the preset number is a value greater than or equal to 2, which may be a value of 3, 5, 6, etc., which is not limited in this embodiment.

For the present embodiment, the type of music includes rock, pop, and the like, and the type of the music is determined according to the acquired beat point of the music.

After step S104, the method further includes:

Send the processed live video to the client.

When the video image processing method provided by the present invention is applied to the field of live video, by selecting special effects according to the anchor intention, the type of music, and the will of the viewer, the requirements of the live broadcast can be met, the live broadcast atmosphere can be enhanced, and the interaction between the anchor and the audience can be better promoted. Improve user satisfaction.

Embodiment 2

The embodiment of the invention provides a video image processing method. As shown in FIG. 2, the method includes:

Step S201: Identify an audio feature of the sound in the played video.

Step S202: Download, from the server, the music that matches the audio feature and the special effect corresponding to the beat point of the music.

For the embodiment, the manner in which the user acquires the music to be played in the played video is no longer limited to selecting the existing music file in the terminal or the video application, but by identifying the sound in the played video, and then determining the matching. The music corresponding to the sound is downloaded from the server to realize the acquisition of the music to be played. Specifically, the terminal captures the sound in the played video and extracts the audio feature of the sound, and then sends the audio feature to the server, so that the server performs the audio feature with the music in the preset music library that it holds. The matching is traversed to determine the music corresponding to the audio characteristics of the sound. The sound may be a sound emitted by the user in the playing video, or may be a sound emitted by other terminal devices.

For the present embodiment, before performing an action of recognizing the audio feature of the sound in the played video, it is also necessary to determine whether the user transmits the audio feature recognition request. For example, when the user triggers the listening song function key of the video application interface, the terminal begins to perform the action of recognizing the audio feature of the sound in the playing video.

For the embodiment, the special effect may be a material such as a fireworks, a love, a snowflake, etc., and the specific expression of the material is not limited in this embodiment. Different shape feature parameters may be set for the same material, and the shape feature parameters include a size parameter, a color parameter, and the like, which are not limited in this embodiment. In addition, the beat point in a piece of music includes a strong beat point and a weak beat point, and the material in the special effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different. The special effect corresponding to the beat point of the music downloaded from the server in step 202 refers to the material of the different shape feature parameters corresponding to the strong beat point and the weak beat point of the music respectively, wherein the special effect may be the user advance Setting the special effect corresponding to the music may also be a default effect corresponding to the music by the server.

Step S203: playing the music in the play video, and determining that a beat point occurs at a current play position of the music.

For the present embodiment, when the music is played in the play video by pre-acquiring the corresponding relationship between the music play position and the beat point, whether the beat point occurs in the current play position of the music may be determined according to the corresponding relationship, and the beat is determined. Whether the point is a strong beat or a weak beat.

Step S204: Processing an image in the played video according to the determined special effect corresponding to the beat point to obtain a video image including the special effect.

For the present embodiment, the material in the special effect corresponding to the beat point downloaded from the server in step S202 is acquired, and the material in the play video is synthesized in a layer superposition manner to obtain A video image containing the material in the effect. In other embodiments, the obtaining of the video image including the special effect may also be implemented by integrating the special effect with the image data or modifying the image according to the shape characteristic parameter of the material in the special effect.

The video image processing method provided by the present invention can know the music that the user wants to play by recognizing the sound in the playing video, and the music played by the user in the playing video is no longer limited to the existing music file, and can be quickly sung by the sing song piece. The corresponding music is obtained, and then the corresponding special effect is displayed in the video image according to the beat point of the music during the music playing, thereby realizing playing the music in the playing video and obtaining a video image containing the special effect corresponding to the music beat point. The method expands the way of acquiring music, and can quickly and conveniently obtain the music to be played, the interaction between the user and the video application is strong, and the special effect displayed on the video image is closely related to the beat point of the music obtained through the sound input. This improves the visual and visual appeal of the music played in the video, significantly increasing the interest of the video application and increasing user experience satisfaction.

Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the second embodiment, where

Before the step S203, the method further includes:

Downloading, from the server, a beat point description file that records a correspondence between a music playing position of the music and a beat point;

The determining that a beat point occurs at a current play position of the music includes:

Determining that a beat point occurs at a current play position of the music according to the correspondence in the beat point description file.

For the embodiment, by downloading the preset beat point description file corresponding to the music from the server, when the music is played in the play video, the music play position and the beat point recorded in the beat point description file are acquired. Corresponding relationship determines that a beat point occurs at a current play position of the music, and may also determine whether the beat point is a strong beat point or a weak beat point. The use of the beat point description file can achieve the corresponding relationship quickly and conveniently, and the video image containing the special effect corresponding to the music beat point can be further obtained after a very short file loading time, thereby further improving the user experience satisfaction. degree.

Before the step S203, the method further includes:

Acquiring a music signal of the music; detecting a beat point of the music according to the music signal; recording a correspondence relationship between the music playing position and a beat point;

Based on the correspondence, it is determined that a beat point occurs at a current play position of the music.

For another example, for the beat detecting method, the beat point includes a strong beat point and a weak beat point; the method includes: performing weighting processing on the music signal to obtain a weighted music signal; and according to the weighted music The energy intensity value of the signal detects a strong beat point or a weak beat point at the detection point.

For the embodiment, the correspondence between the music playing position and the beat point can be confirmed by detecting the beat point in real time, and the plurality of beat point detecting methods can quickly and accurately detect the beat point of the music corresponding to the sound in the played video, and then Obtaining a video image containing special effects corresponding to the music beat point can further improve user experience satisfaction.

Obtaining the music signal, determining whether to detect a strong beat point, or detecting a strong beat point and a weak beat point;

If the strong beat point is detected, it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the candidate beat point is obtained according to the energy intensity value of the music signal, and each adjacent two candidate beats are counted according to each candidate beat point. The time interval between the frames in which the points are located, according to the time interval, detecting that the candidate beat points corresponding to the detected points have strong beat points; if the change value detection is used, the candidate beat points are obtained according to the energy variation difference of the music signals of the detected points. According to the candidate beat point, two adjacent music beat signals are taken as starting points of the two adjacent candidate beat points, and according to the comparison result of the two pieces of music signals, a strong beat point corresponding to the candidate beat point detection point is detected. ;

If the strong beat point and the weak beat point are detected, it is judged whether the intensity value detection or the change value detection is used; if the intensity value detection is used, the music signal is weighted to obtain the weighted music signal, according to the weighted music signal. The energy intensity value is detected as a strong beat point or a weak beat point at the detection point; if the change value detection is used, the music signal is filtered, and then subjected to short-time Fourier transform to obtain a spectrum, and the detection is determined according to the spectrum. The energy change value of the point is detected according to the energy change value, and a weak beat point or a strong candidate beat point is detected at the detection point;

For the present embodiment, different beat detection methods are corresponding for different types of required detection beat points and detection criteria.

Obtain the type of special effect required for the video, judge whether to detect the strong beat point according to the type of special effect required by the video, or detect the strong beat point and the weak beat point; if the strong beat point is detected, it is judged whether the intensity value detection or the change value detection is used, including: Detecting a strong beat point, obtaining the type of the music, determining whether to use the intensity value detection or the change value detection according to the type; if detecting the strong beat point and the weak beat point, determining whether to use the intensity value detection or the change value detection, including: if detecting The strong beat point and the weak beat point acquire the type of the music, and determine whether to use the intensity value detection or the change value detection according to the type.

For the present embodiment, the type of the desired beat point can be determined by the type of the effect required for the acquired video. The effect type required for the video is the default effect type selected by the user or the video application. For example, the user wants to have endless effects in the playing video, so it is judged according to the type of special effect required by the video that both the strong beat point and the weak beat point are detected.

For the present embodiment, the detection criteria can be determined by acquiring the type of music corresponding to the sound. For example, the type of music corresponding to the acquired sound is rock, and the music signal of the music type often has a high intensity value, but the change value is not obvious, so according to the type thereof, the beat of the music is detected by detecting the intensity value. point.

For the present embodiment, the type of the desired beat point and the method of detecting the beat point can be selected according to the type of the special effect required for the video and the type of the music, so as to obtain an accurate beat point by using an appropriate method, and the amount of calculation can be reduced. , shortening the detection time, further ensuring the close relationship between the special effects displayed on the video image and the music beat point, and further improving the satisfaction of the user experience.

The step S201 includes:

Receiving, in the live video, a singer song request sent by the client; receiving a singer voice sent by the client; and identifying an audio feature of the singer voice of the viewer;

After the step S204, the method further includes:

Send the processed video image to the client.

For the embodiment, the interaction between the anchor and the viewer is added in the live video. In the process of watching the live video, the viewer can send the audience to the live broadcast in addition to the common interactive behaviors such as speaking, praising and giving gifts. Sing a song request. After the anchor sings a song request by a certain viewer, the viewer client and the live terminal perform a connection, and the live broadcast end receives the singer voice of the viewer in the live video and extracts the audio feature of the voice, and then the The audio features are sent to the server to cause the server to traverse the audio features in the preset music library with which they are stored, thereby determining the music corresponding to the audio features of the viewer's humming sound. The audience can obtain the opportunity for the audience to sing songs by paying. The anchor can also give the sing-song opportunity as a gift to the audience in the live room. The program can meet the needs of the live broadcast, highlight the live atmosphere, and significantly increase the anchor and The interaction between viewers further increases the fun of video applications.

The step S201 includes:

Receiving a humming download song command sent by the anchor in the live video; receiving an anchor humming sound sent by the anchor; and identifying an audio feature of the anchor humming sound.

For the embodiment, the interactive song of the anchor and the viewer is added to the live video, and the anchor can quickly and easily acquire the music corresponding to the humming voice by humming after sending the humming download song command. The live end receives the humming sound of the anchor in the live video and extracts the audio features of the sound, and then sends the audio feature to the server, so that the server matches the audio feature with the music in the preset music library A traversal match is made to determine the music corresponding to the audio characteristics of the anchor's humming sound. The anchor can select the song with the highest audience voice during the live broadcast of the video according to the viewer's barrage information, so as to play the music in the live video and present the video image containing the special effect corresponding to the music beat point, the solution can satisfy the live broadcast. Demand, highlighting the live broadcast atmosphere, significantly increase the interaction between the anchor and the viewer, further increasing the fun of video applications.

Embodiment 3

The embodiment of the invention provides a video image processing method. As shown in FIG. 3, the method includes:

Step S301: Acquire a music signal of music to be played in the played video.

For the embodiment, the music to be played in the playing video may be music selected from pre-stored music in the video application, or may be selected from the pre-stored music of the user terminal to be loaded into the video application, and may also be The live music played by the other device is obtained by the microphone in the video application, and the source of the music is not limited in this embodiment.

For example, when applied to the field of video live broadcast, the music may be selected before the live broadcast of the video or in the live broadcast of the video, and the terminal will immediately acquire the music signal of the selected music and perform subsequent preset steps.

For another example, when applied to recording of a short video, the music is selected before the video is played, that is, before the video is recorded, and the terminal immediately acquires the music signal of the selected music after the selected music, and performs subsequent preset steps. .

Step S302: determining whether a beat point description file for preserving a correspondence relationship between a beat point of the music and a music play position is prestored; if yes, acquiring the beat point description file; if not, detecting the sound according to the music signal The beat point of the music generates a beat point description file according to the corresponding relationship between the detected beat point and the music play position.

For the embodiment, in order to realize adding special effects to each beat point of the corresponding music in the playing video during music playing, it is necessary to determine whether a beat point appears at the current playing position of the music. The present invention uses a beat point description file to determine the correspondence between the beat point of the music and the music play position.

Wherein, determining whether a beat point description file for storing a correspondence relationship between a beat point of the music and a music play position is pre-stored; if yes, acquiring the beat point description file; comprising: determining whether a beat point is pre-stored in the local file a description file, if yes, obtaining the beat point description file from a local file; or determining whether the server pre-stores a beat point description file, and if so, downloading the beat point description file from the server; or determining whether the local file is pre-stored There is a beat point description file, and if so, the beat point description file is obtained from the local file; if not, it is determined whether the server pre-stores the beat point description file, and if so, the beat point description file is downloaded from the server.

For the embodiment, the beat point description file may be obtained from a local file or may be obtained from a server, and the user may also try to obtain a beat point description file from the server after the local acquisition fails.

For the present embodiment, when the existing beat point description file is not obtained, it is necessary to intelligently generate a beat point description file according to the music signal, specifically, detecting a beat point of the music according to the music signal, according to the The corresponding relationship between the detected beat point and the music playback position generates a beat point description file. After the beat point description file is intelligently generated, the beat point description file may also be uploaded to the server for each user to download and use. The detecting a beat point of the music according to the music signal may be implemented by using a plurality of beat point detecting methods.

Step S303: Determine, according to the beat point description file, that a beat point appears in the current play position when playing music in the play video.

For the embodiment, the beat point description file records the correspondence between the beat point of the music and the music play position by using the information symbol that the terminal can read and understand, and when the music is played, by loading the beat point description file The data is analyzed and the data is analyzed. According to the data analysis result, whether the beat point of the current playing position of the music is known or not, and whether the beat point is a strong beat point or a weak beat point is determined.

Step S304: Acquire an effect corresponding to the beat point; process the image in the play video according to the special effect to obtain a video image including the special effect.

For example, in the field of live video, data of the effect can be integrated with the image data to obtain a video image data packet, and the data packet is sent to the client, so that the client displays the video image containing the special effect.

The video image processing method provided by the present invention obtains a beat point description file of the music or intelligently generates a beat point description file according to the music signal, and obtains an effect corresponding to the beat point determined according to the beat point description file, so as to enable the special effect displayed on the video image. It is closely related to the beat point of the selected video playing music, thereby improving the visual and visual appeal of the music played in the video, increasing the interest of the video application and increasing the satisfaction of the user experience.

Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the third embodiment, where

After the acquiring the beat point description file in step S302, the method further includes:

If the beat point description file is a beat point description file downloaded from the server, detecting a beat point of the music according to the music signal, and correcting the beat point description file according to the detected beat point, in the local file The proof point description file after the proofreading is saved.

For the embodiment, the beat point description file downloaded from the server stores the corresponding relationship between the beat point of the music and the music play position, but there may be a corresponding music in the beat point description file and the current music to be played is incomplete. Consistent issues, for example, in a video application, the user can intercept the selected music and choose to play the intercepted music piece in the playing video, while the beat point description file downloaded from the server corresponds to the entire music. The correspondence between the beat point and the music playback position. Therefore, it is necessary to proof the beat point description file according to the detected beat point, and save the proofed beat point description file in the local file for use by the end user.

For the embodiment, by correcting the beat point description file acquired from the server, the accuracy of the beat point description file can be ensured, thereby ensuring the accuracy of the correspondence between the beat point and the music playing position, and further ensuring the video image. The associated effect of the displayed effect is related to the beat point of the selected video playback music.

The detecting, in the step S302, the beat point of the music according to the music signal, comprising:

Determining whether to detect a strong beat point or a strong beat point and a weak beat point according to the acquired music signal;

For the present embodiment, the detection criteria can be judged by the type of music acquired. For example, the type of music acquired is rock, and the music signal of the music type tends to have a high intensity value, but the change value is not obvious, so the beat point of the music is detected by detecting the intensity value according to the type thereof.

For the present embodiment, the type of the desired beat point and the method of detecting the beat point can be selected according to the type of the special effect required for the video and the type of the music, so as to obtain an accurate beat point by using an appropriate method, and the amount of calculation can be reduced. The detection time is shortened, and the special effect displayed on the video image is closely related to the beat point of the selected video playing music, thereby further improving the satisfaction of the user experience.

Before step S304, the method further includes:

Obtaining the special effect information corresponding to the beat point of the music set by the anchor of the live end, saving the correspondence between the beat point and the music play position and the special effect information into the beat point description file; generating a beat point description file containing the special effect information;

Obtaining the special effect corresponding to the beat point in step S304, including:

Obtaining an effect corresponding to the beat point according to the beat point description file containing the special effect information.

For the embodiment, the beat point description file not only stores the correspondence between the beat point and the music play position, but also stores the corresponding relationship between the two and the special effect information. The special effect information carries the material information that is randomly selected or customized by the user, and the shape characteristic parameter set for the material. For the definition of the material and its shape characteristic parameter, refer to the content of step S104 in the first embodiment, where not Let me repeat.

In other embodiments, the special effect information may also be set by a user watching the live broadcast, and the user may send a special effect setting request to the live broadcast terminal to implement the special effect setting.

For the present embodiment, the video image processing method provided by the present invention is applied to the field of live broadcast, and the special effect corresponding to the beat point of the custom setting by the beat point description file can be realized, which can meet the requirement of the live broadcast and further increase the interest of the video application. .

After the generating the beat point description file containing the special effect information, the method further includes:

Sending the music, the beat point description file containing the special effect information, and the special effect corresponding to the beat point to another live end of the continuous wheat, so that another live broadcast end displays the same as the live broadcast end when playing the music Special effects.

For the embodiment, the anchor of the current live broadcast end interacts with the anchor of another live broadcast during the live broadcast of the live broadcast, and the user watching the current broadcast of the active broadcast can simultaneously view the live broadcast of the interactive anchor. By realizing the effect corresponding to the beat point in the multi-end live broadcast, the live broadcast atmosphere can be effectively enhanced, and the popularity value of the live broadcasts of each interactive live end can be improved, which can meet the needs of the live broadcast and further increase the interest of the video application.

Embodiment 4

The music gift processing method provided by the invention is mainly applied to the field of video live broadcasting. In the existing live video broadcast, in order to activate the atmosphere and increase the interaction with the audience in the live broadcast, the anchor usually selects the music corresponding to the live broadcast theme of the live broadcast or the music with the highest audience in the live broadcast during the live broadcast. The audience will respond to the anchor by clicking, speaking or giving a gift. However, the video image in the playing video is often not related to the played music. The music is simply added to the playing video, and there is not enough appeal in the sense of hearing and visual, and the above interaction method is too single and cannot be effective. Increasing the enthusiasm of the audience and the anchor is not enough to meet the needs of live video. In view of the above problems, the present invention combines the interactive behavior of the anchor playing music and the audience to give gifts, and provides a music gift processing method to achieve a significant improvement in the interaction between the viewer and the anchor and the satisfaction of the user experience. The music gift processing method will be elaborated.

An embodiment of the present invention provides a music gift processing method. As shown in FIG. 4, the method includes:

Step S401: Receiving a music gift sent by a viewer in a live video broadcast; the music gift includes music to be played in a live video.

For the present embodiment, the viewer can interact with the anchor by sending a music gift to the anchor during the live video broadcast. The music gift received by the live broadcast end carries music information, and the music corresponding to the music information is music selected by the viewer to be played by the anchor in the live video broadcast.

Wherein, as a preferred embodiment, the music gift further includes special effect information corresponding to a beat point of the music. The special effect information carries material information randomly set by the user terminal or user-defined, and shape characteristic parameters set for the material.

For the embodiment, the special effect may be a material such as a fireworks, a love, a snowflake, etc., and the specific expression of the material is not limited in this embodiment. Different shape feature parameters may be set for the same material, and the shape feature parameters include a size parameter, a color parameter, and the like, which are not limited in this embodiment. In addition, the beat point in a piece of music includes a strong beat point and a weak beat point, and the material in the special effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different.

Step S402: Acquire a music signal of the music, and detect a beat point of the music according to the music signal.

For the present embodiment, after the music signal of the music to be played in the live video is obtained from the music gift, the beat point is detected on the music, wherein the beat point of the music is detected according to the music signal. It can be implemented by a variety of beat point detection methods.

Step S403: When it is determined that the music in the music gift is played in the live video, a beat point appears at the current play position.

For the present embodiment, each beat point detected in step S402 corresponds to a different play position of the music. When the music is played in the live video, if the current play position of the music corresponds to the beat point detected in step S402, then it is determined. Whether the beat point is a strong beat point or a weak beat point.

Step S404: Acquire an effect corresponding to the beat point.

Step S405: Process the image in the live video broadcast according to the special effect, and obtain a live video of the video including the special effect.

For the embodiment, the material of the different shape feature parameters of the special effect corresponding to the special effect information is obtained from the music gift, and the material is combined with the image in the live video broadcast in a layer overlay manner to obtain an inclusion A live video image of the special effects. After the step S405, the method further includes: sending the processed live video of the video to the client. In other embodiments, the video live image including the special effect may be obtained by integrating data of the special effect with the image or modifying the image according to the shape characteristic parameter of the material in the special effect.

For example, the data of the special effect can be integrated with the image data to obtain a live video image data package, and the data packet is sent to the client, so that the client displays the live video image of the video containing the special effect.

For example, by acquiring the shape feature parameter of the material in the special effect, the image is scaled according to the parameter, and after the video live image that can achieve the special effect is obtained, the image is sent to the viewer client, so that the client displays the inclusion. A live video of the effect.

In other embodiments, if the music gift does not carry the special effect information corresponding to the beat point of the music, the material corresponding to the random setting or the anchor custom setting of the live broadcast may be obtained as the special effect corresponding to the beat point. .

The music gift processing method provided by the present invention performs beat point detection on the music in the music gift sent by the viewer to the anchor, and displays the special effect corresponding to the beat point in the video image according to the detected beat point during the playing process. In order to make the special effects displayed on the live video of the video closely related to the beat points of the music in the music gift, thereby improving the auditory and visual appeal of the music played in the live video broadcast; and sending the gift to the audience by playing the music with the anchor. The combination of behaviors enriches the interaction between the viewer and the anchor in the live video broadcast, significantly improves the interaction enthusiasm between the viewer and the anchor, effectively highlights the live broadcast atmosphere, meets the demand for live video, and increases the interest of the live video application, improving the user. Satisfaction of experience.

Another possible implementation manner of the embodiment of the present invention further includes the following steps on the basis of the fourth embodiment, where

After the step S402, the method further includes: generating a beat point description file according to the detected correspondence between the beat point and the music playing position.

Further, the step S403 includes: determining, according to the beat point description file, that a beat point occurs at a current play position when the music in the music gift is played in a live video.

For the embodiment, in order to realize adding special effects to each beat point of the corresponding music in the live video of the video during music playing, it is necessary to determine whether a beat point appears at the current playing position of the music. In this embodiment, a beat point description file is used to determine the correspondence between the beat point of the music and the music play position. The beat point description file records the correspondence between the beat point of the music and the music play position by using the information symbol that the terminal can read and understand, and when the music is played, the data in the file is described by loading the beat point and the data is Perform analysis, according to the data analysis result, whether the beat point of the current playing position of the music is known, and whether the beat point is a strong beat point or a weak beat point.

For this embodiment, the correspondence between the beat point and the music playing position can be quickly and conveniently obtained through the beat point description file after a very short file loading time, thereby satisfying the demand of the live video, and further improving the user experience satisfaction. .

After the generating a beat point description file according to the detected relationship between the beat point and the music playing position, the method further includes: acquiring the special effect information, and saving the corresponding relationship between the beat point and the music playing position and the special effect information to the beat In the point description file; generate a beat point description file containing the effect information.

Further, the step S404 includes: acquiring, according to the beat point description file containing the special effect information, the special effect corresponding to the beat point from the music gift.

For the embodiment, the beat point description file not only stores the correspondence between the beat point and the music play position, but also stores the corresponding relationship between the two and the special effect information. The special effect information carries the material information that is randomly selected or customized by the user, and the shape characteristic parameter that is set for the material. For the specific definition, refer to the content of step S401 in the first embodiment, and details are not described herein again.

For the embodiment, after a very short file loading time, the beat point description file can quickly and conveniently obtain the special effect corresponding to the beat point of the current music playing position, which satisfies the requirement of the live video, and can further improve the user experience satisfaction. degree.

After the step S401, the method further includes:

Determining whether the anchor of the live end performs a preset action of opening the music gift; if yes, continuing the step of acquiring the music signal of the music, detecting a beat point of the music according to the music signal; and transmitting the The audience of the music gift feedbacks the anchor to open the information of the music gift.

For the present embodiment, in a live video application scenario, there are usually multiple viewers simultaneously sending music presents to the anchor, and since the live broadcast time is limited, the anchor often cannot open all the music gifts, so the music beat point detection is performed. Before the subsequent preset steps, you need to determine if the anchor wants to open the currently received music gift.

For the embodiment, the preset action performed by the anchor may be that the preset password is spoken in the live video, or the preset expression or gesture may be made in the live video, and the displayed on the display may also be clicked. Music gift logo.

For example, in the live video broadcast, the music gift appears in the form of a list. When the anchor says the password of “opening the eighth music gift”, the live terminal performs voice recognition on the password, and obtains the corresponding correspondence in the music gift list according to the recognition result. The music signal in the music gift and perform subsequent preset steps.

For example, in the live video broadcast, the music gift appears in the form of a barrage. When the anchor makes a "birth" gesture, the live broadcast terminal determines the anchor instruction to open the currently popped music gift according to the gesture recognition result of the live video image. And then get the music signal in the music gift and perform subsequent preset steps.

For another example, in the live video broadcast, the music gift appears in the form of gift rain at a preset time. When the live terminal receives the instruction of the anchor click terminal display to trigger one of the music gifts, the live terminal immediately acquires the music gift. Music signals and perform subsequent preset steps.

The step S402 includes:

Obtaining the special effect information in the music gift, determining whether to detect a strong beat point or detecting a strong beat point and a weak beat point according to the special effect information; if detecting the strong beat point, determining whether to use the intensity value detection or the change value detection, including If the strong beat point is detected, the type of the music is obtained, and the intensity value detection or the change value detection is used according to the type judgment; if the strong beat point and the weak beat point are detected, whether the intensity value detection or the change value detection is used, including: If a strong beat point and a weak beat point are detected, the type of the music is acquired, and whether the intensity value detection or the change value detection is used is determined according to the type.

For the present embodiment, the type of the desired beat point can be determined by acquiring the special effect information in the music gift. For example, in the music gift sent to the live end, the viewer sets the shape feature parameters of the material for the strong beat point and the weak beat point respectively, and the user wants to see the corresponding beat point when the music gift is opened in the live video broadcast. Cool special effects, so according to the special effects information of the viewer's custom settings, it is judged that both the strong beat point and the weak beat point are detected. For the present embodiment, the detection criteria can be judged by acquiring the type of music in the music gift. For example, the type of music in the acquired music gift is rock, and the music signal of the music type often has a high intensity value, but the change value is not obvious, so the music value is detected by detecting the intensity value according to the type selection. The beat point.

For the embodiment, the type of the desired beat point and the method of detecting the beat point can be selected according to the special effect information and the type of the music in the music gift, so as to obtain an accurate beat point by using an appropriate method, and the operation can be reduced. The quantity, shorten the detection time, further ensure the close relationship between the special effects displayed on the video image and the music beat point, and further improve the satisfaction of the user experience.

Embodiment 5

An embodiment of the present invention provides a mapping processing method for a video image. As shown in FIG. 5, the method includes:

Step S501: Acquire an image in the played video, and add a texture to the image.

For the embodiment, the texture may be a two-dimensional model map based on AR augmented reality, or may be a three-dimensional model map based on AR augmented reality. The AR (Augmented Reality) is an augmented reality technology that combines and displays real world information and virtual world information. Through augmented reality technology, an image based on AR augmented reality is added to the image in the playing video, and the real world image and the computer virtual image in the playing video can be superimposed on the same screen to realize information integration of the real world and the virtual world. Interaction.

For the embodiment, the texture may be a material such as a fireworks, a love, a snowflake, or the like used as a background-like texture, or may be a material such as a corner, a beard, or a glasses used as a face decoration type map, and the specific performance of the material. The form is not limited in this embodiment. Among them, the added map in the play video image can be user-defined settings, or it can be set by the video application by default. After determining the texture, the terminal performs scene recognition on the image in the played video, and then adds a texture to the corresponding position of the video image.

Step S502: Acquire a music signal of music to be played in the played video, and detect a beat point of the music according to the music signal.

Step S503: When it is determined that the music is played in the playing video, a beat point appears at the current playing position.

For the present embodiment, each beat point detected in step S502 corresponds to a different play position of the music. When the music is played in the live video, if the current play position of the music corresponds to the beat point detected in step S502, then it is determined. Whether the beat point is a strong beat point or a weak beat point.

Step S504: Acquire a texture effect corresponding to the beat point.

For the embodiment, the texture effect refers to a texture state when the texture corresponds to different shape feature parameters. The material in the map may be set with different shape feature parameters, and the shape feature parameters include a state parameter, a size parameter, a color parameter, and the like, which are not limited in this embodiment. In addition, the beat point in a piece of music generally includes a strong beat point and a weak beat point, and the material in the texture effect corresponding to the strong beat point and the weak beat point is the same, but the shape characteristic parameters of the material are different. For example, the texture selected by the user is a three-dimensional model of the bear, and different shape feature parameters are set for the strong beat point and the weak beat point for the texture. When the beat point is strong, the three-dimensional model of the bear dances in the video image has a large dance frequency. The dance frequency of dancing is lower at weak beats.

Step S505: Processing a texture in the image according to the texture effect to obtain a video image including the texture effect.

For the embodiment, the material in which the shape parameter is set in the texture effect is obtained, and the material is combined with the image in the play video in a layer superposition manner to obtain the texture effect. Video image of the material in it. In other embodiments, the video image including the texture effect may be obtained by integrating data of the texture effect with the image or modifying the image according to the shape characteristic parameter of the material in the texture effect.

For example, in the field of video live broadcast, data of the texture effect can be integrated with the live video image data to obtain a video image data packet, and the data packet is sent to the client, so that the client displays the video containing the texture effect. image.

For example, by acquiring the shape feature parameter of the material in the texture effect, the image may be scaled according to the shape feature parameter to obtain a video image that can achieve the effect of the special effect.

The method for mapping a video image provided by the present invention adds a texture to a video image, performs beat point detection on the music to be played in the played video, and displays the video image according to the detected beat point during the music playing process. The texture effect corresponding to the beat point is displayed, so that the texture effect displayed on the video image is closely related to the beat point of the selected video playing music, thereby improving the music played in the playing video and the displayed texture in the auditory and visual The appeal of the user; and the user can express their own personality through custom-set music, texture and texture effects and can edit and obtain personalized play video in real time. This method satisfies the user's personalized play video design needs and increases the video. The interest of the app and the interactivity of the app and the user significantly increase the satisfaction of the user experience.

In addition, the method for mapping a video image provided by the present invention, combined with a plurality of beat point detection methods, can quickly and accurately detect the beat point of the selected video playing music, further ensuring the texture effect and selected on the video image. The beat of the video playback music is closely related to further improve the satisfaction of the user experience.

Another possible implementation manner of the embodiment of the present invention further includes the following steps, where

After the step S502, the method further includes: recording a correspondence between the music playing position and the beat point.

The step S503 includes: determining, according to the correspondence, that a beat point occurs at a current play position when playing music in the play video.

The step S501 includes:

Acquiring an image in the played video, identifying a face region in the image, and adding a map to the face region.

For the embodiment, the texture is mainly a face decoration type map, such as a corner, a beard, a glasses, and the like, and the terminal performs face recognition on the image in the played video, thereby adding a map to the corresponding position of the face in the video image. For example, the face area selected in the play video is added with virtual ear, beard and other textures, and the texture will move along with the user's face to realize the combination and interaction of virtual and reality.

For this embodiment, the user can express his own personality by custom setting texture and texture effect and can edit and play the video of the personalized image design in real time, satisfying the user's personalized image design requirement and increasing the interest of the video application. And the interaction of the application with the user, significantly improving the satisfaction of the user experience.

After the step S501, the method further includes:

Confirm that the texture switching instruction is received; switch the map added in the image.

For the present embodiment, when the user is dissatisfied with the texture added in the video image, a texture switching instruction can be issued to the terminal to switch the texture to the texture material that he likes. The manner of the indication map switching may be a voice indication, an action indication, and an input indication, which is not limited in this embodiment.

For example, when the switching of the texture is indicated by voice, the terminal first acquires the sound in the played video, and recognizes the audio feature of the sound, and then switches the audio feature to the preset texture, such as “switching”, The audio features of the passwords such as "change" are matched. If the matching is consistent, it is confirmed that the texture switching instruction is received and the texture is switched.

Further, for example, when the switching of the map is instructed by the operation, the terminal first recognizes the person region in the image, and detects the motion of the person in the person region, and if a preset texture switching operation is detected in the person region, If the action of "hand waving" or "shaking the head" is performed, it is confirmed that the texture switching instruction is received and the texture is switched.

For another example, when inputting to indicate the switching of the texture, the user can switch the current texture by simply re-entering another image in the video application.

For this embodiment, the user can switch the texture according to his or her own wishes conveniently and quickly, further increasing the interest of the video application and the interaction between the application and the user, and significantly improving the satisfaction of the user experience.

Before the step S501, the method further includes:

Determining whether the anchor of the live broadcast end performs a preset increase map request action;

If yes, acquiring a map of the anchor preset; performing the step of acquiring an image in the played video, and adding a map to the image.

For the embodiment, the anchor can obtain the live video of the video including the map by performing a preset add map request during the live broadcast of the video. The manner in which the request is added to the map may be a voice request, an action request, and an input request, and is not limited in this embodiment.

For example, when requesting to add a texture by voice, the terminal first acquires the sound in the played video, and recognizes the audio feature of the sound, and then adds the audio feature to the preset added texture password, such as “map”, “ The audio features of the passwords are matched, and if the matches are consistent, the texture is added to the image of the played video.

For another example, when an action is requested to add a map, the terminal first recognizes a person region in the image, and detects a person motion in the person region, and if a preset texture switching action is detected in the person region, such as Adding a map to the image of the played video, such as "being heart" and "flying kiss".

For another example, when an input is requested to add a texture, the user simply clicks on a texture in the video application to add the texture to the image of the played video.

In other embodiments, the texture may also be selected by a viewer in the live video room, or the anchor may select the hottest texture according to the wishes of the viewer in the live video broadcast. The method can meet the requirements of the live broadcast, highlight the live broadcast atmosphere, and better promote the interaction between the anchor and the audience, and further improve the satisfaction of the user experience.

The step S502 includes:

Obtain the type of texture effect required for the video, judge whether to detect the strong beat point according to the type of texture effect required by the video, or detect the strong beat point and the weak beat point; if the strong beat point is detected, it is judged whether the intensity value detection or the change value detection is used, including If the strong beat point is detected, the type of the music is obtained, and the intensity value detection or the change value detection is used according to the type judgment; if the strong beat point and the weak beat point are detected, whether the intensity value detection or the change value detection is used, including: If a strong beat point and a weak beat point are detected, the type of the music is acquired, and whether the intensity value detection or the change value detection is used is determined according to the type.

For the present embodiment, the type of the desired beat point can be determined by the type of texture effect required for the acquired video. The type of texture effect required for the video is the default effect type selected by the user or the video application. For example, the user wants to have an endless layer of texture effects in the playback video, so it is judged that the strong beat point is detected and the weak beat point is detected according to the type of texture effect required for the video.

For the embodiment, the type of the desired beat point and the method of detecting the beat point can be selected according to the type of the texture effect and the type of the music required by the video, so as to obtain an accurate beat point by using an appropriate method, and the operation can be reduced. The amount, shorten the detection time, further ensure the close relationship between the texture effect displayed on the video image and the music beat point, and further improve the satisfaction of the user experience.

In addition, an embodiment of the present invention provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the program is executed by the processor, the video image processing method according to the first embodiment is implemented, or an embodiment The video image processing method according to the second embodiment, or the video image processing method according to the third embodiment, or the music gift processing method in the live video broadcast according to the fourth embodiment, or the texture processing method of the video image according to the fifth embodiment . The computer readable storage medium includes, but is not limited to, any type of disk (including a floppy disk, a hard disk, an optical disk, a CD-ROM, and a magneto-optical disk), a ROM (Read-Only Memory), and a RAM (Random Access). Memory, RAM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or Light card. That is, the storage device includes any medium that stores or transmits information in a readable form by a device (eg, a computer, a mobile phone), and may be a read only memory, a magnetic disk, an optical disk, or the like.

A computer readable storage medium provided by the embodiment of the present invention can perform beat point detection on music to be played in a play video, and display the video image in the music image according to the detected beat point. The effect corresponding to the beat point, so that the special effect displayed on the video image is closely related to the beat point of the selected video playing music, thereby increasing the visual and visual appeal of the music played in the video, and increasing the video application. Fun and increase the satisfaction of the user experience. In addition, the computer readable storage medium provided by the present invention can also implement a combination of multiple beat point detection methods to quickly and accurately detect the beat point of the selected video playing music; and can select the special effect type and music according to the video. Type to select the type of beat point to be detected and the method of detecting the beat point, so as to achieve an accurate method to obtain accurate beat points, and reduce the amount of calculation, shorten the detection time, and further ensure the special effects and images displayed on the video image. The beat points of the selected video playing music are closely related to further improve the satisfaction of the user experience. In addition, when the computer readable storage medium provided by the present invention is applied to the field of live video, it is also possible to select a special effect according to the anchor intention, the type of music, and the will of the viewer, thereby meeting the requirements of the live broadcast, highlighting the live broadcast atmosphere, and better promoting the anchor. Interact with the audience to further improve the satisfaction of the user experience.

The computer-readable storage medium provided by the embodiment of the present invention may implement the foregoing method embodiments. For the specific function implementation, refer to the description in the method embodiment, and details are not described herein again.

In addition, the embodiment of the present invention further provides a terminal. As shown in FIG. 6, the terminal may include one or more processors 601, and further includes a memory 602, a WiFi (Wireless Fidelity) circuit 603, RF (Radio Frequency) circuit 604, audio circuit 605, sensor 606, output device 607, input device 604, power supply 609, and processor 601 are control centers of the terminals, and the above components are connected by various interfaces and lines. It will be understood by those skilled in the art that the terminal structure shown in FIG. 6 does not constitute a limitation to the terminal, and may include more or less components than those illustrated, or a combination of certain components, or different component arrangements.

The WiFi circuit 603 can provide the user with wireless local area network or internet access; it can include an antenna, a WiFi module, and the like. The RF circuitry 604 can transmit and receive information, or receive and transmit signals during a call; it can include an antenna, at least one amplifier, a tuner, one or more oscillators, a coupler, a duplexer, and the like. The audio circuit 605 can convert the received audio data into an electrical signal for transmission to a speaker, and can also convert the sound signal collected by the microphone into audio data for processing by the processor 601; it can set a speaker, a microphone, a headphone interface, and the like. The sensor 606 can be used to sense an external signal and send it to the processor 601 for processing; it can include a motion sensor, a light sensor, and the like. The output device 607 can be used to display various signals; the display panel can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like. Input device 604 can be used to input information such as numbers and characters; it can be a physical button, a touch panel, or the like. The power supply 609 can provide power to various portions of the terminal and is logically coupled to the processor 609 via a power management system; it can include one or more components such as a DC or AC power source, a charging system, a power status indicator, and the like. The memory 602 can be used to store software programs and modules; it can be a computer readable storage medium, specifically a hard disk, a flash memory, or the like. The processor is the control center of the terminal, and performs various functions of the terminal and processes the terminal data by running or executing software programs and/or modules stored in the memory 602, and calling data stored in the memory 602.

As an embodiment, the terminal includes: one or more processors 601, a memory 602, one or more applications, wherein the one or more applications are stored in the memory 602 and configured to be configured by the one or The plurality of processors 601 are configured to perform the video image processing method according to the first embodiment, or the video image processing method according to the second embodiment, or the video according to the third embodiment. The image processing method, or the music gift processing method in the live video broadcast described in the fourth embodiment, or the texture processing method of the video image described in the fifth embodiment.

The terminal provided by the embodiment of the invention can perform the beat point detection on the music to be played in the played video, and display the beat point corresponding to the beat point in the video image according to the detected beat point during the music playing process. Special effects, so that the special effects displayed on the video image are closely related to the beat point of the selected video playing music, thereby improving the audio and the visual appeal of the music played in the video, increasing the interest of the video application and improving User experience satisfaction. In addition, the terminal provided by the present invention can implement a combination of multiple beat point detection methods to quickly and accurately detect the beat point of the selected video playing music; and can select according to the type of special effect required by the video and the type of music. The type of beat point and the method of detecting the beat point are detected to achieve an accurate beat point by using an appropriate method, and the amount of calculation can be reduced, the detection time is shortened, and the effect displayed on the video image and the selected video play are further ensured. The beat points of the music are closely related to further improve the satisfaction of the user experience. In addition, when the terminal provided by the present invention is applied to the field of live video, it can realize the special effects according to the anchor intention, the type of music, and the will of the viewer, and can meet the requirements of the live broadcast, highlight the live broadcast atmosphere, and better promote the interaction between the anchor and the viewer. Further improve the satisfaction of the user experience.

The terminal provided by the embodiment of the present invention may implement the foregoing method embodiment. For the specific function implementation, refer to the description in the method embodiment, and details are not described herein again.

The above is only a part of the embodiments of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention.

Claims

A video image processing method, comprising the steps of:

Acquiring a music signal of music to be played in the played video, detecting a beat point of the music according to the music signal;

Determining that a beat point occurs at the current playback position when playing music in the played video;

Obtaining an effect corresponding to the beat point;

The image in the played video is processed according to the special effect to obtain a video image including the special effect.
The video image processing method according to claim 1, wherein the video is a live video;

Before the obtaining the special effect corresponding to the beat point, the method further includes:

Determining whether the special effect group is specified in the live broadcast of the live video; if yes, obtaining the specified effect group; otherwise, obtaining the scene corresponding to the live broadcast, and obtaining the preset number of the hottest server in the current live server matching the scene from the live server Special effects group;

Determining whether the music type automatic adaptation special effect function is enabled in the live broadcast; if yes, identifying the type of the music, obtaining an effect group of the type adaptation from the preset number of special effect groups; otherwise, to the client Sending a request for selecting an effect group from the preset number of effect groups;

Determining whether the selection information received from the client according to the request is received; if yes, obtaining an effect group from the preset number of effect groups according to the selection information, otherwise, from the preset number of special effect groups Randomly obtain a special effects group;

The obtaining the special effect corresponding to the beat point includes:

Obtaining an effect corresponding to a beat point from the obtained effect group;

After processing the image in the playing video according to the special effect to obtain the video image including the special effect, the method further includes:

Send the processed live video to the client.
A video image processing method, comprising the steps of:

Identifying audio features of the sound in the played video;

Downloading, from the server, a music that matches the audio feature and an effect corresponding to a beat point of the music;

Playing the music in the play video, determining that a beat point occurs at a current play position of the music;

The image in the played video is processed according to the determined effect corresponding to the beat point to obtain a video image including the special effect.
The video image processing method according to claim 3, wherein the playing the music in the play video, determining that a beat point occurs in a current play position of the music, further comprising:

Downloading, from the server, a beat point description file that records a correspondence between a music playing position of the music and a beat point;

The determining that a beat point occurs at a current play position of the music includes:

Determining that a beat point occurs at a current play position of the music according to the correspondence in the beat point description file.
The video image processing method according to claim 3, wherein the playing the music in the play video, determining that a beat point occurs in a current play position of the music, further comprising:

Acquiring a music signal of the music; detecting a beat point of the music according to the music signal; recording a correspondence between a current play position of the music and a beat point;

The determining that a beat point occurs at a current play position of the music includes:

Based on the correspondence, it is determined that a beat point occurs at a current play position of the music.
The video image processing method according to claim 3, wherein the video is a live video;

The identifying audio features of the sound in the played video includes:

Receiving a request from the client to sing a song in the live video;

Receiving a singer voice sent by the client;

Identifying an audio feature of the viewer's humming sound;

After obtaining the video image containing the special effect, the method further includes:

Send the processed video image to the client.
The video image processing method according to claim 3, wherein the video is a live video;

The identifying audio features of the sound in the played video includes:

Receiving a humming download song command sent by the anchor in the live video;

Receiving the anchor humming sound sent by the anchor;

An audio feature of the anchor humming sound is identified.
A video image processing method, comprising the steps of:

Acquiring a music signal of music to be played in the played video;

Determining whether a beat point description file for storing a correspondence relationship between a beat point of the music and a music play position is prestored; if yes, acquiring the beat point description file; if not, detecting a beat of the music according to the music signal Point, generating a beat point description file according to the detected correspondence between the beat point and the music playing position;

Determining, according to the beat point description file, that a beat point occurs at a current play position when playing music in the play video;

Obtaining an effect corresponding to the beat point; processing an image in the played video according to the special effect to obtain a video image including the special effect.
The video image processing method according to claim 8, wherein the determining whether a beat point description file for storing a correspondence relationship between a beat point of the music and a music play position is prestored; if yes, acquiring the beat point Description file; includes:

Determining whether a beat point description file is pre-stored in the local file, and if so, obtaining the beat point description file from the local file; or

Determining whether the server pre-stores a beat point description file, and if so, downloading the beat point description file from the server; or

Determining whether a beat point description file is pre-stored in the local file, and if so, obtaining the beat point description file from the local file; if not, determining whether the server pre-stores the beat point description file, and if so, downloading the beat point description from the server file.
The video image processing method according to claim 8, wherein after the acquiring the beat point description file, the method further comprises:

If the beat point description file is a beat point description file downloaded from the server, detecting a beat point of the music according to the music signal, and correcting the beat point description file according to the detected beat point, in the local file Save the beat point description file after proofreading.
The video image processing method according to claim 8, wherein the detecting a beat point of the music according to the music signal, and generating a beat point description according to the detected correspondence relationship between the beat point and the music play position After the file, it also includes:

Upload the beat point description file to the server.
The video image processing method according to claim 8, wherein the video is a live video;

Before the obtaining the special effect corresponding to the beat point, the method further includes:

Obtaining the special effect information corresponding to the beat point of the music set by the anchor of the live end, saving the correspondence between the beat point and the music play position and the special effect information into the beat point description file; generating a beat point description file containing the special effect information;

The obtaining the special effect corresponding to the beat point includes:

Obtaining an effect corresponding to the beat point according to the beat point description file containing the special effect information.
The video image processing method according to claim 12, wherein after the generating a beat point description file containing the special effect information, the method further comprises:

Sending the music, the beat point description file containing the special effect information, and the special effect corresponding to the beat point to another live end of the continuous wheat, so that another live broadcast end displays the same as the live broadcast end when playing the music Special effects.
The video image processing method according to claim 1, 3 or 8, wherein the processing of the image in the played video according to the special effect to obtain the video image including the special effect comprises:

Obtaining the material in the special effect, synthesizing the material with the image in the playing video in a layer superposition manner, and obtaining a video image including the special effect.
A music gift processing method in a live video broadcast, characterized in that it comprises:

Receiving a music gift sent by a viewer in a live video; the music gift includes music to be played in a live video;

Acquiring a music signal of the music, and detecting a beat point of the music according to the music signal;

Determining that when playing the music in the music gift in the live video, a beat point appears at the current play position;

Obtaining an effect corresponding to the beat point;

The image in the live video is processed according to the special effect, and a live video of the video including the special effect is obtained.
The music gift processing method according to claim 15, wherein the music gift further includes special effect information corresponding to a beat point of the music;

The obtaining the special effect corresponding to the beat point includes:

Obtaining an effect corresponding to the beat point from the music gift according to the special effect information.
The music gift processing method according to claim 15 or 16, wherein after the image in the live video is processed according to the special effect to obtain the live video image of the special effect, the method further includes:

Send the processed live video image to the client.
The music gift processing method according to claim 16, wherein the acquiring the music signal of the music, after detecting the beat point of the music according to the music signal, further comprising:

Generating a beat point description file according to the detected correspondence between the beat point and the music play position; and determining to play the music in the music gift in the live video broadcast, the beat point appearing at the current play position includes:

According to the beat point description file, when it is determined that the music in the music gift is played in the live video, a beat point appears at the current play position.
The music gift processing method according to claim 18, wherein the generating a beat point description file according to the corresponding relationship between the detected beat point and the music play position, further comprising:

Obtaining the special effect information, saving a correspondence relationship between the beat point and the music playing position and the special effect information to the beat point description file; generating a beat point description file containing the special effect information;

And obtaining, according to the special effect information, the special effects corresponding to the beat point from the music gift, including:

Obtaining an effect corresponding to the beat point from the music gift according to the beat point description file containing the special effect information.
The music gift processing method according to claim 15 or 16, wherein after receiving the music gift sent by the viewer in the live video broadcast, the method further comprises:

Determining whether the anchor of the live broadcast end performs a preset action of opening the music gift;

If so, continuing the step of acquiring the music signal of the music, detecting a beat point of the music according to the music signal; and feeding back to the viewer who sent the music gift the information that the anchor opens the music gift.
The music gift processing method according to claim 15 or 16, wherein the processing the image in the live video according to the special effect to obtain the live video image including the special effect comprises:

The material in the special effect is obtained, and the material is combined with the image in the live video broadcast in a layer superposition manner to obtain a live video image including the special effect.
A texture processing method for a video image, comprising the steps of:

Obtaining an image in a play video, adding a texture to the image;

Acquiring a music signal of music to be played in the played video, detecting a beat point of the music according to the music signal;

Determining that a beat point occurs at the current playback position when playing music in the played video;

Obtaining a texture effect corresponding to the beat point;

The texture in the image is processed according to the texture effect to obtain a video image including the texture effect.
The map processing method according to claim 22, wherein the map comprises: a two-dimensional model map and/or a three-dimensional model map based on AR augmented reality.
The map processing method according to claim 22, wherein the acquiring an image in the played video and adding a texture to the image comprises:

Acquiring an image in the played video, identifying a face region in the image, and adding a map to the face region.
The map processing method according to claim 22, wherein the acquiring the image in the play video, adding the map to the image further comprises:

Confirm that the texture switching instruction is received; switch the map added in the image.
The map processing method according to claim 25, wherein the confirming receipt of the map switching instruction comprises:

Obtaining a sound in the played video, identifying an audio feature of the sound; if the audio feature matches an audio feature of a preset texture switching password, confirming receipt of the texture switching instruction; or

The person area in the image is identified, and if a preset texture switching operation is detected in the person area, it is confirmed that the texture switching instruction is received.
The mapping processing method according to claim 22, wherein the video is a live video;

The obtaining the image in the played video, before adding the map to the image, further includes:

Determining whether the anchor of the live broadcast end performs a preset increase map request action;

If yes, acquiring a map of the anchor preset; performing the step of acquiring an image in the played video, and adding a map to the image.
The video image processing method according to claim 1, 5 or 8, or the music gift processing method according to claim 15 or 16, or the texture processing method according to claim 22, wherein the beat Points include strong beat points and weak beat points;

The detecting a beat point of the music according to the music signal includes:

Filtering the music signal, filtering and performing short-time Fourier transform to obtain a spectrum;

Determining an energy change value of the detection point according to the spectrum;

According to the energy change value, it is detected that a strong beat point or a weak beat point appears at the detection point.
The video image processing method according to claim 1, 5 or 8, or the music gift processing method according to claim 15 or 16, or the texture processing method according to claim 22, wherein the beat Points include strong beat points and weak beat points;

The detecting a beat point of the music according to the music signal includes:

Performing weighting processing on the music signal to obtain a weighted music signal;

According to the energy intensity value of the weighted music signal, it is detected that a strong beat point or a weak beat point occurs at the detection point.
The video image processing method according to claim 1, 5 or 8, or the music gift processing method according to claim 15 or 16, or the texture processing method according to claim 22, wherein the beat Points include strong beat points;

The detecting a beat point of the music according to the music signal includes:

Obtaining a candidate beat point according to the energy intensity value of the music signal;

Calculating a time interval between frames of each adjacent two candidate beat points according to each candidate beat point;

According to the time interval, it is detected that a candidate beat point corresponds to a strong beat point corresponding to the detected point.
The video image processing method according to claim 1, 5 or 8, or the music gift processing method according to claim 15 or 16, or the texture processing method according to claim 22, wherein the beat Points include strong beat points;

The detecting a beat point of the music according to the music signal includes:

Obtaining a beat point according to a difference in energy of the music signal of the detected point;

Obtaining two pieces of music signals according to the candidate beat points, using each adjacent two candidate beat points as a signal starting point;

According to the comparison result of the two pieces of music signals, it is detected that the candidate beat points have strong beat points corresponding to the detected points.
The video image processing method according to claim 1, 5 or 8, or the music gift processing method according to claim 15 or 16, or the texture processing method according to claim 22, wherein The music signal detects the beat point of the music, including:

Obtaining the music signal, determining whether to detect a strong beat point, or detecting a strong beat point and a weak beat point;

If the strong beat point is detected, it is judged whether the intensity value detection or the change value detection is used;

If the intensity value detection is used, the candidate beat point is obtained according to the energy intensity value of the music signal, and the time interval between frames of each adjacent two candidate beat points is counted according to each candidate beat point, and the detection is performed according to the time interval. A strong beat point appears at the detection point corresponding to the candidate beat point;

If the change value detection is used, the candidate beat point is obtained according to the energy variation difference of the music signal of the detection point, and according to the candidate beat point, two adjacent music beat signals are taken as the signal starting point of each adjacent two candidate beat points. According to the comparison result of the two pieces of music signals, it is detected that a candidate beat point corresponds to a strong beat point corresponding to the detected point;

If the strong beat point and the weak beat point are detected, it is judged whether the intensity value detection or the change value detection is used;

If the intensity value detection is used, the music signal is weighted to obtain a weighted music signal, and according to the energy intensity value of the weighted music signal, a strong beat point or a weak beat point is detected at the detection point;

If the change value detection is used, the music signal is filtered, and after filtering, the short-time Fourier transform is performed to obtain a spectrum, and according to the spectrum, the energy change value of the detection point is determined, and according to the energy change value, the weak beat of the detection point is detected. Point or strong candidate beat point;

The determination detects a strong beat point, or detects a strong beat point and a weak beat point, including:

Get the type of texture effect required for the video, judge whether to detect strong beat points according to the type of texture effect required by the video, or detect strong beat points and weak beat points;

If the strong beat point is detected, it is determined whether the intensity value detection or the change value detection is used, including:

If a strong beat point is detected, the type of the music is acquired, and whether the intensity value detection or the change value detection is used according to the type judgment;

If the strong beat point and the weak beat point are detected, it is determined whether the intensity value detection or the change value detection is used, including:

If a strong beat point and a weak beat point are detected, the type of the music is acquired, and whether the intensity value detection or the change value detection is used is determined according to the type.
The video image processing method according to claim 1, or the texture processing method according to claim 22, wherein the detecting the beat point of the music according to the music signal further comprises: recording a location Corresponding relationship between the playing position of the music and the beat point;

Determining that when playing music in the playing video, a beat point appears at the current playing position, including:

According to the correspondence, it is determined that when playing music in the playing video, a beat point appears at the current playing position.
A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which is implemented by a processor to:

A video image processing method according to any one of claims 1-14, 28 to 33;

Or the music gift processing method according to any one of claims 15-21, 28 to 32;

Or the texture processing method according to any one of claims 22 to 33.
A terminal, comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be Executed by one or more processors, the one or more programs configured to:

Performing a video image processing method according to any one of claims 1-14, 28 to 33;

Or the music gift processing method according to any one of claims 15-21, 28 to 32;

Or the texture processing method according to any one of claims 22 to 33.