TW202303526A - Special effect display method, computer equipment and computer-readable storage medium - Google Patents

Abstract

The present disclosure provides a special effect display method, a computer equipment, and a computer-readable storage medium, wherein the method includes: acquiring a first video image of a real anchor during a live broadcast; performing posture detection on the designated limb parts of the real anchor in the first video image to obtain a posture detection result; when it is detected that the real anchor is in a preset posture according to the posture detection result, the target animation special effect of the virtual anchor model corresponding to the real anchor is determined according to the posture detection result; the target animation special effect of the virtual anchor model is displayed in the live broadcast video interface corresponding to the real anchor.

Description

Special effect display method, computer equipment, and computer-readable storage medium

The present invention relates to the technical field of computers, in particular to a method for displaying special effects, computer equipment and computer-readable storage media.

In the current virtual live broadcast process, the anchor can trigger the display of special effect animations by performing trigger operations on the special effect trigger buttons on the live broadcast device. For example, the anchor can manually manipulate the mouse or keyboard to trigger the display of special effects animation; or, the anchor can also click or press the pre-edited shortcut keys in the live broadcast software to trigger and play the special effect animation. Because the related virtual live broadcast solution requires the host to manually trigger the display of special effects animation, which takes up the hand performance of the host during the live broadcast, reduces the interaction efficiency between the host user and the viewer through hand movements, and leads to the use of the live broadcast software by the host user. Bad experience.

Embodiments of the present invention at least provide a method for displaying special effects, computer equipment, and a computer-readable storage medium.

In the first aspect, an embodiment of the present invention provides a method for displaying special effects, including: acquiring a first video image of a real anchor during a live broadcast; Detecting to obtain a posture detection result; in the case of detecting that the real anchor is in a preset posture according to the posture detection result, determining the target animation special effect of the virtual anchor model corresponding to the real anchor according to the posture detection result; The target animation special effect of the virtual anchor model is displayed in the live video interface corresponding to the real anchor.

The embodiment of the present invention is applicable to the field of virtual live broadcast, and can display the virtual anchor model driven by the real anchor in the live video interface, and can display the animation special effects of the virtual anchor model in the live video interface. That is to say, the target animation effect of the virtual anchor model driven by the real anchor can be determined based on the posture detection result by recognizing the pose of the real anchor, and the target animation effect can be displayed in the live video interface. In this way, the target animation effects corresponding to the virtual anchor model can be triggered and displayed on the live video interface through the posture detection results of the real anchor, without relying on external control devices to trigger the display of animation effects, and at the same time, the live broadcast experience of virtual live broadcast users is improved.

In the second aspect, the embodiment of the present invention also provides a special effect display device, including: an acquisition unit, used to acquire the first video image of the real anchor during the live broadcast; a posture detection unit, used to perform the first video image The specified body parts of the real anchor are subjected to posture detection to obtain a posture detection result; the determining unit is configured to, when it is detected that the real anchor is in a preset posture according to the posture detection result, according to the posture detection result Determining the target animation special effect of the virtual anchor model corresponding to the real anchor; a display unit configured to display the target animation special effect of the virtual anchor model in the live video interface corresponding to the real anchor.

In the third aspect, the embodiment of the present invention also provides a computer device, including: a processor, a memory, and a bus bar. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any possible implementation manner of the first aspect are executed.

In the fourth aspect, the embodiment of the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program executes the above-mentioned first aspect or any one of the first aspects when the computer program is run by a processor. steps in a possible implementation.

In the fifth aspect, the embodiment of the present invention also provides a computer program, including computer readable code, when the computer readable code is run in the electronic device, the processor in the computer device executes to realize the above first aspect , or a step in any possible implementation manner in the first aspect.

In a sixth aspect, the present invention provides a computer program product, including computer program instructions. When the computer program instructions are executed by a computer, the above-mentioned first aspect, or the steps in any possible implementation manner of the first aspect are realized.

In order to make the above-mentioned objects, features and advantages of the present invention more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this article only describes an association relationship, which means that there may be three relationships, for example, A and/or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one or any combination of at least two of the plurality, for example, including at least one of A, B, and C, may mean including the composition consisting of A, B, and C Any one or more elements selected in the collection.

After research, it was found that because the related virtual live broadcast solution requires the anchor to manually trigger the display of special effects animation, this live broadcast solution will occupy the hand performance of the anchor during the live broadcast, thereby reducing the interaction efficiency between the anchor user and the viewer through hand movements , thereby reducing the user experience of the live broadcast software.

Based on the above research, the present invention provides a special effect display method, device, computer equipment, storage medium, computer program and computer program product. The technical solution provided by the present invention can be applied in a virtual live broadcast scene. The virtual live broadcast scene can be understood as the use of pre-set virtual anchor models, such as red pandas, little rabbits, cartoon characters, etc. to replace the actual image of the real anchor for live broadcast. At this time, the above-mentioned virtual anchor models are shown in the live video screen . At the same time, the interaction between the real anchor and the audience can also be carried out according to the virtual anchor model.

For example, the camera device of the live broadcast device can collect video images containing the real anchor, and then capture the body of the real anchor contained in the video image, so as to obtain the posture information of the real anchor. After the posture information is determined, a corresponding driving signal can be generated, and the driving signal is used to drive the live broadcast device to display the animation special effect corresponding to the virtual anchor model in the live video screen.

In an optional embodiment, the real anchor may preset a corresponding virtual anchor model, for example, the preset virtual anchor model may be "YYY role model in XXX game". A real anchor can preset one or more virtual anchor models. When starting the virtual live broadcast at the current moment, one can be selected from one or more preset virtual anchor models as the virtual anchor model at the current moment. Wherein, the virtual anchor model may be a 2D model or a 3D model.

In another optional implementation, in addition to the method described above, that is, the real anchor determines the virtual anchor model, after the first video image is acquired, the real anchor model in the first video image can be Reshape the virtual anchor model.

In some embodiments of the present invention, the live broadcast device can identify the real anchor contained in the video image, and reshape the virtual anchor model for the real anchor according to the recognition result. Here, the recognition result may include at least one of the following: the gender of the real anchor, the appearance characteristics of the real anchor, the wearing characteristics of the real anchor, and the like. At this time, the live broadcast device may search for a model matching the recognition result from the virtual anchor model database as the virtual anchor model of the real anchor.

Exemplarily, when the live broadcast device determines according to the recognition result that the peaked cap and clothes worn by the real anchor during the live broadcast are hip-hop-style clothes, it can search from the virtual anchor model library, and compare the searched ones with the The virtual anchor model that matches the "peaked cap" or "hip-hop style" is used as the virtual anchor model of the real anchor.

In some embodiments of the present invention, in addition to searching for a model that matches the recognition result in the virtual anchor model library, the live broadcast device can also build a corresponding model for the real anchor in real time based on the recognition result. Virtual anchor model.

Here, when constructing the virtual anchor model in real time, the virtual anchor model used in the virtual live broadcast initiated by the real anchor in the past can also be used as a reference to construct the virtual anchor model driven by the real anchor at the current moment.

Through the method of determining the virtual anchor model described above, it is possible to customize corresponding virtual anchor models for real anchors, thereby increasing the diversity of virtual anchor models. At the same time, by customizing the virtual anchor model, it can also leave a deeper impression on the audience.

In order to facilitate the understanding of this embodiment, here, a special effect display method disclosed in the embodiment of the present invention is introduced in detail. The execution subject of the special effect display method provided in the embodiment of the present invention is generally a computer device with a certain computing power, for example: The method for displaying special effects may be executed by a terminal device, or a server or other processing device, wherein the terminal device may be a user device, a mobile device, a user terminal, a terminal, a cellular phone, a personal digital assistant, a handheld device, a computing device, a vehicle-mounted devices, wearables, etc. The computer device can be any live broadcast device that supports the installation of virtual live broadcast software. In some possible implementation manners, the method for displaying special effects may be realized by calling a computer-readable instruction stored in a memory by a processor.

Referring to FIG. 1 , it is a flow chart of a method for displaying special effects provided by an embodiment of the present invention. The method includes steps S101 to S107 as follows.

S101: Obtain a first video image of a real anchor during a live broadcast.

S103: Perform posture detection on the designated body parts of the real anchor in the first video image, and obtain a posture detection result.

In the embodiment of the present invention, the live broadcast device can capture the video stream of the real host during the live broadcast through the camera device pre-installed on the live broadcast device, and the first video image is a video frame in the video stream.

Here, the video images of the collected video stream may include the real host's face and upper body parts. In some embodiments, the video image may also include part or all of the hand images. In the actual live broadcast scene, when the real anchor leaves the shooting range of the camera device, or the live broadcast scene of the real anchor is more complicated, the video image often contains incomplete faces and/or incomplete upper body parts .

In the embodiment of the present invention, the designated body parts may be at least part of the designated body parts of the real anchor. Here, the specified body part includes: a head part and an upper body part (two arm parts, a hand part, and an upper body torso part).

It can be understood that, in the case where there are multiple specified body parts, the above posture detection results can be used to characterize at least one of the following: the relative positional relationship between each specified body part, the position of the gesture contained in the first video image Gesture classification results.

S105: If it is detected according to the posture detection result that the real anchor is in a preset posture, determine a target animation special effect of a virtual anchor model corresponding to the real anchor according to the posture detection result.

S107: Display the target animation special effect of the virtual anchor model in the live video interface corresponding to the real anchor.

In the embodiment of the present invention, after determining the posture detection result, the live broadcast device may detect whether the real host is in a preset posture according to the posture detection result. If the real anchor is in the preset posture, it is determined that the real anchor in the first video image satisfies the triggering condition of the animation special effect. At this time, the live broadcast device may determine the target animation special effect matching the first video image, and display the target animation special effect.

The embodiment of the present invention can be applied to the field of virtual live broadcast. The live broadcast device can display the virtual anchor model driven by the real anchor in the live video interface, and can display the animation special effects of the virtual anchor model in the live video interface. Here, the live broadcast device can determine the target animation effect of the virtual anchor model driven by the real anchor based on the posture detection result by recognizing the pose of the real anchor, and display the target animation special effect in the live video interface. In this way, the gesture detection results of the real anchor can be used to trigger the display of the target animation effects corresponding to the virtual anchor model on the live video interface. In this way, there is no need to rely on external control devices to trigger the display of animation effects, and it also improves the virtual live broadcast user experience. Live experience.

In some embodiments of the present invention, for the above step S103, if the posture detection result includes at least one of the body detection result and the gesture classification result, perform a gesture on the specified body part of the real anchor in the first video image Detection, the realization of obtaining the attitude detection result, may include the following process: Step S1031, perform limb detection on the specified limb parts of the real anchor in the first video image, and obtain a limb detection result. Step S1032 , if the body detection result includes a hand detection frame, perform gesture detection on the image within the hand detection frame in the first video image to obtain a gesture classification result. Step S1033, determining the posture detection result according to the body detection result and the gesture classification result.

In the embodiment of the present invention, the live broadcast device may use the body detection model to perform body detection on the specified body parts of the real anchor in the first video image, and obtain body detection results. Here, the body detection result includes at least one of the following: body key points, size of the face frame, position information of the face frame, size of the hand detection frame, and position information of the hand detection frame.

As shown in FIG. 2 , the live broadcast device can determine the body key points of the specified body parts of the real anchor in the first video image through the body detection model. If it is recognized that the first video image contains a clear facial image, at least one of the size of the face frame and the location information of the face frame is obtained. Then, if it is recognized that the first video image contains a clear hand image, at least one of the size of the hand detection frame and the position information of the hand detection frame can be obtained.

In the embodiment of the present invention, when the body detection result includes the hand detection frame, the live broadcast device can also use the gesture recognition model to perform gesture detection on the image located in the hand detection frame to obtain the gesture classification result, Wherein, the gesture classification result is a feature vector, and the feature vector is used to represent the probability that the hand gesture of the real anchor in the first video image belongs to each preset gesture.

In the embodiment of the present invention, after the above body detection result and gesture classification result are obtained, the live broadcast device may determine the body detection result and gesture classification result as the gesture detection result. Then, judge whether the real anchor in the first video image satisfies the triggering condition of the animation special effect according to the body detection result and the gesture classification result. If the triggering condition of the animation special effect is satisfied, the target animation special effect of the virtual anchor model corresponding to the real anchor is determined according to the posture detection result.

In the embodiment of the present invention, if the body detection result does not contain the hand detection frame, the live broadcast device may discard the first video image, and use the next video frame in the video stream as the first video image, The first video image is processed through the steps described above, and the processing process will not be described in detail here.

Here, the body detection result includes body key points, for example, body key points of each specified body part. If the specified limb part includes a head part and an upper body part, the body keys include keys for the head part, as well as keys for both arms in the upper body part, keys for the hands, and keys for the upper torso.

In the above embodiment, by performing limb detection and gesture detection on the first video image, and integrating the limb detection result and gesture classification result, the semantic information used to accurately represent the action semantic information of the real anchor in the first video image can be obtained. Posture detection results. When the target animation special effect is determined based on the posture detection result, the accuracy of the target animation special effect that is triggered to be displayed can be improved.

In an optional implementation manner, in the above step S105, the implementation of detecting that the real anchor is in a preset posture according to the posture detection result may include the following steps: Step S11, according to the body detection result in the posture detection result, judge whether the real anchor in the first video image satisfies the gesture recognition condition, and obtain the judgment result; Step S12, when the judgment result indicates that the real host satisfies the gesture recognition condition, detect whether the gesture indicated by the gesture classification result in the gesture detection result is a preset gesture; Step S13, if it is detected that the gesture indicated by the gesture classification result is the preset gesture, determine that the real anchor is in the preset gesture.

In the embodiment of the present invention, after the live broadcast device determines the body detection result described above, it can judge whether the real host in the first video image meets the gesture recognition condition according to the body detection result.

In the embodiment of the present invention, the live broadcast device can determine the relative positional relationship between the specified limb parts according to the body detection result; according to the relative positional relationship, determine whether the real anchor in the first video image meets the gesture recognition condition.

It can be understood that the above relative positional relationship includes at least one of the following: a relative distance between each designated body part, and an angular relationship between associated body parts in each designated body part. Wherein, the associated body parts may be adjacent specified body parts, or specified body parts of the same type.

In the embodiment of the present invention, the fact that the real anchor in the first video image satisfies the gesture recognition condition may be understood as: the body movements of the real anchor in the first video image are preset body movements.

Therefore, when detecting that the body movement of the real anchor in the first video image is a preset body movement, the live broadcast device may detect whether the gesture made by the real anchor in the first video image is a preset gesture.

In the embodiment of the present invention, when the live broadcast device detects that the gesture made by the real anchor in the first video image is a preset gesture, it can determine that the real anchor is in the preset gesture, and at this time, it can be determined that The real anchor in the first video image satisfies the triggering condition of the animation special effect, and then executes the step of determining the target animation special effect of the virtual live broadcast model corresponding to the real anchor according to the posture detection result.

In the embodiment of the present invention, the live broadcast device can determine the target animation special effect of the virtual anchor model through the combination of the body detection result and the gesture classification result. In this case, not only the gesture of the real anchor is required to be a preset gesture, but also the body movement of the real anchor when making the preset gesture is required to be a preset body movement. In this way, the method of first judging whether the real object in the first video image meets the gesture recognition condition based on the body detection result can improve the efficiency of posture comparison and shorten the time of posture comparison, so that this technical solution can be applied to real-time Highly demanding live broadcast scenarios.

In an optional implementation manner, the implementation of the above step S11, determining whether the real anchor in the first video image satisfies the gesture recognition condition according to the limb detection result in the posture detection result, may include the following process: (1) Determine the relative orientation information between the designated body parts of the real anchor according to the body detection results; (2) According to the relative orientation information and preset orientation information, it is judged whether the real anchor in the first video image satisfies the gesture recognition condition, and the judgment result is obtained; the preset orientation information is used for To characterize the relative orientation relationship between the specified body parts of the real anchor in a preset posture.

In the embodiment of the present invention, when the posture detection result includes the body detection result and the gesture classification result, the live broadcast device can first determine the relative orientation information between the specified body parts of the real anchor according to the body detection result (that is, The relative positional relationship described above), here, the relative orientation information between the specified body parts may include the following relative orientation information: the relative orientation information between the hand and the face, the relative orientation information between the arm and the arm, Relative orientation information between upper body torso and arms, relative orientation information between arms and upper body torso.

Here, the relative orientation information may include at least one of the following: relative distance, relative angle, and relative direction.

In the embodiment of the present invention, the relative distance may include: the relative distance between each specified body part. For example, the relative distance between the center point of the hand detection frame and the center point of the face detection frame is M1 primitive points; the relative distance between the elbow of the left arm of the real anchor and the elbow of the right arm is M2 primitive points; the relative distance between the center point of the upper body torso and the elbow of each arm is M3 primitive points and M4 primitive points.

In the embodiment of the present invention, the relative angle may include: an included angle between various specified body parts. For example, the angle N1 between the center point of the hand detection frame and the center point of the face detection frame and the horizontal line; the angle between the left arm and the right arm of the real anchor is N2; the upper body torso and each The angles between the arms are N3 and N4.

In the embodiment of the present invention, the relative direction may include: direction information between designated body parts. For example, the hand detection frame is at the left position (or, right position, lower position, upper position, etc.) of the face detection frame. The left arm of the real anchor is on the left side of the right arm; the upper body torso is on the right side of the left arm, and on the left side of the right arm, etc.

In the embodiment of the present invention, after the live broadcast device determines the relative orientation information, it can compare the relative orientation information with the preset orientation information to obtain the comparison result, and then judge the real anchor in the first video image according to the comparison result. Whether the gesture recognition condition is met.

In an optional implementation manner, the relative orientation information includes a plurality of first sub-information, and the default orientation information includes a plurality of second sub-information. The realization of comparing the relative orientation information with the default orientation information to obtain the comparison result may include: (a) Pairing the sub-information of the same type among the plurality of first sub-information and the plurality of second sub-information to obtain a plurality of pairs of information to be compared. Wherein, the same type can be understood as corresponding to the same type of specified body parts, and the represented physical meanings are the same. If the first sub-information is the information used to characterize the relative distance between the center point of the hand detection frame and the center point of the face detection frame in the relative orientation information, the second sub-information is also used in the default orientation information Information representing the relative distance between the center point of the hand detection frame and the center point of the face detection frame. (b) Determine the difference between the first sub-information and the second sub-information in each information pair to be compared, so as to obtain a plurality of difference values, and the comparison result includes a plurality of difference values.

In the embodiment of the present invention, after obtaining multiple difference values, the live broadcast device may judge whether the real host in the first video image meets the gesture recognition condition according to the multiple difference values.

In some embodiments of the present invention, the live broadcast device can obtain the judgment result that the real anchor in the first video image satisfies the gesture recognition condition when each difference value is smaller than the preset difference threshold; When at least one of the difference values is greater than or greater than the preset difference threshold, a judgment result is obtained that the real anchor in the first video image does not meet the gesture recognition condition.

In some embodiments of the present invention, the live broadcast device can obtain the satisfaction gesture of the real anchor in the first video image when it is determined that the number of difference values smaller than the preset difference threshold is greater than or equal to the preset number threshold As for the judgment result of the recognition condition, if it is determined that the number of the preset difference threshold is less than the preset number threshold among the plurality of difference values, the judgment result that the real anchor in the first video image does not meet the gesture recognition condition is obtained.

In some embodiments of the present invention, the live broadcast device can determine the weight value corresponding to each difference value, and then perform a weighted summation according to each difference value and the corresponding weight value to obtain a weighted summation calculation result. When the sum calculation result is less than or equal to the preset weighted threshold, it is determined that the real anchor in the first video image meets the gesture recognition condition; when the weighted sum calculation result is greater than the preset weighted threshold, it is determined that the first video image The real anchor in the image does not meet the gesture recognition requirements.

It should be noted that the preset difference threshold, the preset quantity threshold, and the preset weight threshold can be set according to actual needs, which is not limited in this embodiment of the present invention.

In the above embodiment, by comparing the relative orientation information with the preset orientation information, it is determined whether the body movement of the real anchor in the first video image is the preset body movement, so that more accurate body comparison can be obtained As a result, it can be more accurately determined whether the first video image satisfies the gesture recognition condition.

In an optional embodiment, in the above step S105: determining the realization of the target animation special effect of the virtual anchor model corresponding to the real anchor according to the posture detection result may include the following steps: Step S21, based on the posture detection result, determine the first driving information for the animation special effect; wherein, the first driving information is used to indicate the animation jump information of the animation special effect displayed in the live video interface; Step S22, according to the first driving information, determine an animation sequence matching the first driving information among multiple animation sequences corresponding to the posture detection result, and determine the matching animation sequence as The target animation effect.

In the embodiment of the present invention, the first driving information may be a 1×(P+Q) matrix, where P may be the number of preset poses, and Q may be the number of multiple animation sequences corresponding to the preset poses.

Here, the preset posture matching the posture detection result of the real anchor in the first video image can be determined among the plurality of preset postures through the first driving information. In the above 1×(P+Q) matrix, the element corresponding to the matching preset posture can be set to “1”, and the elements corresponding to other preset postures can be set to “0”, where “1” indicates that the preset posture is a posture that matches the posture detection result of the real anchor, and "0" represents that the preset posture is not a posture that matches the posture detection result of the real anchor.

In the above 1×(P+Q) matrix, the quantity of Q is associated with the “matched preset pose”, that is, Q can be understood as multiple animation sequences corresponding to the “matched preset pose”. In the above 1×(P+Q) matrix, elements corresponding to animation sequences matching the first video image may be set to “1”, and other elements may be set to “0”.

In the embodiment of the present invention, after the live broadcast device determines the first driving information, it can determine a preset posture that matches the posture detection result among multiple preset postures according to the first driving information, and In the plurality of animation sequences corresponding to the posture, determine the target animation special effect of the virtual anchor model corresponding to the real anchor.

For any preset posture, the live broadcast device can pre-set multiple stages for the preset posture, for example: an action entry stage, an action hold stage, and an action exit stage. Exemplarily, the preset posture is the posture of the real anchor showing "OK" through hand movements as shown in Figure 2, and the arm of the real anchor is raised to the position of the head and showing the "OK" action can enter the stage for the above action , the real anchor keeping the "OK" action can be the above-mentioned action holding stage, and the real anchor's arm is lowered from the head position is the above-mentioned action exit stage. It should be noted that the live broadcast device may preset a corresponding animation sequence for each stage, and identify which stage the preset gesture is in according to the preset corresponding animation sequence.

In the above embodiments, by determining the first driving information, the live broadcast device can determine the target animation special effect according to the first driving information, thereby simplifying the data format, saving the internal memory of the live broadcast device, and ensuring the smoothness of the live broadcast process.

In an optional implementation manner, step S21, the implementation of determining the first driving information for animation special effects based on the posture detection result may include the following process: (1) Determine at least one video image preceding the first video image in the video stream; (2) Acquiring second driving information for animation special effects determined according to each of the video images, and determining estimated driving information for animation special effects according to the posture detection result; (3) Determine the animation sequence driven by each of the second driving information and the estimated driving information to obtain at least one animation sequence; (4) Determine the driving information corresponding to the animation sequence whose occurrence number meets the preset number requirement in the at least one animation sequence as the first driving information.

In the embodiment of the present invention, the live broadcast device may detect the stability of the frame rate of the live broadcast device, and determine at least one video image in the video stream before the first video image according to the stability of the frame rate.

For example, for a live broadcast device with a relatively stable frame rate, a target time window can be determined. Exemplarily, if the acquisition time corresponding to the first video image is T seconds, then the target time window may be [T-1, T]. That is to say, the live broadcast device may determine the video image within the target time window in the video stream as at least one video image.

For another example, for a live broadcast device with an unstable frame rate, N frames of video images preceding the first video image in the video stream may be acquired as at least one video image.

After obtaining at least one video image, the live broadcast device can obtain second driving information, and the second driving information is determined according to each video image, so that the live broadcast device can obtain at least one second driving information, and according to The pose detection results determine estimated driving information for the animation effect. Afterwards, an animation sequence driven by each of the second driving information and the estimated driving information may be determined, so as to obtain at least one animation sequence.

After obtaining at least one animation sequence, it is possible to determine from at least one animation sequence that the number of occurrences of the animation sequence meets the requirement of the preset number of times, and then determine the driving information corresponding to the animation sequence that meets the requirement of the preset number of times as the first driving Information. Exemplarily, the requirement for the preset number of times may be the highest number of occurrences, that is, the live broadcast device may use the animation sequence with the highest number of occurrences in at least one animation sequence as the animation sequence that meets the requirement for the preset number of times.

For example, starting to collect video streams from time T0 until time T1, 30 frames of video images can be obtained. At this time, the live broadcast device can obtain the second driving information for animation special effects determined according to each video image, and then , according to the second driving information for animation special effects determined by each frame of video images in the 30 frames of video images, 30 second driving information are obtained; each of the 30 second driving information is determined to drive the display 30 animation sequences are obtained; the second driving information corresponding to the animation sequence with the highest number of occurrences among the 30 animation sequences is determined as the first driving information of each frame of video images in the 30 frames of video images.

Since the virtual live broadcast model is determined based on each video image in multiple video images, the animation special effects of the virtual live broadcast model may be different. At this time, the animation special effects displayed in the live video interface may have the problem of shaking. , the technical solution of the present invention proposes a decision stabilization algorithm based on time series, the algorithm first obtains at least one video image in the video stream before the first video image, and then determines the animation for each video image The second driving information of the special effect, and then determine the first driving information according to the second driving information. Through this processing method, the signal jitter can be reduced on the basis of ensuring a low decision-making response delay, thereby improving the accuracy of the animation sequence that triggers the corresponding action. sex.

In an optional embodiment, the above step S22: according to the first driving information, the implementation of determining the animation sequence matching the first driving information among the multiple animation sequences corresponding to the gesture detection result , which can include the following processes: (1) Acquiring the animation state machines of the plurality of animation sequences; the animation state machine is used to represent the jump relationship between the plurality of animation states, and each animation state corresponds to one or more animation sequences; (2) According to the first driving information, determine the next animation state to be jumped to by the animation state machine; (3) According to the animation sequence corresponding to the next animation state to be jumped, determine the animation sequence matching the first driving information.

In the embodiment of the present invention, the live broadcast device may pre-set corresponding animation state machines for multiple animation sequences corresponding to preset poses. In this way, after obtaining the first driving information, the next animation state to be jumped to for the animation state machine can be determined according to the content contained in the first driving information, for example, jumping from the current animation state A to the animation state B. After the next animation state to be jumped is determined, the animation sequence corresponding to the next animation state to be jumped can be determined, and then the animation sequence is determined to be an animation sequence matching the first video image.

In some embodiments, jumping from A to B may be transitioned through animation transition frames. Here, the live broadcast device can realize the playback of the Avatar image skeletal animation or special effect animation through the animation mixing algorithm based on the mixing parameters of the continuous animation clips.

In some embodiments, the animation blending algorithm may include animation algorithms such as skeletal skinning animation algorithm, 2D freeform cartesian algorithm (2D freeform cartesian) and 2D simple directional (2D simple directional) of Mecannim animation. At least one, this embodiment of the present application does not limit it.

In some embodiments, mixing parameters may include parameters such as speed and direction.

It is understandable that the animation state machine contains the transition conditions between animation states. Therefore, the animation state machine is controlled through the driving information to realize the transition of the animation state in the live video interface, so that live broadcast users can More complex body movements are used to achieve smooth transfer between various action states to improve the user's live broadcast experience.

In an optional implementation manner, in the case where the target animation special effects include at least one of body movement special effects and rendering material special effects used to characterize the body movements of the virtual anchor model, the above step S105: in the real anchor corresponding The realization of the target animation effects of the virtual anchor model displayed in the live video interface may include at least one of the following: displaying the physical effect of the physical movement of the virtual anchor model in the live video interface; displaying the rendering material special effect at a target position associated with the physical movement of the virtual anchor model in the live video interface.

In the embodiment of the present invention, the target animation special effect includes the virtual anchor's body movement special effect, for example, the movement special effect of the virtual live broadcast's body performing the "OK" action as shown in FIG. 2 . In addition, the target animation special effects also include rendering material special effects, for example, you can add "rabbit ears" special effects to the virtual anchor.

Here, the target position may be associated with the body motion special effects. For example, for the same rendered material, the display positions (that is, the target positions) in the live video interface are different in the case of different body motion special effects.

In the above embodiments, by displaying body movement special effects and rendering material special effects in the live video interface, the content displayed in the live video interface can be enriched, thereby increasing the fun of the live broadcast and improving the user's live broadcast experience.

In an optional implementation manner, the method further includes: acquiring a virtual live broadcast scene corresponding to the real anchor.

Here, the virtual live broadcast scene may be a game commentary scene, a talent show scene, an emotional expression scene, etc., which is not limited in this embodiment of the present invention.

The above step S105: according to the posture detection result, determine the realization of the target animation special effect of the virtual anchor model corresponding to the real anchor, and may also include the following steps: obtaining the initial animation special effect matching the posture detection result; An animation special effect matching the virtual live broadcast scene is determined as the target animation special effect in the initial animation special effect.

In the embodiment of the present invention, for different virtual live broadcast scenes, the live broadcast device can set different types of animation special effects. Exemplarily, for the "OK" gesture shown in Figure 2, in different virtual live broadcast scenarios, the displayed animation special effects may also be different, for example, in the game commentary scene, the "OK" gesture corresponds to The animation special effects are more in line with the action habits of the game characters.

In the embodiment of the present invention, after obtaining the initial animation special effect matching the posture detection result, the live broadcast device may determine the animation special effect matching the virtual live broadcast scene as the target animation special effect in the initial animation special effect.

For example, the default posture is the "OK" posture as shown in Figure 2, and multiple initial animation special effects can be set in advance for the "OK" posture, for example, initial animation special effects M1, initial animation special effects M2, initial animation special effects M3 . For each initial animation special effect, a corresponding scene label is set to indicate the virtual live broadcast scene to which the initial animation special effect applies.

After obtaining the virtual live broadcast scene corresponding to the real anchor, the virtual live broadcast scene can be matched with the scene label, so as to determine the animation special effect matching the virtual live broadcast scene as the target animation special effect.

In the above-mentioned embodiment, by screening the target animation special effects from the initial animation special effects in the virtual live broadcast scene, it is possible to customize personalized animation special effects for the live broadcast users, so that the determined target animation special effects can better satisfy the user's live broadcast. demand, thereby improving the user's live broadcast experience.

The method for displaying special effects described above will be introduced below in conjunction with specific implementation methods.

Exemplarily, the real anchor is recorded as anchor A, and the virtual anchor model driven by the real anchor is "Princess Rabbit". The live broadcast device selected by anchor A is a smart phone, and a camera is installed on the smart phone. The preset gesture is the gesture shown in FIG. 2 , and the preset gesture is the OK gesture as shown in FIG. 2 .

In the embodiment of the present invention, first, the live broadcast device collects the first video image of the real anchor during the live broadcast through the camera; then, performs body detection on the designated body parts of the anchor A in the first video image, and obtains the body detection result . The body detection result includes at least one of the following: body key points, size of the face frame, position information of the face frame, size of the hand detection frame, and position information of the hand detection frame.

After the body detection result is obtained, if the relative orientation information between anchor A's hand detection frame and face frame is determined according to the body detection result, it is determined that anchor A meets the gesture recognition condition. For example, as shown in FIG. 2 , according to the body detection result, it can be determined that anchor A's hand is on one side of the face and adjacent to the face. At this time, gesture detection may be performed on the image within the hand detection frame in the first video image to obtain a gesture classification result. If it is recognized that the gesture made by anchor A is "OK gesture", and it is determined that anchor A is in the preset posture as shown in FIG. 2 .

Afterwards, the live broadcast device can determine the target animation special effect of the virtual anchor model "Princess Rabbit" corresponding to anchor A according to the posture detection result, and display the target animation special effect of "Princess Rabbit" in the live video interface corresponding to anchor A.

Exemplarily, for the "OK gesture", it can include 3 animation stages: the action of anchor A raising his arm to the head position and showing the "OK" action enters the stage, and the action of anchor A maintaining the "OK" action remains stage, and the action of Anchor A's arm lowering from the head position exits the stage. For each phase, a corresponding animation sequence can be determined for it.

After the live broadcast device obtains the pose detection result, it can also determine at least one video image in the video stream before the first video image, and obtain the second driving information for animation special effects determined according to each video image , the second driving information is used to indicate the animation jump information of the animation special effect displayed in the live video interface, and at the same time, the estimated driving information for the animation special effect can be determined according to the posture detection result. Then, according to the second driving information and the estimated driving information, the animation sequence driven by each driving information can be determined; and then, the animation sequence with the largest number of occurrences can be determined among the determined animation sequences. For example, the animation sequence corresponding to the "motion hold phase" has the highest number of occurrences. Afterwards, the animation sequence corresponding to the "action holding stage" can be played on the live video screen.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same inventive concept, the embodiment of the present invention also provides a special effect display device corresponding to the special effect display method. Since the problem-solving principle of the device in the embodiment of the present invention is similar to the above-mentioned special effect display method in the embodiment of the present invention, the implementation of the device Reference can be made to the implementation of the method, and repeated descriptions will not be repeated.

Referring to Figure 3, it is a schematic diagram of a special effect display device provided by an embodiment of the present invention, the device includes: an acquisition part 41, a gesture detection part 42, a determination part 43, and a display part 44; wherein, The acquisition part 41 is configured to acquire the first video image of the real anchor during the live broadcast; The posture detection part 42 is configured to perform posture detection on the designated limb parts of the real anchor in the first video image, and obtain a posture detection result; The determining part 43 is configured to determine the target animation special effect of the virtual anchor model corresponding to the real anchor according to the posture detection result when it is detected that the real anchor is in a preset posture according to the posture detection result; The display part 44 is configured to display the target animation special effects of the virtual anchor model in the live video interface corresponding to the real anchor.

In a possible implementation manner, the posture detection part 42 is further configured to: when the posture detection result includes at least one of the body detection result and the gesture classification result, specifying the real anchor in the first video image Performing limb detection on limb parts to obtain limb detection results; in the case that the limb detection results include a hand detection frame, performing gesture detection on an image in the first video image that is located in the hand detection frame , to obtain a gesture classification result; determine the posture detection result according to the limb detection result and the gesture classification result.

In a possible implementation manner, the determining part 43 is further configured to: judge whether the real anchor in the first video image satisfies the gesture recognition condition according to the body detection result in the posture detection result, and obtain the judgment result; When the judgment result indicates that the real anchor satisfies the gesture recognition condition, detect whether the gesture indicated by the gesture classification result in the gesture detection result is a preset gesture; when the gesture indicated by the gesture classification result is detected If the gesture is the preset gesture, it is determined that the real anchor is in the preset gesture.

In a possible implementation manner, the determination part 43 is further configured to: determine the relative orientation information between the designated limb parts of the real anchor according to the body detection result; , determine whether the real anchor in the first video image satisfies the gesture recognition condition; the preset orientation information is used to characterize the relative orientation between the designated body parts of the real anchor in a preset posture relation.

In a possible implementation manner, the determining part 43 is further configured to: determine first driving information for animation special effects based on the posture detection result; wherein, the first driving information is used to indicate that in the live video interface Animation jump information of the animation special effects of the displayed virtual live broadcast model; according to the first driving information, determine the animation sequence matching the first driving information among the plurality of animation sequences corresponding to the posture detection results, And determine the matching animation sequence as the target animation special effect.

In a possible implementation manner, the determining part 43 is further configured to: determine at least one video image located before the first video image in the video stream; For the second driving information of the animation special effect, and determine the estimated driving information for the animation special effect according to the posture detection result; determine the animation sequence driven by each driving information in the second driving information and the estimated driving information, At least one animation sequence is obtained; and the driving information corresponding to the animation sequence whose occurrence number meets the preset number requirement in the at least one animation sequence is determined as the first driving information.

In a possible implementation manner, the determining part 43 is further configured to: acquire animation state machines of the multiple animation sequences; the animation state machine is used to characterize the jump relationship between multiple animation states, each of the animation states The animation state corresponds to one or more animation sequences; according to the first driving information, determine the next animation state to be jumped of the animation state machine; according to the animation sequence corresponding to the next animation state to be jumped, determine An animation sequence matching the first driving information.

In a possible implementation manner, the acquiring part 41 is also configured to acquire the virtual live broadcast scene corresponding to the real anchor; the determining part 43 is also configured to: acquire the pose detection result matching An initial animation special effect; in the initial animation special effect, an animation special effect matching the virtual live broadcast scene is determined as the target animation special effect.

In a possible implementation manner, when the target animation special effects include at least one of physical movement special effects and rendering material special effects for representing the physical movement of the virtual anchor model, the determining part 43 is also set to at least one of the following: displaying the physical effect of the physical movement of the virtual anchor model in the live video interface; displaying the rendering material special effect at a target position associated with the physical movement of the virtual anchor model in the live video interface.

For the description of the processing flow of each module in the device and the interaction flow between the modules, reference may be made to the relevant description in the above method embodiment, and details will not be described here.

In the embodiments of the present invention and other embodiments, a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, it may also be a unit, and it may also be a module or non-modular.

Corresponding to the special effect display method in FIG. 1, the embodiment of the present invention also provides a computer device 500, as shown in FIG. 4, which is a schematic structural diagram of the computer device 500 provided by the embodiment of the present invention, including: Processor 51, memory 52, and bus bar 53; Memory 52 is used for storing execution instructions, including internal memory 521 and external memory 522; Internal memory 521 is used for temporarily storing the calculation data in the processor 51, And the data exchanged with external memory 522 such as hard disk, processor 51 carries out data exchange with external memory 522 through internal memory 521, when described computer equipment 500 runs, described processor 51 and described memory 52 Communicate through the bus 53, so that the processor 51 executes the following instructions: Obtain the first video image of the real anchor during the live broadcast; Performing posture detection on the specified body parts of the real anchor in the first video image to obtain a posture detection result; When it is detected that the real anchor is in a preset posture according to the posture detection result, the target animation special effect of the virtual anchor model corresponding to the real anchor is determined according to the posture detection result; The target animation special effect of the virtual anchor model is displayed in the live video interface corresponding to the real anchor.

An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the method for displaying special effects described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiment of the present invention also provides a computer program product, the computer program product carries a program code, and the instructions contained in the program code can be used to execute the steps of the special effect display method described in the above method embodiment, for details, please refer to the above method implementation example, which will not be repeated here.

Wherein, the above-mentioned computer program product can be realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. wait.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or elements can be combined or can be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may also be distributed to multiple network units . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the related technology or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes. media.

Finally, it should be noted that: the above-described embodiments are only specific implementations of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them, and the scope of protection of the present invention is not limited thereto, although referring to the foregoing The embodiment has described the present invention in detail, and those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention Changes can be easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of the present invention within the scope of protection. Therefore, the protection scope of the present invention should be based on the protection scope of the patent application.

Industrial Applicability In the embodiment of the present invention, by identifying the posture of the real anchor, the target animation special effect of the virtual anchor model driven by the real anchor is determined based on the posture detection result, and the target animation special effect is displayed in the live video interface. In this way, the target animation effects corresponding to the virtual anchor model can be triggered and displayed on the live video interface through the posture detection results of the real anchor, which increases the interaction efficiency between the anchor user and the viewer through body movements, and also improves the live broadcast user’s experience. Live experience.

41: Get part 42: Attitude detection part 43: Determining part 44: Display part 500: computer equipment 51: Processor 52: memory 521: internal memory 522: external memory 53: busbar S101~S107: steps

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following will briefly introduce the drawings used in the embodiments. FIG. 1 shows a flow chart of a method for displaying special effects provided by an embodiment of the present invention; Fig. 2 shows a schematic diagram of a posture detection result provided by an embodiment of the present invention; Fig. 3 shows a schematic diagram of a special effect display device provided by an embodiment of the present invention; Fig. 4 shows a schematic diagram of a computer device provided by an embodiment of the present invention.

S101~S107: steps

Claims (11)

A method for displaying special effects, comprising: Obtain the first video image of the real anchor during the live broadcast; Performing posture detection on the specified body parts of the real anchor in the first video image to obtain a posture detection result; When it is detected that the real anchor is in a preset posture according to the posture detection result, the target animation special effect of the virtual anchor model corresponding to the real anchor is determined according to the posture detection result; The target animation special effect of the virtual anchor model is displayed in the live video interface corresponding to the real anchor. The method according to claim 1, wherein the posture detection results include at least one of body detection results and posture classification results; The gesture detection is performed on the specified limb parts of the real anchor in the first video image, and the gesture detection result is obtained, including: Carry out limb detection to the designated limb parts of the real anchor in the first video image, and obtain the limb detection result; If the limb detection result includes a hand detection frame, gesture detection is performed on an image within the hand detection frame in the first video image to obtain a gesture classification result. The method according to claim 1 or 2, wherein the detecting that the real anchor is in a preset posture according to the posture detection result includes: According to the limb detection result in the posture detection result, judge whether the real anchor in the first video image satisfies the gesture recognition condition, and obtain the judgment result; When the judgment result indicates that the real anchor satisfies the gesture recognition condition, detecting whether the gesture indicated by the gesture classification result in the gesture detection result is a preset gesture; If it is detected that the gesture indicated by the gesture classification result is the preset gesture, it is determined that the real anchor is in the preset gesture. According to the method described in claim 3, wherein, according to the limb detection result in the posture detection result, it is judged whether the real anchor in the first video image meets the gesture recognition condition, and the judgment result is obtained, including: determining the relative orientation information between the designated body parts of the real anchor according to the body detection results; According to the relative orientation information and the preset orientation information, it is judged whether the real anchor in the first video image satisfies the gesture recognition condition, and the judgment result is obtained; the preset orientation information is used to represent the real The anchor is in the preset posture, and the relative orientation relationship between the designated body parts. The method according to claim 1 or 2, wherein said determining the target animation effect of the virtual anchor model corresponding to the real anchor according to the posture detection result includes: Based on the posture detection result, determine the first driving information for the animation special effect; wherein, the first driving information is used to indicate the animation jump information of the animation special effect of the virtual live model displayed in the live video interface; According to the first driving information, an animation sequence matching the first driving information is determined among a plurality of animation sequences corresponding to the posture detection result, and the matching animation sequence is determined as the target Animation effects. According to the method described in claim 5, wherein the determining the first driving information for animation special effects based on the posture detection result includes: determining at least one video image preceding the first video image in the video stream; Acquiring second driving information for animation special effects determined according to each of the video images, and determining estimated driving information for animation special effects according to the gesture detection result; determining an animation sequence driven by each of the second driving information and the estimated driving information to obtain at least one animation sequence; The first driving information is determined as the driving information corresponding to the animation sequence whose occurrence number meets the preset requirement in the at least one animation sequence. The method according to claim 5, wherein, according to the first driving information, determining an animation sequence matching the first driving information among the plurality of animation sequences corresponding to the gesture detection results includes : Acquiring the animation state machines of the plurality of animation sequences; the animation state machine is used to represent the jump relationship between the plurality of animation states, each of the animation states corresponds to one or more animation sequences; According to the first driving information, determine the next animation state to be jumped to by the animation state machine; According to the animation sequence corresponding to the next animation state to be jumped, an animation sequence matching the first driving information is determined. The method according to claim 1 or 2, wherein the method further comprises: Obtaining the virtual live broadcast scene corresponding to the real anchor; According to the posture detection result, determining the target animation special effect of the virtual anchor model corresponding to the real anchor includes: Obtain an initial animation special effect matching the posture detection result; An animation special effect matching the virtual live broadcast scene is determined in the initial animation special effect as the target animation special effect. The method according to claim 1 or 2, wherein the target animation special effects include at least one of physical movement special effects and rendering material special effects for representing the physical movement of the virtual anchor model; The target animation effects of the virtual anchor model displayed in the live video interface corresponding to the real anchor include at least one of the following: Displaying the body movement special effects of the body movement of the virtual anchor model in the live video interface; The special effect of the rendering material is displayed at a target position associated with the body movements of the virtual anchor model in the live video interface. A computer device, comprising: a processor, a memory, and a bus bar, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the connection between the processor and the memory Through bus communication, when the machine-readable instructions are executed by the processor, the steps of the method for displaying special effects as described in any one of claims 1 to 9 are executed. A computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of the method for displaying special effects as described in any one of claims 1 to 9 are executed.

Images