WO2020107908A1 - 多用户视频特效添加方法、装置、终端设备及存储介质 - Google Patents

多用户视频特效添加方法、装置、终端设备及存储介质 Download PDF

Info

Publication number
WO2020107908A1
WO2020107908A1 PCT/CN2019/097443 CN2019097443W WO2020107908A1 WO 2020107908 A1 WO2020107908 A1 WO 2020107908A1 CN 2019097443 W CN2019097443 W CN 2019097443W WO 2020107908 A1 WO2020107908 A1 WO 2020107908A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
special effect
effect addition
image frame
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/097443
Other languages
English (en)
French (fr)
Inventor
黄佳斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Publication of WO2020107908A1 publication Critical patent/WO2020107908A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone

Definitions

  • Embodiments of the present disclosure relate to data technology, such as a multi-user video special effect adding method, device, terminal device, and storage medium.
  • video interaction applications can recognize the user's face, and add a static image on the user's head (for example, add a headdress to the hair) or increase facial expression to cover the user's face.
  • This method of adding images is too limited, and at the same time the application scenario is too single to meet the diverse needs of users.
  • Embodiments of the present disclosure provide a multi-user video special effect adding method, device, terminal device, and storage medium.
  • an embodiment of the present disclosure provides a multi-user video special effect addition method, the method includes: identifying, in a plurality of image frames in a video that matches a special effect addition interval, a target user who matches the special effect addition interval At least one human joint point, wherein the video includes a plurality of special effect addition intervals; based on the position information of at least one human joint point of the target user in the plurality of image frames, calculating the target user in the special effect Adding motion feature parameters in the interval; when at least one human joint point of the target user identified in the image frames selected from the plurality of image frames satisfies the preset joint action conditions, the selected image Frame as the target image frame, obtain video effects and special effect addition information that match the joint motion conditions, and add the video effects to the video position associated with the target image frame in the video; according to at least two The user's motion feature parameters and special effect addition information in the matched special effect addition interval, calculate the sports score information of each of the users, and add the sports score information at the sports
  • an embodiment of the present disclosure also provides a multi-user video effect addition device, the device includes: a human joint point recognition module, configured to recognize and identify a plurality of image frames in the video that match the effect addition interval At least one human joint point of the target user whose said special effect adding interval matches, wherein said video includes a plurality of special effect adding intervals; a motion feature parameter calculation module is set to be based on at least one human joint point of said target user in said multiple Location information in an image frame to calculate the motion feature parameters of the target user in the special effect addition interval; the video special effect determination module is set to identify the image identified in the image frames selected from the plurality of image frames When at least one human joint point of the target user satisfies the preset joint motion conditions, the selected image frame is used as the target image frame to obtain video effects and special effect addition information that match the joint motion conditions, and add the Video effects to the video position associated with the target image frame in the video; the sports score information calculation module is set to calculate each feature based on the motion feature parameters
  • an embodiment of the present disclosure also provides a terminal device including: at least one processor; a memory configured to store at least one program; when the at least one program is executed by the at least one processor, Causing the at least one processor to implement the multi-user video special effect adding method described in the embodiments of the present disclosure.
  • an embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the multi-user video special effect adding method described in the embodiment of the present disclosure is implemented.
  • 1a is a flowchart of a method for adding multi-user video special effects provided by an embodiment of the present disclosure
  • FIG. 1b is a schematic diagram of a human joint point provided by an embodiment of the present disclosure.
  • FIG. 2 is a flowchart of a method for adding multi-user video special effects provided by an embodiment of the present disclosure
  • FIG. 3 is a flowchart of a method for adding multi-user video special effects provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic structural diagram of a multi-user video special effect adding device provided by an embodiment of the present disclosure
  • FIG. 5 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • FIG. 1a is a flowchart of a method for adding multi-user video special effects according to an embodiment of the present disclosure. This embodiment can be applied to the case where video special effects are added to multiple different users in a video. This method can be used by multi-user video
  • the special effect adding device is executed.
  • the device may be implemented in at least one of software and hardware.
  • the device may be configured in a terminal device, for example, a computer or the like. As shown in FIG. 1a, the method includes steps S110 to S160.
  • step S110 at least one human joint point of the target user matching the special effect addition interval is identified in a plurality of image frames matching the special effect addition interval in the video, wherein the video includes a plurality of special effect addition intervals.
  • the video is formed by a series of static image frames continuously displayed at a very fast speed.
  • the video can be split into a series of image frames, and the image frames can be edited, thereby realizing the editing operation of the video.
  • the video may be a complete video that has been recorded, or may be a video that is being recorded in real time.
  • the special effect adding interval may be a collection of image frames for adding video effects to a user, and the target user may refer to the special effect adding interval being a target for adding video effects.
  • At least one user can be photographed in a special effect addition interval. In the case where only one user is photographed, the user can be used as the target user matching the special effect addition interval. In the case where at least two users are photographed, optional A user is the target user, or a user is selected as the target user among users other than the target users in the adjacent special effect addition interval matching.
  • the method of selecting one user from multiple users as the target user may be: according to the recognition completeness, confidence level of each user's joint point, or the distance between each user and the device that took the video, select one of the users as a follow-up need to add video Objects for special effects.
  • Human joint points are used to determine the user's motion state in the image frame, such as standing, bowing or jumping, and to determine the user's position, such as the distance between the user and the terminal device, the user and the terminal device The relative position of other objects or the position of the user in the screen shot by the terminal device.
  • FIG. 1b is the human body contour displayed on the video preview interface of the mobile terminal.
  • the circle in the human body contour represents the recognized human joint point, and the line between the two human joint points It is used to represent the body parts of the human body, for example, the line between the wrist joint point and the elbow joint point is used to indicate the arm between the wrist and the elbow.
  • Perform human body joint point recognition operation on each image frame which can identify all human body regions in the image frame, and can perform image segmentation on the image frame according to the depth information contained in the image frame (depth information can be obtained by infrared camera)
  • depth information can be obtained by infrared camera
  • select a human body area from all the human body areas to identify human joint points The human body area with the shortest distance can be selected as the user who needs to identify the human joint point according to the distance between the human body area and the display screen of the terminal device. It is determined by other methods, and there is no restriction on this.
  • the method for identifying the joint points of the human body may be: determining the body part areas (arms, hands, thighs, and feet) belonging to the human body area in the human body area, and calculating the joint points (elbow, (Wrists, knees, etc.), and finally generate a human skeleton system according to the position of each joint point identified, and determine the target human joint point from it as needed.
  • you can use the connection between two target human joint points (such as the connection between the wrist joint point and the elbow joint point to represent the arm between the wrist and elbow), for example, through the two target human joint points
  • the coordinates determine the vector of the line segment composed of two points, and determine the user's body part area's operating state or position.
  • the above-mentioned human body recognition, body part area recognition, and joint point position calculation in the body part area can all use pre-training
  • the deep learning model is implemented, and the deep learning model can be trained according to the deep features extracted from the human body depth information.
  • step S120 according to the position information of at least one human joint point of the target user in the plurality of image frames, the motion feature parameter of the target user in the special effect addition interval is calculated.
  • the position information of the human joint point may refer to the position of the human joint point in the image frame.
  • a coordinate system may be set in the image frame.
  • the position information of the human joint point in the image frame may be Coordinate representation of the gateway.
  • the motion feature parameter may refer to a feature parameter used to represent the target user's motion, and may include at least one of motion speed, motion direction, motion amplitude, and other parameters.
  • the coordinates of the first image frame and the coordinates in the last image frame of the human joint point in the special effect addition interval can determine the total movement distance of the human joint point in the special effect addition interval, divided by the duration of the special effect addition interval,
  • the obtained calculation result is the average speed of the human joint points in the special effect addition interval.
  • step S130 it is judged whether at least one human joint point of the target user identified in the image frames selected from the plurality of image frames satisfies a preset joint motion condition until the judgment of all the plurality of image frames is completed , In the case that at least one human joint point of the target user identified by the image frames selected from the plurality of image frames meets the preset joint action condition, step S140 is executed; selecting from the plurality of image frames If at least one human joint point of the target user identified by the image frame does not satisfy the preset joint motion condition, step S150 is executed.
  • the joint motion condition may refer to the motion of a preset joint point, and may include at least one of the direction of motion, the speed of the motion, and the distance of the motion, for example, the wrist moves downward to the set area range, and the wrist starts at 1 per frame
  • the speed of one pixel moves to the right, or multiple joint points (such as the head, shoulders, and elbows) move downward, and the movement distance of the head joint point is greater than the movement distance of the shoulder joint point, and the movement distance of the shoulder joint point
  • this embodiment of the present disclosure is not limited.
  • each joint motion condition there may be multiple joint motion conditions, and the human joint points defined by each joint motion condition are not the same.
  • the first joint motion condition defines the motion state of the wrist joint point and the motion state of the elbow joint point.
  • the motion conditions of the two joints only limit the motion state of the ankle joint point.
  • the human joint points matching the joint motion conditions meet the joint motion conditions, the video effects corresponding to the joint motion conditions are added to the video.
  • the judgment of different joint motion conditions is independent of each other. Different video effects that match different joint motion conditions.
  • step S140 an image frame corresponding to a human joint point that satisfies the joint action condition is used as a target image frame, video effect and special effect addition information matching the joint action condition are acquired, and the video effect is added to the At the video position associated with the target image frame in the video, step S160 is performed.
  • the image frame is used as the target image frame, and the joint is acquired Video effects with matching action conditions and special effect addition information, and add the video effects to the video position associated with the target image frame in the video.
  • the video effects matching the video effect conditions are added starting from the current image frame that meets the video effect conditions.
  • Video effects are used to add special effects matching the user's actions in the target image frame to achieve interaction with the user. It can refer to at least one of animation effects and music effects. Adding animation effects is used during the display of the target image frame Simultaneously draw at least one of static and dynamic images overlaid on the original content of the target image frame, and add music special effects to play music simultaneously during the display in the target image frame.
  • the special effect addition information may refer to information added to the video special effect in the at least one special effect addition interval, including at least one item of information such as the number of additions of the video special effect, the type of the video special effect, and the level of the video special effect.
  • step S150 the next image frame is acquired, and step S130 is returned to.
  • step S160 according to the motion feature parameters and the special effect addition information of at least two users in the matched special effect addition interval, each user's sports score information is calculated, and at the position where the sports score matching the video is settled To add the sports score information.
  • the corresponding relationship between the sports feature parameter and the score, and the corresponding relationship between the special effect addition information and the score can be preset respectively, and the sports score can be calculated according to the score matching the sports feature parameter and the special effect addition information matching, for example, the sports score matches the sports feature parameter The sum of the scores matched with the effect added information.
  • the sports score calculation method may also be other methods, which is not limited in the embodiments of the present disclosure.
  • the sports score settlement position may refer to the video position where the sports score needs to be displayed in the video.
  • the video recording device matches the video position corresponding to the time point when receiving the sports score calculation instruction input by the user as the sports score settlement position; or At the video position corresponding to the end time point of the second special effect addition interval in the video, this embodiment of the present disclosure does not limit it.
  • the method before identifying at least one human joint point of the target user matching the special effect addition interval in the multiple image frames in the video matching the special effect addition interval, the method further includes : During video recording, at least one image frame in the video is acquired in real time; the adding to the video position associated with the target image frame in the video includes: the video position of the target image frame As a starting point for adding special effects; according to the duration of the special effects of the video effects matching the joint motion conditions, starting from the starting point of adding the special effects, adding the video effects to the image frames matching the duration of the special effects in the video .
  • the video can be captured in real time, and each image frame in the video can be obtained in real time.
  • the starting point for adding special effects may refer to at least one of a starting position and a starting time for adding video effects.
  • the effect duration may refer to the time elapsed from the start position to the end position of the video effect or the time between the start time and the end time.
  • the image frames matching the duration of the special effects may refer to all image frames in the video starting from the starting point of adding special effects, that is, starting from the target image frame to the corresponding ending image frame at the end of the video special effect.
  • the video effect is a music effect. If the duration of a music effect is 3s, in this video, 30 image frames are played in 1s.
  • 90 image frames (including the target image frame) starting from the target image frame ) Is the image frame that matches the duration of the effect.
  • the method for adding multi-user video special effects may further include: during the recording of the video, presenting the image frames in the video in real time in a video preview interface;
  • the addition of the video special effects to the image frames matching the duration of the special effects also includes: presenting in real time the image frames added with the video special effects in the video preview interface.
  • the video preview interface may refer to an interface of a terminal device used for a user to browse videos, where the terminal device may include a server or a client. While shooting the video in real time, the video is displayed in the video preview interface in real time, whereby the user can browse the content of the captured video in real time.
  • the video effects include: at least one of dynamic animation effects and music effects; correspondingly, in the video preview interface, presenting an image frame added with the video effects in real time may include: In the video preview interface, dynamic animation special effects are drawn in real time in image frames, and music special effects are played.
  • the dynamic animation effects are drawn in the image frame displayed in real time, for example, at least one image of musical instruments, backgrounds, characters, etc. is drawn.
  • the video effects include music effects
  • the music effects are played while the image frames are displayed in real time.
  • one terminal device can be used to shoot two users. In the case of multiple users, one terminal device cannot cover all users. In this case, you can select multiple terminal device communication connections and start video recording for multiple users at the same time.
  • terminal device A photographs user A
  • terminal device B photographs user B
  • terminal device A communicates with terminal device B.
  • Terminal device A can display the terminal in the form of a window on the video preview interface Video recorded in device B.
  • FIG. 2 is a flowchart of a method for adding multi-user video special effects provided by an embodiment of the present disclosure. This embodiment is refined based on the solution in the above embodiment.
  • at least one human joint point identifying the target user matching the special effect addition interval is refined into: when it is determined that the special effect addition condition is satisfied.
  • the start and end time points of the first special effect adding interval matching the special effect adding condition according to the current playing progress or recording of the video
  • the progress, the duration of the special effect addition interval, the start and end time points of the first special effect addition interval, and the number of preset special effect addition intervals determine that multiple special effect addition intervals that match the special effect addition conditions are in the video Start and end time points; in the case where the video position of the image frame acquired in the video matches the start and end time point of the target special effect addition interval in the video, identify
  • the method of this embodiment may include steps S201 to S210.
  • step S201 at least one human joint point of the target user matching the special effect addition interval is identified in a plurality of image frames matching the special effect addition interval in the video, wherein the video includes multiple special effect addition intervals.
  • the special effect adding interval For the video, the special effect adding interval, the image frame, the human joint point, the target user, the joint action condition, the video position, and the video special effect in this embodiment, reference may be made to the description in the above embodiment.
  • step S202 according to the position information of at least one human joint point of the target user in the plurality of image frames, a motion feature parameter of the target user in the special effect addition interval is calculated.
  • step S203 when it is determined that the special effect addition condition is satisfied, the start and end time of the first special effect addition interval matching the special effect addition condition is determined according to the current playback progress or recording progress of the video and the duration of the special effect addition interval point.
  • the condition for adding special effects may be the condition that the video recording device starts to recognize the user's human joints and add video effects, for example, whether the video recording device has captured the user, and for example, whether the video recording device (such as a terminal device) has received
  • the video special effects add a start instruction or a motion score calculation start instruction, etc.
  • the start and end time points include a start time point and an end time point.
  • the start time point of the first special effect addition interval can be determined according to the special effect addition condition, and the time point when the special effect addition condition is satisfied can be used as the starting time point of the first special effect addition interval; or the time can be set after the special effect addition condition is satisfied
  • the time point (such as 10 seconds) is the starting time point of the first special effect adding interval.
  • the end time point of the first special effect adding interval can be determined according to the current playing progress or recording progress of the video and the duration of the special effect adding interval, and the current playing progress or recording progress of the video is determined according to the current playing progress or recording progress of the video
  • the end time point of the first special effect addition interval is determined by the starting time point in combination with the duration of the special effect addition interval ;
  • the time is shorter than the duration of the special effect addition interval.
  • the current playback progress or recording progress of the video is taken as the end time of the first special effect adding interval.
  • step S204 according to the current playback progress or recording progress of the video, the duration of the special effect addition interval, the start and end time points of the first special effect addition interval, and the number of preset special effect addition intervals, determine and A plurality of special effect adding intervals matching the special effect adding conditions are at the starting and ending time points in the video.
  • the current playing progress or recording progress of the video After determining the starting and ending time points of the first special effect adding interval, the current playing progress or recording progress of the video, the duration of the special effect adding interval, the starting and ending time points of the first special effect adding interval and the preset special effects can be determined
  • the number of added intervals determines the start and end time points of subsequent added effects in the video.
  • the start time point of each special effect addition interval can coincide with the end time point of the adjacent previous special effect addition interval, or can be set at a time interval (such as 15 second).
  • the ending time point of each special effect adding interval please refer to the method for determining the ending time point of the first special effect adding interval.
  • step S205 determine whether the video position of the image frame acquired in the video in the video matches the start and end time points of multiple special effect addition intervals in the video until all of the multiple image frames If the judgment is completed, when the video position of the image frame acquired in the video in the video matches the start and end time points of multiple effect addition intervals in the video, step S206 is executed; in the video In a case where the video position of the acquired image frame in the video does not match the start and end time points of multiple effect addition intervals in the video, step S207 is executed.
  • step S206 the matched special effect addition interval is used as a target special effect addition interval, a target user matching the target special effect addition interval is identified in the acquired image frame, and at least one human joint of the target user is identified Click to execute step S208.
  • the target special effect addition interval is the special effect addition interval to which the image frame belongs.
  • the special effect addition interval is used as the target special effect addition interval, and at the same time, the image frame Belongs to the target special effect adding interval.
  • Each special effect addition interval corresponds to a target user as a recognition object. For each image frame, according to the special effect addition interval to which the image frame belongs, the target user to be recognized by the image frame and at least one human joint point of the target user are determined.
  • the identification object corresponding to each special effect addition interval may be determined according to the first image frame of the special effect addition interval.
  • the identifying a target user in the image frame that matches the target special effect addition interval may include: when the acquired image frame is the first image frame of the target special effect addition interval Next, acquire a user matching the previous special effect addition interval adjacent to the target special effect addition interval as a screening user, and identify a user who removed the screening user as the target special effect in the acquired image frame Add a target user who matches the interval; in the case that the acquired image frame is not the first image frame of the target effect addition interval, acquire the target user matching the target effect addition interval, and in the acquired image frame Identify the target user.
  • the target user matched in each special effect addition interval is a user selected from multiple users identified from the first image frame in the interval as the target user, and the other image frames in each special effect addition interval are all Take the target user as the identification object.
  • the target user of the special effect addition interval A is regarded as the screening user of the adjacent next special effect addition interval B, and the special effect addition interval B selects one user from the identified multiple users other than the screening user as the special effect addition interval B Target users.
  • step S207 the next image frame is acquired, and step S205 is returned to.
  • step S208 it is determined whether at least one human joint point of the target user satisfies a preset joint motion condition. In a case where at least one human joint point of the target user satisfies a preset joint motion condition, step S209 is executed ; In the case that at least one human joint point of the target user does not satisfy the preset joint action condition, step S207 is executed.
  • step S209 use an image frame corresponding to a human joint point that satisfies the joint motion condition as a target image frame, acquire video effects and special effect addition information that match the joint motion conditions, and add the video effects to the The video position associated with the target image frame in the video.
  • step S210 according to the motion feature parameters and the special effect addition information of at least two users in the matched special effect addition interval, the sports score information of each of the users is calculated, and at the sports score settlement position matching the video To add the sports score information.
  • the target users matching the special effect addition intervals are respectively used as recognition objects, and user identification and human joint point recognition are performed in at least one image frame in each special effect addition interval.
  • the users can be accurately distinguished, and matching video effects can be independently generated for different users to improve the targeting of video effects, and at the same time improve the scene of video interactive applications and the diversification of video effects .
  • FIG. 3 is a flowchart of a method for adding multi-user video special effects according to an embodiment of the present disclosure.
  • This embodiment is refined based on the solution in the above embodiment.
  • the calculation of the motion feature parameters of the target user in the special effect addition interval is refined as:
  • the special effect adding interval according to the position information of at least one human joint point of the target user in the plurality of image frames, calculate at least one human joint point of the target user in the special effect adding interval
  • the unit displacement and the movement displacement to determine the average movement distance and movement distance variance of the target user in the special effect addition interval; according to the average movement distance and the movement distance variance, calculate the target user
  • the special effects add motion feature parameters in the interval.
  • the video effects and the added information of the special effects that match the joint motion conditions are refined into: according to the joint motion information in the joint motion conditions, at least one human joint point of the target user and the joint are determined Matching degree of action information; acquiring video special effects matching the matching degree; using the joint action conditions and the matching degree as special effect addition information.
  • the method of this embodiment may include steps S301 to S310.
  • step S301 in a plurality of image frames in the video that match the special effect addition interval, at least one human joint point of the target user matching the special effect addition interval is identified, wherein the video includes a plurality of special effect addition intervals.
  • the special effect adding interval For the video, the special effect adding interval, the image frame, the human joint point, the target user, the joint action condition, the video position, and the video special effect in this embodiment, reference may be made to the description in the above embodiment.
  • step S302 in the special effect addition interval, according to the position information of at least one human joint point of the target user in the plurality of image frames, calculate at least one human joint point of the target user in the The unit displacement between any two adjacent image frames in the special effect addition interval, and the movement displacement of at least one human joint point of the target user in the special effect addition interval.
  • the unit displacement may refer to the movement distance of at least one human joint point of the target user between any two adjacent image frames in the special effect addition interval.
  • the modulus of the vector determined by the coordinates of each human joint point in the two image frames is calculated as each human joint point between the two image frames
  • the moving distance is the sum of the moving distances of the human joint points and divided by the number of human joint points, and the obtained result is the unit displacement of at least one human joint point of the target user between two image frames.
  • the motion displacement may refer to a movement distance of at least one human joint point of the target user in the special effect addition interval.
  • the sum of the unit displacement between any two image frames in the special effect addition interval is calculated, and the obtained result is used as the motion displacement of at least one human joint point of the target user in the special effect addition interval.
  • step S303 the duration of the special effect addition interval is counted, and the average movement distance and movement distance of the target user in the special effect addition interval are determined according to the duration, the unit displacement, and the movement displacement variance.
  • the special effect addition interval includes a total of N+1 image frames, and the average motion distance mean N can be calculated based on the following formula:
  • s 1 is the movement distance of at least one human joint point of the target user in the special effect adding interval from the first image frame to the adjacent next image frame (second image frame)
  • s 2 is the target user's moving distance in the special effect adding interval
  • s N is the at least one human joint point of the target user in the special effect addition interval from the Nth image frame to the corresponding The moving distance of the next image frame (N+1th image frame).
  • s n is calculated as follows:
  • 1 ⁇ n ⁇ N represents the nth image frame from 1 to N
  • M n represents the number of key points that can be detected at the same time in the nth image frame and the n+1th image frame
  • k j Represents the displacement of the jth detected key point in two image frames.
  • s n The physical meaning of s n is: the average motion displacement of all key points of two adjacent image frames.
  • the variance VAR of the movement distance can be calculated based on the following formula:
  • s i is the moving distance of at least one human joint point of the target user in the special effect addition interval from the i-th image frame to the adjacent next image frame (i+1 image frame).
  • step S304 according to the average motion distance and the variance of the motion distance, a motion feature parameter of the target user in the special effect addition interval is calculated.
  • step S305 it is judged whether at least one human joint point of the target user identified in the image frames selected from the plurality of image frames satisfies a preset joint action condition until the judgment of all the plurality of image frames is completed .
  • step S306 is executed; selecting from the plurality of image frames If at least one human joint point of the target user identified by the image frame does not satisfy the preset joint motion condition, step S307 is executed.
  • step S306 an image frame corresponding to a human joint point that satisfies the joint motion condition is used as a target image frame, and according to the joint motion information in the joint motion condition, at least one human joint point of the target user and the For the matching degree of the joint motion information, step S308 is executed.
  • Joint motion information may refer to standard motion state information expected to be reached by at least one human joint point, and may include information such as angle information or position information.
  • the matching degree may refer to the degree of similarity between the current motion state of the recognized human joint point and the standard motion state in the joint motion information.
  • the confidence calculation can be: a series of standard movements are specified in advance, for example, the raising of the hand, the position of the wrist joint point is higher than the position of the eye joint point, the position of the wrist joint point, the position of the elbow joint point and the shoulder
  • the positions of the gate nodes are on the same line, and at the same time, the straight line determined by the wrist joint point, elbow joint point and shoulder joint point is parallel to the user's standing direction.
  • the position of the wrist joint point is higher than the position of the eye joint point
  • the confidence level is 0
  • the connection between the wrist joint point and the elbow joint point and the elbow joint point and the shoulder joint point are determined The angle between them, the greater the deviation of the angle from 180 degrees (smaller), the lower the confidence (higher).
  • step S307 the next image frame is acquired, and step S305 is returned to.
  • step S308 video special effects matching the matching degree are acquired.
  • step S309 the joint motion conditions and the matching degree are used as special effect addition information.
  • Each joint action condition corresponds to a different score, for example, a jump action is 10 points, a bending action is 8 points, and a manual action is 5 points.
  • step S310 according to the motion feature parameters and the special effect addition information of at least two users in the matched special effect addition interval, calculate the sports score information of each of the users, and at the position where the sports score matching the video is settled To add the sports score information.
  • the sports score calculation is performed for user A and user B.
  • a mobile terminal is used to record video for two users and select a background music, where different background music corresponds to different joint motion conditions.
  • the mobile terminal receives the instruction of the two-player three-wheel two-win mode entered by the user, that is, the condition for adding special effects is satisfied.
  • this mode there are up to three rounds of competition, and each person performs 10 seconds in each round, that is, the number of special effect addition sections is at most 6, and the duration of special effect addition video is 10 seconds.
  • the video preview interface of the mobile terminal Through the video preview interface of the mobile terminal, user A and user B are prompted to enter the video shooting range and start timing for 10 seconds, user A enters the video shooting range and starts dancing, and user B waits outside the video shooting range.
  • the video preview interface of the mobile terminal will display at least one image of the standard motion in the joint motion condition, and the user A can make actions that meet the joint motion condition according to the displayed image.
  • the mobile terminal counts user A's sports score and starts the second time count in this round.
  • User B enters the video shooting range and starts dancing.
  • User A waits outside the video shooting range.
  • the video preview interface of the mobile terminal will display the image of the standard motion in at least one joint motion condition.
  • the image of the standard motion this time may be the same as or different from the image of the last standard motion.
  • the mobile terminal counts user B's sports score.
  • the sports scores of user A and user B are published, and at the corresponding sports score settlement position of the round (such as the end time point of the special effect addition interval), the image of the user with high sports score is displayed through the video preview interface of the mobile terminal , And add and display victory video effects.
  • the sports score Z is calculated based on the following formula:
  • X is the base score
  • Y is the action score.
  • the base score is the sum of the average movement distance and the variance of the movement distance.
  • the action score is the score corresponding to the special effect added information.
  • FIG. 4 is a schematic structural diagram of a device for adding multi-user video special effects according to an embodiment of the present disclosure. This embodiment can be applied to the case where video special effects are added to multiple different users in a video.
  • the device may be implemented in at least one of software and hardware, and the device may be configured in the terminal device. As shown in FIG. 4, the device may include: a human joint point recognition module 410, a motion feature parameter calculation module 420, a video effect determination module 430, and a motion score information calculation module 440.
  • the human joint point recognition module 410 is configured to identify at least one human joint point of the target user matching the special effect addition interval in a plurality of image frames matching the special effect addition interval in the video, wherein the video includes multiple Special effects add interval.
  • the motion feature parameter calculation module 420 is configured to calculate the motion feature parameter of the target user in the special effect addition interval according to the position information of at least one human joint point of the target user in the plurality of image frames.
  • the video effect determination module 430 is configured to, when at least one human joint point of the target user identified in the image frames selected from the plurality of image frames satisfies a preset joint motion condition, select the selected image
  • the frame serves as a target image frame, acquires video effects and special effect addition information that match the joint motion conditions, and adds the video effects to the video position associated with the target image frame in the video.
  • the sports score information calculation module 440 is configured to calculate the sports score information of each user according to the motion feature parameters and special effect addition information of at least two users in the matched special effect addition interval, and to match the movement of the video At the point settlement position, the sports score information is added.
  • At least one human joint point of the target user is identified in each special effect addition interval of the video, and the motion feature parameter of the target user is calculated according to the human joint point, and at the same time, the human joint point is added when the joint motion condition is satisfied
  • Video special effects and obtain special effect addition information calculate each user's sports score according to the target user's motion feature parameters and special effect addition information, and add them to the video, avoiding the situation that the video interaction application's video effect is too single, can be targeted for simultaneous shooting
  • Add matching dynamic special effects to the video of multiple users' joints improve the scenes of video interactive applications and the diversification of video special effects, and increase the flexibility of video to increase special effects.
  • the human joint point recognition module 410 includes a start point determination module for the first special effect addition interval, a start and end point determination module for the special effect addition interval, and a target user determination module.
  • the starting point determination module of the first special effect addition interval is set to determine the first match with the special effect addition condition according to the current playback progress or recording progress of the video and the duration of the special effect addition interval when it is determined that the special effect addition condition is satisfied.
  • the start and end time points of a special effect addition interval is set to determine the first match with the special effect addition condition according to the current playback progress or recording progress of the video and the duration of the special effect addition interval when it is determined that the special effect addition condition is satisfied.
  • the starting and ending point determination module of the special effect adding interval is set to be based on the current playing progress or recording progress of the video, the duration of the special effect adding interval, the starting and ending time points of the first special effect adding interval, and the number of preset special effect adding intervals , Determine the start and end time points of the multiple effect addition intervals matching the special effect addition condition in the video.
  • the target user determination module is configured to set the image frame acquired in the video when the video position of the image frame in the video matches the start and end time points of the target special effect addition interval, in the acquired image In the frame, identify a target user that matches the target special effect addition interval, and identify at least one human joint point of the target user.
  • the target user determination module includes a screening user determination module and a target user acquisition module.
  • the screening user determination module is set to obtain a user who matches the previous special effect addition interval adjacent to the target special effect addition interval as the acquired image frame is the first image frame of the target special effect addition interval as Screen out users, and identify one user who removed the screened user in the acquired image frame as the target user matching the target special effect addition interval.
  • the target user acquisition module is set to acquire the target user matching the target effect addition interval in the case where the acquired image frame is not the first image frame of the target effect addition interval, and in the acquired image frame Identify the target user.
  • the motion feature parameter calculation module 420 includes a displacement calculation module, a motion distance calculation module, and a motion feature parameter determination module.
  • the displacement calculation module is set to calculate at least one human joint point of the target user based on the position information of the at least one human joint point of the target user in the plurality of image frames within the special effect addition interval The unit displacement between any two adjacent image frames in the special effect adding interval, and the movement displacement of at least one human joint point of the target user in the special effect adding interval.
  • the movement distance calculation module is set to count the duration of the special effect addition interval, and determine the average movement distance of the target user in the special effect addition interval according to the duration, the unit displacement and the movement displacement Variance of movement distance.
  • the motion feature parameter determination module is configured to calculate the motion feature parameter of the target user in the special effect addition interval according to the average motion distance and the motion distance variance.
  • the video special effect determination module 430 includes a matching degree determination module, a video special effect acquisition module, and a special effect addition information determination module.
  • the matching degree determination module is configured to determine the degree of matching between the at least one human joint point of the target user and the joint action information according to the joint action information in the joint action condition.
  • the video special effect obtaining module is set to obtain video special effects matching the matching degree.
  • the special effect addition information determination module is configured to use the joint action condition and the matching degree as special effect addition information.
  • the device for adding multi-user video special effects further includes: a real-time image frame acquisition module configured to acquire at least one image frame in the video in real time during video recording; the video special effect determination module 430, including a module for determining the starting point of adding special effects and a module for adding video special effects.
  • the special effect adding starting point determination module is set to use the video position of the target image frame as a special effect adding starting point; the video special effect adding module is set to add the special effect according to the duration of the video effect matching the joint motion condition from the special effect Starting at the starting point, the video effect is added to the image frame in the video that matches the duration of the effect.
  • the device for adding multi-user video special effects further includes: a real-time image frame rendering module, which is configured to present the image frames in the video in real time in a video preview interface during the recording of the video;
  • the real-time video effect rendering module is configured to present the image frames added with the video special effects in real time in the video preview interface.
  • the apparatus for adding multi-user video special effects provided by the embodiments of the present disclosure belongs to the same inventive concept as the above-mentioned method for adding multi-user video special effects.
  • the method for adding multi-user video special effects please refer to the method for adding multi-user video special effects.
  • An embodiment of the present disclosure provides a terminal device.
  • Terminal devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant (PDA), tablet computers (Portable Android Device, PAD), portable multimedia players (Portable Media Player, PMP), mobile terminals such as in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital television (TV), desktop computers, and so on.
  • PDA Personal Digital Assistant
  • PMP portable multimedia players
  • mobile terminals such as in-vehicle terminals (such as in-vehicle navigation terminals)
  • fixed terminals such as digital television (TV), desktop computers, and so on.
  • TV digital television
  • the electronic device shown in FIG. 5 is only an example, and should not bring any limitation to the functions and use scope of the embodiments of the present disclosure.
  • the electronic device 500 may include a processing device (such as a central processing unit, a graphics processor, etc.) 501, which may be stored in a read-only memory (Read-Only Memory, ROM) 502 program or from a storage device 508 loads the program in random access memory (Random Access Memory, RAM) 503 to perform various appropriate actions and processes.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • various programs and data necessary for the operation of the electronic device 500 are also stored.
  • the processing device 501, ROM 502, and RAM 503 are connected to each other via a bus 504.
  • An input/output (Input/Output, I/O) interface 505 is also connected to the bus 504.
  • the following devices can be connected to the I/O interface 505: including an input device 506 such as a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (Liquid Crystal Display, LCD) , An output device 507 of a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509.
  • the communication device 509 may allow the electronic device 500 to perform wireless or wired communication with other devices to exchange data.
  • FIG. 5 shows an electronic device 500 having various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or provided instead.
  • the process described above with reference to the flowchart may be implemented as a computer software program.
  • embodiments of the present disclosure include a computer program product that includes a computer program carried on a computer-readable medium, the computer program containing program code for performing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication device 509, or from the storage device 508, or from the ROM 502.
  • the above-described functions defined in the method of the embodiments of the present disclosure are executed.
  • An embodiment of the present disclosure also provides a computer-readable storage medium, which may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: electrical connections with at least one wire, portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable only Read memory (Erasable Programmable Read-Only Memory, EPROM or flash memory), optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable combination.
  • the computer-readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • the computer-readable signal medium may include a data signal that is propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried.
  • This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: electric wires, optical cables, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • the computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.
  • the above computer-readable medium carries at least one program, and when the above at least one program is executed by the electronic device, the electronic device is caused to: identify the special effect among the multiple image frames matching the special effect addition interval in the video Adding at least one human joint point of the target user whose interval matches, wherein the video includes a plurality of special effect adding intervals; based on the position information of at least one human joint point of the target user in the plurality of image frames, calculating The motion feature parameters of the target user in the special effect addition interval; the case where at least one human joint point of the target user identified in the image frames selected from the plurality of image frames meets the preset joint action conditions Next, use the selected image frame as the target image frame, and obtain video effects and special effect addition information that match the joint motion conditions, and add the video effects to the video position associated with the target image frame in the video At; based on at least two users’ motion feature parameters and special effects addition information in the matching special effect addition interval, calculate the sports score information for each of the users, and add the location at the sports score settlement
  • the computer program code for performing the operations of the present disclosure can be written in one or more programming languages or a combination thereof.
  • the above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, as well as conventional Procedural programming language-such as "C" language or similar programming language.
  • the program code may be executed entirely on the user's computer, partly on the user's computer, as an independent software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (Local Area Network, LAN) or a wide area network (Wide Area Network, WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of the code contains at least one Execute instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks represented in succession may actually be executed in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.
  • each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented with dedicated hardware-based systems that perform specified functions or operations Or, it can be realized by a combination of dedicated hardware and computer instructions.
  • the modules described in the embodiments of the present disclosure may be implemented in software or hardware.
  • the name of the module does not constitute a limitation on the module itself under certain circumstances.
  • the human joint point recognition module can also be described as "in multiple image frames matching the added effect interval in the video, the recognition and "At least one human joint point of the target user whose matched special effect addition interval matches, wherein the video includes a plurality of modules for the special effect addition interval".

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

本公开公开了一种多用户视频特效添加方法、装置、终端设备及存储介质。该方法包括：在视频中与特效添加区间匹配的多个图像帧中，识别与特效添加区间匹配的目标用户的至少一个人体关节点；根据所述目标用户的至少一个人体关节点的位置信息，计算所述目标用户的运动特征参数；在所述多个图像帧中选取的图像帧中识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，将所选取的图像帧作为目标图像帧，并获取视频特效以及所述特效添加信息，添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处；计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。

Description

多用户视频特效添加方法、装置、终端设备及存储介质
本公开要求在2018年11月29日提交中国专利局、申请号为201811446855.1的中国专利申请的优先权，该公开的全部内容通过引用结合在本公开中。
技术领域
本公开实施例涉及数据技术，例如一种多用户视频特效添加方法、装置、终端设备及存储介质。
背景技术
随着通信技术和终端设备设备的发展，各种终端设备例如手机、平板电脑等已经成为了人们工作和生活中不可或缺的一部分，而且随着终端设备的日益普及，视频交互应用成为一种沟通和娱乐的主要渠道。
目前，视频交互应用能够识别出用户面部，并在用户头部上增加静态图像(例如在头发上增加头饰)或者增加面部表情覆盖在用户面部上。这种增加图像的方法过于局限，同时应用场景过于单一，无法满足用户的多样化需求。
发明内容
本公开实施例提供一种多用户视频特效添加方法、装置、终端设备及存储介质。
第一方面，本公开实施例提供了一种多用户视频特效添加方法，该方法包括：在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间；根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数；在所述多个图像帧中选取的图像帧中识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，将所选取的图像帧作为目标图像帧，则获取与所述关节动作条件匹配的视频特效以及特效添加信息，并添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处；根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。
第二方面，本公开实施例还提供了一种多用户视频特效添加装置，该装置包括：人体关节点识别模块，设置为在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间；运动特征参数计算模块，设置为根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数；视频特效确定模块，设置为在所述多个图像帧中选取的图像帧中识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，将所选取的图像帧作为目标图像帧，获取 与所述关节动作条件匹配的视频特效以及特效添加信息，并添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处；运动得分信息计算模块，设置为根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。
第三方面，本公开实施例还提供了一种终端设备，该终端设备包括：至少一个处理器；存储器，设置为存储至少一个程序；所述至少一个程序被所述至少一个处理器执行时，使得所述至少一个处理器实现如本公开实施例所述的多用户视频特效添加方法。
第四方面，本公开实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现如本公开实施例所述的多用户视频特效添加方法。
附图说明
图1a是本公开一实施例提供的一种多用户视频特效添加方法的流程图；
图1b是本公开一实施例提供的一种人体关节点的示意图；
图2是本公开一实施例提供的一种多用户视频特效添加方法的流程图；
图3是本公开一实施例提供的一种多用户视频特效添加方法的流程图；
图4是本公开一实施例提供的一种多用户视频特效添加装置的结构示意图；
图5是本公开一实施例提供的一种终端设备的结构示意图。
具体实施方式
图1a为本公开一实施例提供的一种多用户视频特效添加方法的流程图，本实施例可适用于在视频中针对多个不同用户分别添加视频特效的情况，该方法可以由多用户视频特效添加装置来执行，该装置可以采用软件和硬件中至少之一的方式实现，该装置可以配置于终端设备中，例如典型的是计算机等。如图1a所示，该方法包括步骤S110至步骤S160。
在步骤S110中，在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间。
一般来说，视频是由一系列静态的图像帧以极快的速度连续放映形成。由此，可以将视频拆分成一系列图像帧，并对图像帧进行编辑操作，从而实现对视频的编辑操作。在本公开实施例中，视频可以是一个录制完成的完整视频，也可以是正在实时录制的视频。
其中，特效添加区间可以是针对一个用户进行添加视频特效的图像帧集合，目标用户可以是指特效添加区间为添加视频特效的针对对象。在一个特效添加区间中可以拍摄到至少一个用户，在仅仅拍摄到一个用户的情况下，可以将该用户作为特效添加区间匹配的目标用户，在拍摄到至少两个用户的情况下，可以任选一个用户作为目标用户，或者在除相邻特效添加区间匹配的目标用户之 外的用户中选取一个用户作为目标用户。
从多个用户选择一个用户作为目标用户的方式可以是：根据每个用户的关节点的识别完整度、置信度或者每个用户与拍摄视频的设备的距离，选择其中一个用户作为后续需要添加视频特效的对象。
人体关节点用于确定图像帧中用户的动作状态，例如站立、鞠躬或跳跃等动作状态，以及用于确定用户的位置，例如用户与终端设备之间的距离、用户与终端设备所拍摄到的其他物体的相对位置或用户在终端设备所拍摄到的画面中的位置等位置。
在一个例子中，图1b为在移动终端的视频预览界面中显示的人体轮廓，如图1b所示，人体轮廓中的圆圈表示识别到的人体关节点，两个人体关节点之间的连线用于表示人体的身体部位，例如，手腕关节点和手肘关节点之间的连线用于表示手腕和手肘之间的手臂。
对每个图像帧进行人体关节点识别操作，可以在图像帧中识别出所有人体区域，可以是根据图像帧所包含的深度信息(深度信息可以通过红外线摄像机获取)，对该图像帧进行图像分割，识别出图像帧中所有人体区域。从所有人体区域中选择一个人体区域用于识别人体关节点，可以是根据人体区域与终端设备显示屏幕之间的距离，选择距离最短的人体区域作为需要识别人体关节点的用户，此外还可以选择其他方式确定，对此不做限制。
在确定一个人体区域之后，对该区域进行人体关节点识别，确定属于该用户的所有人体关节点，并可以根据需要从该用户的所有人体关节点中筛选出至少一个目标人体关节点。
其中，识别人体关节点的方法可以是：在人体区域中确定属于该人体区域中的身体部位区域(手臂、手、大腿和脚等)，并在每个身体部位区域计算关节点(手肘、手腕和膝盖等)位置，最后根据识别到的每个关节点位置，生成人体骨架系统，并从中根据需要确定目标人体关节点。此外，可以通过采用两个目标人体关节点的连线(如手腕关节点和手肘关节点之间的连线用于表示手腕和手肘之间的手臂)，例如通过两个目标人体关节点的坐标确定两点构成的线段的向量，判断用户某个身体部位区域的动作状态或位置，上述涉及到的人体识别、身体部位区域识别和身体部位区域中的关节点位置计算均可以采用预先训练的深度学习模型实现，而深度学习模型可以根据由人体深度信息提取出来的深度特征进行训练。
需要说明的是，识别人体关节点的方法还有其他方法，对此本公开实施例不作限制。
在步骤S120中，根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数。
人体关节点的位置信息可以是指人体关节点在图像帧中的位置，在一实施例中，可以在图像帧中设置坐标系，相应的，人体关节点在图像帧中的位置信息可以用人体关节点的坐标表示。运动特征参数可以是指用于表示目标用户运动情况的特征参数，可以包括运动速度、运动方向和运动幅度等参数中至少一 种。
在一个例子中，人体关节点在特效添加区间内首个图像帧的坐标和最后一个图像帧中的坐标可以确定人体关节点的在特效添加区间的总移动距离，除以特效添加区间的时长，得到的计算结果为人体关节点在特效添加区间的平均速度。
在步骤S130中，判断在所述多个图像帧中选取的图像帧识别出的所述目标用户的至少一个人体关节点是否满足预设的关节动作条件，直至所述多个图像帧全部判断完成，在所述多个图像帧中选取的图像帧识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，执行步骤S140；在所述多个图像帧中选取的图像帧识别出的所述目标用户的至少一个人体关节点不满足预设的关节动作条件的情况下，执行步骤S150。
关节动作条件可以是指预设的关节点的动作，可以包括运动的方向、运动的速度和运动的距离等中的至少一种，例如手腕向下运动到设定区域范围，手腕以每帧1个像素的速度向右移动，或者多个关节点(如头部、肩膀和手肘)均向下移动，同时头部关节点的运动距离大于肩膀关节点的运动距离，肩膀关节点的运动距离大于手肘关节点的运动距离等，还有其他动作，对此本公开实施例不作限制。
需要说明的是，可以存在多个关节动作条件，每个关节动作条件限定的人体关节点不全相同，例如，第一关节动作条件限定手腕关节点的运动状态和手肘关节点的运动状态，第二关节动作条件仅限定脚腕关节点的运动状态。在与关节动作条件匹配的人体关节点满足该关节运动条件的情况下，在视频中对应添加与该关节运动条件的视频特效，同时，不同关节运动条件的判断是相互独立的，可以同时添加与不同关节运动条件匹配的不同视频特效。
在步骤S140中，将满足所述关节动作条件的人体关节点对应的图像帧作为目标图像帧，获取与所述关节动作条件匹配的视频特效以及特效添加信息，并添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处，执行步骤S160。
在视频的多个图像帧中的一个图像帧识别出的目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，将该图像帧作为目标图像帧，并获取获取与所述关节动作条件匹配的视频特效以及特效添加信息，并添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处。
在视频中从满足视频特效条件的当前图像帧开始增加与视频特效条件匹配的视频特效。视频特效用于在目标图像帧中添加根据用户动作匹配的特殊效果，以实现与用户交互，可以是指动画特效和音乐特效中至少一种，添加动画特效用于目标图像帧在显示的过程中同时绘制静态和动态图像中至少一种覆盖于目标图像帧原有内容上，添加音乐特效用于在目标图像帧中显示的过程中，同时播放音乐。
特效添加信息可以是指针对至少一个特效添加区间中添加的视频特效的信息，包括视频特效的添加次数、视频特效的类型和视频特效的等级等信息中的 至少一项。
在步骤S150中，获取下一个图像帧，返回执行步骤S130。
在步骤S160中，根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。
可以分别预先设置运动特征参数与分数的对应关系，以及特效添加信息与分数的对应关系，根据运动特征参数匹配的分数以及特效添加信息匹配的分数，计算运动得分，例如运动得分为运动特征参数匹配的分数与特效添加信息匹配的分数之和。其中，运动得分计算方式还可以是其他方式，本公开实施例不做限制。
运动得分结算位置处可以是指视频中需要显示运动得分的视频位置，例如，视频录制设备在接收到用户输入的运动得分计算指令时对应的时间点匹配的视频位置作为运动得分结算位置；或者是在视频中第二个特效添加区间的终止时间点对应的视频位置处，对此，本公开实施例不做限制。
本公开实施例通过在视频的每个特效添加区间中识别目标用户的至少一个人体关节点，并根据人体关节点计算目标用户的运动特征参数，同时在人体关节点满足关节动作条件的情况下，添加视频特效以及获取特效添加信息，根据目标用户的运动特征参数以及特效添加信息计算每个用户的运动得分，并添加至视频中，避免了视频交互应用的视频特效过于单一的情况，可以针对同时拍摄到多个用户的关节点的视频添加匹配的动态特效，并根据每个用户的动态特性以及运动情况，计算每个用户的运动得分并呈现，提高视频交互应用的场景以及视频特效的多样化，提高视频增加特效的灵活性。
在上述实施例的基础上，在一实施例中，在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点之前，还包括：在视频录制过程中，实时获取所述视频中的至少一个图像帧；所述添加至所述视频中与所述目标图像帧关联的视频位置处，包括：将所述目标图像帧的视频位置作为特效添加起点；根据与所述关节动作条件匹配的视频特效的特效持续时间，从所述特效添加起点开始，在所述视频中与所述特效持续时间匹配的图像帧中添加所述视频特效。
在一实施例中，可以实时拍摄视频，并实时获取视频中的每个图像帧。其中，特效添加起点可以是指视频特效添加的起始位置和起始时刻中至少之一。特效持续时间可以是指视频特效的起始位置到结束位置之间经历的时间或起始时刻到结束时刻之间的时间。与特效持续时间匹配的图像帧可以是指在视频中从特效添加起点开始，也就是从目标图像帧开始，一直到该视频特效结束时对应的结束图像帧之间的所有图像帧。例如，视频特效为音乐特效，若一个音乐特效的持续时间为3s，在该视频中，1s播放30个图像帧，按视频播放顺序，从目标图像帧开始的90个图像帧(包括目标图像帧)即为与特效持续时间匹配的图像帧。
由此通过实时拍摄视频，并实时获取视频拆分的一系列图像帧，从而实时 判断拍摄的视频中当前图像帧是否存在满足运动变化条件的目标运动物体，实时添加与所述运动变化条件和/或目标运动物体匹配的视频特效，可以实现在视频录制的同时添加视频特效，提高视频特效的添加效率。
在一实施例中，所述多用户视频特效添加方法还可以包括：在所述视频的录制过程中，在视频预览界面中实时呈现所述视频中的图像帧；在所述视频中与所述特效持续时间匹配的图像帧中添加所述视频特效的同时，还包括：在所述视频预览界面中，实时呈现添加所述视频特效的图像帧。
其中，视频预览界面可以是指用于用户浏览视频的终端设备的界面，其中，终端设备可以包括服务器端或客户端。在实时拍摄视频的同时，将视频实时显示在视频预览界面中，由此，用户可以实时浏览到拍摄的视频的内容。
在一实施例中，所述视频特效包括：动态动画特效和音乐特效中至少一种；相应的，所述在所述视频预览界面中，实时呈现添加所述视频特效的图像帧，可以包括：在所述视频预览界面中，在图像帧中实时绘制动态动画特效，并播放音乐特效。
在一实施例中，在视频特效包括动态动画特效的情况下，在实时显示的图像帧中绘制动态动画特效，例如，绘制乐器、背景和人物等中至少一种图像。在视频特效包括音乐特效的情况下，在图像帧实时显示的同时播放音乐特效。通过设置视频特效包括动态动画特效和音乐特效中至少一种，提高视频特效的多样性。
需要说明的是，可以使用一个终端设备对两个用户进行拍摄。在存在多个用户的情况下，一个终端设备无法涵盖所有用户，此时，可以选择多个终端设备通信连接，并同时开始对多个用户进行视频录制。
在一个例子中，终端设备A对用户A进行拍摄，同时终端设备B对用户B进行拍摄，且终端设备A与终端设备B进行通信连接，终端设备A可以在视频预览界面中以窗口形式展现终端设备B中录制的视频。
图2为本公开一实施例提供的一种多用户视频特效添加方法的流程图。本实施例以上述实施例中的方案为基础进行细化。在本实施例中，将在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点细化为：在确定满足特效添加条件的情况下，根据所述视频的当前播放进度或录制进度以及特效添加区间的时长，确定与所述特效添加条件匹配的首个特效添加区间的起止时间点；根据所述视频的当前播放进度或录制进度、所述特效添加区间的时长、所述首个特效添加区间的起止时间点以及预设的特效添加区间的数量，确定与所述特效添加条件匹配的多个特效添加区间在所述视频中的起止时间点；在所述视频中获取的图像帧在所述视频中的视频位置与目标特效添加区间的所述起止时间点相匹配的情况下，在所获取的图像帧中识别与所述目标特效添加区间匹配的目标用户，并识别所述目标用户的至少一个人体关节点。
相应的，本实施例的方法可以包括步骤S201至步骤S210。
在步骤S201中，在视频中与特效添加区间匹配的多个图像帧中，识别与所 述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间。
本实施例中的视频、特效添加区间、图像帧、人体关节点、目标用户、关节动作条件、视频位置和视频特效等均可以参考上述实施例中的描述。
在步骤S202中，根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数。
在步骤S203中，在确定满足特效添加条件的情况下，根据所述视频的当前播放进度或录制进度以及特效添加区间的时长，确定与所述特效添加条件匹配的首个特效添加区间的起止时间点。
特效添加条件可以是视频录制设备开始进行用户的人体关节点识别，并添加视频特效的条件，例如，可以是视频录制设备是否拍摄到用户，又如，视频录制设备(如终端设备)是否接收到视频特效添加开始指令或运动得分计算开始指令等。
起止时间点包括起始时间点和终止时间点。首个特效添加区间的起始时间点可以根据特效添加条件确定，可以将满足特效添加条件时的时间点作为首个特效添加区间的起始时间点；或者可以在满足特效添加条件之后设定时间(如10秒)所在的时间点作为首个特效添加区间的起始时间点。
首个特效添加区间的终止时间点可以根据所述视频的当前播放进度或录制进度以及特效添加区间的时长确定，在根据所述视频的当前播放进度或录制进度确定视频的当前播放进度或录制进度所在时间点与首个特效添加区间的起始时间点之间的时间大于等于特效添加区间的时长的情况下，首个特效添加区间的终止时间点由起始时间点结合特效添加区间的时长确定；在根据所述视频的当前播放或录制进度确定视频的当前播放或录制进度所在时间点与首个特效添加区间的起始时间点之间的时间小于特效添加区间的时长的情况下，将所述视频的当前播放进度或录制进度所在时间点作为首个特效添加区间的终止时间点。
在步骤S204中，根据所述视频的当前播放进度或录制进度、所述特效添加区间的时长、所述首个特效添加区间的起止时间点以及预设的特效添加区间的数量，确定与所述特效添加条件匹配的多个特效添加区间在所述视频中的起止时间点。
在确定首个特效添加区间的起止时间点之后，可以根据所述视频的当前播放进度或录制进度、所述特效添加区间的时长、所述首个特效添加区间的起止时间点以及预设的特效添加区间的数量，确定视频中后续的特效添加区间的起止时间点。其中，每个特效添加区间的起始时间点可以与相邻的前一个特效添加区间的终止时间点重合，或者可以与相邻的前一个特效添加区间的终止时间点间隔设定时间(如15秒)。每个特效添加区间的终止时间点可以参考首个特效添加区间的终止时间点的确定方式。
在步骤S205中，判断在所述视频中获取的图像帧在所述视频中的视频位置是否与所述视频中多个特效添加区间的所述起止时间点匹配，直至所述多个图像帧全部判断完成，在所述视频中获取的图像帧在所述视频中的视频位置与所 述视频中多个特效添加区间的所述起止时间点匹配的情况下，执行步骤S206；在所述视频中获取的图像帧在所述视频中的视频位置与所述视频中多个特效添加区间的所述起止时间点不匹配的情况下，执行步骤S207。
在步骤S206中，将匹配的所述特效添加区间作为目标特效添加区间，在所获取的图像帧中识别与所述目标特效添加区间匹配的目标用户，并识别所述目标用户的至少一个人体关节点，执行步骤S208。
目标特效添加区间为图像帧所属的特效添加区间。在所述视频中获取的图像帧在所述视频中的视频位置与一个特效添加区间的所述起止时间点相匹配的情况下，将该特效添加区间作为目标特效添加区间，同时，该图像帧属于目标特效添加区间。每个特效添加区间对应一个目标用户作为识别对象，针对每个图像帧，根据图像帧所属的特效添加区间，确定图像帧需要识别的目标用户，以及该目标用户的至少一个人体关节点。
每个特效添加区间对应的识别对象可以根据该特效添加区间的首个图像帧确定。在一实施例中，所述在所述图像帧中识别与所述目标特效添加区间匹配的目标用户，可以包括：在所获取的图像帧是所述目标特效添加区间的首个图像帧的情况下，获取与所述目标特效添加区间相邻的前一特效添加区间匹配的用户作为筛除用户，并在所获取的图像帧中识别除去所述筛除用户的一个用户作为与所述目标特效添加区间匹配的目标用户；在所获取的图像帧不是所述目标特效添加区间的首个图像帧的情况下，获取与所述目标特效添加区间匹配的目标用户，并在所获取的图像帧中识别所述目标用户。
在一实施例中，每个特效添加区间匹配的目标用户均为从区间内首个图像帧识别到的多个用户中选取的一个用户作为目标用户，每个特效添加区间内的其他图像帧均以该目标用户作为识别对象。
由于视频中多个特效添加区间存在均识别到相同目标用户，为了避免上述情况，使任意相邻两个特效添加区间对应的目标用户不同。将特效添加区间A的目标用户作为相邻的后一个特效添加区间B的筛除用户，特效添加区间B在从识别到的除了筛除用户以外的多个用户中选取一个用户作为特效添加区间B的目标用户。
通过在每个特效添加区间中以不全相同的目标用户为识别对象，并在每个特效添加区间中每个图像帧中以同一个目标用户为识别对象，实现在视频中分别以不同用户作为识别对象添加视频特效，从而实现多用户视频特效添加的应用场景。
在步骤S207中，获取下一个图像帧，返回执行步骤S205。
在步骤S208中，判断所述目标用户的至少一个人体关节点是否满足预设的关节动作条件，在所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，执行步骤S209；在所述目标用户的至少一个人体关节点不满足预设的关节动作条件的情况下，执行步骤S207。
在步骤S209中，将满足所述关节动作条件的人体关节点对应的图像帧作为目标图像帧，获取与所述关节动作条件匹配的视频特效以及特效添加信息，并 添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处。
在步骤S210中，根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。
本公开实施例通过在视频中设置多个特效添加区间，分别将特效添加区间匹配的目标用户作为识别对象，在每个特效添加区间中的至少一个图像帧中进行用户识别以及人体关节点识别，可以在视频中拍摄到多个用户的情况下，对用户进行准确区分，并独立针对不同用户生成匹配的视频特效，提高视频特效的针对性，同时提高视频交互应用的场景以及视频特效的多样化。
图3为本公开一实施例提供的一种多用户视频特效添加方法的流程图。本实施例以上述实施例中的方案为基础进行细化。在本实施例中，将根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数细化为：在所述特效添加区间内，根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户的至少一个人体关节点在所述特效添加区间内任意相邻两个图像帧之间的单位位移，以及所述目标用户的至少一个人体关节点在所述特效添加区间内的运动位移；统计所述特效添加区间的持续时间，并根据所述持续时间、所述单位位移和所述运动位移，确定所述目标用户在所述特效添加区间的平均运动距离和运动距离方差；根据所述平均运动距离和所述运动距离方差，计算所述目标用户在所述特效添加区间内的运动特征参数。同时，将获取与所述关节动作条件匹配的视频特效以及所述特效添加信息细化为：根据所述关节动作条件中关节动作信息，确定所述目标用户的至少一个人体关节点与所述关节动作信息的匹配程度；获取与所述匹配程度匹配的视频特效；将所述关节动作条件以及所述匹配程度作为特效添加信息。
相应的，本实施例的方法可以包括步骤S301至步骤S310。
在步骤S301中，在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间。
本实施例中的视频、特效添加区间、图像帧、人体关节点、目标用户、关节动作条件、视频位置和视频特效等均可以参考上述实施例中的描述。
在步骤S302中，在所述特效添加区间内，根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户的至少一个人体关节点在所述特效添加区间内任意相邻两个图像帧之间的单位位移，以及所述目标用户的至少一个人体关节点在所述特效添加区间内的运动位移。
单位位移可以是指目标用户的至少一个人体关节点在特效添加区间内任意相邻两个图像帧之间的移动距离。在一实施例中，针对目标用户的至少一个人体关节点，计算每个人体关节点在两个图像帧中的坐标确定的向量的模，作为每个人体关节点在两个图像帧之间的移动距离，统计人体关节点的移动距离之和，并除以人体关节点的数量，得到的结果作为目标用户的至少一个人体关节 点在两个图像帧之间的单位位移。
运动位移可以是指，目标用户的至少一个人体关节点在特效添加区间内的移动距离。在一实施例中，计算特效添加区间中任意两个图像帧之间的单位位移之和，得到的结果作为目标用户的至少一个人体关节点在特效添加区间内的运动位移。
在步骤S303中，统计所述特效添加区间的持续时间，并根据所述持续时间、所述单位位移和所述运动位移，确定所述目标用户在所述特效添加区间的平均运动距离和运动距离方差。
特效添加区间总共包括N+1个图像帧，可以基于如下公式计算平均运动距离mean N：
Figure PCTCN2019097443-appb-000001
其中，s 1为特效添加区间中目标用户的至少一个人体关节点由第一图像帧到相邻的后一个图像帧(第二图像帧)的运动距离，s 2为特效添加区间中目标用户的至少一个人体关节点由第二图像帧到相邻的后一个图像帧(第三图像帧)的运动距离，s N为特效添加区间中目标用户的至少一个人体关节点由第N图像帧到相邻的后一个图像帧(第N+1图像帧)的运动距离。
在一实施例中，s n的计算方式如下：
Figure PCTCN2019097443-appb-000002
其中，1≤n≤N，代表1到N个中的第n个图像帧，M n代表在第n个图像帧和第n+1个图像帧同时能检测出的关键点个数，k j代表检测出的关键点的第j个在两个图像帧中的位移。
s n的物理意义是：相邻两个图像帧所有关键点的平均运动位移。
需要说明的是，运动距离的计算方法可以参照前述单位位移的计算方式。
可以基于如下公式计算运动距离方差VAR：
Figure PCTCN2019097443-appb-000003
其中，s i为特效添加区间中目标用户的至少一个人体关节点由第i个图像帧到相邻的后一个图像帧(第i+1个图像帧)的移动距离。
在步骤S304中，根据所述平均运动距离和所述运动距离方差，计算所述目标用户在所述特效添加区间内的运动特征参数。
在步骤S305中，判断在所述多个图像帧中选取的图像帧识别出的所述目标用户的至少一个人体关节点是否满足预设的关节动作条件，直至所述多个图像帧全部判断完成，在所述多个图像帧中选取的图像帧识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，执行步骤S306；在所述多个图像帧中选取的图像帧识别出的所述目标用户的至少一个人体关节点不满足预设的关节动作条件的情况下，执行步骤S307。
在步骤S306中，将满足所述关节动作条件的人体关节点对应的图像帧作为 目标图像帧，根据所述关节动作条件中关节动作信息，确定所述目标用户的至少一个人体关节点与所述关节动作信息的匹配程度，执行步骤S308。
关节动作信息可以是指至少一个人体关节点预期达到的标准动作状态信息，可以包括角度信息或位置信息等信息。
匹配程度可以是指识别到的人体关节点的当前动作状态与关节动作信息中标准动作状态的相似程度。
在一实施例中，可以通过计算动作的置信度，确定匹配程度。例如，若一个关节动作条件对应的分数为100分，在用户做出的动作标准的情况下，计算得到的置信度大于等于0.9，匹配程度为100％，最后用户得到的分数为100％*100＝100分；在用户做出的动作比较标准的情况下，计算得到的置信度大于等于0.7且小于0.9，匹配程度为80％，最后用户得到的分数为80％*100＝80，在用户做出的动作比不标准的情况下，计算得到的置信度大于等于0.5且小于0.7，最后用户得到的分数为60％*100＝60。
其中，置信度的计算可以是：预先规定一系列的标准动作，例如，举手动作，手腕关节点的位置比眼睛关节点的位置高，手腕关节点的位置、手肘关节点的位置和肩膀关节点的位置在同一条线上，同时，由手腕关节点、手肘关节点和肩膀关节点确定的直线，与用户的站立方向平行。
此时，可以先检测手腕关节点的位置与眼睛关节点的位置距离差，在手腕关节点的位置高于眼睛关节点的位置的情况下，距离差越大(小)，置信度越高(低)，在手腕关节点的位置低于或等于眼睛关节点的位置的情况下，置信度为0；其次确定手腕关节点和手肘关节点连线与手肘关节点和肩膀关节点连线之间的夹角，该夹角与180度的偏离程度越大(小)，置信度越低(高)。
在步骤S307中，获取下一个图像帧，返回执行步骤S305。
在步骤S308中，获取与所述匹配程度匹配的视频特效。
在步骤S309中，将所述关节动作条件以及所述匹配程度作为特效添加信息。
每个关节动作条件对应不同的分数，例如跳跃动作为10分，弯腰动作为8分，举手动作为5分。计算匹配程度与关节动作条件对应的分数的乘积，可以确定特效添加信息对应的分数。例如，识别到用户的弯腰动作，匹配程度为80％，则该用户得到的分数为80％*8＝6.4。
在步骤S310中，根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。
在一个具体的例子中，针对用户A和用户B进行运动得分计算。采用移动终端对两个用户进行视频录制，选择一个背景音乐，其中，不同的背景音乐对应有不同的关节动作条件。移动终端接收到用户输入的双人斗舞三轮两胜制模式的指令，即满足特效添加条件。该模式下最多进行三轮比拼，每轮每个人10秒的时间进行表演，即特效添加区间的数量最多为6，特效添加视频的时长为10秒。
通过移动终端的视频预览界面提示用户A和用户B进入视频拍摄范围中， 并开始计时10秒，用户A进入视频拍摄范围并开始跳舞，用户B在视频拍摄范围外等待。在10秒内，移动终端的视频预览界面会显示至少一个关节动作条件中标准动作的图像，用户A可以按照显示的图像做出满足该关节动作条件动作。当本轮属于用户A的时间计时结束后，移动终端统计用户A的运动得分，并开始本轮的第二次计时，用户B进入视频拍摄范围并开始跳舞，用户A在视频拍摄范围外等待，同样，移动终端的视频预览界面会显示至少一个关节动作条件中标准动作的图像，此次标准动作的图像可以与上次标准动作的图像相同，也可以不同。当本轮属于用户B的时间计时结束后，移动终端统计用户B的运动得分。此时，公布用户A和用户B的运动得分，并在该轮对应的运动得分结算位置处(如特效添加区间的终止时间点)，通过移动终端的视频预览界面显示运动得分高的用户的图像，以及添加并显示胜利的视频特效。
基于如下公式计算运动得分Z：
Z＝X*0.3+Y*0.7
其中，X为基础得分，Y为动作得分。基础得分为平均运动距离和运动距离方差之和。动作得分为特效添加信息对应的分数。
此后，每轮结束(即第二特效添加区间的终止时间点对应的视频位置处)后移动终端统计本轮运动得分并显示。在本次录制的视频中，采用三轮两胜制度，率先取得两次胜利的用户作为本次斗舞模式的胜利者，通过移动终端的视频预览界面显示斗舞胜利的用户的图像，以及添加并显示胜利的视频特效。图4为本公开一实施例提供的一种多用户视频特效添加装置的结构示意图，本实施例可适用于在视频中针对多个不同用户分别添加视频特效的情况。该装置可以采用软件和硬件中至少之一的方式实现，该装置可以配置于终端设备中。如图4所示，该装置可以包括：人体关节点识别模块410、运动特征参数计算模块420、视频特效确定模块430和运动得分信息计算模块440。
人体关节点识别模块410，设置为在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间。
运动特征参数计算模块420，设置为根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数。
视频特效确定模块430，设置为在所述多个图像帧中选取的图像帧中识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，将所选取的图像帧作为目标图像帧，获取与所述关节动作条件匹配的视频特效以及特效添加信息，并添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处。
运动得分信息计算模块440，设置为根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。
本公开实施例通过在视频的每个特效添加区间中识别目标用户的至少一个 人体关节点，并根据人体关节点计算目标用户的运动特征参数，同时在人体关节点满足关节动作条件的情况下添加视频特效以及获取特效添加信息，根据目标用户的运动特征参数以及特效添加信息计算每个用户的运动得分，并添加至视频中，避免了视频交互应用的视频特效过于单一的情况，可以针对同时拍摄到多个用户的关节点的视频添加匹配的动态特效，提高视频交互应用的场景以及视频特效的多样化，提高视频增加特效的灵活性。
在一实施例中，所述人体关节点识别模块410，包括首个特效添加区间起始点确定模块、特效添加区间起止点确定模块及目标用户确定模块。首个特效添加区间起始点确定模块，设置为在确定满足特效添加条件的情况下，根据所述视频的当前播放进度或录制进度以及特效添加区间的时长，确定与所述特效添加条件匹配的首个特效添加区间的起止时间点。特效添加区间起止点确定模块，设置为根据所述视频的当前播放进度或录制进度、所述特效添加区间的时长、所述首个特效添加区间的起止时间点以及预设的特效添加区间的数量，确定与所述特效添加条件匹配的多个特效添加区间在所述视频中的起止时间点。目标用户确定模块，设置为在所述视频中获取的图像帧所述图像帧在所述视频中的视频位置与目标特效添加区间的所述起止时间点相匹配的情况下，在所获取的图像帧中识别与所述目标特效添加区间匹配的目标用户，并识别所述目标用户的至少一个人体关节点。
在一实施例中，所述目标用户确定模块，包括筛除用户确定模块及目标用户获取模块。筛除用户确定模块，设置为在所获取的图像帧是所述目标特效添加区间的首个图像帧的情况下，获取与所述目标特效添加区间相邻的前一特效添加区间匹配的用户作为筛除用户，并在所获取的图像帧中识别除去所述筛除用户的一个用户作为与所述目标特效添加区间匹配的目标用户。目标用户获取模块，设置为在所获取的图像帧不是所述目标特效添加区间的首个图像帧的情况下，获取与所述目标特效添加区间匹配的目标用户，并在所获取的图像帧中识别所述目标用户。
在一实施例中，所述运动特征参数计算模块420，包括位移计算模块、运动距离计算模块及运动特征参数确定模块。位移计算模块，设置为在所述特效添加区间内，根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户的至少一个人体关节点在所述特效添加区间内任意相邻两个图像帧之间的单位位移，以及所述目标用户的至少一个人体关节点在所述特效添加区间内的运动位移。运动距离计算模块，设置为统计所述特效添加区间的持续时间，并根据所述持续时间、所述单位位移和所述运动位移，确定所述目标用户在所述特效添加区间的平均运动距离和运动距离方差。运动特征参数确定模块，设置为根据所述平均运动距离和所述运动距离方差，计算所述目标用户在所述特效添加区间内的运动特征参数。
在一实施例中，所述视频特效确定模块430，包括匹配程度确定模块、视频特效获取模块及特效添加信息确定模块。匹配程度确定模块，设置为根据所述关节动作条件中关节动作信息，确定所述目标用户的至少一个人体关节点与所 述关节动作信息的匹配程度。视频特效获取模块，设置为获取与所述匹配程度匹配的视频特效。特效添加信息确定模块，设置为将所述关节动作条件以及所述匹配程度作为特效添加信息。
在一实施例中，所述多用户视频特效添加装置，还包括：图像帧实时获取模块，设置为在视频录制过程中，实时获取所述视频中的至少一个图像帧；所述视频特效确定模块430，包括特效添加起点确定模块及视频特效添加模块。特效添加起点确定模块，设置为将所述目标图像帧的视频位置作为特效添加起点；视频特效添加模块，设置为根据与所述关节动作条件匹配的视频特效的特效持续时间，从所述特效添加起点开始，在所述视频中与所述特效持续时间匹配的图像帧中添加所述视频特效。
在一实施例中，所述多用户视频特效添加装置，还包括：图像帧实时呈现模块，设置为在所述视频的录制过程中，在视频预览界面中实时呈现所述视频中的图像帧；视频特效实时呈现模块，设置为在所述视频预览界面中，实时呈现添加所述视频特效的图像帧。
本公开实施例提供的多用户视频特效添加装置，与上述多用户视频特效添加方法属于同一发明构思，未在本公开实施例中详尽描述的技术细节可参见上述多用户视频特效添加方法。
本公开一实施例提供了一种终端设备，下面参考图5，其示出了适于用来实现本公开实施例的电子设备(例如客户端或服务器端)500的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，PDA)、平板电脑(Portable Android Device，PAD)、便携式多媒体播放器(Portable Media Player，PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字电视(Television，TV)、台式计算机等等的固定终端。图5示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图5所示，电子设备500可以包括处理装置(例如中央处理器、图形处理器等)501，其可以根据存储在只读存储器(Read-Only Memory，ROM)502中的程序或者从存储装置508加载到随机访问存储器(Random Access Memory，RAM)503中的程序而执行各种适当的动作和处理。在RAM 503中，还存储有电子设备500操作所需的各种程序和数据。处理装置501、ROM 502以及RAM503通过总线504彼此相连。输入/输出(Input/Output，I/O)接口505也连接至总线504。
通常，以下装置可以连接至I/O接口505：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置506；包括例如液晶显示器(Liquid Crystal Display，LCD)、扬声器、振动器等的输出装置507；包括例如磁带、硬盘等的存储装置508；以及通信装置509。通信装置509可以允许电子设备500与其他设备进行无线或有线通信以交换数据。虽然图5示出了具有各种装置的电子设备500，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置509从网络上被下载和安装，或者从存储装置508被安装，或者从ROM 502被安装。在该计算机程序被处理装置501执行的情况下，执行本公开实施例的方法中限定的上述功能。
本公开一实施例还提供了一种计算机可读存储介质，计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有至少一个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、射频(Radio Frequency，RF)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有至少一个程序，在上述至少一个程序被该电子设备执行的情况下，使得该电子设备：在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间；根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数；在所述多个图像帧中选取的图像帧中识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，将所选取的图像帧作为目标图像帧，并获取与所述关节动作条件匹配的视频特效以及特效添加信息，添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处；根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置 处，添加所述运动得分信息。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network，LAN)或广域网(Wide Area Network，WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含至少一个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的模块可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，模块的名称在某种情况下并不构成对该模块本身的限定，例如，人体关节点识别模块还可以被描述为“在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间的模块”。

Claims (16)

  1. 一种多用户视频特效添加方法，包括：
    在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间；
    根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数；
    在所述多个图像帧中选取的图像帧中识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，将所选取的图像帧作为目标图像帧，获取与所述关节动作条件匹配的视频特效以及特效添加信息，并添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处；
    根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。
  2. 根据权利要求1所述的方法，其中，所述在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，包括：
    在确定满足特效添加条件的情况下，根据所述视频的当前播放进度或录制进度，以及特效添加区间的时长，确定与所述特效添加条件匹配的首个特效添加区间的起止时间点；
    根据所述视频的当前播放进度或录制进度、所述特效添加区间的时长、所述首个特效添加区间的起止时间点以及预设的特效添加区间的数量，确定与所述特效添加条件匹配的多个特效添加区间在所述视频中的起止时间点；
    在所述视频中获取的图像帧在所述视频中的视频位置与目标特效添加区间的所述起止时间点相匹配的情况下，在所获取的图像帧中识别与所述目标特效添加区间匹配的目标用户，并识别所述目标用户的至少一个人体关节点。
  3. 根据权利要求2所述的方法，其中，所述在所获取的图像帧中识别与所述目标特效添加区间匹配的目标用户，包括：
    在所获取的图像帧是所述目标特效添加区间的首个图像帧的情况下，获取与所述目标特效添加区间相邻的前一特效添加区间匹配的用户作为筛除用户，并在所获取的图像帧中识别除去所述筛除用户的一个用户作为与所述目标特效添加区间匹配的目标用户；
    在所获取的图像帧不是所述目标特效添加区间的首个图像帧的情况下，获取与所述目标特效添加区间匹配的目标用户，并在所获取的图像帧中识别所述目标用户。
  4. 根据权利要求1所述的方法，其中，所述根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数，包括：
    在所述特效添加区间内，根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户的至少一个人体关节点在所述特 效添加区间内任意相邻两个图像帧之间的单位位移，以及所述目标用户的至少一个人体关节点在所述特效添加区间内的运动位移；
    统计所述特效添加区间的持续时间，并根据所述持续时间，所述单位位移和所述运动位移，确定所述目标用户在所述特效添加区间的平均运动距离和运动距离方差；
    根据所述平均运动距离和所述运动距离方差，计算所述目标用户在所述特效添加区间内的运动特征参数。
  5. 根据权利要求1所述的方法，其中，所述获取与所述关节动作条件匹配的视频特效以及特效添加信息，包括：
    根据所述关节动作条件中关节动作信息，确定所述目标用户的至少一个人体关节点与所述关节动作信息的匹配程度；
    获取与所述匹配程度匹配的视频特效；
    将所述关节动作条件以及所述匹配程度作为特效添加信息。
  6. 根据权利要求1-5任一项所述的方法，在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点之前，还包括：
    在视频录制过程中，实时获取所述视频中的至少一个图像帧；
    所述添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处，包括：
    将所述目标图像帧的视频位置作为特效添加起点；
    根据与所述关节动作条件匹配的视频特效的特效持续时间，从所述特效添加起点开始，在所述视频中与所述特效持续时间匹配的图像帧中添加所述视频特效。
  7. 根据权利要求6所述的方法，还包括：
    在所述视频的录制过程中，在视频预览界面中实时呈现所述视频中的图像帧；
    在所述视频中与所述特效持续时间匹配的图像帧中添加所述视频特效的同时，还包括：
    在所述视频预览界面中，实时呈现添加所述视频特效的图像帧。
  8. 一种多用户视频特效添加装置，包括：
    人体关节点识别模块，设置为在视频中与特效添加区间匹配的多个图像帧中，识别与所述特效添加区间匹配的目标用户的至少一个人体关节点，其中，所述视频包括多个特效添加区间；
    运动特征参数计算模块，设置为根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户在所述特效添加区间内的运动特征参数；
    视频特效确定模块，设置为在所述多个图像帧中选取的图像帧中识别出的所述目标用户的至少一个人体关节点满足预设的关节动作条件的情况下，将所选取的图像帧作为目标图像帧，获取与所述关节动作条件匹配的视频特效以及 特效添加信息，并添加所述视频特效至所述视频中与所述目标图像帧关联的视频位置处；
    运动得分信息计算模块，设置为根据至少两个用户在匹配的特效添加区间内的运动特征参数以及特效添加信息，计算每个所述用户的运动得分信息，并在与所述视频匹配的运动得分结算位置处，添加所述运动得分信息。
  9. 根据权利要求8所述的装置，其中，所述人体关节点识别模块，包括：
    首个特效添加区间起始点确定模块，设置为在确定满足特效添加条件的情况下，根据所述视频的当前播放进度或录制进度，以及特效添加区间的时长，确定与所述特效添加条件匹配的首个特效添加区间的起止时间点；
    特效添加区间起止点确定模块，设置为根据所述视频的当前播放进度或录制进度、所述特效添加区间的时长、所述首个特效添加区间的起止时间点以及预设的特效添加区间的数量，确定与所述特效添加条件匹配的多个特效添加区间在所述视频中的起止时间点；
    目标用户确定模块，设置为在所述视频中获取的图像帧所述图像帧在所述视频中的视频位置与目标特效添加区间的所述起止时间点相匹配的情况下，在所获取的图像帧中识别与所述目标特效添加区间匹配的目标用户，并识别所述目标用户的至少一个人体关节点。
  10. 根据权利要求9所述的装置，其中，所述目标用户确定模块，包括：
    筛除用户确定模块，设置为在所获取的图像帧是所述目标特效添加区间的首个图像帧的情况下，获取与所述目标特效添加区间相邻的前一特效添加区间匹配的用户作为筛除用户，并在所获取的图像帧中识别除去所述筛除用户的一个用户作为与所述目标特效添加区间匹配的目标用户；
    目标用户获取模块，设置为在所获取的图像帧不是所述目标特效添加区间的首个图像帧的情况下，获取与所述目标特效添加区间匹配的目标用户，并在所获取的图像帧中识别所述目标用户。
  11. 根据权利要求8所述的装置，其中，所述运动特征参数计算模块，包括：
    位移计算模块，设置为在所述特效添加区间内，根据所述目标用户的至少一个人体关节点在所述多个图像帧中的位置信息，计算所述目标用户的至少一个人体关节点在所述特效添加区间内任意相邻两个图像帧之间的单位位移，以及所述目标用户的至少一个人体关节点在所述特效添加区间内的运动位移；
    运动距离计算模块，设置为统计所述特效添加区间的持续时间，并根据所述持续时间、所述单位位移和所述运动位移，确定所述目标用户在所述特效添加区间的平均运动距离和运动距离方差；
    运动特征参数确定模块，设置为根据所述平均运动距离和所述运动距离方差，计算所述目标用户在所述特效添加区间内的运动特征参数。
  12. 根据权利要求8所述的装置，其中，所述视频特效确定模块，包括：
    匹配程度确定模块，设置为根据所述关节动作条件中关节动作信息，确定所述目标用户的至少一个人体关节点与所述关节动作信息的匹配程度；
    视频特效获取模块，设置为获取与所述匹配程度匹配的视频特效；
    特效添加信息确定模块，设置为将所述关节动作条件以及所述匹配程度作为特效添加信息。
  13. 根据权利要求8-12任一项所述的装置，还包括：
    图像帧实时获取模块，设置为在视频录制过程中，实时获取所述视频中的至少一个图像帧；
    所述视频特效确定模块，包括：
    特效添加起点确定模块，设置为将所述目标图像帧的视频位置作为特效添加起点；
    视频特效添加模块，设置为根据与所述关节动作条件匹配的视频特效的特效持续时间，从所述特效添加起点开始，在所述视频中与所述特效持续时间匹配的图像帧中添加所述视频特效。
  14. 根据权利要求13所述的装置，还包括：
    图像帧实时呈现模块，设置为在所述视频的录制过程中，在视频预览界面中实时呈现所述视频中的图像帧；
    视频特效实时呈现模块，设置为在所述视频预览界面中，实时呈现添加所述视频特效的图像帧。
  15. 一种终端设备，包括：
    至少一个处理器；
    存储器，设置为存储至少一个程序；
    所述至少一个程序被所述至少一个处理器执行时，使得所述至少一个处理器实现如权利要求1-7任一项所述的多用户视频特效添加方法。
  16. 一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现如权利要求1-7任一项所述的多用户视频特效添加方法。
PCT/CN2019/097443 2018-11-29 2019-07-24 多用户视频特效添加方法、装置、终端设备及存储介质 Ceased WO2020107908A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811446855.1A CN109525891B (zh) 2018-11-29 2018-11-29 多用户视频特效添加方法、装置、终端设备及存储介质
CN201811446855.1 2018-11-29

Publications (1)

Publication Number Publication Date
WO2020107908A1 true WO2020107908A1 (zh) 2020-06-04

Family

ID=65794652

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/097443 Ceased WO2020107908A1 (zh) 2018-11-29 2019-07-24 多用户视频特效添加方法、装置、终端设备及存储介质

Country Status (2)

Country Link
CN (1) CN109525891B (zh)
WO (1) WO2020107908A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114125351A (zh) * 2020-08-28 2022-03-01 华为技术有限公司 视频互动方法和设备
CN114973085A (zh) * 2022-05-24 2022-08-30 入微智能科技(南京)有限公司 一种运动视频课程库建立方法、系统、设备及介质
CN115619960A (zh) * 2021-07-15 2023-01-17 北京小米移动软件有限公司 图像处理的方法、装置及电子设备
CN116016817A (zh) * 2023-01-29 2023-04-25 北京达佳互联信息技术有限公司 视频剪辑方法、装置、电子设备及存储介质
CN116137672A (zh) * 2021-11-18 2023-05-19 脸萌有限公司 视频生成方法、装置、设备、存储介质及程序产品
CN119854429A (zh) * 2025-01-14 2025-04-18 北京字跳网络技术有限公司 特效视频生成方法、装置、电子设备及存储介质

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109525891B (zh) * 2018-11-29 2020-01-21 北京字节跳动网络技术有限公司 多用户视频特效添加方法、装置、终端设备及存储介质
CN109889892A (zh) * 2019-04-16 2019-06-14 北京字节跳动网络技术有限公司 视频效果添加方法、装置、设备及存储介质
CN110189364B (zh) * 2019-06-04 2022-04-01 北京字节跳动网络技术有限公司 用于生成信息的方法和装置，以及目标跟踪方法和装置
CN110298327B (zh) * 2019-07-03 2021-09-03 北京字节跳动网络技术有限公司 一种视觉特效处理方法及装置、存储介质与终端
CN111083354A (zh) * 2019-11-27 2020-04-28 维沃移动通信有限公司 一种视频录制方法及电子设备
CN111416991B (zh) * 2020-04-28 2022-08-05 Oppo（重庆）智能科技有限公司 特效处理方法和设备，及存储介质
CN112418322B (zh) * 2020-11-24 2024-08-06 苏州爱医斯坦智能科技有限公司 影像数据处理方法、装置、电子设备及存储介质
CN115278041B (zh) * 2021-04-29 2024-02-27 北京字跳网络技术有限公司 图像处理方法、装置、电子设备以及可读存储介质
CN115988227A (zh) * 2021-10-14 2023-04-18 北京字跳网络技术有限公司 直播间的特效播放方法、系统及设备
CN114866687B (zh) * 2022-03-28 2024-09-24 北京达佳互联信息技术有限公司 同框视频拍摄方法、装置、电子设备及介质
CN115278082B (zh) * 2022-07-29 2024-06-04 维沃移动通信有限公司 视频拍摄方法、视频拍摄装置及电子设备

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110069888A1 (en) * 2009-09-22 2011-03-24 Samsung Electronics Co., Ltd. Image processing apparatus and method
CN104623910A (zh) * 2015-01-15 2015-05-20 西安电子科技大学 舞蹈辅助特效伴侣系统及实现方法
CN107968921A (zh) * 2017-11-23 2018-04-27 乐蜜有限公司 视频生成方法、装置和电子设备
CN108289180A (zh) * 2018-01-30 2018-07-17 广州市百果园信息技术有限公司 根据肢体动作处理视频的方法、介质和终端装置
CN108371814A (zh) * 2018-01-04 2018-08-07 乐蜜有限公司 多人体感舞蹈的实现方法、装置、电子设备及存储介质
CN108615055A (zh) * 2018-04-19 2018-10-02 咪咕动漫有限公司 一种相似度计算方法、装置及计算机可读存储介质
CN108874120A (zh) * 2018-03-29 2018-11-23 北京字节跳动网络技术有限公司 人机交互系统、方法、计算机可读存储介质及交互装置
CN109525891A (zh) * 2018-11-29 2019-03-26 北京字节跳动网络技术有限公司 多用户视频特效添加方法、装置、终端设备及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8942428B2 (en) * 2009-05-01 2015-01-27 Microsoft Corporation Isolate extraneous motions
CN104598867B (zh) * 2013-10-30 2017-12-01 中国艺术科技研究所 一种人体动作自动评估方法及舞蹈评分系统
CN104792327B (zh) * 2015-04-13 2017-10-31 云南大学 一种基于移动设备的运动轨迹对比方法
CN106022305A (zh) * 2016-06-07 2016-10-12 北京光年无限科技有限公司 一种用于智能机器人的动作对比方法以及机器人
CN107952238B (zh) * 2017-11-23 2020-11-17 香港乐蜜有限公司 视频生成方法、装置和电子设备
CN107920269A (zh) * 2017-11-23 2018-04-17 乐蜜有限公司 视频生成方法、装置和电子设备

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110069888A1 (en) * 2009-09-22 2011-03-24 Samsung Electronics Co., Ltd. Image processing apparatus and method
CN104623910A (zh) * 2015-01-15 2015-05-20 西安电子科技大学 舞蹈辅助特效伴侣系统及实现方法
CN107968921A (zh) * 2017-11-23 2018-04-27 乐蜜有限公司 视频生成方法、装置和电子设备
CN108371814A (zh) * 2018-01-04 2018-08-07 乐蜜有限公司 多人体感舞蹈的实现方法、装置、电子设备及存储介质
CN108289180A (zh) * 2018-01-30 2018-07-17 广州市百果园信息技术有限公司 根据肢体动作处理视频的方法、介质和终端装置
CN108874120A (zh) * 2018-03-29 2018-11-23 北京字节跳动网络技术有限公司 人机交互系统、方法、计算机可读存储介质及交互装置
CN108615055A (zh) * 2018-04-19 2018-10-02 咪咕动漫有限公司 一种相似度计算方法、装置及计算机可读存储介质
CN109525891A (zh) * 2018-11-29 2019-03-26 北京字节跳动网络技术有限公司 多用户视频特效添加方法、装置、终端设备及存储介质

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114125351A (zh) * 2020-08-28 2022-03-01 华为技术有限公司 视频互动方法和设备
CN115619960A (zh) * 2021-07-15 2023-01-17 北京小米移动软件有限公司 图像处理的方法、装置及电子设备
CN116137672A (zh) * 2021-11-18 2023-05-19 脸萌有限公司 视频生成方法、装置、设备、存储介质及程序产品
CN114973085A (zh) * 2022-05-24 2022-08-30 入微智能科技(南京)有限公司 一种运动视频课程库建立方法、系统、设备及介质
CN116016817A (zh) * 2023-01-29 2023-04-25 北京达佳互联信息技术有限公司 视频剪辑方法、装置、电子设备及存储介质
CN119854429A (zh) * 2025-01-14 2025-04-18 北京字跳网络技术有限公司 特效视频生成方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN109525891B (zh) 2020-01-21
CN109525891A (zh) 2019-03-26

Similar Documents

Publication Publication Date Title
WO2020107908A1 (zh) 多用户视频特效添加方法、装置、终端设备及存储介质
US20210029305A1 (en) Method and apparatus for adding a video special effect, terminal device and storage medium
CN109462776B (zh) 一种视频特效添加方法、装置、终端设备及存储介质
CN111857923B (zh) 特效展示方法、装置、电子设备及计算机可读介质
US9626103B2 (en) Systems and methods for identifying media portions of interest
US12427432B2 (en) Game live broadcast interaction method and apparatus
CN112560605B (zh) 交互方法、装置、终端、服务器和存储介质
CN109474850B (zh) 运动像素视频特效添加方法、装置、终端设备及存储介质
TW202105331A (zh) 一種人體關鍵點檢測方法及裝置、電子設備和電腦可讀儲存介質
CN113630615B (zh) 直播间虚拟礼物展示方法及装置
CN109348277B (zh) 运动像素视频特效添加方法、装置、终端设备及存储介质
CN109600559B (zh) 一种视频特效添加方法、装置、终端设备及存储介质
WO2019100754A1 (zh) 人体动作的识别方法、装置和电子设备
CN110188719A (zh) 目标跟踪方法和装置
CN113920226B (zh) 用户交互方法、装置、存储介质及电子设备
CN120532141B (zh) 游戏辅助方法与装置
CN115393766A (zh) 视频处理方法、装置、电子设备和存储介质
CN110189364A (zh) 用于生成信息的方法和装置，以及目标跟踪方法和装置
CN114797096A (zh) 虚拟对象的控制方法、装置、设备及存储介质
WO2022260589A1 (zh) 触碰动画显示方法、装置、设备及介质
WO2023116562A1 (zh) 图像展示方法、装置、电子设备及存储介质
CN112733575B (zh) 图像处理方法、装置、电子设备及存储介质
CN115914498A (zh) 视频处理方法、装置、设备和存储介质
CN114979745B (zh) 视频处理方法、装置、电子设备及可读存储介质
CN118105689A (zh) 基于虚拟现实的游戏处理方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19891508

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 30.09.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19891508

Country of ref document: EP

Kind code of ref document: A1