CN109600559B - Video special effect adding method and device, terminal equipment and storage medium - Google Patents
Video special effect adding method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN109600559B CN109600559B CN201811447969.8A CN201811447969A CN109600559B CN 109600559 B CN109600559 B CN 109600559B CN 201811447969 A CN201811447969 A CN 201811447969A CN 109600559 B CN109600559 B CN 109600559B
- Authority
- CN
- China
- Prior art keywords
- video
- image frame
- special effect
- human body
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2621—Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The disclosure discloses a video special effect adding method and device, terminal equipment and a storage medium. The method comprises the following steps: acquiring at least one image frame in a video, and identifying at least one target human body joint point of a user in the image frame; if the target human body joint point identified in the target image frame meets the preset joint position condition, acquiring a video special effect matched with the joint position condition; adding a video special effect matched with the joint position condition at a video position in the video associated with the target image frame. The embodiment of the disclosure can add the matched dynamic special effect aiming at the joint point of the user, and improve the scene diversity of the video interaction application.
Description
Technical Field
The present disclosure relates to data technologies, and in particular, to a method and an apparatus for adding a video special effect, a terminal device, and a storage medium.
Background
With the development of communication technology and terminal devices, various terminal devices such as mobile phones, tablet computers, etc. have become an indispensable part of people's work and life, and with the increasing popularity of terminal devices, video interactive application has become a main channel for communication and entertainment.
Currently, video interactive applications are able to recognize a user's face and add still images to the user's head (e.g., add headwear to the hair) or add facial expressions to overlay the user's face. The method for adding the image is too limited, and meanwhile, the application scene is too single, so that the diversified requirements of users cannot be met.
Disclosure of Invention
The embodiment of the disclosure provides a video special effect adding method and device, a terminal device and a storage medium, which can add a matched dynamic special effect aiming at a joint point of a user and improve the scene diversification of video interaction application.
In a first aspect, an embodiment of the present disclosure provides a video special effect adding method, where the method includes:
acquiring at least one image frame in a video, and identifying at least one target human body joint point of a user in the image frame;
if the target human body joint point identified in the target image frame meets the preset joint position condition, acquiring a video special effect matched with the joint position condition;
adding a video special effect matched with the joint position condition at a video position in the video associated with the target image frame.
Further, the acquiring at least one image frame in the video includes:
in the video recording process, at least one image frame in the video is acquired in real time;
the adding of the video special effect matched with the joint position condition at the video position associated with the target image frame in the video comprises:
taking the video position of the target image frame as a special effect adding starting point;
adding the video special effect in an image frame matched with the special effect duration in the video from the special effect adding starting point according to the special effect duration of the video special effect matched with the joint position condition.
Further, the determining that the target human body joint point identified in the target image frame meets the preset joint position condition includes:
when the position of the target human body joint point in the image frame is in a set position range matched with the joint position condition and the position of the target human body joint point in the previous image frame of the image frame is not in the set position range, determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets a preset joint position condition.
Further, the determining that the target human body joint point identified in the target image frame meets the preset joint position condition includes:
when the position of the target human body joint point in the image frame is in the set position range matched with the joint position condition and the image frame does not comprise a video special effect matched with the joint position condition, determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets the preset joint position condition.
Further, the set position range includes a set plane position range or a set space position range.
Further, the video special effect adding method further includes:
in the recording process of the video, presenting image frames in the video in real time in a video preview interface;
adding the video special effect to the image frame matched with the special effect duration in the video, and simultaneously:
and presenting the image frames added with the video special effect in real time in the video preview interface.
Further, the video special effects include: dynamic animation effects, and/or musical effects;
the presenting, in the video preview interface, the image frame added with the video special effect in real time includes:
and in the video preview interface, drawing a dynamic animation special effect in the image frame in real time, and playing a music special effect.
In a second aspect, an embodiment of the present disclosure further provides a video special effect adding apparatus, where the apparatus includes:
the target human body joint point identification module is used for acquiring at least one image frame in a video and identifying at least one target human body joint point of a user in the image frame;
the video special effect acquisition module is used for acquiring a video special effect matched with a joint position condition if the target human body joint point identified in the target image frame is determined to meet the preset joint position condition;
and the video special effect adding module is used for adding a video special effect matched with the joint position condition at a video position in the video associated with the target image frame.
Further, the target human joint point identification module includes:
the image frame real-time acquisition module is used for acquiring at least one image frame in the video in real time in the video recording process;
the video special effect adding module comprises:
the special effect adding starting point determining module is used for taking the video position of the target image frame as a special effect adding starting point;
and the video special effect real-time adding module is used for adding the video special effect in an image frame matched with the special effect duration in the video from the special effect adding starting point according to the special effect duration of the video special effect matched with the joint position condition.
Further, the video special effect obtaining module includes: and the target human body joint point entering state judging module is used for determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets the preset joint position condition when the position of the target human body joint point in the image frame is in the set position range matched with the joint position condition and the position of the target human body joint point in the previous image frame of the image frame is not in the set position range.
Further, the video special effect obtaining module includes: and the target human body joint point position state judging module is used for determining the image frame as a target image frame when the position of the target human body joint point in the image frame is in a set position range matched with the joint position condition and the image frame does not comprise a video special effect matched with the joint position condition, and meanwhile, determining that the target human body joint point identified in the target image frame meets a preset joint position condition.
Further, the set position range includes a set plane position range or a set space position range.
Further, the video special effect adding apparatus further includes:
the image frame real-time presenting module is used for presenting the image frames in the video in real time in a video preview interface in the recording process of the video;
and the video special effect real-time presenting module is used for presenting the image frames added with the video special effect in real time in the video preview interface.
Further, the video special effects include: dynamic animation effects, and/or musical effects;
the video special effect real-time presentation module comprises: and the special effect display and play module is used for drawing a dynamic animation special effect in real time in the image frame in the video preview interface and playing a music special effect.
In a third aspect, an embodiment of the present disclosure further provides a terminal device, where the terminal device includes:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a video special effects addition method as described in embodiments of the present disclosure.
In a fourth aspect, the disclosed embodiments also provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the video special effect adding method according to the disclosed embodiments.
According to the embodiment of the method and the device, the target human body joint point identified in the image frame of the video meets the joint position condition, the action special effect matched with the joint position condition is added to the video, the problem that the video special effect of video interaction application is too single is solved, the video special effect is added according to the action of the user, and the flexibility of the video for increasing the special effect is improved.
Drawings
Fig. 1a is a flowchart of a video special effect adding method according to an embodiment of the present disclosure;
FIG. 1b is a schematic view of a human joint according to an embodiment of the present disclosure;
fig. 2a is a flowchart of a video special effect adding method according to a second embodiment of the disclosure;
fig. 2b is a schematic diagram of a set position range according to a second embodiment of the disclosure;
fig. 2c is a schematic diagram of another set position range provided in the second embodiment of the present disclosure;
FIG. 2d is a schematic view of a joint point of a human body according to a second embodiment of the disclosure;
FIG. 2e is a schematic view of another human joint point provided in the second embodiment of the present disclosure;
fig. 3 is a flowchart of a video special effect adding method according to a third embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a video special effect adding apparatus according to a fourth embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a terminal device provided in the fifth embodiment of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the disclosure and are not limiting of the disclosure. It should be further noted that, for the convenience of description, only some of the structures relevant to the present disclosure are shown in the drawings, not all of them.
Example one
Fig. 1a is a flowchart of a video special effect adding method according to an embodiment of the present disclosure, where the present embodiment is applicable to a case of adding a video special effect in a video, and the method may be executed by a video special effect adding apparatus, which may be implemented in a software and/or hardware manner, and the apparatus may be configured in a terminal device, such as a computer, for example. As shown in fig. 1a, the method specifically includes the following steps:
s110, at least one image frame in the video is obtained, and at least one target human body joint point of the user is identified in the image frame.
In general, video is formed by a series of still image frames that are projected in succession at extremely fast speeds. Therefore, the video can be split into a series of image frames, and the image frames are edited, so that the video is edited. When a plurality of users exist in the image frame, one of the users can be selected as an object to be added with a video special effect subsequently according to the recognition completeness and the confidence of the joint point of each user or the distance between each user and the video shooting device. The human body joint points are used for determining the action state of the user in the image frame, such as the action state of standing, bowing or jumping, and for determining the position information of the user, such as the distance between the user and the terminal device, the relative position of the user and other objects shot by the terminal device, or the position of the user in the picture shot by the terminal device.
In a specific example, as shown in fig. 1b, in the mobile terminal, the human body contour is specifically shown as the figure, wherein a circle in the human body contour represents the identified human body joint point, and a line between two human body joint points is used for representing a body part of the human body, for example, a line between a wrist joint point and an elbow joint point is used for representing an arm between a wrist and an elbow.
The human body joint point recognition operation is performed on each image frame, so that all human body regions can be recognized in the image frame, and specifically, the image frame is subjected to image segmentation according to depth information (the depth information can be acquired by an infrared camera) contained in the image frame, so that all human body regions in the image frame are recognized. The method includes selecting a human body region from all human body regions for identifying human body joint points, specifically selecting the human body region with the shortest distance as a user needing to identify the human body joint points according to the distance between the human body region and a display screen of the terminal device, and determining the human body joint points in other modes, which is not limited specifically. After the human body region is determined, human body joint point identification is carried out on the human body region, all human body joint points belonging to the user are determined, and at least one target human body joint point can be further screened out from all human body joint points of the user according to requirements.
The method for identifying the human body joint points specifically comprises the following steps: determining body part areas (arms, hands, thighs, feet and the like) belonging to the body area in the body area, calculating positions of joint points (elbows, wrists, knees and the like) in each body part area, finally generating a body skeleton system according to the positions of the identified joint points, and determining target body joint points according to needs. In addition, the motion state or position information of a certain body part area of the user can be further judged by adopting a connecting line of two target body joint points (for example, the connecting line between a wrist joint point and an elbow joint point is used for representing an arm between a wrist and an elbow), for example, a vector of a line segment formed by two points is determined by coordinates of the two target body joint points, the related human body recognition, the body part area recognition and the joint point position calculation in the body part area can be realized by adopting a pre-trained deep learning model, and the deep learning model can be trained according to a depth characteristic extracted from the human body depth information.
It should be noted that there are other methods for identifying human body joint points, and the embodiments of the present disclosure are not particularly limited.
S120, judging whether the identified target human body joint point meets a preset joint position condition or not in the image frames selected from the at least one image frame until all the at least one image frame is judged, and if so, executing S130; otherwise, S140 is performed.
The target human body joint points identified by all image frames in the video need to be judged, and specifically, the judgment can be performed by selecting the image frames one by one.
The joint position condition may refer to a position requirement of at least one target human joint point, which is used to start adding a video special effect, for example, placing the left hand at a center position of a picture shot by a camera, and may also refer to a relative position requirement of two target human joint points, for example, placing the left hand in a region where a left eye joint point is located, and there are other specific position regions, which is not particularly limited to this disclosed embodiment.
It should be noted that, one joint position condition corresponds to one set position range and corresponds to one video special effect at the same time, if two joint position conditions exist, and when two target human body joint points respectively satisfy the set position ranges corresponding to the two joint position conditions, two video special effects corresponding to the two joint position conditions may be added in the image frame at the same time.
S130, taking the image frame corresponding to the target human body joint point meeting the preset joint position condition as a target image frame, acquiring a video special effect matched with the joint position, and executing S150.
When the joint position condition is met, a video special effect matched with the joint position condition is added in the video from the current image frame. The video special effect is used for adding a special effect matched according to the user action in the target image frame so as to realize interaction with the user, specifically, the special effect can be an animation special effect and/or a music special effect, the animation special effect is added to be used for simultaneously drawing a static image and/or a dynamic image to cover the original content of the target image frame in the display process of the target image frame, and the music special effect is added to be used for simultaneously playing music in the display process of the target image frame.
And S140, acquiring the next image frame and returning to execute S120.
S150, adding a video special effect matched with the joint position condition at the video position associated with the target image frame in the video.
The video position is used to represent the position of the image frame in the video. The image frames split from the video can be arranged according to the video playing sequence, so that the video position can also be used for representing the playing time of the image frames in the video playing process, and the playing time can refer to the specific time relative to the starting time of video playing. A series of image frames split from a video can be numbered according to a playing sequence, specifically: the first played image frame is the 1 st frame, the image frame played after the 1 st frame image frame is the 2 nd frame, and so on, all the image frames split in the video are numbered. For example, the video may be split into 100 frames, each image frame corresponds to a sequence number, and specifically, the target image frame may be the 50 th frame.
After determining the video position of the target image frame, a video special effect is added at the video position. In fact, the video special effect can be represented in a code form, and the video special effect is added at the video position, that is, the code segment corresponding to the video special effect is added in the code segment corresponding to the target image frame, so that the video special effect is added in the target image frame.
According to the method and the device, the target human body joint point identified in the image frame of the video meets the joint position condition, the action special effect matched with the joint position condition is added to the video, the problem that the video special effect of video interaction application is too single is solved, the video special effect is added according to the action of the user, the richness of the video interaction application is improved, and meanwhile the flexibility of the video for increasing the special effect is improved.
Example two
Fig. 2a is a flowchart of a video special effect adding method according to a second embodiment of the disclosure. The present embodiment is embodied on the basis of various alternatives in the above-described embodiments. In this embodiment, at least one image frame in the captured video is embodied as: and in the video recording process, at least one image frame in the video is acquired in real time. Meanwhile, adding a video special effect matched with the joint position condition at the video position associated with the target image frame in the video is embodied as follows: taking the video position of the target image frame as a special effect adding starting point; adding the video special effect in an image frame matched with the special effect duration in the video from the special effect adding starting point according to the special effect duration of the video special effect matched with the joint position condition.
Correspondingly, the method of the embodiment may include:
s210, in the video recording process, at least one image frame in the video is obtained in real time, and at least one target human body joint point of a user is identified in the image frame.
The video can be shot in real time, and each image frame in the video can be acquired in real time.
The video, the image frame, the target human joint point, the joint position condition, the video position, the video special effect, and the like in the present embodiment can all refer to the description in the above embodiments.
S220, judging whether the identified target human body joint point meets a preset joint position condition or not in the image frames selected from the at least one image frame until all the at least one image frame is judged, and if so, executing S230; otherwise, S240 is performed.
And S230, taking the image frame corresponding to the target human body joint point meeting the preset joint position condition as a target image frame, acquiring a video special effect matched with the joint position condition, and executing S250.
The target human body joint point satisfies a preset joint position condition, which may mean that at least one target human body joint point is continuously located within a set position range, or that at least one target human body joint point enters or exits from the set position range. Specifically, the set position range includes a set plane position range or a set spatial position range, where the set plane position range may refer to a position range in a plane that is the same as or parallel to the video shooting plane, and when the position of the shot object mapped to the plane is within the set position range, it is determined that the object satisfies the set plane position range; and the set spatial position range refers to a position range in a space photographed in the video, and when the photographed object is within the set spatial position range, it is determined that the object satisfies the set spatial position range. That is, the set plane position range does not contain depth information, while the set spatial position range contains depth information. In a specific example, three boxes are set planar position ranges as shown in fig. 2b, and three boxes are set spatial position ranges within a cube as shown in fig. 2 c.
Optionally, the determining that the target human body joint point identified in the target image frame meets the preset joint position condition may include: when the position of the target human body joint point in the image frame is in a set position range matched with the joint position condition and the position of the target human body joint point in the previous image frame of the image frame is not in the set position range, determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets a preset joint position condition.
Specifically, the entering state of the target human body joint point into the set position range is used as a preset joint position condition, and when the target human body joint point is in the set position range in the current image frame and is not in the set position range in the previous image frame of the current image frame, the target human body joint point is determined to enter the set position range from outside the set position range, so that the target human body joint point is determined to have the entering state aiming at the set position range, and the target human body joint point is determined to meet the preset joint position condition. In a specific example, as shown in fig. 2 d-2 e, the set position range is 5 dashed rectangles, the sizes of all the rectangular areas may not be all the same, and correspondingly, the video effects corresponding to all the rectangular areas may be the same or not may be all the same, the left-hand wrist joint point in fig. 2d is outside the set position range, the left-hand wrist joint point in fig. 2e is within the set position range, and when the position of the left-hand wrist joint point of the user is changed from the position shown in fig. 2d to the position shown in fig. 2e, it is determined that the left-hand wrist joint point of the user enters the set position range from outside the set position range, so as to determine that the left-hand wrist joint point of the user meets the preset joint position condition.
Optionally, the determining that the target human body joint point identified in the target image frame meets the preset joint position condition may include: when the position of the target human body joint point in the image frame is in the set position range matched with the joint position condition and the image frame does not comprise a video special effect matched with the joint position condition, determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets the preset joint position condition.
Specifically, the position state in which the target human body joint point is kept within the set position range is taken as a preset joint position condition. When the target human body joint point is always in the set position range, if the video special effect is added or updated all the time, the video special effect is displayed repeatedly and rapidly, so that a user cannot distinguish each video special effect, for example, 50 continuous frames, each image frame starts to display the same video special effect, and at the moment, the user views the superposed picture of the same animation special effect and simultaneously hears the superposed sound effect of the music special effect. Therefore, when the target human body joint point is always within the set position range and the corresponding video special effect is played or the video special effect does not exist, the video special effect is added to avoid fast and repeatedly displaying the same video special effect. In a specific example, as shown in fig. 2e, the wrist joint point of the user is located within a set position range, and if there is no video special effect corresponding to the set position range in the image frame at this time, a video special effect is correspondingly added; if the image frame has the video special effect corresponding to the set position range, no processing is performed.
The joint position condition may be a condition in which the target human body joint point is determined to satisfy the joint position condition when all of the plurality of target human body joint points satisfy the joint position condition.
S240, acquiring a next image frame, and returning to S220.
And S250, taking the video position of the target image frame as a special effect adding starting point, and adding the video special effect in an image frame matched with the special effect duration in the video from the special effect adding starting point according to the special effect duration of the video special effect matched with the joint position condition.
The special effect addition starting point may refer to a starting position and/or a starting time of video special effect addition. The effect duration may refer to the time elapsed between the start position to the end position or the time between the start time to the end time of the video effect. The image frames that match the special effect duration may refer to all image frames in the video between the start of the special effect addition, i.e. from the target image frame, and the corresponding end image frame up to the end of the video special effect. For example, if the video special effect is a music special effect, and the duration of a music special effect is 3s, in the video, 30 image frames are played in 1s, and 90 image frames (including the target image frame) from the target image frame are image frames matched with the duration of the special effect in the video playing sequence.
According to the video recording method and device, the video is shot in real time, the series of image frames split from the video are obtained in real time, whether the joint point of the target human body in the shot video meets the joint position condition or not is judged in real time, and further, the video special effect is added in real time under the condition that the joint position condition is met, so that the video special effect can be added while the video is recorded, and the adding efficiency of the video special effect is improved.
On the basis of the foregoing embodiment, optionally, the video special effect adding method may further include: in the recording process of the video, presenting image frames in the video in real time in a video preview interface; correspondingly, when the video special effect is added to the image frame matched with the special effect duration in the video, the method further comprises the following steps: and presenting the image frames added with the video special effect in real time in the video preview interface.
The video preview interface may refer to an interface of a terminal device for a user to browse a video, where the terminal device may include a server-side device or a client-side device. The video is displayed in the video preview interface in real time while the video is shot in real time, so that the user can browse the content of the shot video in real time.
When the video special effect is added in real time, the video special effect is displayed in a video preview interface along with the video, so that a user can browse the video with the video effect added in real time, the efficiency of adding the video special effect is improved, and the user experience is improved.
Optionally, the video special effect includes: dynamic animation effects, and/or musical effects; correspondingly, the presenting, in the video preview interface, the image frame to which the video special effect is added in real time may include: and in the video preview interface, drawing a dynamic animation special effect in the image frame in real time, and playing a music special effect.
Specifically, when the video effect includes a dynamic animated effect, the dynamic animated effect is drawn in an image frame displayed in real time, for example, at least one image of a musical instrument, a background, a character, and the like is drawn. When the video special effect comprises a music special effect, the music special effect is played while the image frame is displayed in real time. The diversity of the video special effects is improved by setting the video special effects to include dynamic animation special effects and/or music special effects.
EXAMPLE III
Fig. 3 is a flowchart of a video special effect adding method according to a third embodiment of the present disclosure. The present embodiment is embodied on the basis of various alternatives in the above-described embodiments. In this embodiment, the determining that the target human joint point identified in the target image frame satisfies the preset joint position condition is embodied in determining the image frame as the target image frame when the position of the target human joint point in the image frame is within the set position range and the position of the target human joint point in the previous image frame of the image frame is not within the set position range matching the joint position condition, and simultaneously determining that the target human joint point identified in the target image frame satisfies the preset joint position condition.
Correspondingly, the method of the embodiment may include:
s310, in the video recording process, at least one image frame in the video is obtained in real time, the image frame in the video is presented in real time in a video preview interface, and at least one target human body joint point of a user is identified in the image frame.
The video, the image frame, the target human body joint point, the joint position condition, the video position, the video special effect, the set position range, and the like in the present embodiment can all refer to the description in the above embodiments.
S320, judging whether the position of the target human body joint point in the image frame is in a set position range matched with the joint position condition or not in the image frame selected from the at least one image frame until the at least one image frame is completely judged, and if so, executing S330; otherwise, S340 is performed.
S330, judging whether the position of the target human body joint point in the previous image frame of the image frame is in the set position range, if so, executing S340; otherwise, S350 is performed.
And S340, acquiring the next image frame, and returning to the step S320.
And S350, taking the image frame as a target image frame, and acquiring a video special effect matched with the joint position condition.
And S360, taking the video position of the target image frame as a special effect adding starting point, adding the video special effect in the image frame matched with the special effect duration in the video from the special effect adding starting point according to the special effect duration of the video special effect matched with the joint position condition, and presenting the image frame added with the video special effect in real time in the video preview interface.
Example four
Fig. 4 is a schematic structural diagram of a video special effect adding apparatus according to an embodiment of the present disclosure, which is applicable to a case of adding a video special effect in a video. The apparatus may be implemented in software and/or hardware, and may be configured in a terminal device. As shown in fig. 4, the apparatus may include: a target human body joint point recognition module 410, a video special effect acquisition module 420 and a video special effect addition module 430.
A target human body joint point identification module 410, configured to acquire at least one image frame in a video, and identify at least one target human body joint point of a user in the image frame;
a video special effect obtaining module 420, configured to obtain a video special effect matched with a joint position condition if it is determined that a target human body joint point identified in a target image frame meets a preset joint position condition;
a video special effect adding module 430, configured to add a video special effect matching the joint position condition at a video position in the video associated with the target image frame.
According to the embodiment of the method and the device, the target human body joint point identified in the image frame of the video meets the joint position condition, the action special effect matched with the joint position condition is added to the video, the problem that the video special effect of video interaction application is too single is solved, the video special effect is added according to the action of the user, and the flexibility of the video for increasing the special effect is improved.
Further, the target human joint point identification module 410 includes: the image frame real-time acquisition module is used for acquiring at least one image frame in the video in real time in the video recording process; the video special effect adding module 430 includes: the special effect adding starting point determining module is used for taking the video position of the target image frame as a special effect adding starting point; and the video special effect real-time adding module is used for adding the video special effect in an image frame matched with the special effect duration in the video from the special effect adding starting point according to the special effect duration of the video special effect matched with the joint position condition.
Further, the video special effect obtaining module 420 includes: and the target human body joint point entering state judging module is used for determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets a preset joint position condition when the position of the target human body joint point in the image frame is within the set position range and the position of the target human body joint point in the previous image frame of the image frame is not within the set position range matched with the joint position condition.
Further, the video special effect obtaining module 420 includes: and the target human body joint point position state judging module is used for determining the image frame as a target image frame when the position of the target human body joint point in the image frame is in a set position range matched with the joint position condition and the image frame does not comprise a video special effect matched with the joint position condition, and meanwhile, determining that the target human body joint point identified in the target image frame meets a preset joint position condition.
Further, the set position range includes a set plane position range or a set space position range.
Further, the video special effect adding apparatus further includes: the image frame real-time presenting module is used for presenting the image frames in the video in real time in a video preview interface in the recording process of the video; and the video special effect real-time presenting module is used for presenting the image frames added with the video special effect in real time in the video preview interface.
Further, the video special effects include: dynamic animation effects, and/or musical effects;
the video special effect real-time presentation module comprises: and the special effect display and play module is used for drawing a dynamic animation special effect in real time in the image frame in the video preview interface and playing a music special effect.
The video special effect adding device provided by the embodiment of the disclosure and the video special effect adding method provided by the first embodiment belong to the same inventive concept, and technical details which are not described in detail in the embodiment of the disclosure can be referred to in the first embodiment, and the first embodiment and the second embodiment of the disclosure have the same beneficial effects.
EXAMPLE five
The present disclosure provides a terminal device, and referring to fig. 5 below, a schematic structural diagram of an electronic device (e.g., a client or a server) 500 suitable for implementing the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a Personal Digital Assistant (PDA), a tablet computer (PAD), a Portable Multimedia Player (PMP), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 5, electronic device 500 may include a processing means (e.g., central processing unit, graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Generally, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and a communication device 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 illustrates an electronic device 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or installed from the storage means 508, or installed from the ROM 502. The computer program performs the above-described functions defined in the methods of the embodiments of the present disclosure when executed by the processing device 501.
EXAMPLE six
Embodiments of the present disclosure also provide a computer readable storage medium, which may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, Radio Frequency (RF), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring at least one image frame in a video, and identifying at least one target human body joint point of a user in the image frame; if the target human body joint point identified in the target image frame meets the preset joint position condition, acquiring a video special effect matched with the joint position condition; adding a video special effect matched with the joint position condition at a video position in the video associated with the target image frame.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a module does not in some cases constitute a limitation of the module itself, for example, the target human joint identification module may also be described as a "module that takes at least one image frame in a video and identifies at least one target human joint of a user in said image frame".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Claims (16)
1. A video special effect adding method, comprising:
acquiring at least one image frame in a video, and identifying at least one target human body joint point of a user in the image frame;
if the target human body joint point identified in the target image frame meets the preset joint position condition, acquiring a video special effect matched with the joint position condition; wherein the target human body joint point satisfying the preset joint position condition comprises: at least one target human body joint point is continuously positioned in the set position range, or at least one target human body joint point enters or exits the set position range;
adding a video special effect matched with the joint position condition at a video position in the video associated with the target image frame;
the adding of the video special effect matched with the joint position condition at the video position associated with the target image frame in the video comprises:
taking the video position of the target image frame as a special effect adding starting point;
adding the video special effect in an image frame matched with the special effect duration in the video from the special effect adding starting point according to the special effect duration of the video special effect matched with the joint position condition.
2. The method of claim 1, wherein said obtaining at least one image frame in a video comprises:
and in the video recording process, at least one image frame in the video is acquired in real time.
3. The method of claim 1, wherein determining that the target human joint identified in the target image frame satisfies a preset joint position condition comprises:
when the position of the target human body joint point in the image frame is in a set position range matched with the joint position condition and the position of the target human body joint point in the previous image frame of the image frame is not in the set position range, determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets a preset joint position condition.
4. The method of claim 1, wherein determining that the target human joint identified in the target image frame satisfies a preset joint position condition comprises:
when the position of the target human body joint point in the image frame is in the set position range matched with the joint position condition and the image frame does not comprise a video special effect matched with the joint position condition, determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets the preset joint position condition.
5. The method of claim 3 or 4, wherein the set position range comprises a set planar position range or a set spatial position range.
6. The method of claim 2, further comprising:
in the recording process of the video, presenting image frames in the video in real time in a video preview interface;
adding the video special effect to the image frame matched with the special effect duration in the video, and simultaneously:
and presenting the image frames added with the video special effect in real time in the video preview interface.
7. The method of claim 6, wherein the video special effect comprises: dynamic animation effects, and/or musical effects;
the presenting, in the video preview interface, the image frame added with the video special effect in real time includes:
and in the video preview interface, drawing a dynamic animation special effect in the image frame in real time, and playing a music special effect.
8. A video special effect adding apparatus, comprising:
the target human body joint point identification module is used for acquiring at least one image frame in a video and identifying at least one target human body joint point of a user in the image frame;
the video special effect acquisition module is used for acquiring a video special effect matched with a joint position condition if the target human body joint point identified in the target image frame is determined to meet the preset joint position condition; wherein the target human body joint point satisfying the preset joint position condition comprises: at least one target human body joint point is continuously positioned in the set position range, or at least one target human body joint point enters or exits the set position range;
the video special effect adding module is used for adding a video special effect matched with the joint position condition at a video position in the video related to the target image frame;
the video special effect adding module comprises:
the special effect adding starting point determining module is used for taking the video position of the target image frame as a special effect adding starting point;
and the video special effect real-time adding module is used for adding the video special effect in an image frame matched with the special effect duration in the video from the special effect adding starting point according to the special effect duration of the video special effect matched with the joint position condition.
9. The apparatus of claim 8, wherein the target human joint identification module comprises:
and the image frame real-time acquisition module is used for acquiring at least one image frame in the video in real time in the video recording process.
10. The apparatus of claim 8, wherein the video special effects acquisition module comprises:
and the target human body joint point entering state judging module is used for determining the image frame as a target image frame and simultaneously determining that the target human body joint point identified in the target image frame meets the preset joint position condition when the position of the target human body joint point in the image frame is in the set position range matched with the joint position condition and the position of the target human body joint point in the previous image frame of the image frame is not in the set position range.
11. The apparatus of claim 8, wherein the video special effects acquisition module comprises:
and the target human body joint point position state judging module is used for determining the image frame as a target image frame when the position of the target human body joint point in the image frame is in a set position range matched with the joint position condition and the image frame does not comprise a video special effect matched with the joint position condition, and meanwhile, determining that the target human body joint point identified in the target image frame meets a preset joint position condition.
12. The apparatus of claim 10 or 11, wherein the set position range comprises a set planar position range or a set spatial position range.
13. The apparatus of claim 9, further comprising:
the image frame real-time presenting module is used for presenting the image frames in the video in real time in a video preview interface in the recording process of the video;
and the video special effect real-time presenting module is used for presenting the image frames added with the video special effect in real time in the video preview interface.
14. The apparatus of claim 13, wherein the video effect comprises: dynamic animation effects, and/or musical effects;
the video special effect real-time presentation module comprises:
and the special effect display and play module is used for drawing a dynamic animation special effect in real time in the image frame in the video preview interface and playing a music special effect.
15. A terminal device, comprising:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the video effects addition method of any of claims 1-7.
16. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a video effect addition method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811447969.8A CN109600559B (en) | 2018-11-29 | 2018-11-29 | Video special effect adding method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811447969.8A CN109600559B (en) | 2018-11-29 | 2018-11-29 | Video special effect adding method and device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109600559A CN109600559A (en) | 2019-04-09 |
| CN109600559B true CN109600559B (en) | 2021-07-23 |
Family
ID=65959384
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811447969.8A Active CN109600559B (en) | 2018-11-29 | 2018-11-29 | Video special effect adding method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109600559B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112106347A (en) * | 2019-08-30 | 2020-12-18 | 深圳市大疆创新科技有限公司 | Image generation method, image generation equipment, movable platform and storage medium |
| CN110582021B (en) * | 2019-09-26 | 2021-11-05 | 深圳市商汤科技有限公司 | Information processing method and device, electronic equipment and storage medium |
| CN110740262A (en) * | 2019-10-31 | 2020-01-31 | 维沃移动通信有限公司 | Background music adding method and device and electronic equipment |
| CN111611941B (en) * | 2020-05-22 | 2023-09-19 | 腾讯科技（深圳）有限公司 | Special effect processing method and related equipment |
| CN112560605B (en) * | 2020-12-02 | 2023-04-18 | 北京字节跳动网络技术有限公司 | Interaction method, device, terminal, server and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002298557A (en) * | 2001-03-30 | 2002-10-11 | Fantekkusu:Kk | System for editing nonlinear video, recording medium and photograph holder |
| EP2028619A1 (en) * | 2006-05-25 | 2009-02-25 | NEC Corporation | Video image special effect detecting device, special effect detecting method, special effect detecting program and video image reproducing device |
| CN103020648A (en) * | 2013-01-09 | 2013-04-03 | 北京东方艾迪普科技发展有限公司 | Method and device for identifying action types, and method and device for broadcasting programs |
| CN107038455A (en) * | 2017-03-22 | 2017-08-11 | 腾讯科技（深圳）有限公司 | A kind of image processing method and device |
| WO2018053257A1 (en) * | 2016-09-16 | 2018-03-22 | Second Spectrum, Inc. | Methods and systems of spatiotemporal pattern recognition for video content development |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108182396B (en) * | 2017-12-25 | 2022-06-28 | 中国电子科技集团公司电子科学研究院 | Method and device for automatically identifying photographing behavior |
| CN108289180B (en) * | 2018-01-30 | 2020-08-21 | 广州市百果园信息技术有限公司 | Method, medium, and terminal device for processing video according to body movement |
| CN108537867B (en) * | 2018-04-12 | 2020-01-10 | 北京微播视界科技有限公司 | Video rendering method and device according to user limb movement |
| CN108615055B (en) * | 2018-04-19 | 2021-04-27 | 咪咕动漫有限公司 | Similarity calculation method and device and computer readable storage medium |
| CN108712661B (en) * | 2018-05-28 | 2022-02-25 | 广州虎牙信息科技有限公司 | Live video processing method, device, equipment and storage medium |
-
2018
- 2018-11-29 CN CN201811447969.8A patent/CN109600559B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002298557A (en) * | 2001-03-30 | 2002-10-11 | Fantekkusu:Kk | System for editing nonlinear video, recording medium and photograph holder |
| EP2028619A1 (en) * | 2006-05-25 | 2009-02-25 | NEC Corporation | Video image special effect detecting device, special effect detecting method, special effect detecting program and video image reproducing device |
| CN103020648A (en) * | 2013-01-09 | 2013-04-03 | 北京东方艾迪普科技发展有限公司 | Method and device for identifying action types, and method and device for broadcasting programs |
| WO2018053257A1 (en) * | 2016-09-16 | 2018-03-22 | Second Spectrum, Inc. | Methods and systems of spatiotemporal pattern recognition for video content development |
| CN107038455A (en) * | 2017-03-22 | 2017-08-11 | 腾讯科技（深圳）有限公司 | A kind of image processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109600559A (en) | 2019-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109462776B (en) | Video special effect adding method and device, terminal equipment and storage medium | |
| US20210029305A1 (en) | Method and apparatus for adding a video special effect, terminal device and storage medium | |
| CN109600559B (en) | Video special effect adding method and device, terminal equipment and storage medium | |
| CN109525891B (en) | Multi-user video special effect adding method and device, terminal equipment and storage medium | |
| US20200234478A1 (en) | Method and Apparatus for Processing Information | |
| WO2020186935A1 (en) | Virtual object displaying method and device, electronic apparatus, and computer-readable storage medium | |
| CN109474850B (en) | Motion pixel video special effect adding method and device, terminal equipment and storage medium | |
| CN110188719B (en) | Target tracking method and device | |
| CN109348277B (en) | Motion pixel video special effect adding method and device, terminal equipment and storage medium | |
| CN112199016B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN113467603B (en) | Audio processing method and device, readable medium and electronic equipment | |
| US20230421716A1 (en) | Video processing method and apparatus, electronic device and storage medium | |
| CN110163066B (en) | Multimedia data recommendation method, device and storage medium | |
| WO2022007565A1 (en) | Image processing method and apparatus for augmented reality, electronic device and storage medium | |
| CN112672185B (en) | Augmented reality-based display method, device, equipment and storage medium | |
| CN109982130A (en) | A kind of video capture method, apparatus, electronic equipment and storage medium | |
| CN110930220A (en) | Display method, display device, terminal equipment and medium | |
| CN114630057B (en) | Method and device for determining special effect video, electronic equipment and storage medium | |
| CN108537149B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN110189364B (en) | Method and device for generating information, and target tracking method and device | |
| CN111447379B (en) | Method and device for generating information | |
| CN109636917B (en) | Three-dimensional model generation method, device and hardware device | |
| CN116527993A (en) | Video processing method, apparatus, electronic device, storage medium and program product | |
| CN114697568A (en) | Special effect video determination method and device, electronic equipment and storage medium | |
| CN110188833B (en) | Method and apparatus for training a model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |