CN111611941A - Special effect processing method and related equipment - Google Patents

Special effect processing method and related equipment Download PDF

Info

Publication number
CN111611941A
CN111611941A CN202010443569.0A CN202010443569A CN111611941A CN 111611941 A CN111611941 A CN 111611941A CN 202010443569 A CN202010443569 A CN 202010443569A CN 111611941 A CN111611941 A CN 111611941A
Authority
CN
China
Prior art keywords
action
special effect
accuracy
initialized
sequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010443569.0A
Other languages
Chinese (zh)
Other versions
CN111611941B (en
Inventor
吴雪蕾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202010443569.0A priority Critical patent/CN111611941B/en
Publication of CN111611941A publication Critical patent/CN111611941A/en
Application granted granted Critical
Publication of CN111611941B publication Critical patent/CN111611941B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Social Psychology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to the technical field of artificial intelligence, and provides a special effect processing method and related equipment, wherein the method comprises the following steps: carrying out image detection on an original video to obtain at least two target video frames including a moving target; performing motion recognition on the moving target based on at least two target video frames; when an action recognition result is obtained, determining an action sequence of the initialized special effect; performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining the action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining the special effect accuracy of the initialized special effect according to the action accuracy obtained by calculation; and acquiring a target special effect associated with the initialized special effect based on the special effect accuracy for displaying. The implementation of the method and the device improves the accuracy of special effect matching and enhances the interactivity of the special effect matching process.

Description

Special effect processing method and related equipment
Technical Field
The application relates to the technical field of artificial intelligence, in particular to a special effect processing method and related equipment.
Background
When the video shooting is carried out by adopting the equipment at present, if a special effect is added, the video editor is adopted for editing after the video shooting is finished, and an additional function carried by a shooting application program can be adopted for processing in the video shooting process; such an additional function for adding a special effect is generally to match the special effect according to the motion by recognizing the motion of the target object shown in the captured video image.
However, in the prior art, when the motion is recognized, only a single motion is generally recognized, and then the special effect is matched based on the single motion, and the method for matching the special effect has low accuracy and weak interactivity.
Disclosure of Invention
The application provides a special effect processing method and related equipment, which can solve at least one technical problem. The technical scheme is as follows:
in a first aspect, a special effect processing method is provided, including: carrying out image detection on an original video to obtain at least two target video frames including a moving target; performing motion recognition on the moving target based on the at least two target video frames; when an action recognition result is obtained, determining an action sequence of the initialized special effect; performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining special effect accuracy of the initialized special effect according to the calculated action accuracy; and acquiring a target special effect associated with the initialization special effect based on the special effect accuracy for displaying.
With reference to the first aspect, in a first implementation manner of the first aspect, the performing motion recognition on the moving object based on the at least two target video frames includes any one of: sequentially identifying the action change trend of the moving target in two adjacent front and back target video frames to finish the action identification of the moving target; and selecting at least two video frames to be identified from the at least two target video frames at a preset frequency, and identifying the action change trend of the moving target in the at least two video frames to be identified so as to finish the action identification of the moving target.
With reference to the first aspect, in a second implementation manner of the first aspect, the determining, when the motion recognition result is obtained, a motion sequence for initializing a special effect includes any one of: when an action recognition result is obtained, determining at least one action sequence of the initialized special effect based on the first action in the action recognition result according to time sequence; when the action recognition result is obtained, taking a preset action sequence of the special effect as an action sequence of the initialized special effect; the preset special effects comprise at least one.
With reference to the first aspect, in a third implementation manner of the first aspect, the performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and a corresponding action matched in the action recognition result, and determining the special effect accuracy of the initialized special effect according to the calculated action accuracy includes: sequentially matching each action in the action sequence of the initialized special effect with each action in the action recognition result according to time sequence; respectively calculating action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result; and when each action in the action sequence of the initialized special effect is matched with the corresponding action in the action recognition result, determining the special effect accuracy of the initialized special effect according to the calculated action accuracy.
With reference to the first aspect, in a fourth implementation manner of the first aspect, the action accuracy includes an action point location accuracy, an action trigger time accuracy, and an action duration accuracy; determining a special effect accuracy of the initialized special effect according to the calculated action accuracy, comprising: calculating the sum of the action point positions, the sum of the action trigger time and the sum of the action duration time of all actions in the action sequence of the initialized special effect; and calculating the sum of the accuracy of the point positions of the actions, the sum of the accuracy of the triggering time of the actions and the sum of the accuracy of the duration time of the actions based on preset weights to obtain the accuracy of the special effect of the initialization special effect.
With reference to the second implementation manner of the first aspect, in a fifth implementation manner of the first aspect, when an initialization special effect is included, the obtaining, based on the special effect accuracy, a target special effect associated with the initialization special effect for presentation includes: and adjusting the display effect of the initialized special effect based on the special effect accuracy, and displaying the initialized special effect after the display effect is adjusted as a target special effect.
With reference to the second implementation manner of the first aspect, in a sixth implementation manner of the first aspect, when at least two initialization effects are included, the obtaining, based on the effect accuracy, a target effect associated with the initialization effects for presentation includes: and determining the initialized special effect with the highest special effect accuracy as a target special effect, and displaying the target special effect.
In a second aspect, there is provided a special effect processing apparatus including: the detection module is used for carrying out image detection on an original video and acquiring at least two target video frames comprising moving targets; the identification module is used for performing action identification on the moving target based on the at least two target video frames; the determining module is used for determining an action sequence of the initialized special effect when the action recognition result is obtained; the calculation module is used for performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining the special effect accuracy of the initialized special effect according to the calculated action accuracy; and the display module is used for acquiring a target special effect associated with the initialization special effect based on the special effect accuracy and displaying the target special effect.
With reference to the second aspect, in a first implementation manner of the second aspect, the identification module includes any one of the following units: the first identification unit is used for sequentially identifying the action change trend of the moving target in the front and rear two adjacent target video frames so as to finish action identification on the moving target; and the second identification unit is used for selecting at least two video frames to be identified from the at least two target video frames at a preset frequency, and identifying the action change trend of the moving target in the at least two video frames to be identified so as to finish the action identification of the moving target.
With reference to the second aspect, in a second implementation manner of the second aspect, the determining module includes any one of: the first determining unit is used for determining at least one action sequence of the initialized special effect based on the first action sequenced according to time in the action recognition result when the action recognition result is obtained; the second determining unit is used for taking a preset special effect action sequence as an initialized special effect action sequence when the action recognition result is obtained; the preset special effects comprise at least one.
With reference to the second aspect, in a third implementation manner of the second aspect, the calculation module includes: the matching unit is used for sequentially matching each action in the action sequence of the initialized special effect with each action in the action recognition result according to time sequence; the action calculation unit is used for respectively calculating the action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result; and the special effect calculating unit is used for determining the special effect accuracy of the initialized special effect according to the calculated action accuracy when each action in the action sequence of the initialized special effect is matched with the corresponding action in the action recognition result.
With reference to the second aspect, in a fourth implementation form of the second aspect, the action accuracy includes an action point location accuracy, an action trigger time accuracy, and an action duration accuracy; the calculation module comprises: the first calculation unit is used for calculating the sum of the action point positions, the sum of the action trigger time and the sum of the action duration time of all actions in the action sequence of the initialized special effect; and the second calculation unit is used for calculating the sum of the accuracy of the action point position, the sum of the accuracy of the action trigger time and the sum of the accuracy of the action duration time based on preset weight to obtain the accuracy of the special effect of the initialization special effect.
With reference to the second implementation manner of the second aspect, in a fifth implementation manner of the second aspect, when an initialized special effect is included, the display module includes a first display unit, configured to adjust a display effect of the initialized special effect based on the special effect accuracy, and display the initialized special effect after the display effect is adjusted as a target special effect.
With reference to the second implementation manner of the second aspect, in a sixth implementation manner of the second aspect, when at least two initialized special effects are included, the display module includes a second display unit, configured to determine the initialized special effect with the highest special effect accuracy as the target special effect, and display the target special effect.
In a third aspect, an electronic device is provided, which includes: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the special effect processing method according to any one of the embodiments of the first aspect and the first aspect is executed.
In a fourth aspect, a computer-readable storage medium is provided, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by a processor to implement the special effects processing method according to the first aspect and any of the embodiments of the first aspect.
The beneficial effect that technical scheme that this application provided brought is:
the method comprises the steps of carrying out image detection on an original video, obtaining at least two target video frames comprising a moving target, and carrying out action recognition on the moving target based on the at least two target video frames; when the action recognition result is obtained, determining an action sequence for initializing a special effect, namely triggering special effect recognition corresponding to a series of actions; performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining the special effect accuracy of the initialized special effect according to the calculated action accuracy, wherein the special effect accuracy of the initialized special effect is determined by the action accuracy of at least one action; and acquiring a target special effect associated with the initialized special effect based on the special effect accuracy for displaying. According to the method and the device, the action sequence corresponding to the initialized special effect and comprising at least one action is set, the action of the moving target in the original video is identified, the action identification result comprising at least one action is obtained, each action in the action sequence of the initialized special effect is matched with each action in the action identification result, and finally the accuracy of the special effect is determined according to the action accuracy between each action in the action sequence of the initialized special effect and the corresponding action in the action identification result, so that the target special effect is obtained according to the accuracy of the special effect to be displayed, the accuracy of special effect matching is improved, and the interactivity in the special effect matching process is enhanced.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic flowchart of a special effect processing method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a special effect processing method according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a special effect processing method according to an embodiment of the present application;
fig. 4 is a schematic flowchart of a special effect processing method according to an embodiment of the present application;
fig. 5 is a schematic flowchart of a special effect processing method according to an embodiment of the present application;
fig. 6 is a schematic flowchart of a special effect processing method according to an embodiment of the present application;
fig. 7 is a schematic flowchart of a special effect processing method according to an embodiment of the present application;
fig. 8 is a schematic diagram illustrating a special effect displayed on a display interface according to an embodiment of the present application;
fig. 9 is a flowchart illustrating an application of a special effect processing method according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of a special effect processing apparatus according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. Computer Vision technology (CV) is a subject of artificial intelligence technology to study how to make a machine "see", and more specifically, it refers to a technology that a camera and a Computer are used to replace human eyes to perform machine Vision such as recognition, tracking and measurement on a target, and further perform graphic processing, so that the Computer processing becomes an image more suitable for human eyes to observe or to transmit to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technologies, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also include common biometric technologies such as face recognition, gesture recognition, fingerprint recognition, and the like.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
In the special effect processing method according to the embodiment of the present application, the method may be specifically executed by an electronic device according to the embodiment of the present application, and specifically, the electronic device may be a mobile terminal. As shown in fig. 1, the embodiment of the present application includes the following steps:
s101, carrying out image detection on an original video to obtain at least two target video frames including a moving target;
s102, performing motion recognition on the moving target based on the at least two target video frames;
s103, when the action recognition result is obtained, determining an action sequence of the initialized special effect;
s104, performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining special effect accuracy of the initialized special effect according to the calculated action accuracy;
s105, acquiring a target special effect associated with the initialization special effect based on the special effect accuracy and displaying the target special effect.
In step S101, performing image detection on an original video, and acquiring at least two target video frames including a moving target; specifically, the original video is content shot by electronic equipment; when the image detection is carried out on the original video, the image detection can be carried out on the video frame of the original video by adopting a model designed by combining an optimized lightweight network hufflenet block structure with a multi-scale feature fusion method. The moving target is a target object for image detection, and the moving target can be a human hand, a human face, a limb, an object with biological characteristics and the like. In the shooting process of the video, the moving object does not always appear in the shot video image, at the moment, the image detection is carried out on the original video in view of reducing the calculation amount of the subsequent motion recognition of the moving object, and at least two object video frames comprising the moving object are obtained to be used as the basis for the subsequent motion recognition. In an embodiment, assuming that a moving target is a human hand, when an original video is acquired, gesture detection is performed on each video frame in the original video in sequence, and if the human hand is detected, gesture tracking is triggered (gesture tracking refers to obtaining an initial gesture region of a certain video frame through a detection algorithm for a video stream at one end, or predicting a gesture region of a certain video frame, so as to obtain an accurate gesture region of a next video frame); when the gesture tracking fails, carrying out image detection on the next video frame again; the embodiment of the application obtains at least two target video frames comprising moving targets through gesture tracking.
In step S102, performing motion recognition on the moving object based on the at least two target video frames; specifically, an action recognition model designed by an optimized lightweight network shuffle block structure is adopted to carry out action recognition on a moving target; and if the moving target is a human hand, designing the model as a gesture recognition model so as to recognize the gesture of the human hand. Specifically, a key point positioning method is adopted for motion recognition; according to a face detector blazeface network structure, an action recognition model is designed aiming at a key point positioning task, and if a moving object is a human hand, a gesture detection blazehand model is designed. When the moving object is a human hand, because the degree of freedom of the movement of the finger joints is large, in order to restrict the position relationship between the local joint points and the joint points of the whole hand, the regression loss of local multiple joint points (such as 21 joint points) and the structure loss of the global joint points are designed so as to enhance the regression stability of the key points of the model gesture. In the embodiment of the application, motion recognition is performed on video frames including moving objects, specifically, a category of the moving object in the target video frames is recognized, if the moving object is a human hand, a gesture of a certain target video frame is recognized as a fist, and by recognizing the target video frames of at least two frames, at least one motion of the moving object can be determined to be a motion of punching a fist, punching a fist and the like.
In step S103, when a motion recognition result is obtained, determining a motion sequence for initializing a special effect; specifically, when motion recognition is performed on a moving target, when at least one motion displayed by the moving target in shooting is determined, special effect matching is triggered. In the implementation of the application, one special effect corresponds to a plurality of actions, namely, one special effect is triggered after a plurality of continuous actions are displayed through a moving target; for example, a special effect named as the cheerful palm, and the corresponding actions include the fist starting, the fist extending forwards and backwards, the palm extending out and the like; when the moving target shows a plurality of corresponding actions, the action recognition structure of the moving target is matched with the special effect of the Buddha palm, and the special effect of the huge fingered citron appears on the display interface. In step S103, when the motion recognition result is acquired, the motion sequence of the initialization special effect is determined so that the motion in the motion sequence of the initialization special effect is matched with the motion in the motion recognition result. Alternatively, when the moving object is a human hand, the initialized special effect may be a special effect related to swordsmen. The swordsman sport related special effect consists of a series of swordsman actions, and each action is provided with corresponding completion time, action point position accuracy and duration time accuracy; the method specifically combines common swordsmen with special-effect action sequences by disassembling action details. The initialization special effect can be a set special effect, and an interface for setting variables can also be provided externally, so that the initialization special effect is set according to the special effect material.
In step S104, performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining special effect accuracy of all actions in the action sequence of the initialized special effect completed by the moving target according to the calculated action accuracy; specifically, each action in the action sequence of the initialized special effect is matched and calculated with each action in an action recognition structure through an action recognizer; the action recognizer is used for judging whether the current action accords with an action mode preset by a system (action in an action series with an initialized special effect), and judging whether the action is accurate (whether point position of the action is accurate, whether action trigger time is accurate, whether action duration is accurate, and three variables are provided with accuracy values). Calculating the special effect accuracy of the initialized special effect according to the action accuracy by using the special effect recognizer; specifically, the special effect recognizer comprises a sequence of action recognizers, and the accuracy of one action sequence is judged; when the motion recognition result is obtained in step S103, the special effect recognizer is started, the motions are recognized one by one according to the sequence in the motion recognizer sequence, and the special effect accuracy of the initialized special effect is calculated according to the point location accuracy, the trigger time accuracy, and the duration accuracy of each motion.
In step S105, obtaining a target special effect associated with the initialization special effect based on the special effect accuracy for displaying; specifically, the target special effect associated with the initialization special effect is acquired and displayed based on the special effect accuracy of the motion completed by the moving target calculated in step S104. Optionally, a standard value of the effect accuracy of the initialized effect is set in the effect recognizer, when it is determined that the current moving target completes the effect accuracy of the action sequence corresponding to the initialized effect through step S104, a deviation value between the effect accuracy of the actually completed action and the set standard value may be calculated, and the target effect associated with the initialized effect is obtained according to the deviation value and displayed. The target special effect is formed by adjusting or selecting the initialized special effect through special effect accuracy. The special effect accuracy can be expressed by a normalized value. For example, the initialization special effect is that a halo appears around the moving target behind the moving target, when the accuracy of the special effect is 0.8, the size and/or rendering degree of the halo is adjusted to 80% of the original halo display effect, and the adjusted halo is displayed as the target special effect.
In the embodiment, an action sequence corresponding to the initialized special effect and including at least one action is set, the action of the moving target in the original video is identified to obtain an action identification result including at least one action, each action in the action sequence of the initialized special effect is matched with each action in the action identification result, and finally the accuracy of the special effect is determined according to the action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action identification result, so that the target special effect is obtained according to the accuracy of the special effect to be displayed, the accuracy of special effect matching is improved, and the interactivity in the special effect matching process is enhanced.
In an embodiment, as shown in fig. 2, the step S102 performs motion recognition on the moving object based on the at least two target video frames, including any one of the following:
s201, sequentially identifying the motion change trend of a moving target in two adjacent front and back target video frames to finish motion identification of the moving target;
s202, at least two video frames to be identified are selected from the at least two target video frames at a preset frequency, and the action change trend of the moving target in the at least two video frames to be identified is identified so as to complete the action identification of the moving target.
Specifically, when the motion of the moving object is identified, in step S201, the motion change trend of the moving object in two adjacent front and back target video frames may be identified in sequence; in consideration of reducing the complexity of the calculation, at step S202, at least two video frames to be identified are selected from at least two target video frames including a moving target at a predetermined frequency, and a motion change trend of the moving target in the at least two video frames to be identified is identified; and if 7 frames are taken as one period, extracting the first frame, the seventh frame, the fourteenth frame and other target video frames from the at least two target video frames for identification. In an embodiment, the at least two target video frames are consecutive video frames each comprising a moving target.
In one embodiment, as shown in fig. 3, when the motion recognition result is obtained, step S103 determines that the motion sequence of the initialization special effect includes any one of:
s301, when an action recognition result is obtained, determining at least one action sequence of an initialized special effect based on a first action in the action recognition result according to time sequence;
s302, when the action recognition result is obtained, taking a preset action sequence of the special effect as an action sequence of the initialized special effect; the preset special effects comprise at least one.
In step S301, when a motion recognition result is obtained, at least one motion sequence for initializing a special effect is determined based on a first motion sorted according to time in the motion recognition result; specifically, when the motion recognition result is acquired, at least one motion sequence for initializing a special effect is determined based on the acquired first motion. For example, if the first motion in the current motion recognition result starts as a fist, a motion sequence of the first motion relative to the fist is obtained from the stored plurality of special effects as a motion sequence for initializing the special effects. Optionally, a guidance for triggering the special effect is displayed on a display interface shot by the terminal, if the special effect like a cheer palm is to be triggered, a fist starting action of a user is guided, when the user finishes the action, an action sequence for initializing the special effect is determined, that is, the action sequence corresponding to the cheer palm special effect is used as an action sequence for initializing the special effect, and the action recognizer and the special effect recognizer are triggered. That is, in step S301, the motion sequence for initializing the special effect is related to the first motion completed by the moving object.
In step S302, when the motion recognition result is obtained, taking a preset special-effect motion sequence as an initialized special-effect motion sequence; the preset special effects comprise at least one; specifically, when the motion recognition result is obtained, the preset special effect motion sequence is used as the special effect initialization motion sequence, that is, the special effect initialization motion sequence is irrelevant to the motion currently shown by the moving object, and the special effect initialization motion sequence is the preset special effect motion sequence. The preset action sequence of the special effect comprises at least one of the following: (1) a fixed sequence of actions, such as a fixed fist-palm-fist sequence of actions; (2) randomly generating action sequences, for example, a plurality of actions are included in an action pool, and randomly capturing any number of actions to combine into the action sequences; (3) the method is characterized in that the method is a special effect action sequence in a special effect pool, for example, the special effect pool comprises 3 special effects, and a special effect a, a special effect b and a special effect c respectively correspond to respective action sequences. In one embodiment, in step S302, taking the action sequence of the preset special effect as the action sequence of initializing the special effect includes taking the action sequences of all the special effects in the special effect pool as the action sequence of initializing the special effect.
In one embodiment, as shown in fig. 4, the step S104 performs matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determines action accuracy between each action in the action sequence of the initialized special effect and a corresponding action in the action recognition result, and determines the special effect accuracy of the initialized special effect according to the calculated action accuracy, including:
s401, sequentially matching each action in the action sequence of the initialized special effect with each action in the action recognition result according to time sequence;
s402, respectively calculating the action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result;
s403, when each action in the action sequence of the initialized special effect is matched with the corresponding action in the action recognition result, determining the special effect accuracy of the initialized special effect according to the calculated action accuracy.
Specifically, the motion recognition and the motion matching are performed while the motion object shows the motion, that is, after the motion sequence of the initialized special effect is determined in step S103, the motion sequence of the initialized special effect is used as a reference to perform the matching with the motion completed by the motion object. Specifically, when one action in the action sequence of the initialization special effect is completed, the action accuracy is calculated for the action; namely, the motion accuracy calculation of each motion is completed while the motion matching is performed in sequence. In one embodiment, in step S401, if it is assumed that the third action in the action sequence of the initialized special effect cannot be matched with the action in the action recognition result, the action matching operation is exited, i.e. the action recognizer and the move recognizer are turned off, and the subsequent steps are not executed; namely, the process of action matching is performed in sequence, and when any action in the action sequence of the initialized special effect cannot be completed, the subsequent actions are not matched any more. Alternatively, when any one of the motions in the motion sequence of the initialized special effect cannot be completed, the value corresponding to the motion accuracy of the motion matching is set to 0 in step S402, and the matching of the subsequent motion is continued. In step S402, the motion accuracy of the motion that the moving object has completed in the motion sequence of the initialized special effect is calculated, one motion in the motion sequence of the initialized special effect corresponds to only one motion in the motion recognition result, and motion matching is performed in chronological order. In step S403, when it is determined that each action in the action sequence of the initialized special effect matches a corresponding action in the action recognition result, the special effect accuracy of the initialized special effect is determined according to the calculated action accuracy of each action.
In an embodiment, when only one action sequence for initializing the special effect is included, the user can be guided to make an action in the action sequence for initializing the special effect by displaying an action guide on a display interface of the terminal. If the initialized special effect is the palm like the god, the corresponding action sequence comprises three actions, wherein the fist is started, the fist is extended from the front to the back, and the palm is extended. Assuming that the user triggers the matching of the motions of the cheerful palm special effect through the motion of completing the fist starting, step S402 will calculate the motion accuracy between the first motion of the initialization special effect and the first motion completed by the moving target; a guide diagram of the second action fist extending from front to back is displayed on the terminal interface, the user is guided to complete the second action in the action sequence of the initialized special effect, and when the second action of the moving target is determined to be matched with the second action of the initialized special effect in the step S401, the action accuracy between the second action of the initialized special effect and the second action completed by the moving target is calculated through the step S402; displaying a guide graph of the third action extending out of the palm on the terminal interface, guiding the user to finish the third action in the action sequence of the initialized special effect, and calculating the action accuracy between the third action of the initialized special effect and the third action finished by the moving target through the step S402 when the step S401 determines that the third action of the moving target is matched with the third action of the initialized special effect; at this time, step S403 determines that each action in the action sequence of the initialized special effect matches a corresponding action in the action recognition result, and determines the special effect accuracy of the initialized special effect according to the action accuracies of the above three actions.
In one embodiment, as shown in FIG. 5, the motion accuracy includes motion point location accuracy, motion trigger time accuracy, and motion duration accuracy; step S104 determines a special effect accuracy of the initialization special effect according to the calculated motion accuracy, including:
s501, calculating the sum of the motion point positions, the sum of the motion trigger time and the sum of the motion duration time of all the motions in the motion sequence of the initialized special effect;
s502, calculating the action point position accuracy sum, the action trigger time accuracy sum and the action duration time accuracy sum based on preset weight to obtain the special effect accuracy of the initialization special effect.
Specifically, the action accuracy is composed of three variables, namely action point location accuracy, action trigger time accuracy and action duration accuracy; the action point location accuracy can be determined by key point location; the action triggering time accuracy is determined by the display time stamp corresponding to the first frame of video frame in each action; the action duration accuracy is determined by the time period corresponding to the first frame to the last frame of video for completing each action. The motion accuracy of each motion may be expressed as a normalized value, and the effect accuracy determined from the motion accuracy of each motion is illustrated in conjunction with table 1 below:
TABLE 1
Figure BDA0002504816630000141
As can be seen from table 1, the action sequence for initializing the special effect includes 3 actions, and in the action recognition result, 3 actions correspond to the action sequence. The above values can be calculated in an absolute value manner, and taking the motion duration accuracy as an example, assuming that the standard duration for completing the first motion in the initialization special effect is 5ms, if the duration for completing the first motion by the motion target is 4ms or 6ms, the motion duration accuracy of the first motion is 0.8(1- |4-5|/5 ═ 0.8; 1- |6-5 |/0.8). The other two variables (action point location accuracy, action trigger time accuracy) are calculated in the same way. In step S501, the accuracy of action point positions, the accuracy of action trigger time and the accuracy of action duration time of all actions in the action sequence of the initialized special effect are calculated to be 2.4, 2.5 and 2.7 respectively; in step S502, the effect accuracy of the initialization effect calculated based on the preset weight of each variable is 2.51. When the accuracy values of the three variables are expressed as normalized values, the standard value of the effect accuracy is 3. When the action sequence of two or more initialization effects is determined in step S103, each initialization effect has its corresponding effect accuracy.
Optionally, when only one action sequence for initializing a special effect is determined in step S103, a standard value may be calculated when calculating accuracy values corresponding to three variables (action point location accuracy, action trigger time accuracy, and action duration accuracy); taking the action duration accuracy as an example, the standard value range of the action duration of the second action in the initialized special effect is 5ms-10ms, and if the moving target takes 7ms when the second action is completed, the action duration accuracy of the second action is 1; namely, the accuracy is 1 as long as the action duration of the moving target for completing the second action is within the range of the standard value; if the action duration time of the moving target for completing the second action is not within the standard value range, the moving target is judged not to complete the second action in the initialized special effect, and at the moment, the action recognizer and the special effect recognizer are quitted, and the action recognition is determined to fail. The other two variables (action point location accuracy, action trigger time accuracy) are calculated in the same way. When two or more action sequences of the initialized special effects are determined in step S103, assuming that there are three action sequences of the initialized special effects (action sequences corresponding to the initialized special effect a, the initialized special effect B, and the initialized special effect C, respectively), and the first action completed by the moving target is matched with the first action of the three initialized special effects, the action accuracy of the first action of the three initialized special effects is 1; when the second action completed by the moving target is only matched with the initialization special effect A and the initialization special effect B, the action matching of the initialization special effect C is quitted, and the action accuracy of the initialization special effect A and the initialization special effect B relative to the second action is 3; when the third action completed by the moving target is only matched with the initialized special effect A, the action matching of the initialized special effect B is quitted, the action accuracy of the initialized special effect A relative to the third action is 3, and at the moment, if the initialized special effect A also comprises the fourth action and more than the fourth action, the action matching with the action completed by the moving target is continued; if there are only three actions for initializing the special effect a, the accuracy of the special effect corresponding to the initialized special effect a is 3 for the action completed by the moving object at this time. Since the motion accuracy is composed of three variables, the standard value of the special effect accuracy is 3.
In an embodiment, as shown in fig. 6, when an initialization special effect is included, step S105 obtains a target special effect associated with the initialization special effect based on the special effect accuracy for displaying, including:
s601, adjusting the display effect of the initialized special effect based on the special effect accuracy, and displaying the initialized special effect after the display effect is adjusted as a target special effect.
Specifically, when the action sequence of the initialized special effect determined in step S103 includes only one, the presentation effect of the initialized special effect is adjusted based on the special effect accuracy, and the adjusted initialized special effect is taken as the target special effect. For example, assuming that the standard value of the accuracy of the special effect of the initialized special effect is 3, and the actual value of the accuracy of the special effect calculated based on the step S104 is 2.51, the display effect of the initialized special effect is adjusted by the ratio of the actual value to the standard value; if the display effect of the initialized special effect under the standard is that a halo with the size of 7x7 appears, the adjusted display effect of the initialized special effect is that a halo with the size of 5.8x5.8 appears; and displaying the adjusted halo as a target special effect on a display interface.
In an embodiment, as shown in fig. 7, when at least two initialization special effects are included, step S105 obtains a target special effect associated with the initialization special effect for presentation based on the special effect accuracy, which includes:
s701, determining the initialized special effect with the highest special effect accuracy as a target special effect, and displaying the target special effect.
Specifically, when two or more of the action sequences of the initialized special effects determined in step S103 include, the initialized special effect with the highest actual value of the accuracy of the special effects calculated in step S104 is determined as the target special effect; if the three initialized special effects are determined in step S103, and the initialized special effects a, B and C correspond to each other, then respectively calculating the accuracy values of the initialized special effects when the moving target completes the action, assuming that the accuracy of the initialized special effect a corresponding to the completion action of the moving target is 2.7, the accuracy of the initialized special effect B corresponding to the completion action of the moving target is 2.63, and the accuracy of the initialized special effect C corresponding to the completion action of the moving target is 2.82; the initialization special effect C is acquired as the target special effect for display. In one embodiment, as shown in fig. 8, assume that the initialized special effect C corresponds to a motion sequence related to a positive finger in the swordsman movement, a positive finger related special effect appears on the display interface.
In an application example, as shown in fig. 9, it is assumed that the moving object is a human hand. A user starts a video shooting application program on a terminal to shoot a video; in the shooting process, a user wants to increase the interestingness of a shot video by adding a special effect, if a positive finger special effect related to the swordsman movement is added (the action sequence of the positive finger special effect comprises two actions, namely, stretching out a finger and keeping a preset time, and assuming that the preset time is 3 seconds), the fist is displayed in a video shooting area, and special effect matching is triggered (a plurality of special effect triggering gestures can be displayed on a terminal display interface, the user triggers different special effect matching by displaying different gestures, and the triggering gesture of the positive finger special effect is set as the fist); at the moment, a motion guide diagram of a first motion appears on the terminal display interface (the motion guide diagram can be characters and can also be a motion path; if the characters can be prompts such as 'fist starting posture' and the like), when the user completes the current first motion according to the motion guide diagram, the motion guide diagram of a second motion appears on the display interface (the characters in the motion guide diagram of the second motion can be expressed as 'extending one finger and keeping for 3 seconds'; the terminal simultaneously calculates the motion accuracy of the user for completing the first motion); when the user completes the second action according to the action guide diagram (the terminal calculates the action accuracy of the user for completing the second action, calculates the special effect accuracy based on the action accuracy of each action, further adjusts the display effect of the one-finger special effect based on the special effect accuracy, and displays the one-finger special effect after the display effect is adjusted as the target special effect), a sticker with the one-finger special effect appears on the terminal display interface, as shown in fig. 8.
Optionally, the gesture for triggering special effect matching may be a first action in an action sequence for initializing a special effect, that is, a user triggers special effect matching of a corresponding special effect by completing the first action. In the embodiment, the user triggers special effect matching of a positive finger special effect by completing the action of starting the fist; at this time, the terminal display interface displays a motion guide diagram of a second motion (extending a finger and keeping a preset duration) in the motion sequence (the terminal simultaneously calculates the motion accuracy of the user for completing the first motion); the subsequent steps are the same as those in the above embodiments, and are not described herein again.
In one embodiment, as shown in fig. 10, there is provided a special effects processing apparatus 100 including:
the detection module 101 is configured to perform image detection on an original video, and acquire at least two target video frames including a moving target;
an identification module 102, configured to perform motion identification on the moving object based on the at least two target video frames;
the determining module 103 is configured to determine, when the motion recognition result is obtained, a motion sequence of the initialization special effect;
a calculating module 104, configured to perform matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determine action accuracy between each action in the action sequence of the initialized special effect and a corresponding action matched in the action recognition result, and determine special effect accuracy of the initialized special effect according to the calculated action accuracy;
a display module 105, configured to obtain a target special effect associated with the initialization special effect based on the special effect accuracy for displaying.
In one embodiment, the identification module 102 includes any one of the following: the first identification unit is used for sequentially identifying the action change trend of the moving target in the front and rear two adjacent target video frames so as to finish action identification on the moving target; and the second identification unit is used for selecting at least two video frames to be identified from the at least two target video frames at a preset frequency, and identifying the action change trend of the moving target in the at least two video frames to be identified so as to finish the action identification of the moving target.
In an embodiment, the determining module 103 includes any one of the following units: the first determining unit is used for determining at least one action sequence of the initialized special effect based on the first action sequenced according to time in the action recognition result when the action recognition result is obtained; the second determining unit is used for taking a preset special effect action sequence as an initialized special effect action sequence when the action recognition result is obtained; the preset special effects comprise at least one.
In one embodiment, the calculation module 104 includes: the matching unit is used for sequentially matching each action in the action sequence of the initialized special effect with each action in the action recognition result according to time sequence; the action calculation unit is used for respectively calculating the action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result; and the special effect calculating unit is used for determining the special effect accuracy of the initialized special effect according to the calculated action accuracy when each action in the action sequence of the initialized special effect is matched with the corresponding action in the action recognition result.
In an embodiment, the action accuracy comprises an action point location accuracy, an action trigger time accuracy, and an action duration accuracy; the calculation module 104 includes a first calculation unit, configured to calculate a sum of motion point location accuracy, a sum of motion trigger time accuracy, and a sum of motion duration accuracy of all motions in the motion sequence of the initialization special effect; and the second calculation unit is used for calculating the sum of the accuracy of the action point position, the sum of the accuracy of the action trigger time and the sum of the accuracy of the action duration time based on preset weight to obtain the accuracy of the special effect of the initialization special effect.
In an embodiment, when the display module 105 includes an initialization special effect, the display module includes a first display unit, configured to adjust a display effect of the initialization special effect based on the accuracy of the special effect, and display the initialization special effect after the adjustment of the display effect as a target special effect.
In an embodiment, when at least two initialized special effects are included, the display module 105 includes a second display unit, configured to determine the initialized special effect with the highest special effect accuracy as the target special effect, and display the target special effect.
The special effect processing apparatus of the embodiment of the present application may execute the special effect processing method provided in the embodiment of the present application, and the implementation principles thereof are similar, the actions executed by each module in the special effect processing apparatus in each embodiment of the present application correspond to the steps in the special effect processing method in each embodiment of the present application, and for detailed functional descriptions of each module of the special effect processing apparatus, reference may be specifically made to the descriptions in the corresponding special effect processing method shown in the foregoing, and details are not described here again.
Based on the same principle as the method shown in the embodiments of the present application, there is also provided in the embodiments of the present application an electronic device, which may include but is not limited to: a processor and a memory; a memory for storing computer operating instructions; and the processor is used for executing the special effect processing method shown in the embodiment by calling the computer operation instruction.
In an alternative embodiment, an electronic device is provided, as shown in fig. 11, the electronic device 4000 shown in fig. 11 comprising: a processor 4001 and a memory 4003. Processor 4001 is coupled to memory 4003, such as via bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004. In addition, the transceiver 4004 is not limited to one in practical applications, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.
The Processor 4001 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application specific integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 4001 may also be a combination that performs a computational function, including, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, or the like. The processor 4001 executes the special effect processing method shown in the above embodiments by calling a computer operation instruction, the special effect processing method including: carrying out image detection on an original video to obtain at least two target video frames including a moving target; performing motion recognition on the moving target based on the at least two target video frames; when an action recognition result is obtained, determining an action sequence of the initialized special effect; performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining special effect accuracy of the initialized special effect according to the calculated action accuracy; and acquiring a target special effect associated with the initialization special effect based on the special effect accuracy for displaying.
Bus 4002 may include a path that carries information between the aforementioned components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus, an EISA (extended industry Standard Architecture) bus, or the like. The bus 4002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 11, but this is not intended to represent only one bus or type of bus.
The Memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, an EEPROM (Electrically erasable programmable Read Only Memory), a CD-ROM (Compact Read Only Memory) or other optical disk storage, optical disk storage (including Compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to.
The memory 4003 is used for storing application codes for executing the scheme of the present application, and the execution is controlled by the processor 4001. Processor 4001 is configured to execute application code stored in memory 4003 to implement what is shown in the foregoing method embodiments.
Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 11 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a computer, enables the computer to execute the corresponding content in the foregoing method embodiments.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present application may be implemented by software or hardware. The name of a module does not in some cases form a limitation of the module itself, for example, a detection module may also be described as a module for performing image detection on an original video and acquiring at least two target video frames including a moving target.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the disclosure. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims (15)

1. A special effect processing method is characterized by comprising the following steps:
carrying out image detection on an original video to obtain at least two target video frames including a moving target;
performing motion recognition on the moving target based on the at least two target video frames;
when an action recognition result is obtained, determining an action sequence of the initialized special effect;
performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining special effect accuracy of the initialized special effect according to the calculated action accuracy;
and acquiring a target special effect associated with the initialization special effect based on the special effect accuracy for displaying.
2. The method of claim 1, wherein the performing motion recognition on the moving object based on the at least two target video frames comprises any one of:
sequentially identifying the action change trend of the moving target in two adjacent front and back target video frames to finish the action identification of the moving target;
and selecting at least two video frames to be identified from the at least two target video frames at a preset frequency, and identifying the action change trend of the moving target in the at least two video frames to be identified so as to finish the action identification of the moving target.
3. The method according to claim 1, wherein the determining, when the motion recognition result is obtained, a motion sequence for initializing a special effect includes any one of:
when an action recognition result is obtained, determining at least one action sequence of the initialized special effect based on the first action in the action recognition result according to time sequence;
when the action recognition result is obtained, taking a preset action sequence of the special effect as an action sequence of the initialized special effect; the preset special effects comprise at least one.
4. The method of claim 1, wherein the performing a match calculation on each action in the sequence of actions of the initialized special effect and each action in the action recognition result, determining an action accuracy between each action in the sequence of actions of the initialized special effect and a corresponding action matched in the action recognition result, and determining a special effect accuracy of the initialized special effect according to the calculated action accuracy comprises:
sequentially matching each action in the action sequence of the initialized special effect with the action sorted according to time in the action recognition result;
respectively calculating action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result;
and when each action in the action sequence of the initialized special effect is matched with the corresponding action in the action recognition result, determining the special effect accuracy of the initialized special effect according to the calculated action accuracy.
5. The method of claim 1, wherein the action accuracy comprises an action point location accuracy, an action trigger time accuracy, and an action duration accuracy; determining a special effect accuracy of the initialized special effect according to the calculated action accuracy, comprising:
calculating the sum of the action point positions, the sum of the action trigger time and the sum of the action duration time of all actions in the action sequence of the initialized special effect;
and calculating the sum of the accuracy of the point positions of the actions, the sum of the accuracy of the triggering time of the actions and the sum of the accuracy of the duration time of the actions based on preset weights to obtain the accuracy of the special effect of the initialization special effect.
6. The method of claim 3, wherein when an initialization effect is included, said obtaining a target effect associated with the initialization effect for presentation based on the effect accuracy comprises:
and adjusting the display effect of the initialized special effect based on the special effect accuracy, and displaying the initialized special effect after the display effect is adjusted as a target special effect.
7. The method of claim 3, wherein when at least two initialization effects are included, the obtaining a target effect associated with the initialization effect for presentation based on the effect accuracy comprises:
and determining the initialized special effect with the highest special effect accuracy as a target special effect, and displaying the target special effect.
8. A special effect processing apparatus, comprising:
the detection module is used for carrying out image detection on an original video and acquiring at least two target video frames comprising moving targets;
the identification module is used for performing action identification on the moving target based on the at least two target video frames;
the determining module is used for determining an action sequence of the initialized special effect when the action recognition result is obtained;
the calculation module is used for performing matching calculation on each action in the action sequence of the initialized special effect and each action in the action recognition result, determining action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result, and determining the special effect accuracy of the initialized special effect according to the calculated action accuracy;
and the display module is used for acquiring a target special effect associated with the initialization special effect based on the special effect accuracy and displaying the target special effect.
9. The apparatus of claim 8, wherein the identification module comprises any one of:
the first identification unit is used for sequentially identifying the action change trend of the moving target in the front and rear two adjacent target video frames so as to finish action identification on the moving target;
and the second identification unit is used for selecting at least two video frames to be identified from the at least two target video frames at a preset frequency, and identifying the action change trend of the moving target in the at least two video frames to be identified so as to finish the action identification of the moving target.
10. The apparatus of claim 8, wherein the determining module comprises any one of:
the first determining unit is used for determining at least one action sequence of the initialized special effect based on the first action sequenced according to time in the action recognition result when the action recognition result is obtained;
the second determining unit is used for taking a preset special effect action sequence as an initialized special effect action sequence when the action recognition result is obtained; the preset special effects comprise at least one.
11. The apparatus of claim 8, wherein the computing module comprises:
the matching unit is used for sequentially matching each action in the action sequence of the initialized special effect with each action in the action recognition result according to time sequence;
the action calculation unit is used for respectively calculating the action accuracy between each action in the action sequence of the initialized special effect and the corresponding action matched in the action recognition result;
and the special effect calculating unit is used for determining the special effect accuracy of the initialized special effect according to the calculated action accuracy when each action in the action sequence of the initialized special effect is matched with the corresponding action in the action recognition result.
12. The apparatus of claim 8, wherein the action accuracy comprises an action point location accuracy, an action trigger time accuracy, and an action duration accuracy; the calculation module comprises:
the first calculation unit is used for calculating the sum of the action point positions, the sum of the action trigger time and the sum of the action duration time of all actions in the action sequence of the initialized special effect;
and the second calculation unit is used for calculating the sum of the accuracy of the action point position, the sum of the accuracy of the action trigger time and the sum of the accuracy of the action duration time based on preset weight to obtain the accuracy of the special effect of the initialization special effect.
13. The apparatus of claim 10, wherein when the initial effect is included, the display module includes a first display unit configured to adjust a display effect of the initial effect based on the effect accuracy, and display the initial effect with the adjusted display effect as the target effect.
14. An electronic device, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: executing the special effects processing method according to any one of claims 1 to 7.
15. A computer readable storage medium, characterized in that it stores at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the special effects processing method according to any one of claims 1 to 7.
CN202010443569.0A 2020-05-22 2020-05-22 Special effect processing method and related equipment Active CN111611941B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010443569.0A CN111611941B (en) 2020-05-22 2020-05-22 Special effect processing method and related equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010443569.0A CN111611941B (en) 2020-05-22 2020-05-22 Special effect processing method and related equipment

Publications (2)

Publication Number Publication Date
CN111611941A true CN111611941A (en) 2020-09-01
CN111611941B CN111611941B (en) 2023-09-19

Family

ID=72199320

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010443569.0A Active CN111611941B (en) 2020-05-22 2020-05-22 Special effect processing method and related equipment

Country Status (1)

Country Link
CN (1) CN111611941B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112333473A (en) * 2020-10-30 2021-02-05 北京字跳网络技术有限公司 Interaction method, interaction device and computer storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017152794A1 (en) * 2016-03-10 2017-09-14 Zhejiang Shenghui Lighting Co., Ltd. Method and device for target tracking
WO2019024750A1 (en) * 2017-08-03 2019-02-07 腾讯科技（深圳）有限公司 Video communications method and apparatus, terminal, and computer readable storage medium
CN109600559A (en) * 2018-11-29 2019-04-09 北京字节跳动网络技术有限公司 A kind of special video effect adding method, device, terminal device and storage medium
CN109618183A (en) * 2018-11-29 2019-04-12 北京字节跳动网络技术有限公司 A kind of special video effect adding method, device, terminal device and storage medium
CN110222576A (en) * 2019-05-07 2019-09-10 北京字节跳动网络技术有限公司 Punch action recognition methods, device and electronic equipment
CN110472531A (en) * 2019-07-29 2019-11-19 腾讯科技（深圳）有限公司 Method for processing video frequency, device, electronic equipment and storage medium
WO2020020156A1 (en) * 2018-07-23 2020-01-30 腾讯科技（深圳）有限公司 Video processing method and apparatus, terminal device, server, and storage medium
CN110913205A (en) * 2019-11-27 2020-03-24 腾讯科技（深圳）有限公司 Video special effect verification method and device
CN111104930A (en) * 2019-12-31 2020-05-05 腾讯科技（深圳）有限公司 Video processing method and device, electronic equipment and storage medium

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017152794A1 (en) * 2016-03-10 2017-09-14 Zhejiang Shenghui Lighting Co., Ltd. Method and device for target tracking
WO2019024750A1 (en) * 2017-08-03 2019-02-07 腾讯科技（深圳）有限公司 Video communications method and apparatus, terminal, and computer readable storage medium
WO2020020156A1 (en) * 2018-07-23 2020-01-30 腾讯科技（深圳）有限公司 Video processing method and apparatus, terminal device, server, and storage medium
CN109600559A (en) * 2018-11-29 2019-04-09 北京字节跳动网络技术有限公司 A kind of special video effect adding method, device, terminal device and storage medium
CN109618183A (en) * 2018-11-29 2019-04-12 北京字节跳动网络技术有限公司 A kind of special video effect adding method, device, terminal device and storage medium
CN110222576A (en) * 2019-05-07 2019-09-10 北京字节跳动网络技术有限公司 Punch action recognition methods, device and electronic equipment
CN110472531A (en) * 2019-07-29 2019-11-19 腾讯科技（深圳）有限公司 Method for processing video frequency, device, electronic equipment and storage medium
CN110913205A (en) * 2019-11-27 2020-03-24 腾讯科技（深圳）有限公司 Video special effect verification method and device
CN111104930A (en) * 2019-12-31 2020-05-05 腾讯科技（深圳）有限公司 Video processing method and device, electronic equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112333473A (en) * 2020-10-30 2021-02-05 北京字跳网络技术有限公司 Interaction method, interaction device and computer storage medium
CN112333473B (en) * 2020-10-30 2022-08-23 北京字跳网络技术有限公司 Interaction method, interaction device and computer storage medium

Also Published As

Publication number Publication date
CN111611941B (en) 2023-09-19

Similar Documents

Publication Publication Date Title
CN109584276B (en) Key point detection method, device, equipment and readable medium
CN106250938B (en) Target tracking method, augmented reality method and device thereof
WO2019245768A1 (en) System for predicting articulated object feature location
CN109657533A (en) Pedestrian recognition methods and Related product again
CN112148197A (en) Augmented reality AR interaction method and device, electronic equipment and storage medium
JP6571108B2 (en) Real-time 3D gesture recognition and tracking system for mobile devices
CN110610154A (en) Behavior recognition method and apparatus, computer device, and storage medium
CN110986969B (en) Map fusion method and device, equipment and storage medium
US10990170B2 (en) Eye tracking method, electronic device, and non-transitory computer readable storage medium
CN113034652A (en) Virtual image driving method, device, equipment and storage medium
CN111273772B (en) Augmented reality interaction method and device based on slam mapping method
CN108096833B (en) Motion sensing game control method and device based on cascade neural network and computing equipment
CN113792651B (en) Gesture interaction method, device and medium integrating gesture recognition and fingertip positioning
CN112927259A (en) Multi-camera-based bare hand tracking display method, device and system
CN112882576B (en) AR interaction method and device, electronic equipment and storage medium
CN107368181B (en) Gesture recognition method and device
CN112381071A (en) Behavior analysis method of target in video stream, terminal device and medium
US20230401799A1 (en) Augmented reality method and related device
CN114241597A (en) Posture recognition method and related equipment thereof
US20220300774A1 (en) Methods, apparatuses, devices and storage media for detecting correlated objects involved in image
CN111611941B (en) Special effect processing method and related equipment
US20200150765A1 (en) Systems and methods for generating haptic effects based on visual characteristics
WO2024022301A1 (en) Visual angle path acquisition method and apparatus, and electronic device and medium
EP3639193A1 (en) Human feedback in 3d model fitting
CN115497094A (en) Image processing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40028078

Country of ref document: HK

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant