WO2022062680A1 - 动画生成方法、装置、系统及存储介质 - Google Patents

动画生成方法、装置、系统及存储介质 Download PDF

Info

Publication number
WO2022062680A1
WO2022062680A1 PCT/CN2021/110349 CN2021110349W WO2022062680A1 WO 2022062680 A1 WO2022062680 A1 WO 2022062680A1 CN 2021110349 W CN2021110349 W CN 2021110349W WO 2022062680 A1 WO2022062680 A1 WO 2022062680A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
virtual
virtual character
animation
real
Prior art date
Application number
PCT/CN2021/110349
Other languages
English (en)
French (fr)
Inventor
柴金祥
赵文平
金师豪
刘博�
朱曈晖
谭宏冰
熊兴堂
王从艺
王志勇
Original Assignee
魔珐(上海)信息科技有限公司
上海墨舞科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 魔珐(上海)信息科技有限公司, 上海墨舞科技有限公司 filed Critical 魔珐(上海)信息科技有限公司
Priority to US18/028,472 priority Critical patent/US11893670B2/en
Publication of WO2022062680A1 publication Critical patent/WO2022062680A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/205—3D [Three Dimensional] animation driven by audio data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/80—2D [Two Dimensional] animation, e.g. using sprites
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/292—Multi-camera tracking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/176—Dynamic expression
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Definitions

  • the present disclosure relates to the technical field of animation, and in particular, to an animation generation method, device, system and storage medium.
  • animation production a lot of animations are very complex production work done in a highly condensed schedule.
  • the traditional animation production process is a typical linear process, similar to the pipeline, including the early stage, the middle stage and the later stage.
  • the animation production effect is limited by the drawing level and efficiency of the animator.
  • the technical solution includes:
  • an animation generation method comprising:
  • the real feature data of the real object includes the action data and the facial data of the real object in the performance process
  • An animation of the virtual character is generated according to the target feature data.
  • the method further includes:
  • the reference data includes sound recording data and/or virtual camera pose data of the real object during the performance;
  • An animation of the virtual character is generated according to the target feature data and the reference data.
  • the target feature data and the reference data both carry a time code
  • generating the animation of the virtual character according to the target feature data and the reference data includes:
  • the animation of the virtual character is generated according to the target feature data and the reference data after the alignment process.
  • generating the animation of the virtual character according to the target feature data and the reference data after the alignment processing includes:
  • an animation picture is obtained according to the virtual camera pose data and the target feature data, and the virtual camera pose data is used to indicate a preview camera angle of view of the animation picture to be generated;
  • an animation video of the virtual character is generated.
  • the acquiring real feature data of real objects includes:
  • the motion data includes body motion data and/or gesture motion data
  • the facial data of the real object is acquired, where the facial data includes expression data and/or eye gaze data.
  • the acquiring the motion data of the real object includes:
  • the position data corresponding to each of a plurality of optical marking points preset on the hand of the real object is obtained, and the gesture action data of the real object is determined according to the position data corresponding to each of the plurality of optical marking points.
  • the acquiring the facial data of the real object includes:
  • a face video frame of the real object is acquired, where the face video frame is a video frame including the face of the real object, and the face video frame is used to indicate the face data of the real object.
  • the determining the target feature data of the virtual character according to the real feature data includes:
  • the virtual object is a virtual model obtained by restoring and reconstructing the real object
  • the virtual feature data includes motion data and facial data of the virtual object
  • performing redirection processing on the virtual feature data to obtain the target feature data of the virtual character includes:
  • performing redirection processing on the action data of the virtual object to obtain the action data of the virtual character including:
  • the action data of the virtual object is redirected to the virtual character to obtain the action data of the virtual character.
  • performing redirection processing on the face data of the virtual object to obtain the face data of the virtual character including:
  • the facial data of the virtual object is redirected to the virtual character to obtain the facial data of the virtual character.
  • the method further includes:
  • the skin motion of the virtual character is driven and displayed.
  • the method before generating the animation of the virtual character according to the target feature data, the method further includes:
  • Video recording data carrying a time code is acquired, where the video recording data includes video data obtained by recording the performance content of the real object.
  • the method further includes:
  • An animation of the virtual character is generated according to the target feature data and the prop motion data.
  • an animation generating apparatus comprising:
  • an acquisition module for acquiring the real feature data of the real object, the real feature data including the action data and the facial data of the real object in the performance process;
  • a determining module configured to determine target feature data of the virtual character according to the real feature data, the virtual character is a preset animation model, and the target feature data includes the action data and facial data of the virtual character;
  • the generating module is used for generating the animation of the virtual character according to the target feature data.
  • a computer device comprising: a processor; a memory for storing processor-executable instructions;
  • processor is configured to:
  • the real feature data of the real object includes the action data and the facial data of the real object in the performance process
  • An animation of the virtual character is generated according to the target feature data.
  • an animation generation system comprising:
  • a motion-capture garment wherein a plurality of optical marking points are arranged on the motion-capture garment;
  • a first camera where the first camera is used to capture motion data when a real object is performing
  • a helmet a second camera is arranged on the helmet, and the second camera is used to capture the facial data of the real object when performing;
  • a non-volatile computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor.
  • the embodiment of the present disclosure acquires real feature data of a real object, where the real feature data includes motion data and facial data of the real object in the performance process; the target feature data of the virtual character is determined according to the real feature data, and the virtual character is a preset animation model, The target feature data includes the action data and face data of the virtual character; according to the target feature data, the animation of the virtual character is generated; that is, the animation of the virtual character is generated by the performance of the real object, on the one hand, the situation of manual drawing is avoided, and the animation is improved.
  • the delicate performance of real objects can be directly transferred to virtual characters, and the skeletal movements and facial emotions of virtual characters are more realistic and vivid, ensuring the effect of animation production.
  • FIG. 1 shows a schematic structural diagram of a computer device provided by an exemplary embodiment of the present disclosure
  • FIG. 2 shows a flowchart of an animation generation method provided by an exemplary embodiment of the present disclosure
  • FIG. 3 shows a schematic structural diagram of a computer device provided by another exemplary embodiment of the present disclosure
  • FIG. 4 shows a flowchart of an animation generation method provided by another exemplary embodiment of the present disclosure
  • FIG. 5 shows a schematic structural diagram of an animation generating apparatus provided by an exemplary embodiment of the present disclosure.
  • FIG. 1 shows a schematic structural diagram of a computer device provided by an exemplary embodiment of the present disclosure.
  • the animation generation method in the embodiment of the present disclosure may be executed by a computer device.
  • a computer device may be a processing system that includes multiple devices or systems.
  • the computer equipment is a server, or a server cluster composed of several servers, or a cloud computing service center. This embodiment of the present disclosure does not limit this.
  • the computer device includes a processor 110 , a memory 120 and a communication interface 130 .
  • the structure shown in FIG. 1 does not constitute a limitation on the computer device, and may include more or less components than the one shown, or combine some components, or arrange different components. in:
  • the processor 110 is the control center of the computer equipment, using various interfaces and lines to connect various parts of the entire computer equipment, by running or executing the software programs and/or modules stored in the memory 120, and calling the data stored in the memory 120. , perform various functions of computer equipment and process data, so as to carry out overall control of computer equipment.
  • the processor 110 may be implemented by a CPU, or may be implemented by a graphics processor (Graphics Processing Unit, GPU).
  • Memory 120 may be used to store software programs and modules.
  • the processor 110 executes various functional applications and data processing by executing software programs and modules stored in the memory 120 .
  • the memory 120 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, a virtual module, and an application program (such as neural network model training, etc.) required for at least one function; Data created by the use of computer equipment, etc.
  • the memory 120 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable).
  • SRAM Static Random Access Memory
  • Electrically Erasable Programmable Read-Only Memory Electrically Erasable
  • memory 120 may also include a memory controller to provide processor 110 access to memory 120 .
  • the processor 110 is configured to perform the following functions: acquiring real feature data of a real object, where the real feature data includes motion data and facial data of the real object during the performance; determining the target feature data of the virtual character according to the real feature data, and the virtual character It is a preset animation model, and the target feature data includes action data and face data of the virtual character; according to the target feature data, an animation of the virtual character is generated.
  • the animation generation method provided by the embodiment of the present disclosure can be applied to the production of film and television previs, the production of animated dramas, the production of game CG, game animation, game action, and the production of virtual animation short videos, and is also the technical basis of virtual live broadcast.
  • the animation generation method provided by the embodiment of the present disclosure is applied to the application field of offline production of virtual character performance animation, especially the field of three-dimensional animation. This embodiment of the present disclosure does not limit this.
  • FIG. 2 shows a flowchart of an animation generation method provided by an exemplary embodiment of the present disclosure. This embodiment is exemplified by using the method in the computer device shown in FIG. 1 . The method includes the following steps.
  • Step 201 Obtain real feature data of a real object, where the real feature data includes motion data and facial data of the real object during the performance.
  • the computer equipment captures the skeletal movement of the real object through the optical capture device to obtain the motion data of the real object; at the same time, the facial emotion of the real object is captured through the optical capture device to obtain the real object facial data.
  • the optical capture device includes at least one of an infrared camera, an RGB camera and a depth camera.
  • the embodiment of the present disclosure does not limit the type of the optical capture device.
  • Real objects are movable objects in the real environment.
  • the real object is a person.
  • This embodiment of the present disclosure does not limit this.
  • the following description only takes a real object as a character as an example.
  • the real feature data includes the action data and face data of the real object during the performance.
  • the action data is used to indicate the skeletal movement of the real object
  • the face data is used to indicate the facial emotion of the real object.
  • the motion data of the real object includes body motion data and/or gesture motion data, the body motion data is used to indicate the body motion of the real object, and the gesture motion data is used to indicate the hand motion of the real object.
  • the limbs in the embodiments of the present disclosure are body parts other than the hands in the body, that is, the body of the real object includes the limbs of the real object and the hands other than the limbs.
  • the facial data of the real object includes expression data and/or eye gaze data
  • the expression data is used to indicate the facial expression of the real object
  • the eye gaze data is used to indicate the eye state of the real object.
  • Step 202 Determine target feature data of the virtual character according to the real feature data, where the virtual character is a preset animation model, and the target feature data includes motion data and facial data of the virtual character.
  • the computer equipment converts the real feature data of the real object into the target feature data of the virtual character.
  • the virtual character is a preset three-dimensional or two-dimensional animation model.
  • An avatar is a movable object in a virtual environment.
  • the virtual character is a virtual character, a virtual animal, a virtual pet or other objects in virtual form.
  • the target feature data of the virtual character includes action data and facial data of the virtual character.
  • the motion data of the virtual character includes body motion data and/or gesture motion data
  • the facial data of the virtual character includes expression data and/or eye gaze data.
  • the target feature data corresponds to the real feature data, and the meaning of the target feature data can be compared with the relevant description of the real feature data, which will not be repeated here.
  • the facial data of the real object acquired by the computer device may be acquired in units of frames, and the subsequent determination of the facial data of the virtual character according to the real feature data may also be correspondingly transformed in units of frames. This embodiment of the present disclosure does not limit this.
  • Step 203 generating an animation of the virtual character according to the target feature data.
  • the computer device generates the animation of the virtual character in real time or offline according to the target feature data of the virtual character.
  • the animation of the virtual character includes three-dimensional or two-dimensional animation.
  • the animation of the virtual character is a three-dimensional animation as an example for description.
  • the computer device displays a target user interface, the target user interface including the generated three-dimensional animation of the virtual character.
  • the target user interface may also include a three-dimensional animation of a virtual character displayed in a virtual environment, where the virtual environment is a three-dimensional virtual environment.
  • the virtual environment is a created scene for virtual characters to perform activities.
  • the virtual environment may be a simulated environment of the real world, a semi-simulated and semi-fictional environment, or a purely fictional environment.
  • the real object is a character, such as an actor.
  • the actor performs the performance required by the character in the plot, including body movements, gestures, expressions and
  • the corresponding capture device captures the actor's body movements, gestures, expressions and eyes
  • the computer device obtains the actor's real feature data, and converts the actor's real feature data into the target feature data of the virtual character. , that is, transfer the body movements and gestures of the actor to the virtual character, transfer the expressions and eyes of the actor to the face of the virtual character, and generate and display the three-dimensional animation of the virtual character based on the target feature data.
  • the animation generation method is a performance-based animation production method.
  • the real feature data of the real object that is, the motion data and the facial data of the real object
  • the data determines the target feature data of the virtual character, and generates the animation of the virtual character according to the target feature data; that is, the animation of the virtual character is generated by the performance of the real object, on the one hand, the situation of manual drawing is avoided, and the efficiency of animation generation is improved;
  • the delicate performance of real objects can be directly transferred to virtual characters, and the skeletal movements and facial emotions of virtual characters are more realistic and vivid, ensuring the effect of animation production.
  • the embodiment of the present disclosure provides an animation generation system
  • the animation generation system includes: a motion capture clothing, on which a plurality of optical markers are set; a first camera, the first camera is used to capture the actions of a real object when performing data; helmet, a second camera is arranged on the helmet, and the second camera is used to capture the facial data of the real object when performing; the first camera and the second camera respectively establish a communication connection with the computer equipment, and the computer equipment is used to execute the implementation of the present disclosure
  • the animation generation method provided by the example.
  • At least two first cameras are set to capture the action data of the real object when performing, and a plurality of first cameras can be set according to the actual situation; at least one second camera is set to capture the facial data of the real object when it is performing , and a plurality of second cameras can be set according to the actual situation.
  • FIG. 3 shows a schematic structural diagram of a computer device provided by another exemplary embodiment of the present disclosure.
  • the computer device 10 includes a motion capture system 20 , a motion redirection system 30 , a face capture system 40 , a face redirection system 50 , a prop capture system 60 , a prop redirection system 62 , a sound recording system 64 , and a virtual camera pose tracking system 70 , reference video recording system 80 and animation composition system 92 .
  • the motion capture system 20 includes a body capture system 21 and a gesture capture system 22 , and the body capture system 21 and the gesture capture system 22 are respectively connected to the motion redirection system 30 .
  • the limb capture system 21 is used to obtain position data corresponding to a plurality of optical markers preset on the limbs of the real object, and determine the limb movement data of the real object according to the position data corresponding to the multiple optical markers;
  • the motion data is used for limb reconstruction to obtain limb motion data of the virtual object.
  • the virtual object is a virtual model obtained by restoring and reconstructing the real object.
  • the gesture capture system 22 is used to obtain position data corresponding to multiple optical markers preset on the hands of the real object, and determine the gesture motion data of the real object according to the position data corresponding to the multiple optical markers;
  • the gesture action data is used for hand reconstruction to obtain the gesture action data of the virtual object.
  • the body capture system 21 is used for body capture by the first camera; meanwhile, the gesture capture system 22 is used for gesture capture by the first camera.
  • the first camera is an infrared camera.
  • the motion redirection system 30 is configured to perform redirection processing on the motion data of the virtual object to obtain motion data of the virtual character, where the motion data includes body motion data and/or gesture motion data.
  • the face capture system 40 includes an expression capture system 41 and an eye capture system 42 , and the expression capture system 41 and the eye capture system 42 are respectively connected to the face redirection system 50 .
  • the facial expression capture system 41 is used to obtain the facial video frame of the real object, and the facial video frame is a video frame including the face of the real object, and the facial video frame is used to indicate the facial data of the real object; According to the facial data of the real object, face reconstruction is performed to obtain Facial data of virtual objects.
  • the expression capturing system 41 is configured to acquire the facial video frame of the real subject through the second camera on the helmet of the real subject.
  • the second camera is a head-mounted RGB camera, or an RGBD camera.
  • the face redirection system 50 is used for redirecting the face data of the virtual object to obtain the face data of the virtual character.
  • the redirection process in the face redirection process is also called expression transfer process.
  • the motion data in the embodiments of the present disclosure includes body motion data and/or gesture motion data
  • the facial data includes expression data and/or eye gaze data.
  • the action data of the real object includes body action data and/or gesture action data of the real object
  • the facial data of the real object includes expression data and/or eye gaze data of the real object.
  • the motion data of the virtual object includes body motion data and/or gesture motion data of the virtual object
  • the facial data of the virtual object includes expression data and/or eye gaze data of the virtual object.
  • the motion data of the virtual character includes body motion data and/or gesture motion data of the virtual character
  • the facial data of the virtual character includes expression data and/or gaze data of the virtual character.
  • the action redirection system 30 , the face redirection system 50 , the prop redirection system 62 , the sound recording system 64 , and the virtual camera pose tracking system 70 are respectively connected with the animation synthesis system 92 .
  • the action redirection system 30 is also used for inputting the action data of the virtual character obtained after redirection into the animation synthesis system 92 .
  • the face redirection system 50 is also used for inputting the face data of the virtual character obtained after the redirection into the animation synthesis system 92 .
  • the prop capture system 60 is connected to the prop redirection system 62 .
  • the prop capture system 60 is used to acquire prop motion data of real props used by real objects in the performance process.
  • the prop capturing system 60 is used for capturing the movement of the real props used during the performance of the real object to obtain prop motion data of the real prop.
  • the prop motion data of the real prop is used to indicate the movement of the real prop, and the real prop may be a football, a basketball, a knife, a sword, a staircase, etc.
  • the embodiment of the present disclosure does not limit the type of the real prop.
  • the prop capture system 60 is used for prop capture through the first camera.
  • the first camera is an infrared camera.
  • the prop capturing system 60 is configured to obtain position data corresponding to the preset optical markers on the real props, determine the prop motion data of the real props according to the position data corresponding to the optical markers; Reconstruct to get the prop motion data of the virtual intermediate prop.
  • the virtual intermediate props are virtual models obtained by restoring and reconstructing real props.
  • the prop redirection system 62 is used for redirecting the prop motion data of the virtual intermediate prop to obtain the prop motion data of the virtual prop used by the virtual character. Wherein, the prop motion data of the virtual prop is used to indicate the movement of the virtual prop.
  • the prop redirection system 62 is also used to input the prop motion data of the virtual prop obtained after the redirection into the animation synthesis system 92 .
  • the sound recording system 64 is used for recording the sound of real objects during the performance to obtain sound recording data, and inputting the sound recording data into the animation synthesis system 92 .
  • the virtual camera pose tracking system 70 is used for capturing virtual cameras to obtain virtual camera pose data, and inputting the virtual camera pose data into the animation synthesis system 92 .
  • the virtual camera pose data is used to indicate the preview camera angle of the animation picture to be generated.
  • the virtual camera pose data includes: virtual camera position, virtual camera direction, and virtual camera parameters, such as virtual camera parameters including focal length.
  • the reference video recording system 80 is used to capture the performance content of the real object to obtain video recording data.
  • the video recording data can be used as reference data for the post-production of the animation to be generated. That is, the video recording data is the reference data for the animation composition system 92 .
  • body capture and gesture capture, expression capture and eye capture, sound recording, reference video recording, and prop capture are performed simultaneously.
  • body capture and gesture capture, expression capture and eye capture, sound recording, reference video recording and prop capture are done through different systems, due to communication delays, different signals may be out of sync, and the final animation is The above systems need to be completely synchronized. Therefore, a time code synchronization system 90 is added to the entire computer device 10, and each system in the computer device 10 is synchronized based on the same time code.
  • the animation synthesis system 92 is also called a rendering engine, and is used to combine the imported multiple data (including body motion data, gesture motion data, expression data, eye data, voice recording data, virtual camera pose data, and prop motion data of virtual props). ) is synchronized according to the time code, and after synchronization, multiple imported data are synthesized and rendered to obtain an animation video, and the generated animation video is displayed.
  • imported multiple data including body motion data, gesture motion data, expression data, eye data, voice recording data, virtual camera pose data, and prop motion data of virtual props.
  • a body motion capture system and a gesture motion capture system can be combined into one system, that is, a motion capture system, and an expression capture system and an eye-catching system can be combined into one system, that is, a face capture system.
  • the motion capture system and the motion redirection system can be combined into one system, namely the motion processing system, the face capture system and the face redirection system can be combined into one system, the emotion processing system, and the prop capture system and the prop redirection system can be combined into one system, namely the Prop handling system.
  • the above-mentioned systems can also all be combined into one system. This embodiment does not limit this.
  • FIG. 4 shows a flowchart of an animation generation method provided by another exemplary embodiment of the present disclosure. This embodiment is exemplified by using the method in the computer device shown in FIG. 3 . The method includes the following steps.
  • Step 401 Acquire motion data of a real object, where the motion data includes body motion data and/or gesture motion data.
  • the motion capture system acquires motion data of the real object, and the motion data includes body motion data and/or gesture motion data.
  • motion capture is to record the action data of the actor.
  • Motion capture is captured by devices worn or attached to actors.
  • the actor is wearing a suit with camera tracking markers or a suit with built-in sensors, and the motion capture process is completed by the reflection of the camera tracking markers or the movement of the sensor.
  • the position data corresponding to each of a plurality of preset optical marker points on the limb of the real object is obtained, and the limb motion data of the real object is determined according to the position data corresponding to each of the plurality of optical marker points; and/ Or, the position data corresponding to each of the plurality of optical marker points preset on the hand of the real object is obtained, and the gesture action data of the real object is determined according to the position data corresponding to each of the plurality of optical marker points.
  • the real object wears a set of motion capture clothing provided with a plurality of optical marking points, and the motion capture clothing covers the limbs and hands of the real object.
  • the motion capture clothing includes clothing covering the limbs of the real subject and gloves covering the hands of the real subject.
  • the motion capture system acquires motion data of the real object through the first optical capture device.
  • the first optical capture device is an infrared camera.
  • the motion capture system captures the positions of multiple reflective points (including multiple optical markers and multiple optical markers) on the real object through an infrared camera, and calculates the body of the real object and tracks the movements of the real object in real time. That is, the motion capture system determines the body of the real object and the motion data of the real object according to the positions of multiple reflection points, and the motion data includes body motion data and gesture motion data.
  • the limb motion data is used to indicate the limb motion of the real object, and the limb motion data includes three-dimensional position coordinates and motion parameters of each joint point on the limb of the real object.
  • the joint points are key joint points at preset positions, such as head, neck, shoulders, arms, legs, and so on.
  • the gesture action data is used to indicate the hand action of the real object, and the gesture action data includes the three-dimensional position coordinates and motion parameters of each joint point on the hand of the real object.
  • Step 402 Determine the motion data of the virtual character according to the motion data of the real object.
  • the motion capture system converts the motion data of the real object into the motion data of the virtual object, and the motion redirection system redirects the motion data of the virtual object to obtain the motion data of the virtual character.
  • the virtual object is a virtual model obtained by restoring and reconstructing the real object.
  • the virtual object is a virtual model obtained by performing one-to-one restoration and reconstruction of the real object.
  • the virtual object is a three-dimensional or two-dimensional virtual model.
  • the following description is only given by taking the virtual object as a three-dimensional virtual model as an example.
  • the meaning of the action data of the virtual object can be compared with the relevant description of the action data of the real object, and will not be repeated here.
  • the action redirection system performs redirection processing on the action data of the virtual object to obtain the action data of the virtual character, including: obtaining a first correspondence between the skeleton data of the virtual object and the skeleton data of the virtual character
  • the skeleton data is used to indicate the topological structure feature of the skeleton; according to the first corresponding relationship, the action data of the virtual object is redirected to the virtual character to obtain the action data of the virtual character.
  • a first correspondence between the skeleton data of the virtual object and the skeleton data of the virtual character is established.
  • the skeleton data of the virtual object is used to indicate the topological structure feature of the skeleton of the virtual object
  • the skeleton data of the virtual character is used to indicate the topological structure feature of the skeleton of the virtual character.
  • the topological features of the bones are used to indicate the distribution of the bones and the connection state between the bones.
  • the action redirection system redirects and processes the action data of the virtual object to the virtual character according to the first corresponding relationship, and obtains the action data of the virtual character, including: according to the first corresponding relationship, redirecting and processing the limb movement data of the virtual object to the virtual character.
  • the body motion data of the virtual character is obtained; and/or, the gesture motion data of the virtual object is redirected to the virtual character to obtain the gesture motion data of the virtual character.
  • the skin motion of the virtual character is driven and displayed according to the motion data of the virtual character and the binding relationship between the bones and the skin of the virtual character.
  • the binding relationship between the bones and the skin of the virtual character is preset.
  • Step 403 Acquire facial data of the real object, where the facial data includes expression data and/or eye gaze data.
  • a face video frame of a real object is acquired, the face video frame is a video frame including the face of the real object, and the face video frame is used to indicate the face data of the real object.
  • the face capture system acquires the face video frames of the real object through the second optical capture device.
  • the expression data of the real object is used to indicate the facial expression of the real object, and the expression data includes three-dimensional position coordinates and motion parameters of each feature point on the face of the real object.
  • Each feature point is the contour on the face of the real object and each feature point on the facial features.
  • the eye eye data of the real object is used to indicate the eye state of the real object, and the eye eye data includes three-dimensional position coordinates and motion parameters of each feature point on the eyeball of the real object.
  • This embodiment does not limit the data structures of the expression data and the eye gaze data.
  • Step 404 Determine the face data of the virtual character according to the face data of the real object.
  • the face capture system converts the face data of the real object into the face data of the virtual object, and the face redirection system reorients the face data of the virtual object to obtain the face data of the virtual character.
  • the virtual object is a virtual model obtained by restoring and reconstructing the real object.
  • the facial data of the virtual object includes expression data and/or eye gaze data of the virtual character. There is a mapping relationship between the face data of the virtual object and the face data of the real object. The meaning of the face data of the virtual object can be compared with the relevant description of the face data of the real object, and will not be repeated here.
  • the face capture system converts the face data of the real object into the face data of the virtual object, including: the face capture system invokes the first preset face processing model to output the face model of the virtual object according to the face data of the real object, and the face Models are used to indicate facial data of virtual objects.
  • the face video frame of the real object is a video frame including the face of the real object
  • the face video frame is data in a two-dimensional form
  • the face model of the virtual object is used to indicate the expression data and/or eye data of the virtual character
  • the face The model is data in three-dimensional form
  • the first preset facial processing model is used to convert the two-dimensional facial video frame of the real object into the three-dimensional facial model of the virtual object.
  • the first preset facial processing model is a pre-trained neural network model, which is used to represent the correlation between the facial video frame of the real object and the facial model of the virtual object.
  • the first preset facial processing model is a preset mathematical model
  • the first preset facial processing model includes model coefficients between the facial video frame of the real object and the facial model of the virtual object.
  • the model coefficients can be fixed values or dynamically modified values.
  • the facial redirection system performs redirection processing on the facial data of the virtual object to obtain the facial data of the avatar, including: obtaining the second correspondence between the facial data of the virtual object and the facial data of the avatar, the facial data using It is used to indicate the facial structure feature and the emotional style feature; according to the second correspondence, the facial data of the virtual object is redirected to the virtual character, and the facial data of the virtual character is obtained.
  • the facial data of the virtual object is used to indicate the facial structure feature and emotional style feature of the virtual object
  • the facial data of the virtual character is used to indicate the facial structure feature and emotional style feature of the virtual character.
  • the facial structure features are used to indicate the contour of the face and the distribution of facial features.
  • Emotional style features are used to indicate the emotions embodied by multiple feature points on the face, such as happy, sad, helpless and so on.
  • the facial redirection system redirects the facial data of the virtual object to the virtual character according to the second corresponding relationship, and obtains the facial data of the virtual character, including: according to the second corresponding relationship, replaying the facial expression data of the virtual object.
  • Orientation processing is performed on the virtual character to obtain the expression data of the virtual character; and/or, the eye gaze data of the virtual object is redirected and processed on the virtual character to obtain the gaze data of the virtual character.
  • the facial redirection system performs redirection processing on the facial data of the virtual object to obtain the facial data of the virtual character, including: the facial redirection system calls the second preset facial processing model to output to obtain the virtual character according to the facial data of the virtual object. facial data.
  • the second preset facial processing model is a pre-trained neural network model, which is used to represent the correlation between the facial data of the virtual object and the facial data of the virtual character.
  • the second preset facial processing model is a preset mathematical model
  • the second preset facial processing model includes model coefficients between the facial data of the virtual object and the facial data of the virtual character.
  • the model coefficients can be fixed values or dynamically modified values.
  • the redirection processing in the face redirection process is also called expression migration processing, and the embodiment of the present disclosure does not limit the specific implementation of the expression migration processing.
  • the face capture system captures the actor's expression and eyes during the performance through the head-mounted RGB camera worn by the actor.
  • a video the video includes a plurality of facial video frames; for each facial video frame, a three-dimensional facial model of a virtual object is reconstructed, and the three-dimensional facial model of the virtual object is redirected to obtain the facial data of the virtual character.
  • Face tracing method mark a number of points on the actor's face, capture the face, and obtain face information
  • face no tracing method there is no mark on the actor's face, and the algorithm is used to directly extract information on the actor's face , to capture the face and obtain the face information.
  • a single camera or multiple cameras can be used to capture the face.
  • a single camera is light and easy to wear, and it can also achieve the result of multiple cameras. Multiple cameras can capture face data from multiple angles. For capture devices, RGB cameras and/or RGBD cameras may be employed.
  • Step 405 Obtain reference data, where the reference data includes sound recording data and/or virtual camera pose data of the real object during the performance.
  • the sound recording system records the sound of the real object to obtain the sound recording data of the real object
  • the virtual camera pose tracking system monitors the virtual camera. Capture the virtual camera pose data.
  • the virtual camera pose data is used to indicate the preview camera angle of the animation picture to be generated.
  • the preview camera angle is the angle when the virtual character and/or other scene information is observed through the virtual camera in the virtual environment. That is, the animation picture to be generated is the animation picture collected by observing the virtual character from the perspective of the virtual camera.
  • the virtual camera pose data includes: virtual camera position, virtual camera direction, and virtual camera parameters, such as virtual camera parameters including focal length.
  • steps 401 and 402 the process of capturing and redirecting motion data shown in steps 401 and 402 is the same as the process of capturing and redirecting facial data shown in steps 403 and 404, and the acquisition of reference data shown in step 405. Processes can be executed in parallel, in no particular order.
  • Step 406 generating an animation of the virtual character according to the target feature data and the reference data.
  • the animation synthesis system generates animations of virtual characters in real time or offline according to target feature data and reference data.
  • the animation of the virtual character includes three-dimensional or two-dimensional animation.
  • the sound recording system After the sound recording system records the sound recording data of the real object, the sound recording data is recorded into the animation synthesis system.
  • the animation synthesis system determines the sound data of the virtual character according to the input sound recording data.
  • the sound recording data of the real object is the sound data of the virtual character, or the sound recording data of the real object is subjected to preset sound processing to obtain the sound data of the virtual character, or the sound recording data is replaced with dubbing data to obtain the virtual character. sound data.
  • This embodiment of the present disclosure does not limit this.
  • the virtual camera pose tracking system After the virtual camera pose tracking system captures the virtual camera pose data, the virtual camera pose data is entered into the animation synthesis system.
  • the animation synthesis system determines the preview camera angle of the animation to be generated according to the entered virtual camera pose data.
  • the target feature data and the reference data both carry a time code
  • the animation synthesis system generates an animation of the virtual character according to the target feature data and the reference data, including: according to the respective time codes corresponding to the target feature data and the reference data, the target feature data and the reference data.
  • the feature data and the reference data are aligned; according to the aligned target feature data and the reference data, an animation of the virtual character is generated.
  • the target feature data and the reference data after the alignment process are time-synchronized data.
  • motion data, face data, voice recording data and virtual camera pose data all carry time codes
  • the animation synthesis system will import motion data, face data, voice recording data and virtual camera pose data according to the time code. Alignment processing, compositing and rendering after alignment processing to obtain animation video.
  • the animation synthesis system generates an animation of the virtual character according to the target feature data and the reference data after the alignment process, including: after the alignment process, according to the virtual camera pose data and the target feature data, obtaining an animation picture, the virtual camera pose
  • the data is used to indicate the preview camera angle of the animation picture to be generated; the animation picture is rendered to obtain the rendering result; the animation video of the virtual character is generated according to the rendering result and the sound recording data.
  • the virtual camera pose data includes: virtual camera position, virtual camera direction, and virtual camera parameters, such as virtual camera parameters including focal length.
  • the animation synthesis system obtains video recording data carrying a time code, and the video recording data includes video data obtained by recording the performance content of a real object.
  • the video recording data can be used as reference data for the animation to be generated.
  • the animation synthesis system After the animation synthesis system generates the animation of the virtual character, the animation of the virtual character is displayed. It should be noted that, for the relevant details of the animation of the virtual character displayed on the display screen, reference may be made to the relevant descriptions in the foregoing embodiments, which will not be repeated here.
  • the motion capture system captures the positions of multiple reflection points on the actor through an infrared camera.
  • the action data of the actor is reconstructed into action data of the virtual actor model at the positions of the multiple reflection points, and the action redirection system performs redirection processing on the action data of the virtual actor model to obtain the action data of the virtual animation character.
  • the face capture system obtains the actor's face video frame through the head-mounted RGB camera or RGBD camera worn by the actor, and converts the actor's face video frame into the face data of the virtual actor model.
  • the directional processing obtains the motion data of the virtual animation character.
  • the voice recording system While capturing the actor's skeletal movements and the actor's facial emotions, the voice recording system records the actor's voice to obtain voice recording data, and the virtual camera pose tracking system records the virtual camera pose and motion trajectory to obtain virtual camera pose data.
  • the various systems described above are synchronized based on the same time code.
  • the animation synthesis system obtains multiple imported data, including motion data (body movement data and gesture action data), facial data (expression data and eye data), voice recording data, and virtual camera pose data.
  • the animation synthesis system will The imported data is synchronized according to the time code, and after synchronization, an animation is generated according to the imported data, and the generated animation is displayed.
  • the method further includes: acquiring prop motion data of real props used by real objects in the performance process; determining prop motion data of virtual props used by the virtual character according to the real prop data; Motion data to generate animations of virtual characters.
  • the prop capture system obtains the prop motion data of the real prop used by the real object; the prop capture system converts the prop motion data of the real prop into the prop motion data of the virtual intermediate prop, and the prop redirection system converts the virtual intermediate The prop motion data of the prop is redirected to obtain the prop motion data of the virtual prop.
  • the real prop is a football
  • the actor plays the football
  • the football will move such as moving and rotating.
  • the prop motion data of the intermediate soccer ball the prop redirection system redirects the prop motion data of the virtual intermediate soccer ball to obtain the prop motion data of the virtual soccer ball.
  • the real prop is a sword
  • the actor swings the sword
  • the sword moves
  • the prop capture system captures the movement of the sword to obtain the prop motion data of the sword, and reconstructs the prop according to the prop motion data of the sword to obtain the virtual middle sword.
  • Prop motion data; the prop redirection system redirects the prop motion data of the virtual middle sword to obtain the prop motion data of the virtual sword.
  • prop capture system performs prop capture
  • reference motion capture system performs motion capture
  • the manner in which the prop redirection system performs redirection processing may be analogous to the manner in which the reference action redirection system performs redirection processing, and details are not described herein again.
  • the prop motion data of the virtual prop also carries a time code
  • the animation synthesis system aligns the target feature data, the reference data and the prop motion data according to the respective time codes corresponding to the target feature data, reference data and prop motion data. ; According to the target feature data, reference data and prop motion data after alignment processing, the animation of the virtual character is generated.
  • the animation synthesis system obtains an animation picture according to the virtual camera pose data, target feature data and prop motion data after the alignment processing; renders the animation picture to obtain a rendering result; and generates a virtual character according to the rendering result and the sound recording data. animation video.
  • the embodiment of the present disclosure also converts real feature data into virtual feature data of a virtual object, where the virtual object is a virtual model obtained by restoring and reconstructing the real object, and the virtual feature data includes motion data and facial data of the virtual object ; wherein, the action data includes body action data and/or gesture action data, and the facial data includes expression data and/or eye data; it can more accurately reflect the details of the skeletal movements and facial emotions of the virtual character, so that the generated virtual character is more Vivid and natural, ensuring the animation effect of virtual characters.
  • the embodiment of the present disclosure also obtains reference data, including sound recording data and/or virtual camera pose data of the real object during the performance; according to the target feature data and the reference data, the animation of the virtual character is generated in real time;
  • the WYSIWYG method of virtual shooting can see the performance of real objects in real time on the spot, and can confirm the performance on the spot, which improves the shooting efficiency.
  • the target feature data and the reference data both carry time codes, and the target feature data and the reference data are aligned according to the respective time codes corresponding to the target feature data and the reference data; according to the aligned target feature data and reference data to generate the animation of the virtual character; the skeletal movement, facial emotion, sound and virtual camera pose of the virtual character are synchronized, which enriches the display details of the animation of the virtual character and ensures the natural fluency of the animation. , which further ensures the display effect of the animation.
  • the animation generation method can be used in the field of performance animation.
  • the animation generation method mentioned above can realize the capture of a single person or a plurality of people, that is, the output of a single virtual character or the output of multiple virtual characters can be realized in the same picture.
  • interactions between actors such as hugs, handshakes, etc., can be captured, and the interactions of virtual characters are output according to the interactions between multiple actors.
  • the animation generation method supports offline mode and real-time online mode.
  • the animation data can be processed and adjusted offline by using the data processing tool in the animation processing system 10 .
  • the animation data can be refined using the animation refinement tool in the animation processing system 10, allowing the animator to improve the animation quality and control the animation style.
  • Offline processing and animation refinement is available for both body and face.
  • offline animation production can be applied to character animation in film and television animation, game animation, virtual short video or character animation in variety shows, etc.
  • the animation generation method can be used for real-time animation, support real-time live broadcast, and real-time interaction of virtual characters, etc. such as interactions between virtual characters.
  • FIG. 5 shows a schematic structural diagram of an animation generating apparatus provided by an exemplary embodiment of the present disclosure.
  • the animation generating apparatus can be implemented as all or a part of the user equipment through software, hardware and a combination of the two.
  • the apparatus includes: an acquisition module 510 , a determination module 520 and a generation module 530 .
  • the acquisition module 510 is used to acquire the real feature data of the real object, and the real feature data includes the action data and the facial data of the real object in the performance process;
  • the determination module 520 is used to determine the target feature data of the virtual character according to the real feature data, the virtual character is a preset animation model, and the target feature data includes the action data and facial data of the virtual character;
  • the generating module 530 is configured to generate the animation of the virtual character according to the target feature data.
  • the obtaining module 510 is further configured to obtain reference data, where the reference data includes sound recording data and/or virtual camera pose data of the real object during the performance;
  • the generating module 530 is further configured to generate the animation of the virtual character according to the target feature data and the reference data.
  • the target feature data and the reference data both carry time codes
  • the generating module 530 is further configured to:
  • the target feature data and the reference data are aligned
  • the animation of the virtual character is generated according to the target feature data and reference data after the alignment processing.
  • the generating module 530 is further configured to:
  • an animation picture is obtained according to the virtual camera pose data and the target feature data, and the virtual camera pose data is used to indicate the preview camera angle of the animation picture to be generated;
  • the obtaining module 510 is further configured to:
  • the motion data includes body motion data and/or gesture motion data;
  • Acquire facial data of real objects including facial expression data and/or eye gaze data.
  • the obtaining module 510 is further configured to:
  • the position data corresponding to each of the plurality of optical markers preset on the hand of the real object is acquired, and the gesture action data of the real object is determined according to the position data corresponding to each of the plurality of optical markers.
  • the obtaining module 510 is further configured to:
  • a face video frame of the real object is acquired, the face video frame is a video frame including the face of the real object, and the face video frame is used to indicate the face data of the real object.
  • the determining module 520 is further configured to:
  • the virtual object is a virtual model obtained by restoring and reconstructing the real object, and the virtual feature data includes motion data and facial data of the virtual object;
  • the target feature data of the virtual character is obtained by redirecting the virtual feature data.
  • the determining module 520 is further configured to:
  • the motion data including body motion data and/or gesture motion data
  • the facial data of the virtual object is redirected to obtain the facial data of the virtual character, and the facial data includes expression data and/or eye data.
  • the determining module 520 is further configured to:
  • the skeleton data is used to indicate the topological structure feature of the skeleton
  • the action data of the virtual object is redirected to the virtual character to obtain the action data of the virtual character.
  • the determining module 520 is further configured to:
  • the facial data of the virtual object is redirected to the virtual character to obtain the facial data of the virtual character.
  • the apparatus further includes: a display module
  • the display module is used for driving and displaying the skin motion of the virtual character according to the action data of the virtual character and the binding relationship between the bones and the skin of the virtual character.
  • the apparatus further includes: a recording module
  • the recording module is used for acquiring video recording data carrying a time code, and the video recording data includes video data obtained by recording the performance content of a real object.
  • the acquiring module 510 is further configured to acquire the prop motion data of the real props used by the real object in the performance process;
  • the determining module 520 is further configured to determine the prop motion data of the virtual prop used by the virtual character according to the real prop data;
  • the generating module 530 is further configured to generate the animation of the virtual character according to the target feature data and the prop motion data of the virtual prop.
  • An embodiment of the present disclosure further provides a computer device, the computer device comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: implement the above-mentioned method embodiments by The steps performed by the computer equipment.
  • the embodiment of the present disclosure also provides an animation generation system, the animation generation system includes:
  • a motion-capture garment wherein a plurality of optical marking points are arranged on the motion-capture garment;
  • a first camera where the first camera is used to capture motion data when a real object is performing
  • a helmet a second camera is arranged on the helmet, and the second camera is used to capture the facial data of the real object when performing;
  • a computer device where the computer device is configured to perform the steps performed by the computer device in each of the foregoing method embodiments.
  • Embodiments of the present disclosure further provide a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implement the methods in the foregoing method embodiments.
  • the present disclosure may be a system, method and/or computer program product.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present disclosure.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM erasable programmable read only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disk read only memory
  • DVD digital versatile disk
  • memory sticks floppy disks
  • mechanically coded devices such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • Computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.
  • the computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • Computer program instructions for carrying out operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages.
  • Source or object code written in any combination, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect).
  • LAN local area network
  • WAN wide area network
  • custom electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) can be personalized by utilizing state information of computer readable program instructions.
  • Computer readable program instructions are executed to implement various aspects of the present disclosure.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium storing the instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Computation (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种动画生成方法、装置、系统及存储介质，涉及动画技术领域。所述方法包括：获取真实对象的真实特征数据，所述真实特征数据包括所述真实对象在表演过程中的动作数据和面部数据（201）；根据所述真实特征数据确定虚拟角色的目标特征数据，所述虚拟角色为预设的动画模型，所述目标特征数据包括所述虚拟角色的动作数据和面部数据（202）；根据所述目标特征数据，生成所述虚拟角色的动画（203）。上述方法通过采用真实对象的表演生成虚拟角色的动画，在一方面，避免了手动绘制的情况，提高了动画生成的效率；在另一方面，可以将真实对象的细腻表演直接迁移到虚拟角色上，虚拟角色的骨骼动作和面部情绪更加真实生动，保证了动画制作效果。

Description

动画生成方法、装置、系统及存储介质 技术领域
本公开涉及动画技术领域，尤其涉及一种动画生成方法、装置、系统及存储介质。
背景技术
在动画制作中，很多动画都要在高度浓缩的时间表内完成非常复杂的制作工作。传统的动画制作流程是典型的线性流程，类似于流水线，包括了前期、中期和后期，动画制作效果受限于动画师的绘制水平和效率。
与传统制作流程相比，使用虚拟制作动画已是主流模式。虚拟制作动画在各个方面的技术需求也大大的增加，如何使用虚拟制作动画以提高动画制作效率，保证动画制作效果，相关技术中尚未提供一种合理且有效的技术方案。
发明内容
有鉴于此，本公开提出了一种动画生成方法、装置、系统及存储介质。所述技术方案包括：
根据本公开的一方面，提供了一种动画生成方法，所述方法包括：
获取真实对象的真实特征数据，所述真实特征数据包括所述真实对象在表演过程中的动作数据和面部数据；
根据所述真实特征数据确定虚拟角色的目标特征数据，所述虚拟角色为预设的动画模型，所述目标特征数据包括所述虚拟角色的动作数据和面部数据；
根据所述目标特征数据，生成所述虚拟角色的动画。
在一种可能的实现方式中，所述方法还包括：
获取参考数据，所述参考数据包括所述真实对象在表演过程中的声音录制数据和/或虚拟相机位姿数据；
所述根据所述目标特征数据，生成所述虚拟角色的动画，包括：
根据所述目标特征数据和所述参考数据，生成所述虚拟角色的动画。
在另一种可能的实现方式中，所述目标特征数据和所述参考数据均携带有时间码，所述根据所述目标特征数据和所述参考数据，生成所述虚拟角色的动画，包括：
根据所述目标特征数据和所述参考数据各自对应的所述时间码，将所述目标特征数据和所述参考数据进行对齐处理；
根据对齐处理后的所述目标特征数据和所述参考数据，生成所述虚拟角色的动画。
在另一种可能的实现方式中，所述根据对齐处理后的所述目标特征数据和所述参考数据，生成所述虚拟角色的动画，包括：
在对齐处理后根据所述虚拟相机位姿数据和所述目标特征数据，得到动画画面，所述虚拟相机位姿数据用于指示待生成的动画画面的预览相机视角；
对所述动画画面进行渲染得到渲染结果；
根据所述渲染结果和所述声音录制数据，生成所述虚拟角色的动画视频。
在另一种可能的实现方式中，所述获取真实对象的真实特征数据，包括：
获取所述真实对象的所述动作数据，所述动作数据包括肢体动作数据和/或手势动作数据；以及，
获取所述真实对象的所述面部数据，所述面部数据包括表情数据和/或眼神数据。
在另一种可能的实现方式中，所述获取所述真实对象的所述动作数据，包括：
获取所述真实对象的肢体上预设的多个光学标记点各自对应的位置数据，根据所述多个光学标记点各自对应的所述位置数据确定所述真实对象的所述肢体动作数据；和/或，
获取所述真实对象的手部上预设的多个光学标记点各自对应的位置数据，根据所述多个光学标记 点各自对应的所述位置数据确定所述真实对象的所述手势动作数据。
在另一种可能的实现方式中，所述获取所述真实对象的所述面部数据，包括：
获取所述真实对象的面部视频帧，所述面部视频帧为包括所述真实对象的面部的视频帧，所述面部视频帧用于指示所述真实对象的所述面部数据。
在另一种可能的实现方式中，所述根据所述真实特征数据确定虚拟角色的目标特征数据，包括：
将所述真实特征数据转化为虚拟对象的虚拟特征数据，所述虚拟对象为对所述真实对象进行还原重建得到的虚拟模型，所述虚拟特征数据包括所述虚拟对象的动作数据和面部数据；
将所述虚拟特征数据进行重定向处理得到所述虚拟角色的所述目标特征数据。
在另一种可能的实现方式中，所述将所述虚拟特征数据进行重定向处理得到所述虚拟角色的所述目标特征数据，包括：
将所述虚拟对象的动作数据进行重定向处理，得到所述虚拟角色的动作数据，所述动作数据包括肢体动作数据和/或手势动作数据；以及，
将所述虚拟对象的面部数据进行重定向处理，得到所述虚拟角色的面部数据，所述面部数据包括表情数据和/或眼神数据。
在另一种可能的实现方式中，所述将所述虚拟对象的动作数据进行重定向处理，得到所述虚拟角色的动作数据，包括：
获取所述虚拟对象的骨骼数据与所述虚拟角色的骨骼数据之间的第一对应关系，所述骨骼数据用于指示骨骼的拓扑结构特征；
根据所述第一对应关系，将所述虚拟对象的动作数据重定向处理至所述虚拟角色上，得到所述虚拟角色的动作数据。
在另一种可能的实现方式中，所述将所述虚拟对象的面部数据进行重定向处理，得到所述虚拟角色的面部数据，包括：
获取所述虚拟对象的面部数据与所述虚拟角色的面部数据之间的第二对应关系，所述面部数据用于指示面部结构特征和情绪风格特征；
根据所述第二对应关系，将所述虚拟对象的面部数据重定向处理至所述虚拟角色上，得到所述虚拟角色的所述面部数据。
在另一种可能的实现方式中，所述根据所述真实特征数据确定虚拟角色的目标特征数据之后，还包括：
根据所述虚拟角色的动作数据、以及所述虚拟角色的骨骼和蒙皮的绑定关系，驱动并显示所述虚拟角色的蒙皮运动。
在另一种可能的实现方式中，所述根据所述目标特征数据，生成所述虚拟角色的动画之前，还包括：
获取携带有时间码的视频录制数据，所述视频录制数据包括对所述真实对象的表演内容进行录制得到的视频数据。
在另一种可能的实现方式中，所述方法还包括：
获取所述真实对象在表演过程中所用的真实道具的道具运动数据；
根据所述真实道具数据确定所述虚拟角色所用的虚拟道具的道具运动数据；
所述根据所述目标特征数据，生成所述虚拟角色的动画，包括：
根据所述目标特征数据和所述道具运动数据，生成所述虚拟角色的动画。
根据本公开的另一方面，提供了一种动画生成装置，所述装置包括：
获取模块，用于获取真实对象的真实特征数据，所述真实特征数据包括所述真实对象在表演过程中的动作数据和面部数据；
确定模块，用于根据所述真实特征数据确定虚拟角色的目标特征数据，所述虚拟角色为预设的动画模型，所述目标特征数据包括所述虚拟角色的动作数据和面部数据；
生成模块，用于根据所述目标特征数据，生成所述虚拟角色的动画。
根据本公开的另一方面，提供了一种计算机设备，所述计算机设备包括：处理器；用于存储处理器可执行指令的存储器；
其中，所述处理器被配置为：
获取真实对象的真实特征数据，所述真实特征数据包括所述真实对象在表演过程中的动作数据和面部数据；
根据所述真实特征数据确定虚拟角色的目标特征数据，所述虚拟角色为预设的动画模型，所述目标特征数据包括所述虚拟角色的动作数据和面部数据；
根据所述目标特征数据，生成所述虚拟角色的动画。
根据本公开的另一方面，提供了一种动画生成系统，所述动画生成系统包括：
动捕服装，所述动捕服装上设置有多个光学标记点；
第一相机，所述第一相机用于捕捉真实对象表演时的动作数据；
头盔，所述头盔上设置有第二相机，所述第二相机用于捕捉所述真实对象表演时的面部数据；
计算机设备，所述计算机设备用于执行上述的方法。
根据本公开的另一方面，提供了一种非易失性计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时实现上述的方法。
本公开实施例获取真实对象的真实特征数据，真实特征数据包括真实对象在表演过程中的动作数据和面部数据；根据真实特征数据确定虚拟角色的目标特征数据，虚拟角色为预设的动画模型，目标特征数据包括虚拟角色的动作数据和面部数据；根据目标特征数据，生成虚拟角色的动画；即采用真实对象的表演生成虚拟角色的动画，在一方面，避免了手动绘制的情况，提高了动画生成的效率；在另一方面，可以将真实对象的细腻表演直接迁移到虚拟角色上，虚拟角色的骨骼动作和面部情绪更加真实生动，保证了动画制作效果。
附图说明
包含在说明书中并且构成说明书的一部分的附图与说明书一起示出了本公开的示例性实施例、特征和方面，并且用于解释本公开的原理。
图1示出了本公开一个示例性实施例提供的计算机设备的结构示意图；
图2示出了本公开一个示例性实施例提供的动画生成方法的流程图；
图3示出了本公开另一个示例性实施例提供的计算机设备的结构示意图；
图4示出了本公开另一个示例性实施例提供的动画生成方法的流程图；
图5示出了本公开一个示例性实施例提供的动画生成装置的结构示意图。
具体实施方式
以下将参考附图详细说明本公开的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
另外，为了更好的说明本公开，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本公开同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本公开的主旨。
请参考图1，其示出了本公开一个示例性实施例提供的计算机设备的结构示意图。
本公开实施例中的动画生成方法可以由计算机设备执行。
计算机设备可以是包括多个设备或者系统的处理系统。比如，计算机设备为一台服务器，或者由若干台服务器组成的服务器集群，或者是一个云计算服务中心。本公开实施例对此不加以限定。为了方便说明，仅以计算机设备为一台服务器为例进行介绍。如图1所示，计算机设备包括处理器110、存储器120以及通信接口130。本领域技术人员可以理解，图1中示出的结构并不构成对该计算机设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。其中：
处理器110是计算机设备的控制中心，利用各种接口和线路连接整个计算机设备的各个部分，通过运行或执行存储在存储器120内的软件程序和/或模块，以及调用存储在存储器120内的数据，执行计算机设备的各种功能和处理数据，从而对计算机设备进行整体控制。处理器110可以由CPU实现，也可以由图形处理器(Graphics Processing Unit，GPU)实现。
存储器120可用于存储软件程序以及模块。处理器110通过运行存储在存储器120的软件程序以及模块，从而执行各种功能应用以及数据处理。存储器120可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、虚拟模块和至少一个功能所需的应用程序(比如神经网络模型训练等)等；存储数据区可存储根据计算机设备的使用所创建的数据等。存储器120可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(Static Random Access Memory，SRAM)，电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)，可擦除可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)，可编程只读存储器(Programmable Read-Only Memory，PROM)，只读存储器(Read Only Memory，ROM)，磁存储器，快闪存储器，磁盘或光盘。相应地，存储器120还可以包括存储器控制器，以提供处理器110对存储器120的访问。
其中，处理器110用于执行以下功能：获取真实对象的真实特征数据，真实特征数据包括真实对象在表演过程中的动作数据和面部数据；根据真实特征数据确定虚拟角色的目标特征数据，虚拟角色为预设的动画模型，目标特征数据包括虚拟角色的动作数据和面部数据；根据目标特征数据，生成虚拟角色的动画。
本公开实施例提供的动画生成方法可以应用于影视previs制作，动画番剧制作，游戏CG、游戏动画、游戏动作的制作，虚拟动画短视频的制作，同时，也是虚拟直播的技术基础。比如，本公开实施例提供的动画生成方法应用于虚拟人物的表演动画离线制作的应用领域，特别是三维动画领域。本公开实施例对此不加以限定。
下面，采用几个示例性实施例对本公开实施例提供的动画生成方法进行介绍。
请参考图2，其示出了本公开一个示例性实施例提供的动画生成方法的流程图，本实施例以该方法用于图1所示的计算机设备中来举例说明。该方法包括以下几个步骤。
步骤201，获取真实对象的真实特征数据，真实特征数据包括真实对象在表演过程中的动作数据和面部数据。
在真实对象的表演过程中，计算机设备通过光学捕捉设备对真实对象的骨骼动作进行捕捉，得到该真实对象的动作数据；同时，通过光学捕捉设备对真实对象的面部情绪进行捕捉，得到该真实对象的面部数据。
可选的，光学捕捉设备包括红外相机、RGB相机和深度相机中的至少一种。本公开实施例对光学捕捉设备的类型不加以限定。
真实对象为在真实环境中的可活动对象。比如，真实对象为人物。本公开实施例对此不加以限定。下面仅以真实对象为人物为例进行说明。
真实特征数据包括真实对象在表演过程中的动作数据和面部数据，动作数据用于指示真实对象的骨骼动作，面部数据用于指示真实对象的面部情绪。
其中，真实对象的动作数据包括肢体动作数据和/或手势动作数据，肢体动作数据用于指示真实对象的肢体动作，手势动作数据用于指示真实对象的手部动作。
需要说明的是，本公开实施例中肢体为身体中除了手部以外的身体部位，即真实对象的身体包括 真实对象的肢体和除肢体以外的手部。
真实对象的面部数据包括表情数据和/或眼神数据，表情数据用于指示真实对象的面部表情，眼神数据用于指示真实对象的眼球状态。
步骤202，根据真实特征数据确定虚拟角色的目标特征数据，虚拟角色为预设的动画模型，目标特征数据包括虚拟角色的动作数据和面部数据。
计算机设备将真实对象的真实特征数据转化为虚拟角色的目标特征数据。
可选的，虚拟角色为预设的三维或者二维动画模型。虚拟角色为在虚拟环境中的可活动对象。可选的，虚拟角色为虚拟人物、虚拟动物、虚拟宠物或者其他虚拟形态的对象。
虚拟角色的目标特征数据包括虚拟角色的动作数据和面部数据。虚拟角色的动作数据包括肢体动作数据和/或手势动作数据，虚拟角色的面部数据包括表情数据和/或眼神数据。
目标特征数据与真实特征数据是相对应的，目标特征数据的含义可类比参考真实特征数据的相关描述，在此不再赘述。
需要说明的是，计算机设备获取真实对象的面部数据可以是以帧为单位获取的，后续根据真实特征数据确定虚拟角色的面部数据也可以是以帧为单位对应转化的。本公开实施例对此不加以限定。
步骤203，根据目标特征数据，生成虚拟角色的动画。
可选的，计算机设备根据虚拟角色的目标特征数据，实时或者离线生成虚拟角色的动画。
虚拟角色的动画包括三维或者二维动画。下面，为了方便说明，仅以虚拟角色的动画为三维动画为例进行说明。
可选的，计算机设备显示目标用户界面，该目标用户界面包括生成的虚拟角色的三维动画。该目标用户界面还可以包括在虚拟环境中展示的虚拟角色的三维动画，虚拟环境为三维虚拟环境。虚拟环境为营造出的供虚拟角色进行活动的场景。该虚拟环境可以是对真实世界的仿真环境，也可以是半仿真半虚构的环境，还可以是纯虚构的环境。
在一个示意性的例子中，真实对象为人物，比如演员，演员按照预设的剧本和分镜要求，根据导演的指导，将剧情中角色所需要的表演，包括肢体动作、手势动作、表情和眼神表演出来，相应的捕捉设备对该演员的肢体动作、手势动作、表情和眼神进行捕捉，计算机设备获得到该演员的真实特征数据，将该演员的真实特征数据转化为虚拟人物的目标特征数据，即将演员的肢体动作和手势动作转移到虚拟人物上，将演员的表情和眼神转移到虚拟人物的面部，基于该目标特征数据生成并显示该虚拟人物的三维动画。
综上所述，本公开实施例提供的动画生成方法为基于表演的动画制作方法，在真实对象的表演过程中，获取真实对象的真实特征数据即真实对象的动作数据和面部数据；根据真实特征数据确定虚拟角色的目标特征数据，根据目标特征数据生成虚拟角色的动画；即采用真实对象的表演生成虚拟角色的动画，在一方面，避免了手动绘制的情况，提高了动画生成的效率；在另一方面，可以将真实对象的细腻表演直接迁移到虚拟角色上，虚拟角色的骨骼动作和面部情绪更加真实生动，保证了动画制作效果。
本公开实施例提供了一种动画生成系统，该动画生成系统包括：动捕服装，动捕服装上设置有多个光学标记点；第一相机，第一相机用于捕捉真实对象表演时的动作数据；头盔，头盔上设置有第二相机，第二相机用于捕捉真实对象表演时的面部数据；第一相机和第二相机分别与计算机设备建立有通信连接，计算机设备用于执行本公开实施例提供的动画生成方法。其中，设置至少两个第一相机用于捕捉真实对象表演时的动作数据，根据实际情况可设置多个数量的第一相机；设置至少一个第二相机，用于捕捉真实对象表演时的面部数据，根据实际情况可设置多个数量的第二相机。
请参考图3，其示出了本公开另一个示例性实施例提供的计算机设备的结构示意图。
该计算机设备10包括动作捕捉系统20、动作重定向系统30、面部捕捉系统40、面部重定向系统50、道具捕捉系统60、道具重定向系统62、声音录制系统64、虚拟相机位姿跟踪系统70、参考视频录制系 统80和动画合成系统92。
动作捕捉系统20包括肢体捕捉系统21和手势捕捉系统22，肢体捕捉系统21和手势捕捉系统22分别与动作重定向系统30相连。
肢体捕捉系统21用于获取真实对象的肢体上预设的多个光学标记点各自对应的位置数据，根据多个光学标记点各自对应的位置数据确定真实对象的肢体动作数据；根据真实对象的肢体动作数据进行肢体重建，得到虚拟对象的肢体动作数据。
其中，虚拟对象为对真实对象进行还原重建得到的虚拟模型。
手势捕捉系统22用于获取真实对象的手部上预设的多个光学标记点各自对应的位置数据，根据多个光学标记点各自对应的位置数据确定真实对象的手势动作数据；根据真实对象的手势动作数据进行手部重建，得到虚拟对象的手势动作数据。
可选的，肢体捕捉系统21用于通过第一相机进行肢体捕捉；同时，手势捕捉系统22用于通过第一相机进行手势捕捉。示意性的，第一相机为红外相机。
动作重定向系统30用于将虚拟对象的动作数据进行重定向处理，得到虚拟角色的动作数据，动作数据包括肢体动作数据和/或手势动作数据。
面部捕捉系统40包括表情捕捉系统41和眼神捕捉系统42，表情捕捉系统41和眼神捕捉系统42分别与面部重定向系统50相连。
表情捕捉系统41用于获取真实对象的面部视频帧，面部视频帧为包括真实对象的面部的视频帧，面部视频帧用于指示真实对象的面部数据；根据真实对象的面部数据进行面部重建，得到虚拟对象的面部数据。
可选的，表情捕捉系统41用于通过真实对象的头盔上的第二相机获取真实对象的面部视频帧。比如，第二相机为头戴式RGB相机，或者RGBD相机。
面部重定向系统50用于将虚拟对象的面部数据进行重定向处理，得到虚拟角色的面部数据。本公开实施例中，面部重定向过程中的重定向处理也称为表情迁移处理。
需要说明的是，本公开实施例中的动作数据包括肢体动作数据和/或手势动作数据，面部数据包括表情数据和/或眼神数据。即，真实对象的动作数据包括真实对象的肢体动作数据和/或手势动作数据，真实对象的面部数据包括真实对象的表情数据和/或眼神数据。虚拟对象的动作数据包括虚拟对象的肢体动作数据和/或手势动作数据，虚拟对象的面部数据包括虚拟对象的表情数据和/或眼神数据。虚拟角色的动作数据包括虚拟角色的肢体动作数据和/或手势动作数据，虚拟角色的面部数据包括虚拟角色的表情数据和/或眼神数据。动作重定向系统30、面部重定向系统50、道具重定向系统62、声音录制系统64、虚拟相机位姿跟踪系统70分别与动画合成系统92相连。
动作重定向系统30还用于将重定向后得到的虚拟角色的动作数据输入至动画合成系统92中。
面部重定向系统50还用于将重定向后得到的虚拟角色的面部数据输入至动画合成系统92中。
道具捕捉系统60与道具重定向系统62相连。道具捕捉系统60用于获取真实对象在表演过程中所用的真实道具的道具运动数据。
道具捕捉系统60用于在真实对象的表演过程中，捕捉所用的真实道具的运动得到真实道具的道具运动数据。其中，真实道具的道具运动数据用于指示真实道具的运动，真实道具可以是足球、篮球、刀、剑和楼梯等，本公开实施例对真实道具的类型不加以限定。
可选地，道具捕捉系统60用于通过第一相机进行道具捕捉。示意性的，第一相机为红外相机。
可选地，道具捕捉系统60用于获取真实道具上预设的光学标记点对应的位置数据，根据光学标记点对应的位置数据确定真实道具的道具运动数据；根据真实道具的道具运动数据进行道具重建，得到虚拟中间道具的道具运动数据。
其中，虚拟中间道具为对真实道具进行还原重建得到的虚拟模型。
道具重定向系统62用于将虚拟中间道具的道具运动数据进行重定向处理，得到虚拟角色所用的虚拟道具的道具运动数据。其中，虚拟道具的道具运动数据用于指示虚拟道具的运动。
道具重定向系统62还用于将重定向后得到的虚拟道具的道具运动数据输入至动画合成系统92中。
声音录制系统64用于对真实对象在表演过程中的声音进行录制得到声音录制数据，并将声音录制数据输入至动画合成系统92中。
虚拟相机位姿跟踪系统70用于捕捉虚拟相机得到虚拟相机位姿数据，并将虚拟相机位姿数据输入至动画合成系统92中。虚拟相机位姿数据用于指示待生成的动画画面的预览相机视角。其中，虚拟相机位姿数据包括：虚拟相机位置、虚拟相机方向，以及虚拟相机参数，比如虚拟相机参数包括焦距。
参考视频录制系统80用于对真实对象的表演内容进行拍摄得到视频录制数据。视频录制数据可以作为待生成动画的后期制作的参考数据。即视频录制数据是动画合成系统92的参考数据。
可选的，本公开实施例中的肢体捕捉和手势捕捉，表情捕捉和眼神捕捉，声音录制，参考视频录制以及道具捕捉是同时进行的。但由于肢体捕捉和手势捕捉，表情捕捉和眼神捕捉，声音录制，参考视频录制以及道具捕捉是通过不同的系统完成的，由于通讯上的延迟，可能导致不同信号不同步，而最后生成的动画是需要做到上述各个系统是完全同步的，所以，在整个计算机设备10中加入了时间码同步系统90，计算机设备10中的各个系统基于相同的时间码，进行同步。
动画合成系统92也称为渲染引擎，用于将导入的多个数据(包括肢体动作数据、手势动作数据、表情数据、眼神数据、声音录制数据、虚拟相机位姿数据、虚拟道具的道具运动数据)按照时间码进行同步，在同步后将导入的多个数据进行合成并进行渲染得到动画视频，并显示出生成的动画视频。
需要说明的一点是，上述各个系统中涉及的步骤的实现细节可参考下述实施例中的相关描述，在此先不介绍。
需要说明的另一点是，上述实施例提供的系统在实现其功能时，仅以上述各个系统的划分进行举例说明，实际应用中，可以根据实际需要而将上述功能分配由不同的系统完成，以完成以上描述的全部或者部分功能。比如，肢体动作捕捉系统和手势动作捕捉系统可以合并为一个系统即动作捕捉系统，表情捕捉系统和眼神捕捉系统可以合并为一个系统即面部捕捉系统。动作捕捉系统和动作重定向系统可以合并为一个系统即动作处理系统，面部捕捉系统和面部重定向系统可以合并为一个系统即情绪处理系统，道具捕捉系统和道具重定向系统可以合并为一个系统即道具处理系统。上述各个系统还可以全部合并为一个系统。本实施例对此不加以限定。
请参考图4，其示出了本公开另一个示例性实施例提供的动画生成方法的流程图，本实施例以该方法用于图3所示的计算机设备中来举例说明。该方法包括以下几个步骤。
步骤401，获取真实对象的动作数据，动作数据包括肢体动作数据和/或手势动作数据。
在真实对象的表演过程中，动作捕捉系统获取真实对象的动作数据，动作数据包括肢体动作数据和/或手势动作数据。
以真实对象为人物(比如演员)为例，动作捕捉是记录演员的动作数据。动作捕捉是由穿戴或者贴附在演员身上的装置来采集。比如，演员穿着一套带有相机跟踪标记的衣服或者是内置传感器的衣服，通过相机跟踪标记的反光或者传感器的移动来完成动作捕捉过程。
在一种可能的实现方式中，获取真实对象的肢体上预设的多个光学标记点各自对应的位置数据，根据多个光学标记点各自对应的位置数据确定真实对象的肢体动作数据；和/或，获取真实对象的手部上预设的多个光学标记点各自对应的位置数据，根据多个光学标记点各自对应的位置数据确定真实对象的手势动作数据。
可选的，真实对象穿着一套设置有多个光学标记点的动捕服装，动捕服装覆盖在该真实对象的肢体和手部上。示意性的，动捕服装包括覆盖在该真实对象的肢体上的衣服和覆盖在该真实对象的手部上的手套。
可选的，真实对象的衣服上预设的多个光学标记点与真实对象的肢体的多个关节点存在一一对应的关系。真实对象的手套上预设的多个光学标记点与真实对象的手部的多个关节点存在一一对应的关系。
动作捕捉系统通过第一光学捕捉设备获取真实对象的动作数据。示意性的，第一光学捕捉设备为红外相机。动作捕捉系统通过红外相机捕捉真实对象身上的多个反光点(包括：多个光学标记点和多个光学标记点)位置，并实时解算出真实对象的身材和跟踪真实对象的动作。即动作捕捉系统根据多个反光点位置确定真实对象的身材和真实对象的动作数据，动作数据包括肢体动作数据和手势动作数据。
可选的，肢体动作数据用于指示真实对象的肢体动作，肢体动作数据包括真实对象的肢体上的各个关节点的三维位置坐标和运动参数。示意性的，关节点为预设位置上的关键关节点，比如头部、颈部、肩膀、手臂、腿部等等。
手势动作数据用于指示真实对象的手部动作，手势动作数据包括真实对象的手部上的各个关节点的三维位置坐标和运动参数。
步骤402，根据真实对象的动作数据，确定虚拟角色的动作数据。
动作捕捉系统将真实对象的动作数据转化为虚拟对象的动作数据，动作重定向系统将虚拟对象的动作数据进行重定向处理得到虚拟角色的动作数据。
其中，虚拟对象为对真实对象进行还原重建得到的虚拟模型。可选的，虚拟对象为对真实对象进行一比一还原重建得到的虚拟模型。
可选的，虚拟对象为三维或者二维虚拟模型。下面仅以虚拟对象为三维虚拟模型为例进行说明。虚拟对象的动作数据与真实对象的动作数据存在映射关系，虚拟对象的动作数据的含义可类比参考真实对象的动作数据的相关描述，在此不再赘述。
在一种可能的实现方式中，动作重定向系统将虚拟对象的动作数据进行重定向处理得到虚拟角色的动作数据，包括：获取虚拟对象的骨骼数据与虚拟角色的骨骼数据之间的第一对应关系，骨骼数据用于指示骨骼的拓扑结构特征；根据第一对应关系，将虚拟对象的动作数据重定向处理至虚拟角色上，得到虚拟角色的动作数据。
在将虚拟对象的动作数据进行重定向处理，得到虚拟角色的动作数据之前，建立虚拟对象的骨骼数据与虚拟角色的骨骼数据之间的第一对应关系。其中，虚拟对象的骨骼数据用于指示虚拟对象的骨骼的拓扑结构特征，虚拟角色的骨骼数据用于指示虚拟角色的骨骼的拓扑结构特征。
其中，骨骼的拓扑结构特征用于指示骨骼的分布情况和骨骼之间的连接状态。
动作重定向系统根据第一对应关系，将虚拟对象的动作数据重定向处理至虚拟角色上，得到虚拟角色的动作数据，包括：根据第一对应关系，将虚拟对象的肢体动作数据重定向处理至虚拟角色上，得到虚拟角色的肢体动作数据；和/或，将虚拟对象的手势动作数据重定向处理至虚拟角色上，得到虚拟角色的手势动作数据。
可选的，在根据真实对象的动作数据，确定虚拟角色的动作数据之后，根据虚拟角色的动作数据、以及虚拟角色的骨骼和蒙皮的绑定关系，驱动并显示虚拟角色的蒙皮运动。其中，虚拟角色的骨骼和蒙皮的绑定关系是预先设置的。
步骤403，获取真实对象的面部数据，面部数据包括表情数据和/或眼神数据。
在一种可能的实现方式中，获取真实对象的面部视频帧，面部视频帧为包括真实对象的面部的视频帧，面部视频帧用于指示所述真实对象的面部数据。
可选的，面部捕捉系统通过第二光学捕捉设备获取真实对象的面部视频帧。
可选的，真实对象的表情数据用于指示真实对象的面部表情，表情数据包括真实对象的面部上的各个特征点的三维位置坐标和运动参数。各个特征点为真实对象的面部上的轮廓和五官上的各个特征点。
可选的，真实对象的眼神数据用于指示真实对象的眼球状态，眼神数据包括真实对象的眼球上的各个特征点的三维位置坐标和运动参数。本实施例对表情数据和眼神数据的数据结构不加以限定。
步骤404，根据真实对象的面部数据，确定虚拟角色的面部数据。
面部捕捉系统将真实对象的面部数据转化为虚拟对象的面部数据，面部重定向系统将虚拟对象的 面部数据进行重定向处理得到虚拟角色的面部数据。其中，虚拟对象为对真实对象进行还原重建得到的虚拟模型。
其中，虚拟对象的面部数据包括虚拟角色的表情数据和/或眼神数据。虚拟对象的面部数据与真实对象的面部数据存在映射关系，虚拟对象的面部数据的含义可类比参考真实对象的面部数据的相关描述，在此不再赘述。
可选的，面部捕捉系统将真实对象的面部数据转化为虚拟对象的面部数据，包括：面部捕捉系统根据真实对象的面部数据，调用第一预设面部处理模型输出得到虚拟对象的面部模型，面部模型用于指示虚拟对象的面部数据。
可选的，真实对象的面部视频帧为包括真实对象的面部的视频帧，面部视频帧为二维形式的数据，虚拟对象的面部模型用于指示虚拟角色的表情数据和/或眼神数据，面部模型为三维形式的数据，第一预设面部处理模型用于将真实对象的二维的面部视频帧转化为虚拟对象的三维的面部模型。
可选的，第一预设面部处理模型为预先训练的神经网络模型，用于表示真实对象的面部视频帧与虚拟对象的面部模型之间的相关关系。
可选的，第一预设面部处理模型为预设的数学模型，该第一预设面部处理模型包括真实对象的面部视频帧与虚拟对象的面部模型之间的模型系数。模型系数可以为固定值，也可以是动态修改的值。
可选的，面部重定向系统将虚拟对象的面部数据进行重定向处理得到虚拟角色的面部数据，包括：获取虚拟对象的面部数据与虚拟角色的面部数据之间的第二对应关系，面部数据用于指示面部结构特征和情绪风格特征；根据第二对应关系，将虚拟对象的面部数据重定向处理至虚拟角色上，得到虚拟角色的面部数据。
在将虚拟对象的面部数据进行重定向处理，得到虚拟角色的面部数据之前，建立虚拟对象的面部数据与虚拟角色的面部数据之间的第二对应关系。其中，虚拟对象的面部数据用于指示虚拟对象的面部结构特征和情绪风格特征，虚拟角色的面部数据用于指示虚拟角色的面部结构特征和情绪风格特征。
其中，面部结构特征用于指示面部的轮廓和五官的分布情况。情绪风格特征用于指示面部上的多个特征点所体现的情绪，比如开心、难过、无奈等等。
可选的，面部重定向系统根据第二对应关系，将虚拟对象的面部数据重定向处理至虚拟角色上，得到虚拟角色的面部数据，包括：根据第二对应关系，将虚拟对象的表情数据重定向处理至虚拟角色上，得到虚拟角色的表情数据；和/或，将虚拟对象的眼神数据重定向处理至虚拟角色上，得到虚拟角色的眼神数据。
可选的，面部重定向系统将虚拟对象的面部数据进行重定向处理得到虚拟角色的面部数据，包括：面部重定向系统根据虚拟对象的面部数据，调用第二预设面部处理模型输出得到虚拟角色的面部数据。
可选的，第二预设面部处理模型为预先训练的神经网络模型，用于表示虚拟对象的面部数据与虚拟角色的面部数据之间的相关关系。
可选的，第二预设面部处理模型为预设的数学模型，该第二预设面部处理模型包括虚拟对象的面部数据与虚拟角色的面部数据之间的模型系数。模型系数可以为固定值，也可以是动态修改的值。
需要说明的是，面部重定向过程中的重定向处理也称为表情迁移处理，本公开实施例对表情迁移处理的具体实现方式不加以限定。以真实对象为人物(比如演员)为例，第二光学捕捉设备为头戴式RGB相机为例，面部捕捉系统通过演员佩戴的头戴式RGB相机，捕捉演员在表演过程中的表情和眼神得到一个视频，视频包括多个面部视频帧；对于每帧面部视频帧，重建得到虚拟对象的三维的面部模型，将虚拟对象的三维的面部模型进行重定向处理，得到虚拟角色的面部数据。
对于捕捉演员在表演过程中的表情和眼神，可采用以下方法进行捕捉。脸上描点法，在演员的脸上标记处若干个标记点，捕捉人脸，获得人脸信息；脸上不描点法：演员的脸上无标记点，运用算法直接在演员的脸上提取信息，捕捉人脸，获得人脸信息。在人脸捕捉过程中，可以采用单个相机或者 多个相机对人脸进行捕捉。单个相机轻便易戴，也可以达到多个相机的结果，多个相机可以实现多个角度的人脸数据的捕捉。对于捕捉设备，可以采用RGB相机和/或RGBD相机。
步骤405，获取参考数据，参考数据包括真实对象在表演过程中的声音录制数据和/或虚拟相机位姿数据。
在真实对象的表演过程中，在对真实对象的动作数据和面部数据进行捕捉的同时，声音录制系统将真实对象的声音进行录制得到真实对象的声音录制数据，虚拟相机位姿跟踪系统对虚拟相机进行捕捉得到虚拟相机位姿数据。
即在真实对象的表演过程中，同步捕捉一个虚拟相机，对虚拟相机的位姿和运动轨迹进行记录得到虚拟相机位姿数据。虚拟相机位姿数据用于指示待生成的动画画面的预览相机视角。预览相机视角是在虚拟环境中通过虚拟相机对虚拟角色和/或其他场景信息进行观察时的角度。即待生成的动画画面是以虚拟相机的视角对虚拟角色进行观察所采集到的动画画面。其中，虚拟相机位姿数据包括：虚拟相机位置、虚拟相机方向，以及虚拟相机参数，比如虚拟相机参数包括焦距。
需要说明的是，步骤401和步骤402所示的动作数据的捕捉和重定向过程，与步骤403和步骤404所示的面部数据的捕捉和重定向过程、与步骤405所示的参考数据的获取过程可以并列执行，不分先后顺序。
步骤406，根据目标特征数据和参考数据，生成虚拟角色的动画。
动画合成系统根据目标特征数据和参考数据，实时或者离线生成虚拟角色的动画。可选的，虚拟角色的动画包括三维或者二维动画。
声音录制系统录制得到真实对象的声音录制数据后，将声音录制数据录入至动画合成系统。动画合成系统根据录入的声音录制数据确定虚拟角色的声音数据。
可选的，真实对象的声音录制数据即为虚拟角色的声音数据，或者将真实对象的声音录制数据进行预设声音处理得到虚拟角色的声音数据，或者将声音录制数据替换为配音数据得到虚拟角色的声音数据。本公开实施例对此不加以限定。
虚拟相机位姿跟踪系统捕捉得到虚拟相机位姿数据后，将虚拟相机位姿数据录入至动画合成系统。动画合成系统根据录入的虚拟相机位姿数据确定待生成的动画的预览相机视角。
可选的，目标特征数据和参考数据均携带有时间码，动画合成系统根据目标特征数据和参考数据，生成虚拟角色的动画，包括：根据目标特征数据和参考数据各自对应的时间码，将目标特征数据和参考数据进行对齐处理；根据对齐处理后的目标特征数据和参考数据，生成虚拟角色的动画。
其中，对齐处理后的目标特征数据和参考数据是时间上同步的数据。
示意性的，动作数据、面部数据、声音录制数据和虚拟相机位姿数据均携带有时间码，动画合成系统将导入的动作数据、面部数据、声音录制数据和虚拟相机位姿数据按照时间码进行对齐处理，对齐处理后再进行合成和渲染得到动画视频。
可选的，动画合成系统根据对齐处理后的目标特征数据和参考数据，生成虚拟角色的动画，包括：在对齐处理后根据虚拟相机位姿数据和目标特征数据，得到动画画面，虚拟相机位姿数据用于指示待生成的动画画面的预览相机视角；对动画画面进行渲染得到渲染结果；根据渲染结果和声音录制数据，生成虚拟角色的动画视频。其中，虚拟相机位姿数据包括：虚拟相机位置、虚拟相机方向，以及虚拟相机参数，比如虚拟相机参数包括焦距。
可选的，动画合成系统获取携带有时间码的视频录制数据，视频录制数据包括对真实对象的表演内容进行录制得到的视频数据。视频录制数据可以作为待生成的动画的参考数据。
动画合成系统生成虚拟角色的动画后，显示该虚拟角色的动画。需要说明的是，在显示屏上显示该虚拟角色的动画的相关细节可参考上述实施例中的相关描述，在此不再赘述。
在一个示意性的例子中，以真实对象为演员，虚拟对象为虚拟演员模型，虚拟角色为预设的虚拟动画角色为例，动作捕捉系统通过红外相机捕捉演员身上的多个反光点位置，根据多个反光点位置将演员的动作数据重建为虚拟演员模型的动作数据，动作重定向系统将虚拟演员模型的动作数据进行重 定向处理得到虚拟动画角色的动作数据。面部捕捉系统通过演员佩戴的头戴式RGB相机或者RGBD相机获取演员的面部视频帧，将演员的面部视频帧转化为虚拟演员模型的面部数据，面部重定向系统将虚拟演员模型的面部数据进行重定向处理得到虚拟动画角色的动作数据。在捕捉演员骨骼动作和演员面部情绪的同时，声音录制系统对演员的声音进行录制得到声音录制数据，虚拟相机位姿跟踪系统对虚拟相机的位姿和运动轨迹进行记录得到虚拟相机位姿数据。上述的各个系统基于相同的时间码进行同步。动画合成系统获取导入的多个数据，多个数据包括动作数据(肢体动作数据和手势动作数据)、面部数据(表情数据和眼神数据)、声音录制数据、虚拟相机位姿数据，动画合成系统将导入的多个数据按照时间码进行同步，在同步后根据导入的多个数据生成动画，并显示出生成的动画。
可选地，该方法还包括：获取真实对象在表演过程中所用的真实道具的道具运动数据；根据真实道具数据确定虚拟角色所用的虚拟道具的道具运动数据；根据目标特征数据和虚拟道具的道具运动数据，生成虚拟角色的动画。
在真实对象的表演过程中，道具捕捉系统获取真实对象所用的真实道具的道具运动数据；道具捕捉系统将真实道具的道具运动数据转化为虚拟中间道具的道具运动数据，道具重定向系统将虚拟中间道具的道具运动数据进行重定向处理得到虚拟道具的道具运动数据。
在一个示意性的例子中，真实道具为足球，演员踢足球，足球会运动比如移动和旋转，道具捕捉系统捕捉足球的运动得到足球的道具运动数据，根据足球的道具运动数据进行道具重建得到虚拟中间足球的道具运动数据；道具重定向系统将虚拟中间足球的道具运动数据进行重定向处理得到虚拟足球的道具运动数据。
在另一个示意性的例子中，真实道具为刀剑，演员挥动刀剑，刀剑会运动，道具捕捉系统捕捉刀剑的运动得到刀剑的道具运动数据，根据刀剑的道具运动数据进行道具重建得到虚拟中间刀剑的道具运动数据；道具重定向系统将虚拟中间刀剑的道具运动数据进行重定向处理得到虚拟刀剑的道具运动数据。
需要说明的一点是，道具捕捉系统进行道具捕捉的方式可类比参考动作捕捉系统进行动作捕捉的方式。道具重定向系统进行重定向处理的方式可类比参考动作重定向系统进行重定向处理的方式，在此不再赘述。
需要说明的另一点是，动作数据的捕捉和重定向过程，与面部数据的捕捉和重定向过程、与参考数据的获取过程、与道具运动数据的捕捉和重定向过程可以并列执行，不分先后顺序。
可选地，虚拟道具的道具运动数据也携带有时间码，动画合成系统根据目标特征数据、参考数据和道具运动数据各自对应的时间码，将目标特征数据、参考数据和道具运动数据进行对齐处理；根据对齐处理后的目标特征数据、参考数据和道具运动数据，生成虚拟角色的动画。
可选地，动画合成系统在对齐处理后根据虚拟相机位姿数据、目标特征数据和道具运动数据，得到动画画面；对动画画面进行渲染得到渲染结果；根据渲染结果和声音录制数据，生成虚拟角色的动画视频。
需要说明的是，动画合成系统根据对齐处理后的目标特征数据、参考数据和道具运动数据生成虚拟角色的动画的方式，可类比参考根据对齐处理后的目标特征数据和参考数据生成虚拟角色的动画的方式，在此不再赘述。
综上所述，本公开实施例还通过将真实特征数据转化为虚拟对象的虚拟特征数据，虚拟对象为对真实对象进行还原重建得到的虚拟模型，虚拟特征数据包括虚拟对象的动作数据和面部数据；其中，动作数据包括肢体动作数据和/或手势动作数据，面部数据包括表情数据和/或眼神数据；能够更加准确地反映出虚拟角色的骨骼动作和面部情绪的细节，使得生成的虚拟角色更加生动自然，保证了虚拟角色的动画效果。
本公开实施例还通过获取参考数据，参考数据包括真实对象在表演过程中的声音录制数据和/或虚拟相机位姿数据；根据目标特征数据和参考数据，实时生成虚拟角色的动画；即采用了虚拟拍摄的所见即所得的方式，可以在现场实时的看到真实对象的表演，可以现场确认表演，提高了拍摄效率。
本公开实施例还通过目标特征数据和参考数据均携带有时间码，根据目标特征数据和参考数据各自对应的时间码，将目标特征数据和参考数据进行对齐处理；根据对齐处理后的目标特征数据和参考数据，生成虚拟角色的动画；使得虚拟角色的骨骼动作、面部情绪、声音和虚拟相机位姿是同步的，在丰富了虚拟角色的动画的显示细节的同时，保证了动画的自然流畅度，进一步保证了动画的显示效果。
动画生成方法可用于表演动画领域。上述提及的动画生成方法可以实现单人的捕捉，也可以实现多人的捕捉，即在同一画面中可以实现单个虚拟角色的输出也可以实现多个虚拟角色的输出。在多人捕捉的情况下，可以捕捉演员之间的互动，例如，拥抱，握手等，根据多个演员之间的互动输出虚拟角色的互动。
动画生成方法支持离线模式和实时在线模式。在离线模式中，可以利用动画处理系统10中的数据处理工具，对动画数据进行离线处理和调整。可以利用动画处理系统10中的动画精修工具，对动画数据进行精修，容许动画师提升动画质量和控制动画风格。对于身体和人脸都可以进行离线处理和动画精修。而且离线动画制作可以应用在比如影视动画里的角色动画，游戏动画，虚拟短视频或者综艺里角色动画等等。在实时模式中，动画生成方法可以用于实时动画，支撑实时直播，以及虚拟角色实时互动等等。例如虚拟角色之间的互动。
以下为本公开实施例的装置实施例，对于装置实施例中未详细阐述的部分，可以参考上述方法实施例中公开的技术细节。
请参考图5，其示出了本公开一个示例性实施例提供的动画生成装置的结构示意图。该动画生成装置可以通过软件、硬件以及两者的组合实现成为用户设备的全部或一部分。该装置包括：获取模块510、确定模块520和生成模块530。
获取模块510，用于获取真实对象的真实特征数据，真实特征数据包括真实对象在表演过程中的动作数据和面部数据；
确定模块520，用于根据真实特征数据确定虚拟角色的目标特征数据，虚拟角色为预设的动画模型，目标特征数据包括虚拟角色的动作数据和面部数据；
生成模块530，用于根据目标特征数据，生成虚拟角色的动画。
在一种可能的实现方式中，
获取模块510，还用于获取参考数据，参考数据包括真实对象在表演过程中的声音录制数据和/或虚拟相机位姿数据；
生成模块530，还用于根据目标特征数据和参考数据，生成虚拟角色的动画。
在另一种可能的实现方式中，目标特征数据和参考数据均携带有时间码，生成模块530，还用于：
根据目标特征数据和参考数据各自对应的时间码，将目标特征数据和参考数据进行对齐处理；
根据对齐处理后的目标特征数据和参考数据，生成虚拟角色的动画。
在另一种可能的实现方式中，生成模块530，还用于：
在对齐处理后根据虚拟相机位姿数据和目标特征数据，得到动画画面，虚拟相机位姿数据用于指示待生成的动画画面的预览相机视角；
对动画画面进行渲染得到渲染结果；
根据渲染结果和声音录制数据，生成虚拟角色的动画视频。
在另一种可能的实现方式中，获取模块510，还用于：
获取真实对象的动作数据，动作数据包括肢体动作数据和/或手势动作数据；以及，
获取真实对象的面部数据，面部数据包括表情数据和/或眼神数据。
在另一种可能的实现方式中，获取模块510，还用于：
获取真实对象的肢体上预设的多个光学标记点各自对应的位置数据，根据多个光学标记点各自对应的位置数据确定真实对象的肢体动作数据；和/或，
获取真实对象的手部上预设的多个光学标记点各自对应的位置数据，根据多个光学标记点各自对应的位置数据确定真实对象的手势动作数据。
在另一种可能的实现方式中，获取模块510，还用于：
获取真实对象的面部视频帧，面部视频帧为包括真实对象的面部的视频帧，面部视频帧用于指示真实对象的面部数据。
在另一种可能的实现方式中，确定模块520，还用于：
将真实特征数据转化为虚拟对象的虚拟特征数据，虚拟对象为对真实对象进行还原重建得到的虚拟模型，虚拟特征数据包括虚拟对象的动作数据和面部数据；
将虚拟特征数据进行重定向处理得到虚拟角色的目标特征数据。
在另一种可能的实现方式中，确定模块520，还用于：
将虚拟对象的动作数据进行重定向处理，得到虚拟角色的动作数据，动作数据包括肢体动作数据和/或手势动作数据；以及，
将虚拟对象的面部数据进行重定向处理，得到虚拟角色的面部数据，面部数据包括表情数据和/或眼神数据。
在另一种可能的实现方式中，确定模块520，还用于：
获取虚拟对象的骨骼数据与虚拟角色的骨骼数据之间的第一对应关系，骨骼数据用于指示骨骼的拓扑结构特征；
根据第一对应关系，将虚拟对象的动作数据重定向处理至虚拟角色上，得到虚拟角色的动作数据。
在另一种可能的实现方式中，确定模块520，还用于：
获取虚拟对象的面部数据与虚拟角色的面部数据之间的第二对应关系，面部数据用于指示面部结构特征和情绪风格特征；
根据第二对应关系，将虚拟对象的面部数据重定向处理至虚拟角色上，得到虚拟角色的面部数据。
在另一种可能的实现方式中，该装置还包括：显示模块；
显示模块，用于根据虚拟角色的动作数据、以及虚拟角色的骨骼和蒙皮的绑定关系，驱动并显示虚拟角色的蒙皮运动。
在另一种可能的实现方式中，该装置还包括：录制模块；
录制模块，用于获取携带有时间码的视频录制数据，视频录制数据包括对真实对象的表演内容进行录制得到的视频数据。
在另一种可能的实现方式中，
获取模块510，还用于获取真实对象在表演过程中所用的真实道具的道具运动数据；
确定模块520，还用于根据真实道具数据确定虚拟角色所用的虚拟道具的道具运动数据；
生成模块530，还用于根据目标特征数据和虚拟道具的道具运动数据，生成虚拟角色的动画。
需要说明的是，上述实施例提供的装置在实现其功能时，仅以上述各个功能模块的划分进行举例说明，实际应用中，可以根据实际需要而将上述功能分配由不同的功能模块完成，即将设备的内容结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
本公开实施例还提供了一种计算机设备，所述计算机设备包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为：实现上述各个方法实施例中由计算机设备执行的步骤。
本公开实施例还提供了一种动画生成系统，所述动画生成系统包括：
动捕服装，所述动捕服装上设置有多个光学标记点；
第一相机，所述第一相机用于捕捉真实对象表演时的动作数据；
头盔，所述头盔上设置有第二相机，所述第二相机用于捕捉所述真实对象表演时的面部数据；
计算机设备，所述计算机设备用于执行上述各个方法实施例中由计算机设备执行的步骤。
本公开实施例还提供了一种非易失性计算机可读存储介质，其上存储有计算机程序指令，计算机程序指令被处理器执行时实现上述各个方法实施例中的方法。
本公开可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里参照根据本公开实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一 个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (18)

  1. 一种动画生成方法，其特征在于，所述方法包括：
    获取真实对象的真实特征数据，所述真实特征数据包括所述真实对象在表演过程中的动作数据和面部数据；
    根据所述真实特征数据确定虚拟角色的目标特征数据，所述虚拟角色为预设的动画模型，所述目标特征数据包括所述虚拟角色的动作数据和面部数据；
    根据所述目标特征数据，生成所述虚拟角色的动画。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    获取参考数据，所述参考数据包括所述真实对象在表演过程中的声音录制数据和/或虚拟相机位姿数据；
    所述根据所述目标特征数据，生成所述虚拟角色的动画，包括：
    根据所述目标特征数据和所述参考数据，生成所述虚拟角色的动画。
  3. 根据权利要求2所述的方法，其特征在于，所述目标特征数据和所述参考数据均携带有时间码，所述根据所述目标特征数据和所述参考数据，生成所述虚拟角色的动画，包括：
    根据所述目标特征数据和所述参考数据各自对应的所述时间码，将所述目标特征数据和所述参考数据进行对齐处理；
    根据对齐处理后的所述目标特征数据和所述参考数据，生成所述虚拟角色的动画。
  4. 根据权利要求3所述的方法，其特征在于，所述根据对齐处理后的所述目标特征数据和所述参考数据，生成所述虚拟角色的动画，包括：
    在对齐处理后根据所述虚拟相机位姿数据和所述目标特征数据，得到动画画面，所述虚拟相机位姿数据用于指示待生成的动画画面的预览相机视角；
    对所述动画画面进行渲染得到渲染结果；
    根据所述渲染结果和所述声音录制数据，生成所述虚拟角色的动画视频。
  5. 根据权利要求1所述的方法，其特征在于，所述获取真实对象的真实特征数据，包括：
    获取所述真实对象的所述动作数据，所述动作数据包括肢体动作数据和/或手势动作数据；以及，
    获取所述真实对象的所述面部数据，所述面部数据包括表情数据和/或眼神数据。
  6. 根据权利要求5所述的方法，其特征在于，所述获取所述真实对象的所述动作数据，包括：
    获取所述真实对象的肢体上预设的多个光学标记点各自对应的位置数据，根据所述多个光学标记点各自对应的所述位置数据确定所述真实对象的所述肢体动作数据；和/或，
    获取所述真实对象的手部上预设的多个光学标记点各自对应的位置数据，根据所述多个光学标记点各自对应的所述位置数据确定所述真实对象的所述手势动作数据。
  7. 根据权利要求5所述的方法，其特征在于，所述获取所述真实对象的所述面部数据，包括：
    获取所述真实对象的面部视频帧，所述面部视频帧为包括所述真实对象的面部的视频帧，所述面部视频帧用于指示所述真实对象的所述面部数据。
  8. 根据权利要求1所述的方法，其特征在于，所述根据所述真实特征数据确定虚拟角色的目标特征数据，包括：
    将所述真实特征数据转化为虚拟对象的虚拟特征数据，所述虚拟对象为对所述真实对象进行还原重建得到的虚拟模型，所述虚拟特征数据包括所述虚拟对象的动作数据和面部数据；
    将所述虚拟特征数据进行重定向处理得到所述虚拟角色的所述目标特征数据。
  9. 根据权利要求8所述的方法，其特征在于，所述将所述虚拟特征数据进行重定向处理得到所述虚拟角色的所述目标特征数据，包括：
    将所述虚拟对象的动作数据进行重定向处理，得到所述虚拟角色的动作数据，所述动作数据包括肢体动作数据和/或手势动作数据；以及，
    将所述虚拟对象的面部数据进行重定向处理，得到所述虚拟角色的面部数据，所述面部数据包括表情数据和/或眼神数据。
  10. 根据权利要求9所述的方法，其特征在于，所述将所述虚拟对象的动作数据进行重定向处理，得到所述虚拟角色的动作数据，包括：
    获取所述虚拟对象的骨骼数据与所述虚拟角色的骨骼数据之间的第一对应关系，所述骨骼数据用于指示骨骼的拓扑结构特征；
    根据所述第一对应关系，将所述虚拟对象的动作数据重定向处理至所述虚拟角色上，得到所述虚拟角色的动作数据。
  11. 根据权利要求9所述的方法，其特征在于，所述将所述虚拟对象的面部数据进行重定向处理，得到所述虚拟角色的面部数据，包括：
    获取所述虚拟对象的面部数据与所述虚拟角色的面部数据之间的第二对应关系，所述面部数据用于指示面部结构特征和情绪风格特征；
    根据所述第二对应关系，将所述虚拟对象的面部数据重定向处理至所述虚拟角色上，得到所述虚拟角色的所述面部数据。
  12. 根据权利要求1至11任一所述的方法，其特征在于，所述根据所述真实特征数据确定虚拟角色的目标特征数据之后，还包括：
    根据所述虚拟角色的动作数据、以及所述虚拟角色的骨骼和蒙皮的绑定关系，驱动并显示所述虚拟角色的蒙皮运动。
  13. 根据权利要求1至11任一所述的方法，其特征在于，所述根据所述目标特征数据，生成所述虚拟角色的动画之前，还包括：
    获取携带有时间码的视频录制数据，所述视频录制数据包括对所述真实对象的表演内容进行录制得到的视频数据。
  14. 根据权利要求1至11任一所述的方法，其特征在于，所述方法还包括：
    获取所述真实对象在表演过程中所用的真实道具的道具运动数据；
    根据所述真实道具数据确定所述虚拟角色所用的虚拟道具的道具运动数据；
    所述根据所述目标特征数据，生成所述虚拟角色的动画，包括：
    根据所述目标特征数据和所述虚拟道具的道具运动数据，生成所述虚拟角色的动画。
  15. 一种动画生成装置，其特征在于，所述装置包括：
    获取模块，用于获取真实对象的真实特征数据，所述真实特征数据包括所述真实对象在表演过程中的动作数据和面部数据；
    确定模块，用于根据所述真实特征数据确定虚拟角色的目标特征数据，所述虚拟角色为预设的动画模型，所述目标特征数据包括所述虚拟角色的动作数据和面部数据；
    生成模块，用于根据所述目标特征数据，生成所述虚拟角色的动画。
  16. 一种计算机设备，其特征在于，所述计算机设备包括：处理器；用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为：
    获取真实对象的真实特征数据，所述真实特征数据包括所述真实对象在表演过程中的动作数据和面部数据；
    根据所述真实特征数据确定虚拟角色的目标特征数据，所述虚拟角色为预设的动画模型，所述目标特征数据包括所述虚拟角色的动作数据和面部数据；
    根据所述目标特征数据，生成所述虚拟角色的动画。
  17. 一种动画生成系统，其特征在于，所述动画生成系统包括：
    动捕服装，所述动捕服装上设置有多个光学标记点；
    第一相机，所述第一相机用于捕捉真实对象表演时的动作数据；
    头盔，所述头盔上设置有第二相机，所述第二相机用于捕捉所述真实对象表演时的面部数据；
    计算机设备，所述计算机设备用于执行上述权利要求1至14中任意一项所述的方法。
  18. 一种非易失性计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时实现权利要求1至14中任意一项所述的方法。
PCT/CN2021/110349 2020-09-25 2021-08-03 动画生成方法、装置、系统及存储介质 WO2022062680A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/028,472 US11893670B2 (en) 2020-09-25 2021-08-03 Animation generation method, apparatus and system, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011023780.3 2020-09-25
CN202011023780.3A CN111968207B (zh) 2020-09-25 2020-09-25 动画生成方法、装置、系统及存储介质

Publications (1)

Publication Number Publication Date
WO2022062680A1 true WO2022062680A1 (zh) 2022-03-31

Family

ID=73386809

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/110349 WO2022062680A1 (zh) 2020-09-25 2021-08-03 动画生成方法、装置、系统及存储介质

Country Status (3)

Country Link
US (1) US11893670B2 (zh)
CN (1) CN111968207B (zh)
WO (1) WO2022062680A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116051700A (zh) * 2023-04-03 2023-05-02 北京墨境天合数字图像科技有限公司 一种虚拟拍摄方法、装置、电子设备及存储介质

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111968207B (zh) 2020-09-25 2021-10-29 魔珐(上海)信息科技有限公司 动画生成方法、装置、系统及存储介质
CN112215930A (zh) * 2020-10-19 2021-01-12 珠海金山网络游戏科技有限公司 一种数据处理方法与装置
CN112446938B (zh) * 2020-11-30 2023-08-18 重庆空间视创科技有限公司 基于多模态的虚拟主播系统及方法
CN112669413A (zh) * 2021-01-05 2021-04-16 北京诺亦腾科技有限公司 基于动作捕捉的动画制作系统和动画制作方法
CN112927331B (zh) * 2021-03-31 2023-09-22 腾讯科技（深圳）有限公司 角色模型的动画生成方法和装置、存储介质及电子设备
CN113313794B (zh) * 2021-05-19 2022-11-08 深圳市慧鲤科技有限公司 动画迁移方法和装置、设备及存储介质
CN114638918B (zh) * 2022-01-26 2023-03-28 武汉艺画开天文化传播有限公司 一种实时表演捕捉虚拟直播与录制系统
CN114879877B (zh) * 2022-05-23 2023-03-28 北京新唐思创教育科技有限公司 状态数据同步方法、装置、设备及存储介质
WO2024000480A1 (zh) * 2022-06-30 2024-01-04 中国科学院深圳先进技术研究院 3d虚拟对象的动画生成方法、装置、终端设备及介质
CN115661308B (zh) * 2022-11-03 2024-03-19 北京百度网讯科技有限公司 驱动数字人的方法、装置、电子设备和存储介质
CN115908766B (zh) * 2023-02-20 2023-05-26 北京红棉小冰科技有限公司 三维虚拟人物图像的生成方法、装置及电子设备
CN117998166A (zh) * 2024-04-02 2024-05-07 腾讯科技（深圳）有限公司 视频生成模型的训练方法、装置、设备、存储介质和产品

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8854376B1 (en) * 2009-07-30 2014-10-07 Lucasfilm Entertainment Company Ltd. Generating animation from actor performance
CN107274464A (zh) * 2017-05-31 2017-10-20 珠海金山网络游戏科技有限公司 一种实时交互3d动画的方法、装置和系统
US20190004639A1 (en) * 2017-06-29 2019-01-03 Microsoft Technology Licensing, Llc Providing living avatars within virtual meetings
CN110517337A (zh) * 2019-08-29 2019-11-29 成都数字天空科技有限公司 动画角色表情生成方法、动画制作方法及电子设备
CN110650354A (zh) * 2019-10-12 2020-01-03 苏州大禹网络科技有限公司 虚拟动漫角色直播方法、系统、设备及存储介质
CN111968207A (zh) * 2020-09-25 2020-11-20 魔珐(上海)信息科技有限公司 动画生成方法、装置、系统及存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008156437A1 (en) * 2006-04-10 2008-12-24 Avaworks Incorporated Do-it-yourself photo realistic talking head creation system and method
US9177387B2 (en) * 2003-02-11 2015-11-03 Sony Computer Entertainment Inc. Method and apparatus for real time motion capture
GB2404315A (en) * 2003-07-22 2005-01-26 Kelseus Ltd Controlling a virtual environment
EP2600316A1 (en) * 2011-11-29 2013-06-05 Inria Institut National de Recherche en Informatique et en Automatique Method, system and software program for shooting and editing a film comprising at least one image of a 3D computer-generated animation
US9811937B2 (en) * 2015-09-29 2017-11-07 Disney Enterprises, Inc. Coordinated gesture and locomotion for virtual pedestrians
WO2017137948A1 (en) * 2016-02-10 2017-08-17 Vats Nitin Producing realistic body movement using body images
CN107274466A (zh) * 2017-05-31 2017-10-20 珠海金山网络游戏科技有限公司 一种实时全身动作捕捉的方法、装置和系统
CN108564642A (zh) * 2018-03-16 2018-09-21 中国科学院自动化研究所 基于ue引擎的无标记表演捕捉系统
US10650564B1 (en) * 2019-04-21 2020-05-12 XRSpace CO., LTD. Method of generating 3D facial model for an avatar and related device
CN110599573B (zh) * 2019-09-03 2023-04-11 电子科技大学 一种基于单目相机的人脸实时交互动画的实现方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8854376B1 (en) * 2009-07-30 2014-10-07 Lucasfilm Entertainment Company Ltd. Generating animation from actor performance
CN107274464A (zh) * 2017-05-31 2017-10-20 珠海金山网络游戏科技有限公司 一种实时交互3d动画的方法、装置和系统
US20190004639A1 (en) * 2017-06-29 2019-01-03 Microsoft Technology Licensing, Llc Providing living avatars within virtual meetings
CN110517337A (zh) * 2019-08-29 2019-11-29 成都数字天空科技有限公司 动画角色表情生成方法、动画制作方法及电子设备
CN110650354A (zh) * 2019-10-12 2020-01-03 苏州大禹网络科技有限公司 虚拟动漫角色直播方法、系统、设备及存储介质
CN111968207A (zh) * 2020-09-25 2020-11-20 魔珐(上海)信息科技有限公司 动画生成方法、装置、系统及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116051700A (zh) * 2023-04-03 2023-05-02 北京墨境天合数字图像科技有限公司 一种虚拟拍摄方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
US11893670B2 (en) 2024-02-06
CN111968207B (zh) 2021-10-29
CN111968207A (zh) 2020-11-20
US20230274484A1 (en) 2023-08-31

Similar Documents

Publication Publication Date Title
WO2022062680A1 (zh) 动画生成方法、装置、系统及存储介质
WO2022062678A1 (zh) 虚拟直播方法、装置、系统及存储介质
US9939887B2 (en) Avatar control system
Latoschik et al. FakeMi: A fake mirror system for avatar embodiment studies
CN107231531A (zh) 一种网络虚拟现实技术与实拍结合影视制作系统
CN107274464A (zh) 一种实时交互3d动画的方法、装置和系统
US20160350957A1 (en) Multitrack Virtual Puppeteering
Gonzalez-Franco et al. Movebox: Democratizing mocap for the microsoft rocketbox avatar library
Sanna et al. A kinect-based interface to animate virtual characters
Baker The History of Motion Capture within the Entertainment Industry
KR101276429B1 (ko) 모션 캡처에서의 안구 운동 데이터 대체 장치 및 컴퓨터 판독 가능 저장 매체
Kang et al. Real-time animation and motion retargeting of virtual characters based on single rgb-d camera
Huang et al. A process for the semi-automated generation of life-sized, interactive 3D character models for holographic projection
Beimler et al. Smurvebox: A smart multi-user real-time virtual environment for generating character animations
Cannavò et al. Ar-mocap: Using augmented reality to support motion capture acting
KR101859318B1 (ko) 360˚ 가상카메라를 활용한 영상콘텐츠 제작 방법
Akinjala et al. Animating human movement & gestures on an agent using Microsoft kinect
US20220408031A1 (en) Systems and Methods for Video Editing and Effects
Wang et al. Applying motion capture in computer animation education
TWI814318B (zh) 用於使用模擬角色訓練模型以用於將遊戲角色之臉部表情製成動畫之方法以及用於使用三維（３ｄ）影像擷取來產生遊戲角色之臉部表情之標籤值之方法
Törmänen Comparison of entry level motion capture suits aimed at indie game production
Wang et al. Body and Face Animation Based on Motion Capture
Gomide Motion capture and performance
Zhu Application of Motion Capture Technology in 3D Animation Creation
Ndubuisi et al. Model Retargeting Motion Capture System Based on Kinect Gesture Calibration

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21871057

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 01.09.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21871057

Country of ref document: EP

Kind code of ref document: A1