WO2024000480A1 - Procédé et appareil de génération d'animation d'objet virtuel 3d, dispositif terminal et support - Google Patents

Procédé et appareil de génération d'animation d'objet virtuel 3d, dispositif terminal et support Download PDF

Info

Publication number
WO2024000480A1
WO2024000480A1 PCT/CN2022/103020 CN2022103020W WO2024000480A1 WO 2024000480 A1 WO2024000480 A1 WO 2024000480A1 CN 2022103020 W CN2022103020 W CN 2022103020W WO 2024000480 A1 WO2024000480 A1 WO 2024000480A1
Authority
WO
WIPO (PCT)
Prior art keywords
skeleton
animation
preset
target
frame
Prior art date
Application number
PCT/CN2022/103020
Other languages
English (en)
Chinese (zh)
Inventor
陈恭斌
王磊
刘薰裕
程俊
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to PCT/CN2022/103020 priority Critical patent/WO2024000480A1/fr
Publication of WO2024000480A1 publication Critical patent/WO2024000480A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings

Definitions

  • the present application belongs to the field of image processing technology, and in particular relates to a method, device, terminal equipment and medium for animation generation of 3D virtual objects.
  • 3D Three-dimensional (Threee Dimensions, 3D) technology is widely used in games, movies, virtual reality (Virtual Reality, VR) and other fields.
  • 3D technology it is usually necessary to design 3D virtual objects and a series of actions of the 3D virtual objects according to the application scenarios of the 3D technology to generate animations of the 3D virtual objects.
  • 3D technology when 3D technology is applied in the game field, it is usually necessary to design a 3D game character and a series of actions of the 3D game character based on the 3D game scene, and generate animation of the 3D game character based on a series of actions of the 3D game character.
  • the usual method is to first create key animation frames corresponding to key actions of the 3D virtual object, and then generate animation frames corresponding to other actions of the 3D virtual object based on the key animation frames.
  • the production of key animation frames requires professionals, and the production process of key animation frames is complex and time-consuming, which reduces the efficiency of animation generation of 3D virtual objects.
  • the purpose of the embodiments of the present application is to provide an animation generation method, device, terminal device and medium for 3D virtual objects, aiming to solve the existing problem of low efficiency in animation generation of 3D virtual objects.
  • embodiments of the present application provide a method for generating animations of 3D virtual objects, including:
  • the preset object model includes a preset skeleton
  • Skeleton redirection is performed on the target skeleton of the target virtual object based on the preset skeleton to obtain a redirected skeleton;
  • the structure of the target skeleton is the same as or similar to the structure of the preset skeleton;
  • generating a first animation sequence corresponding to the reference object based on the three-dimensional reconstruction information of the reference object in all the video frames and a preset object model includes:
  • all the first animation frames are sequentially composed into the first animation sequence.
  • the three-dimensional reconstruction information includes grid point information, bone key point information, root node position information and shooting parameter information; the video frame is based on each frame
  • the three-dimensional reconstruction information of the reference object in the first animation sequence is used to set the structural parameters of the preset object model in the first animation frame of each frame of the first animation sequence, including:
  • For each video frame determine the spatial position of the preset object model in the corresponding first animation frame according to the shooting parameters of the reference object in the video frame and the position information of the root node;
  • the posture of the preset object model in the corresponding first animation frame is set.
  • the method before performing skeleton redirection on the target skeleton of the target virtual object based on the preset skeleton, the method further includes:
  • the target object model includes an initial skeleton
  • corresponding bones in the initial skeleton are set to obtain the target bones of the target virtual object.
  • the preset object model further includes a preset mesh; the target virtual object model further includes a target mesh; and the pair of objects based on the preset skeleton is The target skeleton of the target virtual object performs skeleton redirection to obtain the redirected skeleton, including:
  • the three-dimensional reconstruction of the reference object in each video frame of the target video is performed sequentially to obtain the three-dimensional reconstruction information of the reference object in each video frame.
  • All video frames of the target video are imported into the preset three-dimensional reconstruction model frame by frame for processing, and the three-dimensional reconstruction information of the reference object in each video frame is obtained respectively.
  • an animation generation device for 3D virtual objects including:
  • a three-dimensional reconstruction unit configured to sequentially perform three-dimensional reconstruction of the reference object in each video frame of the target video, and obtain three-dimensional reconstruction information of the reference object in each video frame;
  • An animation sequence generation unit configured to generate a first animation sequence corresponding to the reference object based on the three-dimensional reconstruction information of the reference object in all the video frames and a preset object model; the preset object model includes a preset skeleton;
  • a skeleton redirection unit configured to perform skeleton redirection on the target skeleton of the target virtual object based on the preset skeleton to obtain a redirected skeleton;
  • the structure of the target skeleton is the same as or similar to the structure of the preset skeleton;
  • An animation redirection unit is configured to perform animation redirection on the redirected skeleton based on the first animation sequence to obtain a second animation sequence corresponding to the target virtual object.
  • the animation sequence generating unit includes:
  • the first acquisition unit is used to acquire the preset object model
  • a parameter setting unit configured to separately set the structural parameters of the preset object model in each first animation frame of the first animation sequence based on the three-dimensional reconstruction information of the reference object in each frame of the video frame. Make settings;
  • a sorting unit configured to sequence all the first animation frames into the first animation sequence according to a preset first animation frame rate.
  • embodiments of the present application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor.
  • the processor executes the computer program, the Steps in any optional manner as described above in the first aspect.
  • embodiments of the present application provide a computer-readable storage medium that stores a computer program.
  • the computer program is executed by a processor, any optional method as described in the first aspect is implemented. Methods.
  • embodiments of the present application provide a computer program product, which when the computer program product is run on a terminal device, causes the terminal device to execute the method described in the above first aspect or any alternative method of the first aspect.
  • the 3D virtual object animation generation method, device, terminal equipment, computer-readable storage medium and computer program product provided by the embodiments of the present application have the following beneficial effects:
  • the animation generation method of 3D virtual characters obtains the three-dimensional reconstruction information of the reference object in each video frame by sequentially performing three-dimensional reconstruction of the reference object in each video frame of the target video, and then based on all
  • the three-dimensional reconstruction information of the reference object in the video frame and the preset object model generate a first animation sequence corresponding to the reference object; since the structure of the preset skeleton of the preset object model is the same or similar to the structure of the target skeleton of the target virtual object, therefore , convert the target video into the corresponding first animation sequence, which can provide a reference basis for the subsequent animation simulation of the target virtual object; then, first perform skeleton redirection on the target skeleton of the target virtual object based on the preset skeleton to obtain the redirected skeleton, and then Perform animation redirection on the redirected skeleton based on the first animation sequence to obtain the second animation sequence corresponding to the target virtual object, so that the actions of the reference object in the target video can be mapped to the virtual object.
  • this solution can be used to automatically realize the target video
  • the animation sequence of the target virtual object is obtained by simulating the motion of the reference object. Compared with the traditional method, there is no need for professionals to produce key animation frames of the target virtual object, thus improving the efficiency of animation generation of the target virtual object.
  • Figure 1 is an implementation flow chart of a method for generating animations of 3D virtual objects provided by an embodiment of the present application
  • Figure 2 is a schematic diagram of a target virtual object provided by an embodiment of the present application.
  • Figure 3 is a schematic diagram of a first animation sequence generation process provided by an embodiment of the present application.
  • Figure 4 is an implementation flow chart of a method for generating a first animation sequence provided by an embodiment of the present application
  • Figure 5 is an implementation flow chart of a skeleton redirection process provided by an embodiment of the present application.
  • Figure 6 is a schematic diagram of a second animation sequence generation process provided by an embodiment of the present application.
  • Figure 7 is a schematic structural diagram of a 3D virtual object animation generation device provided by an embodiment of the present application.
  • Figure 8 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • 3D virtual objects provided by the embodiment of the present application can be applied in three-dimensional (Three Dimensions, 3D) digital creation or virtual reality (Virtual Reality, VR) scenes.
  • 3D digital creation can include but is not limited to 3D film and television creation or 3D game creation.
  • the animation generation method of 3D virtual objects can be used to generate an animation sequence of 3D virtual objects in the game.
  • the animation generation method of 3D virtual objects can be used to generate an animation sequence of 3D virtual objects in the game.
  • you need to construct a target 3D virtual character in a 3D game scene you can first record a video containing the character.
  • the postures of the characters in different video frames in the video can be different.
  • Three-dimensional reconstruction obtain the three-dimensional reconstruction information of the character in each video frame, and then generate the first animation sequence corresponding to the character based on the three-dimensional reconstruction information of the character in all video frames and the preset object model; due to the skeleton structure of the preset object model It is the same as or similar to the skeleton structure of the target 3D virtual character.
  • converting the video into the first animation sequence can provide a reference basis for subsequent construction of the animation model of the target 3D virtual character; after that, the prediction based on the preset object model is first Assume that the skeleton performs skeleton redirection on the target skeleton of the target 3D virtual character to obtain the redirected skeleton, and then performs animation redirection on the redirected skeleton based on the first animation sequence to obtain the second animation sequence corresponding to the target 3D virtual character, so that the redirected skeleton can be obtained
  • the actions of the characters in the video are mapped to a certain target 3D virtual character in the 3D game scene.
  • the execution subject of the animation generation method for 3D virtual objects provided by the embodiments of the present application may be a terminal device.
  • Terminal devices can include electronic devices such as mobile phones, tablets, laptops, and desktop computers. Please refer to Figure 1.
  • Figure 1 is an implementation flow chart of a 3D virtual object animation generation method provided by an embodiment of the present application.
  • the 3D virtual object animation generation method may include S101 ⁇ S104, as detailed below:
  • the terminal device can obtain the target video.
  • the target video may include several video frames, and each video frame may include a reference object.
  • the postures of the reference objects in different video frames may be the same or different.
  • the reference object refers to the object that the target virtual object refers to when simulating actions. Therefore, the skeleton structure of the reference object needs to be the same or similar to the skeleton structure of the target virtual object.
  • the target virtual object may be any 3D virtual object among the 3D virtual objects that need to be created. For example, when the target virtual object is a human-like object (for example, 21 or 22 in Figure 2), the reference object may be a human.
  • the target video can be a video containing a reference object recorded in real time, a video containing a reference object that is pre-recorded or produced, a video containing a reference object downloaded from the Internet, or it can be obtained through other methods. Videos containing reference objects. There are no special restrictions on how to obtain the target video.
  • the terminal device After the terminal device obtains the target video, it can process the target video into frames to obtain each video frame in the target video. For example, as shown in Figure 3, if 31 in Figure 3 is a target video, the terminal device can obtain multi-frame video frames 32 after performing frame segmentation processing on the target video 31.
  • the terminal device After the terminal device obtains each video frame in the target video, it can sequentially perform three-dimensional reconstruction of the reference object in each video frame.
  • the purpose of the terminal device performing three-dimensional reconstruction of the reference object in each video frame is to obtain the three-dimensional reconstruction information of the reference object in each video frame.
  • performing three-dimensional reconstruction of the reference objects in each video frame sequentially means performing three-dimensional reconstruction of the reference objects in each video frame according to the order in which each video frame is arranged in the target video.
  • the terminal device can use a three-dimensional reconstruction model to process each video frame in sequence to obtain the three-dimensional reconstruction information of the reference object in each video frame.
  • the 3D reconstruction model can be used to quickly and efficiently obtain the 3D reconstruction information of the reference object in each video frame.
  • the three-dimensional reconstruction model can be obtained by training a deep learning model using a preset sample data set.
  • Each piece of data in the preset sample data set includes a sample image and three-dimensional reconstruction information of the sample image.
  • the sample image is an image containing a reference object.
  • this preset sample data set to train the deep learning model, the sample image in each sample data is used as the input of the deep learning model, and the three-dimensional reconstruction information of the sample image in each sample data is used as the input of the deep learning model.
  • Output train the deep learning model, so that after the deep learning model training is completed, the deep learning model can learn the correspondence between the image containing the reference object and the three-dimensional reconstruction information of the reference object.
  • the terminal device can determine the trained deep learning model as a three-dimensional reconstruction model.
  • the three-dimensional reconstruction information of the reference object may include: grid point information, bone key point information, root node information, and shooting parameter information.
  • the grid point information is used to describe the outline of the reference object.
  • Skeleton keypoint information is used to describe the pose of the reference object.
  • the bone key point information may be the axis angle information of the bone key point.
  • the bone key points can include the starting point and end point of each bone of the human body.
  • the axis angle information of the bone key point is used to describe the three-dimensional space rotation information of the bone key point relative to its parent node.
  • the root node information is used to describe the pelvic position of the reference object.
  • the root node information can be described by the coordinates of the reference object's pelvic key points in the preset coordinate system.
  • the preset coordinate system can be set according to actual needs.
  • the shooting parameter information is used to describe the camera parameters when the reference object is photographed.
  • the shooting parameter information may include but is not limited to the zoom ratio of the camera.
  • the terminal device after the terminal device obtains the three-dimensional reconstruction information of the reference object in each video frame, it can store the bone key point information and root node information of the reference object in each video frame in an array respectively. And the arrays corresponding to all video frames are stored in the first preset file according to the order of each video frame in the target video.
  • the first preset file may be a file in PKL format.
  • PKL format is a file format used by python software to save files
  • python is a computer programming language.
  • a first animation sequence corresponding to the reference object is generated based on the three-dimensional reconstruction information of the reference object in all the video frames and a preset object model; the preset object model includes a preset skeleton.
  • Default object models refer to standard 3D reference object templates.
  • Preset object models can be described by preset skeletons and preset meshes. Among them, the preset skeleton is the same as or similar to the skeleton of the reference object. For example, if the reference object is a human, the default object model is a standard 3D human body template.
  • the preset object model has a preset initial posture.
  • 33 in Figure 3 is a standard 3D human body template, and its current posture is the above-mentioned initial posture.
  • the terminal device can adjust the posture and/or position of the preset object model according to the three-dimensional reconstruction information of the reference object in each video frame to obtain the first animation frame corresponding to each video frame and the third animation frame corresponding to all video frames.
  • One animation frame constitutes the first animation sequence.
  • the terminal device adjusts the posture and/or position of the preset object model 33 according to the three-dimensional reconstruction information of the reference object in each video frame 32 to obtain the first animation frame 34 corresponding to each frame 32 .
  • the first animation frames 34 corresponding to all the video frames 32 constitute the first animation sequence.
  • S102 may include S41 ⁇ S43 as shown in Figure 4, as detailed below:
  • the preset object model can be stored in the object model library in advance. Based on this, the terminal device can obtain the pre-stored preset object model from the object model library.
  • setting the structural parameters of the preset object model in each first animation frame may include setting the position and posture of the preset object model in each animation frame.
  • the terminal device can first generate multiple first animation frames containing the preset object model, and the number of the first animation frames is consistent with the number of video frames included in the target video; and then set the first animation frame of each frame.
  • the position information of the preset object in the first animation frame can be determined by the shooting parameters of the reference object in the corresponding video frame and the position information of the root node; the posture of the preset object model can be determined by the grid points of the reference object in the corresponding video frame. information and bone keypoint information determined.
  • S42 may include the following steps:
  • Step a For each video frame, determine the spatial position of the preset object model in the corresponding first animation frame according to the shooting parameters and the position information of the root node of the reference object.
  • the terminal device after the terminal device obtains the shooting parameters and the location information of the root node of the reference object in each video frame, it can combine weak perspective projection based on the scaling ratio in the shooting parameters and the location information of the root node. Relationship, calculate the spatial position of the preset object model in the first animation frame corresponding to each video frame.
  • Step b Adjust the root node position of the preset object model in the first animation frame based on the spatial position of the preset object model in the corresponding first animation frame.
  • the terminal device After the terminal device obtains the spatial position of the preset object model in the first animation frame of each frame, it adjusts the position of the root node of the preset object model in the first animation frame of each frame so that the root node of the preset object model The position of the node is consistent with the spatial position of the preset object model in the corresponding first animation frame.
  • Step c Set the posture of the preset object model in the corresponding first animation frame based on the grid point information and the skeletal key point information.
  • the terminal device After the terminal device adjusts the spatial position of the preset object model in the first animation frame of each frame, it can based on the grid point information and bone key point information of the reference object in the video frame corresponding to the first animation frame of each frame, The posture of the preset object model in the first animation frame of each frame is adjusted so that the posture of the preset object model in the first animation frame of each frame is the same as the posture of the reference object model in the corresponding video frame.
  • the terminal device can first calculate the axis angle of each bone key point of the preset object model in the first animation frame of each frame based on the bone key point information of the reference object in the video frame corresponding to the first animation frame of each frame. Adjust so that the axis angle of each bone key point of the preset object model in the first animation frame of each frame is the same as the axis angle of each bone key point of the reference object model in the corresponding video frame; after that, the terminal device can be based on The grid point information of the reference object in the video frame corresponding to the first animation frame of each frame is adjusted to the grid points of the preset object model in the first animation frame of each frame, so that the first animation frame of each frame The grid points of the preset object model in are the same as the grid points of the reference object model in the corresponding video frame.
  • the preset first animation frame rate can be set according to actual needs.
  • the preset first animation frame rate may be the same as the frame rate of the target video, or the preset first animation frame rate may be n times the frame rate of the target video, and n may be an integer greater than 1.
  • the terminal device can sort all the first animation frames according to the order of the video frames corresponding to the first animation frames of each frame in the target video, and all the sorted first animation frames constitute the first animation sequence.
  • the first animation frame corresponding to the first video frame is regarded as the first frame of the first animation sequence
  • the first animation frame corresponding to the second video frame is regarded as the second frame of the first animation sequence
  • the corresponding first animation frames of each of the above video frames can be composed into a first animation sequence in the above order.
  • the terminal device After the terminal device obtains the first animation sequence, it can store the first animation sequence in the second preset file.
  • the second preset file may be a file in FBX format.
  • FBX format is a file format used by Filmbox software to save files.
  • Filmbox is a software that allows mutual guidance of models, materials, actions and camera information between 3D design software.
  • the video frame can be The action pose of the reference object in the animation is quickly transferred to the preset object in the first animation frame.
  • the first animation frames of each frame after the above-mentioned structural parameter setting are arranged in order. All the first animation frames after sorting constitute the first animation sequence, so that the target video can be mapped to the corresponding animation sequence, and each first animation frame can be mapped to the corresponding animation sequence.
  • the posture and position of the reference object in the video frame are quickly mapped to the preset object model in the corresponding first animation frame, so that the preset object model in the first animation frame of each frame has the same characteristics as the reference object in the corresponding video frame. attitude and position.
  • the target virtual object can be any 3D virtual object that needs to be created in the target scene, for example, it can be a 3D virtual character.
  • Target scenes can include 3D game creation scenes or VR scenes, etc.
  • Each target virtual object corresponds to a default target object model.
  • the target object model can be described by an initial skeleton and an initial mesh.
  • the terminal device can first determine the target skeleton of the target virtual object based on the preset skeleton and the initial skeleton of each target virtual object, and then perform skeleton redirection on the target skeleton of the target virtual object based on the above preset skeleton to obtain the redirected skeleton.
  • the structure of the target skeleton is the same as or similar to the structure of the preset skeleton.
  • the skeleton reorientation process on the target skeleton refers to the process of corresponding the same bone key points of the target skeleton with the preset skeleton.
  • the terminal device can obtain the target skeleton of each target virtual object through S51 ⁇ S53 as shown in Figure 5, as detailed below:
  • bones to be merged are determined from the preset skeleton, and the bones to be merged are merged to obtain merged bones.
  • the above-mentioned skeleton to be merged can be determined based on the actual situation of the preset skeleton.
  • the bones to be merged in different skeletons can be different.
  • the embodiment of the present application does not specifically limit the merging rules and merging methods of the bones to be merged in the preset skeleton.
  • the skeleton composed of the merged bones can describe the action posture of the reference object.
  • a target object model of the target virtual object is obtained; the target object model includes an initial skeleton.
  • the target object model of the target virtual object can be pre-stored in the virtual object model library corresponding to the target scene. Based on this, the terminal device can obtain the target object model of the target virtual object from the virtual object model library.
  • the corresponding bones in the initial skeleton are set based on the merged bones to obtain the target skeleton of the target virtual object.
  • the terminal device after the terminal device merges the bones to be merged in the preset skeleton, it can merge the corresponding bones in the initial skeleton based on the merge rules and merge methods of the merged bones, and the initial skeleton after the skeleton is merged Identifies the target skeleton as the target virtual object.
  • the naming rules of the key points of the target skeleton and the naming rules of the key points of the preset skeleton may be different, it is necessary to distinguish the key points of the target skeleton and the key points of the preset skeleton. Bone key points are matched to match the target skeleton with the same bone key points of the preset skeleton, thereby reorienting the target skeleton.
  • Animation redirection refers to copying action data from one skeleton to another. Therefore, animation redirection of the redirected skeleton based on the first animation sequence means copying the action data of the preset object model in the first animation sequence from The preset skeleton is copied to the redirected skeleton of the target object model, thereby obtaining a second animation sequence of the target object model.
  • the terminal device may first generate multiple frames of second animation frames containing the target object model after skeleton redirection, where the number of second animation frames is consistent with the number of first animation frames; and then set each The initial position of the target object model in the second animation frame of each frame.
  • the initial position of the target object model in the second animation frame of each frame can be set at the center of the second animation frame of each frame; and then based on each frame The position and posture of the preset object model in the first animation frame are adjusted to the movement and posture of the target object model in the second animation frame.
  • the position of the target object model in the second animation frame can be determined by the position of the root node of the preset object model in the corresponding first animation frame; the posture of the target object model can be determined by the posture of the preset object model in the corresponding first animation frame.
  • S104 may include the following steps:
  • Step d For each second animation frame, adjust the spatial position of the target object model in the second animation frame according to the spatial position of the preset object model in the first animation frame of each frame.
  • the terminal device can obtain the spatial position of the preset object model in the first animation frame of each frame, and adjust the spatial position of the target object model in the second animation frame corresponding to each frame, so that the preset object
  • the spatial position of the model in the first animation frame is the same as the spatial position of the target object model in the second animation frame, that is, even if the position of the root node of the preset object in the first animation frame is the same as the position of the target object model in the second animation frame
  • the location of the root node is the same.
  • Step e For each second animation frame, the terminal device adjusts the posture of the target object model so that the posture of the target object model is the same as the posture of the reference object model.
  • the terminal device After the terminal device adjusts the spatial position of the target object model in each second animation frame, it can use the grid point information and bone key of the preset object model in the first animation frame corresponding to each second animation frame. point information, adjust the posture of the target object model in the second animation frame of each frame, so that the posture of the target object model in the second animation frame of each frame corresponds to the posture of the preset object model in the first animation frame same.
  • the terminal device may first calculate the axes of each bone key point of the target object model in each second animation frame based on the bone key point information of the reference object in the first animation frame corresponding to each second animation frame. The angle is adjusted so that the axis angle of each bone key point of the target object model in the second animation frame of each frame is the same as the axis angle of each bone key point of the corresponding preset object model in the first animation frame; after that, the terminal The device can adjust the grid points of the target object model in each second animation frame based on the grid point information of the preset object in the first animation frame corresponding to each second animation frame, so that each The grid points of the target object model in the second animation frame are the same as the corresponding grid points of the preset object model in the first animation frame.
  • Step f According to the preset second first animation frame rate, all the adjusted second first animation frames are sequentially composed into the second animation sequence.
  • the preset second animation frame rate can be set according to actual needs.
  • the preset second animation frame rate may be the same as the first animation frame rate, or the preset second animation frame rate may be n times the first animation frame rate, and n may be an integer greater than 1.
  • the terminal device can sort all the second animation frames according to the order in which the second animation frames correspond to the first animation frames, and all the sorted second animation frames constitute the second animation sequence.
  • the second animation frame corresponding to the first animation frame of the first frame is regarded as the first frame of the second animation sequence
  • the second animation frame corresponding to the first animation frame of the second frame is regarded as the second frame of the second animation sequence.
  • the corresponding second animation frames of each of the above first animation frames can be composed in the above order to form a second animation sequence.
  • the terminal device After the terminal device obtains the second animation sequence, it can store the second animation sequence in the third preset file.
  • the third preset file may be a file in FBX format.
  • FBX format is a file format used by filmbox software to save files.
  • Filmbox is a software that allows mutual guidance of models, materials, actions and camera information between 3D design software.
  • the first animation frame obtained according to each video frame can be 621, and all the first animation frames constitute The first animation sequence 61; the second animation frame obtained according to each first animation frame may be 631, and all the second animation frames 631 constitute the second animation sequence 63.
  • the above scheme obtains the three-dimensional reconstruction information of the reference object in each video frame by sequentially performing three-dimensional reconstruction of the reference object in each video frame of the target video, and then based on the three-dimensional reconstruction information of the reference object in all video frames Generate a first animation sequence corresponding to the reference object with the preset object model; since the structure of the preset skeleton of the preset object model is the same or similar to the structure of the target skeleton of the target virtual object, the target video is converted into the corresponding first animation sequence.
  • the animation sequence can provide a reference basis for the subsequent animation simulation of the target virtual object; then, first perform skeleton redirection on the target skeleton of the target virtual object based on the preset skeleton to obtain the redirected skeleton, and then perform the redirected skeleton based on the first animation sequence.
  • Animation redirection obtains the second animation sequence corresponding to the target virtual object, so that the actions of the reference object in the target video can be mapped to the virtual object. That is, this solution can be used to automatically simulate the actions of the reference object in the target video and obtain the target
  • the animation sequence of virtual objects does not require professionals to produce key animation frames of the target virtual object, thereby improving the efficiency of animation generation of the target virtual object.
  • embodiments of the present application further provide an animation generation device for 3D virtual objects that implements the above method embodiments.
  • Figure 7 shows the animation generation method of 3D virtual objects provided by the embodiments of the present application.
  • the animation generation device of 3D virtual objects may include: a three-dimensional reconstruction unit 71 , an animation sequence generation unit 72 , a skeleton reorientation unit 73 and an animation reorientation unit 74 . in:
  • the three-dimensional reconstruction unit 71 is configured to sequentially perform three-dimensional reconstruction of the reference object in each video frame of the target video to obtain three-dimensional reconstruction information of the reference object in each video frame.
  • the animation sequence generating unit 72 is configured to generate a first animation sequence corresponding to the reference object based on the three-dimensional reconstruction information of the reference object in all video frames and a preset object model; the preset object model includes a preset skeleton.
  • the skeleton redirection unit 73 is configured to perform skeleton redirection on the target skeleton of the target virtual object based on the preset skeleton to obtain a redirected skeleton; the structure of the redirected skeleton is the same as or similar to the structure of the preset skeleton.
  • the animation redirection unit 74 is configured to perform animation redirection on the redirected skeleton based on the first animation sequence to obtain a second animation sequence corresponding to the target virtual object.
  • the animation sequence generating unit 72 includes a first obtaining unit, a parameter setting unit and a sorting unit. in:
  • the first acquisition unit is used to acquire the preset object model.
  • the parameter setting unit is configured to separately set the structural parameters of the preset object model in each first animation frame of the first animation sequence based on the three-dimensional reconstruction information of the reference object in each frame of the video frame. Make settings.
  • the sorting unit is configured to sequence all the first animation frames into the first animation sequence according to a preset first animation frame rate.
  • the parameter setting unit includes a position determination unit, a position adjustment unit and an attitude adjustment unit. in:
  • the position determination unit is configured to determine, for each video frame, the space of the preset object model in the corresponding first animation frame according to the shooting parameters of the reference object in the video frame and the position information of the root node. Location.
  • the position adjustment unit is configured to adjust the position of the root node of the preset object model in the corresponding first animation frame based on the spatial position of the preset object model in the corresponding first animation frame.
  • the posture adjustment unit is configured to set the posture of the preset object model in the corresponding first animation frame based on the grid point information and skeletal key point information of the reference object in the video frame.
  • the 3D virtual object animation generation device also includes a bone merging unit, a second acquisition unit, and a bone setting unit. in:
  • the bone merging unit is used to determine the bones to be merged from the preset skeleton, and merge the bones to be merged to obtain the merged bones.
  • the second acquisition unit is used to acquire the target object model of the target virtual object; the target object model includes an initial skeleton.
  • the bone setting unit is used to set corresponding bones in the initial skeleton based on the merged bones to obtain the target skeleton of the target virtual object.
  • the skeleton redirection unit 73 is specifically used for:
  • the three-dimensional reconstruction unit 71 is specifically used for:
  • All video frames of the target video are imported into a preset three-dimensional reconstruction model frame by frame for processing, and the three-dimensional reconstruction information of the reference object in each video frame is obtained respectively.
  • FIG. 8 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device 8 provided by this embodiment may include: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and runable on the processor 80, such as a method for generating animations of 3D virtual objects. corresponding program.
  • the processor 80 executes the computer program 82, the steps in the above embodiment of the animation generation method applied to 3D virtual objects are implemented, such as S101 to S104 shown in FIG. 1 .
  • the processor 80 executes the computer program 82
  • the functions of each module/unit in the above embodiment of the animation generation device for 3D virtual objects are implemented, such as the functions of the units 71 to 74 shown in FIG. 7 .
  • the computer program 82 may be divided into one or more modules/units, and one or more modules/units are stored in the memory 81 and executed by the processor 80 to complete the present application.
  • One or more modules/units may be a series of computer program instruction segments capable of completing specific functions. The instruction segments are used to describe the execution process of the computer program 82 in the terminal device 8 .
  • the computer program 82 can be divided into a three-dimensional reconstruction unit, an animation sequence generation unit, a skeleton redirection unit, and an animation redirection unit.
  • an animation redirection unit For the specific functions of each unit, please refer to the relevant descriptions in the corresponding embodiment of FIG. 7 and will not be described again here. .
  • FIG. 8 is only an example of the terminal device 8 and does not constitute a limitation on the terminal device 8. It may include more or less components than shown in the figure, or some components may be combined, or different components may be used. .
  • Processor 80 may be a central processing unit (central processing unit). processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (field-programmable) gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • CPU central processing unit
  • DSP digital signal processors
  • ASICs application specific integrated circuits
  • FPGA field-programmable gate array
  • a general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
  • the memory 81 may be an internal storage unit of the terminal device 8 , such as a hard disk or memory of the terminal device 8 .
  • the memory 81 may also be an external storage device of the terminal device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card or a flash memory card (flash card) equipped on the terminal device 8. card) etc.
  • the memory 81 may also include both an internal storage unit of the terminal device 8 and an external storage device.
  • the memory 81 is used to store computer programs and other programs and data required by the terminal device.
  • the memory 81 may also be used to temporarily store data that has been output or is to be output.
  • each functional unit in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware.
  • the specific names of each functional unit are only for the convenience of distinguishing each other and are not used to limit the protection scope of the present application.
  • Embodiments of the present application also provide a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program. When executed by a processor, the computer program can implement the steps in each of the above method embodiments.
  • Embodiments of the present application provide a computer program product.
  • the terminal device implements the steps in each of the above method embodiments.

Abstract

La présente demande peut être appliquée au domaine technique du traitement d'image. Sont décrits un procédé et un appareil de génération d'animation d'objet virtuel 3D, un dispositif terminal et un support. Le procédé de génération d'animation consiste à : réaliser successivement une reconstruction tridimensionnelle sur un objet de référence dans chaque trame vidéo d'une vidéo cible pour obtenir des informations de reconstruction tridimensionnelle de l'objet de référence dans chaque trame vidéo; selon les informations de reconstruction tridimensionnelle des objets de référence dans toutes les trames vidéo et un modèle d'objet prédéfini, générer une première séquence d'animation correspondant à l'objet de référence, le modèle d'objet prédéfini comprenant un squelette prédéfini; sur la base du squelette prédéfini, réaliser un reciblage de squelette sur le squelette cible de l'objet virtuel cible pour obtenir un squelette reciblé, la structure du squelette cible étant identique ou similaire à la structure du squelette prédéfini; et, sur la base de la première séquence d'animation, réaliser un reciblage d'animation sur le squelette reciblé pour obtenir une seconde séquence d'animation correspondant à l'objet virtuel cible, ce qui permet d'améliorer l'efficacité de génération d'animation de l'objet virtuel cible.
PCT/CN2022/103020 2022-06-30 2022-06-30 Procédé et appareil de génération d'animation d'objet virtuel 3d, dispositif terminal et support WO2024000480A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/103020 WO2024000480A1 (fr) 2022-06-30 2022-06-30 Procédé et appareil de génération d'animation d'objet virtuel 3d, dispositif terminal et support

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/103020 WO2024000480A1 (fr) 2022-06-30 2022-06-30 Procédé et appareil de génération d'animation d'objet virtuel 3d, dispositif terminal et support

Publications (1)

Publication Number Publication Date
WO2024000480A1 true WO2024000480A1 (fr) 2024-01-04

Family

ID=89383821

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/103020 WO2024000480A1 (fr) 2022-06-30 2022-06-30 Procédé et appareil de génération d'animation d'objet virtuel 3d, dispositif terminal et support

Country Status (1)

Country Link
WO (1) WO2024000480A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150178988A1 (en) * 2012-05-22 2015-06-25 Telefonica, S.A. Method and a system for generating a realistic 3d reconstruction model for an object or being
CN111028317A (zh) * 2019-11-14 2020-04-17 腾讯科技（深圳）有限公司 虚拟对象的动画生成方法、装置、设备及存储介质
CN111968207A (zh) * 2020-09-25 2020-11-20 魔珐(上海)信息科技有限公司 动画生成方法、装置、系统及存储介质
CN113920232A (zh) * 2021-10-29 2022-01-11 上海商汤智能科技有限公司 动画生成方法、装置、计算机设备和存储介质
CN114219878A (zh) * 2021-12-14 2022-03-22 魔珐(上海)信息科技有限公司 虚拟角色的动画生成方法及装置、存储介质、终端

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150178988A1 (en) * 2012-05-22 2015-06-25 Telefonica, S.A. Method and a system for generating a realistic 3d reconstruction model for an object or being
CN111028317A (zh) * 2019-11-14 2020-04-17 腾讯科技（深圳）有限公司 虚拟对象的动画生成方法、装置、设备及存储介质
CN111968207A (zh) * 2020-09-25 2020-11-20 魔珐(上海)信息科技有限公司 动画生成方法、装置、系统及存储介质
CN113920232A (zh) * 2021-10-29 2022-01-11 上海商汤智能科技有限公司 动画生成方法、装置、计算机设备和存储介质
CN114219878A (zh) * 2021-12-14 2022-03-22 魔珐(上海)信息科技有限公司 虚拟角色的动画生成方法及装置、存储介质、终端

Similar Documents

Publication Publication Date Title
WO2022002032A1 (fr) Formation de modèle entraîné par image et génération d'image
JP7337104B2 (ja) 拡張現実によるモデル動画多平面インタラクション方法、装置、デバイス及び記憶媒体
WO2020125567A1 (fr) Procédé de génération d'animation automatique et système de génération d'animation automatique
JP7457806B2 (ja) レンズ分割の実現方法、装置および関連製品
CN112884881B (zh) 三维人脸模型重建方法、装置、电子设备及存储介质
CN112819944B (zh) 三维人体模型重建方法、装置、电子设备及存储介质
US20220126447A1 (en) Data generation method and apparatus, and storage medium
WO2022075859A1 (fr) Mappage de modèle facial à l'aide d'un réseau neuronal formé sur des niveaux variables de détail de balayages faciaux
KR20230162107A (ko) 증강 현실 콘텐츠에서의 머리 회전들에 대한 얼굴 합성
CN115331265A (zh) 姿态检测模型的训练方法和数字人的驱动方法、装置
KR102358464B1 (ko) 인공지능 기술을 이용하여 영상 정보로부터 3d 캐릭터 애니메이션을 자동으로 생성하는 3d 이미지 변환장치 및 이를 포함하는 3d 이미지 변환시스템
CN117372602B (zh) 一种异构三维多对象融合渲染方法、设备及系统
WO2024000480A1 (fr) Procédé et appareil de génération d'animation d'objet virtuel 3d, dispositif terminal et support
WO2024027063A1 (fr) Procédé et appareil de flux continu en direct, support de stockage, dispositif électronique et produit
CN109360274A (zh) 沉浸式虚拟现实构建方法、装置、智能升降桌及存储介质
CN115239856A (zh) 3d虚拟对象的动画生成方法、装置、终端设备及介质
CN113705379A (zh) 一种手势估计方法、装置、存储介质及设备
CN113805532B (zh) 一种制作实体机器人动作的方法及终端
CN116152447B (zh) 一种人脸建模方法、装置、电子设备及存储介质
JP7483301B2 (ja) 画像処理及び画像合成方法、装置及びコンピュータプログラム
WO2024077792A1 (fr) Procédé et appareil de génération de vidéo, dispositif et support de stockage lisible par ordinateur
US20240135616A1 (en) Automated system for generation of facial animation rigs
CN114972615A (zh) 一种光影的绘制方法和装置
WO2024027285A1 (fr) Procédé et appareil de traitement d'expression faciale, dispositif informatique et support de stockage
Malowany Beyond Homographies: Exploration and Analysis of Image Warping for Projection in a Dome

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22948583

Country of ref document: EP

Kind code of ref document: A1