WO2010060113A1 - Génération en temps réel de modèles de personnages 3d prêts pour l' animation - Google Patents

Génération en temps réel de modèles de personnages 3d prêts pour l' animation Download PDF

Info

Publication number
WO2010060113A1
WO2010060113A1 PCT/US2009/065825 US2009065825W WO2010060113A1 WO 2010060113 A1 WO2010060113 A1 WO 2010060113A1 US 2009065825 W US2009065825 W US 2009065825W WO 2010060113 A1 WO2010060113 A1 WO 2010060113A1
Authority
WO
WIPO (PCT)
Prior art keywords
mesh
clothing
template
anatomical
model
Prior art date
Application number
PCT/US2009/065825
Other languages
English (en)
Inventor
Stefano Corazza
Emiliano Gambaretto
Original Assignee
Mixamo, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mixamo, Inc. filed Critical Mixamo, Inc.
Publication of WO2010060113A1 publication Critical patent/WO2010060113A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/60—Methods for processing data by generating or executing the game program
    • A63F2300/66—Methods for processing data by generating or executing the game program for rendering three dimensional images
    • A63F2300/6607—Methods for processing data by generating or executing the game program for rendering three dimensional images for animating game characters, e.g. skeleton kinematics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/16—Cloth

Definitions

  • the present invention relates generally to computer generated graphics, and more specifically to the real time generation of 3D characters for use in animation.
  • FIG. 1 A common process for generating 3D character animations is illustrated in FIG. 1.
  • the process 100 involves creating (102) a mesh of the 3D character.
  • a mesh is a collection of vertices and polygons that define the shape of an object in 3D.
  • a texture is created (104) and is applied (106) to the mesh.
  • the texture defines the appearance of the mesh.
  • a skeleton is created (108) and skinning weights (110) are defined.
  • the skeleton defines the articulated body parts of the mesh and the skinning weights define the deformation of the mesh as a function of the motion of the skeleton.
  • the process of defining a skeleton and skinning weights is often referred to as rigging the 3D character.
  • the rigging of a 3D character is usually quite complex and time-consuming, and requires specific 3D modeling knowledge/education. This represents a barrier to the generation of 3D animation content from a wider audience.
  • most 3D modeling tools require the installation of 3 rd party software, which represents a further barrier to widespread use and diffusion of 3D animation related art.
  • Animation-ready 3D character models are generated in real time based upon a description of a desired 3D character provided via a web based user interface.
  • animation-ready 3D character models are generated by animators for use in animations.
  • animation-ready 3D character models are generated in interactive environments such as video games and are used to generate animations within the game.
  • One embodiment of the invention includes an application server configured to receive the user defined model parameters and the clothing selection via a user interface.
  • the application server includes a generative model and the application server is configured to generate a 3D anatomical mesh based upon the user defined model parameters using the generative model
  • the application server includes at least one clothing mesh template including a clothing mesh, a template skeleton, and skinning weights and the application server is configured to apply the clothing mesh from the clothing mesh template corresponding to the user clothing selection to the generated 3D anatomical mesh to create a clothed mesh
  • the application server is configured to adjust the template skeleton of the clothing mesh template corresponding to the user clothing selection based upon the shape of the clothed mesh
  • the application server is configured to generate skinning weights based upon the skinning weights of the clothing mesh template corresponding to the user clothing selection
  • the application server stores an animation-ready 3D character model including the clothed mesh, the adjusted skeleton, and the generated skinning weights.
  • the clothed mesh has polygon correspondence with the clothing mesh template corresponding to the user clothing selection.
  • the generated skinning weights are the same as the skinning weights of the clothing mesh template corresponding to the user clothing selection.
  • the clothing mesh model includes UV mappings used to map texture to the clothing mesh model
  • the application server is configured to apply modified textures to the clothed mesh utilizing the UV mappings.
  • the user interface is a web based user interface generated by the application server.
  • the application server is configured to generate the animation-ready 3D character model in real time in response to receipt of the user defined model parameters and the clothing selection via the web based user interface.
  • the generative model is produced by principal component analysis of a data set of 3D meshes
  • the application server is configured to transform the user defined model parameters in terms of the principal components of the generative model
  • the 3D anatomical mesh is generated using the transformed user defined model parameters and the generative model.
  • the application server is configured to apply the clothing mesh template to the 3D anatomical mesh using a Mean Value Coordinates calculation that deforms the vertices of the clothing mesh template in response to the location of the vertices of the 3D anatomical mesh.
  • the application server is configured to apply a blend shape process to correct artifacts in the clothed mesh introduced by the Mean
  • the application server is configured to adjust the template skeleton based upon the deformation of the clothing mesh template using a Mean Value Coordinates calculation.
  • the application server is configured to apply the clothing mesh template to the 3D anatomical mesh by morphing the clothing mesh template into the 3D anatomical mesh using a non-rigid mesh registration process.
  • An embodiment of the method of the invention includes receiving model parameters and a clothing selection via a user interface, automatically generating a 3D anatomical mesh based upon the received model parameters, automatically applying a clothing mesh template based upon the clothing selection to the generated 3D anatomical model to produce a clothed mesh, and automatically rigging the clothed mesh by adjusting the template skeleton and generating skinning weights based upon the skinning weight of the clothing mesh template.
  • a further embodiment of the method of the invention includes mapping the model parameters to principal components of a generative model, where the generative model is created by principal component analysis of a data set of 3D meshes; and generating a 3D anatomical mesh based upon the mapped model parameters using the generative model.
  • applying a clothing mesh template based upon the clothing selection to the generated 3D anatomical model to produce a clothed mesh further includes using a Mean Value Coordinates calculation that deform the vertices of the clothing mesh template in response to the location of the vertices of the 3D anatomical mesh.
  • a still further embodiment of the method of the invention also includes applying a blend shape process to correct artifacts in the clothed mesh introduced by the Mean Value Coordinates calculation.
  • adjusting the template skeleton further includes adjusting the joints of the skeleton based upon the deformation of the clothing mesh template using a Mean Value Coordinates calculation.
  • applying a clothing mesh template based upon the clothing selection to the generated 3D anatomical model to produce a clothed mesh further includes applying the clothing mesh template to the 3D anatomical mesh by morphing the clothing mesh template into the 3D anatomical mesh using a non-rigid mesh registration process.
  • the clothed mesh has polygon correspondence with the clothing mesh template selected by the user.
  • the generated skinning weights are the same as the skinning weights of the clothing mesh template.
  • Another embodiment again of the method of the invention further includes modifying the texture of the clothed mesh by applying textures using UV mappings that were used to map texture to the clothing mesh model.
  • Another further embodiment of the method of the invention includes deforming the vertices of the clothing mesh based upon the vertices of the 3D anatomical mesh using a Mean Value Coordinates calculation.
  • Still another further embodiment of the method of the invention includes applying a blend shape process to correct artifacts in the clothed mesh introduced by the Mean Value Coordinates calculation.
  • Yet another further embodiment of the method of the invention includes modifying the location of the joints of the template skeleton based upon the deformation of the template mesh using a Mean Value Coordinate calculation.
  • FIG. 1 is a flow chart illustrating a common process for generating 3D character animations.
  • FIG. 2 is a network diagram illustrating a system for automatically generating animation-ready 3D characters based upon parameters received from a user via the
  • FIG. 3 is a flow chart illustrating a process for generating animation-ready 3D characters in accordance with an embodiment of the invention.
  • FIG. 4 is a flow chart illustrating a process for using Principal Component
  • FIG. 5 is a flow chart illustrating a process for applying a clothing mesh template to a 3D anatomical mesh in accordance with an embodiment of the invention.
  • FIG. 6a is a rendering of a 3D mesh showing an artifact resulting from the application of a clothing mesh template to a 3D anatomical mesh in accordance with an embodiment of the invention.
  • FIG. 6b is a rendering of the 3D mesh shown in FIG. 6b following application of a blend shape process to smooth artifacts introduced by the application of the clothing mesh template to the 3D anatomical mesh.
  • FIG. 7 is a flow chart illustrating a process for removing artifacts from 3D meshes generated by applying a clothing mesh template to a 3D anatomical mesh in accordance with an embodiment of the invention.
  • FIG. 8 is a flow chart illustrating a process for adjusting a template skeleton based upon the shape of a 3D mesh in accordance with an embodiment of the invention.
  • FIG. 9 is a flow chart illustrating a process for applying pre-defined skinning weights to rig a 3D mesh in accordance with an embodiment of the invention.
  • FIG. 10 is a flow diagram conceptually illustrating a process for creating an animation-ready 3D character model in accordance with an embodiment of the invention.
  • An animation-ready 3D character model is typically considered to be a 3D character model for which a mesh, a skeleton, and skinning weights are defined.
  • the skeleton and the skinning weights define the deformation of the mesh during articulation of the body parts of the 3D character. Therefore, the combination of the mesh, skeleton, and skinning weights is sufficient to enable the animation of the 3D character.
  • an animation-ready 3D character model is generated in response to a description of the desired characteristics of the 3D character using a generative model.
  • the generative model is used to generate a 3D anatomical mesh.
  • the 3D anatomical mesh defines the body of the character.
  • Each 3D anatomical mesh generated by the generative model posseses polygon correspondence with a number of clothing mesh templates, which can be applied to the generated 3D mesh.
  • a clothing mesh template is template that is typically created by animoator, in which a clothing mesh is applied to a 3D anatomical mesh and the mesh is rigged with a skeleton and skinning weights.
  • a user can select a clothing mesh template to apply to the generated 3D anatomical mesh.
  • the clothing mesh template includes a template skeleton, which can be automatically adjusted according to the generated 3D anatomical mesh to which the clothing mesh is applied.
  • the result is often referred to as a clothed mesh.
  • a clothed mesh Due to the polygon correspondence between the clothing template mesh and the generated 3D anatomical mesh, he same UV mappings used to apply texture to the clothing mesh template can be used to map textures to the clothed mesh. Therefore, the user can modify the texture of the clothed mesh in real time. For example, the color and/or pattern of the clothing can be changed by the user.
  • the clothing mesh template also includes skinning weights, which can be directly applied to rig the 3D character model due to the polygon correspondence between the generated 3D anatomical mesh and the clothing mesh template.
  • the application of the clothing mesh template to the generated 3D mesh typically involves using the 3D mesh to " drive " the deformation of the clothing mesh to created the clothed mesh.
  • the process of deforming the clothing mesh template based upon the generated 3D anatomical mesh results in artifacts at intersections of body parts of the 3D character. Therefore, many embodiments apply a process to correct mesh flattening and other artifacts introduced by application of the clothing mesh template to the generated 3D mesh.
  • a generative model for 3D characters is created using unsupervised learning or supervised learning. In both cases the learning is achieved using a set of example 3D meshes that can involve labeled data.
  • the generative model enables the generation of new 3D meshes that interpolate or extrapolate the space defined by the example 3D meshes.
  • the labels are used by the generative model to express the space of 3D meshes in terms of meaningful parameters that describe the anatomical characteristics of a specific 3D mesh.
  • the model parameters can include but are not limited to height, weight, and/or attractiveness.
  • a user interface is provided that enables a user to automatically create, view, and animate a 3D character model in real time.
  • Systems and methods for generating animation-ready 3D characters in accordance with embodiments of the invention are discussed further below.
  • FIG. 2 A system for generating an animation-ready 3D character model in accordance with the embodiment of the invention is illustrated in FIG. 2.
  • the system 200 includes an application server 202 that is configured to communicate with a client device 204 via a network such as the Internet 206.
  • the application server is typically configured to provide a user interface to the client device via which a user is able to specify characteristics of a 3D character.
  • the application server includes a generative model.
  • the application server uses the description provided via the client device to create a 3D anatomical mesh.
  • the application server can also include a number of clothing mesh templates.
  • the clothing mesh templates typically include a clothing mesh constrained to have polygon correspondence with the 3D anatomical meshes generated by the generative model, a template skeleton, and a set of skinning weights.
  • the user can specify clothing to apply to the 3D character and the appropriate clothing mesh is applied to the generated 3D anatomical mesh.
  • the application server adjusts the template skeleton for the clothing mesh template in accordance with the shape of the 3D mesh created by the application of the clothing mesh to the 3D anatomical mesh.
  • the application server can then apply the skinning weights from the clothing mesh template to rig the generated 3D character model.
  • the generative model used to produce the 3D anatomical mesh is created using a database of character shapes 208. Any of a variety of processes can be used in the creation of a generative model including supervised learning in which each of the example 3D meshes in a database is labeled and a process such as the process described in U.S. Patent Application Serial No.
  • Polygon correspondence refers to the fact that each 3D mesh includes the same number of polygons and that each polygon has a corresponding polygon in all of the 3D meshes.
  • generating 3D anatomical meshes that have polygon correspondence can provide a number of benefits during the application of a clothing mesh and/or during the application of skinning weights to complete the rigging of the generated 3D character model. Processes for building a generative model and the use of a generative model in the automatic generation of animation-ready 3D character models in accordance with embodiments of the invention are discussed further below.
  • an animation-ready 3D character model can be generated by an animator using a web based user interface provided by the application server and the animation-ready 3D character model can be downloaded to a client device for animation.
  • the client device can also specify motions to animate the generated 3D character model and download both the 3D character model and the motion data used to animate the 3D character model.
  • the animator could specify the motion data manually or generate the motion data automatically using a process such as one of the processes described in accordance with the process described in U.S. Patent Application No.
  • an animation-ready 3D character model in accordance with an embodiment of the invention can be generated in real time is the creation of a 3D character or avatar by a user for use in a 3D interactive application such as a video game.
  • An application server in accordance with an embodiment of the invention can enable a user to specify the characteristics and clothing of a 3D character, and the generated 3D character model can be animated within the 3D video game.
  • Systems and methods in accordance with embodiments of the invention can also be utilized to generate animation-ready 3D character models or animated 3D characters in a variety of contexts including but not limited to those outlined above and contexts in which the generation is performed, offline on a local workstation, and/or in a non-real time environment.
  • a process for generating an animation-ready 3D character model in accordance with an embodiment of the invention is illustrated in FIG. 3.
  • the process 300 commences when a user provides (302) a definition of a desired 3D character.
  • the definition is typically provided using a series of model parameters, where each model parameter describes a particular anatomical characteristic of the 3D character.
  • model parameters include, but are not limited to, height, weight, musculature, and attractiveness.
  • the description of the 3D character is used to automatically generate (304) a 3D anatomical mesh using a generative model.
  • the automatic generation of a 3D anatomical mesh is discussed further below.
  • the user can also specify clothing for the character.
  • the clothing is selected (306) from one of a number of options, where each option corresponds to a clothing mesh template.
  • artifacts such as mesh flattening occur at complex vertices when a clothing mesh is applied to the generated 3D anatomical mesh. Therefore, a process can be applied to correct (308) for introduced artifacts in the clothed mesh of the 3D character. Processes for correcting for introduced artifacts are discussed further below.
  • the rigging of the clothed mesh is then completed by automatically generating skinning weights (312) for the clothed mesh.
  • skinning weights 312
  • polygon correspondence between the clothed mesh and the clothing mesh template enables the clothed mesh to be rigged by simply applying the same skinning weights from the clothing mesh template to the clothed mesh of the 3D character.
  • the automatic rigging of animation-ready 3D character models is also discussed further below.
  • the process 300 illustrated in FIG. 3 requires considerably less specialized knowledge in order to generate an animation-ready 3D character model.
  • a user need simply describe the 3D character using a simple user interface and select clothing from a number of clothing options.
  • the generation of the mesh, skeleton and skinning weights for the 3D character model is entirely automated.
  • the user can modify the description of the character and automatically regenerate the 3D character until satisfied.
  • the computation complexity of generating an animation-ready 3D character model is sufficiently economical that the 3D character model can be generated in real time.
  • any of a variety of other processes involving a user specifying the characteristics and clothing of a 3D character and then automatically generating a 3D anatomical mesh to which clothing meshes can be applied can be utilized in accordance with embodiments of the invention.
  • a generative model can be constructed using a machine learning technology similar to the machine learning technology described in U.S. Patent Application Serial No. 1 1/881 ,172, incorporated by reference above, and further described by Anguelov et al., " SCAPE: Shape Completion and Animation of People " Proceedings of SIGGRAPH Conference (2005), the disclosure of which is also incorporated by reference herein in its entirety.
  • Principal Component Analysis PCA is used to learn a generative model for a particular type of 3D character.
  • a data set of 3D anatomical meshes can be derived from range scans of real individuals, animals, or sculptures, or from synthetic data. To perform PCA, the 3D anatomical meshes in the data set are registered and placed in correspondence with one of the meshes, which is selected as a template mesh, and a single generative or morphable model is created by computing point to point correspondences between the template mesh and the remaining 3D anatomical meshes in the data set.
  • PCA is applied to vertex positions as described by Blanz et al. " A morphable model for the synthesis of 3D faces " , Proceedings of ACM SIGGRAPH (1999), the disclosure of which is incorporated by reference herein in its entirety.
  • PCA is applied to determine the correspondence between the polygons of the template mesh and the other 3D anatomical meshes in the data set as described by Anguelov et al. in the paper incorporated by reference above.
  • An advantage of using PCA based upon polygon correspondence is that all 3D anatomical meshes generated by the anatomical model possess polygon correspondence, which can simplify the processes associated with applying clothing meshes to the generated 3D anatomical mesh, and applying skinning weights to rig the clothed mesh.
  • PCA can be applied to any of a variety of different manually or automatically identified correspondence characteristics.
  • processes other than PCA can be utilized to create a generative model capable of generating 3D anatomical meshes for a 3D character type.
  • anatomical meshes for 3D characters can be generated based upon a specification of principal components.
  • the principal components can be re-parametrized in terms concepts that are meaningful when describing the anatomical shape of a 3D character.
  • the principal components can be projected onto the axis of a multi-dimensional space describing anatomical shape such as, but not limited to, weight, height, muscularity, and attractiveness.
  • Re-parametrization can leverage both automatic and manual labeling of the 3D anatomical mesh data set by enabling each principal component or groups of principal components to be expressed as a function of a set of labels.
  • these functions are automatically learned based upon the relationship between the labels vector and the PCA components of each 3D anatomical mesh in the training data set.
  • the functions are constrained to be linear and a learning process similar to the process described by Allen et al., " The space of human body shapes: reconstruction and parameterization from range scans " , ACM Transactions on Graphics 22, 3 (2003), 587 - 594, the disclosure of which is incorporated by reference herein in its entirety, is utilized.
  • a layer of non-linear transformation can be applied between the principal components and the labels provided to the user.
  • body mass index can be a function of the semantic components height and weight.
  • any of a number of different functions and/or learning processes can be utilized to re-parameterize the principal components in accordance with the requirements of a specific application.
  • a re-parameterized generative model in accordance with embodiments of the invention enables the generation of a 3D anatomical mesh by specification of a few simple parameters corresponding to well known anatomical characteristics. The parameters used to describe the anatomical characteristics can be referred to as model parameters.
  • the generative model enable generation of 3D anatomical meshes similar to the 3D meshes observed in the example data set, but the generative model can also generate new 3D anatomical meshes derived from the example 3D anatomical meshes, and having model parameters inside the observed range of the principal components (interpolation). In addition, the generative model can generate new 3D anatomical models derived from the observed data, and having model parameters that our outside the observed range of the principal components (extrapolation).
  • FIG. 4 A process for creating a generative model capable of generating a 3D anatomical mesh for a specified 3D character type in accordance with an embodiment of the invention is illustrated in FIG. 4.
  • the process 400 includes labeling (402) example 3D anatomical meshes, performing principal component analysis (404) on the example 3D anatomical meshes, and automatically re-parameterizing (406) the principal components of the generative model in terms of model parameters.
  • the result is a generative model defined by the principal components and a set of functions that map model parameters to the principal components.
  • FIG. 4 alternative processes satisfying the requirements of specific applications can be utilized to create a generative model capable of generating 3D anatomical meshes based upon model parameters in accordance with embodiments of the invention.
  • a 3D anatomical mesh for the particular 3D character type can be created by simply specifying desired model parameters for a 3D character that is sought to be generated.
  • the generated 3D anatomical mesh can be used to drive different clothing mesh templates that can be used to customize the appearance of the generated 3D character.
  • the clothing mesh templates are designed by an animator and, as is discussed further below, include a template skeleton and skinning weights.
  • the generative model is created in such a way that all 3D anatomical meshes generated by it include the same number of corresponding polygons.
  • the UV mapping for the texture of the clothing mesh template does not vary based upon the shape of the generated 3D anatomical model and the clothing mesh template can be applied to all 3D anatomical models generated by the generative model. In this way aspects of the appearance of the clothing including but not limited to the color and pattern of the clothing can be interactively edited in real time with the new textures simply being applied using the same UV mappings defined for the clothing mesh template.
  • FIG. 5 A process for applying a clothing mesh template to a generated 3D anatomical model in accordance with an embodiment of the invention is illustrated in FIG. 5.
  • the process 500 includes creating (502) a clothing mesh template.
  • the clothing mesh template is typically manually created.
  • a generative model is used to generate a 3D anatomical mesh, which an animator then uses as the basis for creating and rigging a clothing mesh template in a way that preserves mesh registration.
  • the clothing mesh template has polygon correspondence with the 3D anatomical meshes generated by the generative model and as a result the Mean Value Coordinates algorithm can be utilized to " drive " the deformation of the clothing mesh template using a generated 3D anatomical mesh.
  • the Mean Value Coordinates algorithm is generally defined by Tao et al. " Mean Value Coordinates for Closed Triangular Meshes " , Proceedings of ACM SIGGRAPH (2005), the disclosure of which is incorporated by reference herein in its entirety.
  • the clothed mesh template deformation is driven by the 3D anatomical mesh in the following way: i) for each vertex "i" of the clothed mesh template, the mean value weight (a vector Wi that express the position of vertex "i” as a weighted average of the vertices of the 3D anatomical mesh) is computed; ii) a matrix W is obtained that gives the position of the clothed mesh template vertices as functions of the position of the vertices of the 3D anatomical mesh; iii) applying the positions of the vertices of a specific generated 3D anatomical mesh to the matrix W can provide the deformed vertices of the clothing mesh template, creating what is referred to as a clothed template.
  • the mean value weight a vector Wi that express the position of vertex "i” as a weighted average of the vertices of the 3D anatomical mesh
  • a pre-existing clothing mesh template is created and morphed into the shape of a generated 3D anatomical mesh by virtue of a non-rigid mesh registration process such as the process described in U.S. Patent Application Serial No. 12/156,132 to Corazza et al. entitled “ Automatic Generation of Human Models for Motion Capture, Biomechanics, and Animation " , and filed May 29, 2008, the disclosure of which is incorporated herein by reference in its entirety.
  • any of a variety of processes can be utilized to apply a clothing mesh template to a 3D anatomical mesh in accordance with an embodiment of the invention including process that involve deformation of the clothing mesh template and/or processes that involve morphing of the clothing mesh template.
  • the applied clothing mesh template can be interactively edited (506) until a desired appearance is achieved.
  • the texture of the clothed mesh can be customized by using an input received via a user interface such as a common HSV color representation to map a new color texture to the clothed mesh.
  • the mapping (508) of textures to the clothed mesh is simplified by enforcing polygon correspondence between the clothing mesh template and the 3D anatomical mesh used to create the clothed mesh.
  • the polygon correspondence between the clothing mesh template and the clothed mesh means that texture can be automatically applied to the clothed mesh using the same UV mappings as were used to apply texture to the clothing mesh model. In this way, new textures can be applied to the clothed mesh in real time interactive manner.
  • the temporal dynamic behavior of the moving clothes can be incorporated into the animation of the 3D character by coupling the kinematics of the clothing mesh template to those of the 3D anatomical mesh after a proper rigging is performed.
  • a number of artifacts that can result from the generation of a 3D character and the application of a clothing mesh template may be corrected.
  • FIG. 6a An example of mesh flattening is illustrated in FIG. 6a.
  • the 3D character 600 includes a Mean Value Coordinate artifact in the form of a flattened armpit 602.
  • processes are applied to remove artifacts resulting from the process of applying a clothing model to a 3D anatomical mesh.
  • a professional animator corrects the artifacts that occur after the clothing mesh is applied to a generated 3D anatomical mesh.
  • a blend shape based approach is then used to determine the deformations applied to remove the artifacts as a function of the model parameters used to generate the 3D anatomical mesh.
  • a blend shape process uses the modifications made by the animator to deform the shape of the mesh resulting from the application of a clothing mesh template to a 3D anatomical mesh.
  • the blend shape process can learn the extent to which blending is applied using a model such as, but not limited to, a linear model.
  • the blend shape process is applied to remove artifacts following the application of the clothing mesh to the 3D anatomical mesh.
  • the results of applying a blend shape process to the mesh of the 3D character 600 shown in FIG. 6a in accordance with an embodiment of the invention is illustrated in FIG. 6b.
  • the application of the blend shape process produces a 3D mesh 600 ' for the character in which the mesh flattening artifact that was present in the armpit region 602 ' of the mesh is removed. Similar processes can be utilized in accordance with embodiments of the invention to remove other artifacts.
  • FIG. 7 A process for removing artifacts following application of a clothing mesh template to a 3D anatomical mesh in accordance with an embodiment of the invention is illustrated FIG. 7.
  • the process 700 includes generating (702) 3D meshes by applying clothing mesh templates to 3D anatomical models generated using a variety of different model parameters, deforming (704) the generated 3D meshes to eliminate artifacts, and performing (706) supervised learning to determine blending shapes and blending weights to apply based upon the model parameters of the 3D anatomical meshes.
  • a new 3D clothed mesh can then be generated (708) by applying a clothing mesh template to a newly generated 3D anatomical model, and performing (710) blend shaping of the new 3D mesh shape based upon the model parameters used to generate the new 3D anatomical model.
  • blend shaping other processes suitable for specific applications can be utilized that eliminate artifacts in a 3D mesh resulting from application of a clothing mesh template to a 3D anatomical mesh in accordance with embodiments of the invention.
  • Adaptation of skeleton to clothed mesh In order to make a 3D character mesh created by applying a clothing mesh template to a 3D anatomical mesh animation-ready, the 3D character mesh is rigged by creating a skeleton for the 3D character mesh (i.e. the clothed mesh) and defining skinning weights.
  • a template skeleton is generated for each clothing mesh template and a process is applied to the skeleton to move the skeleton ' s joint centers based upon the shape of the clothed mesh generated by applying the clothing mesh template to a 3D anatomical mesh.
  • the adjustment of the template skeleton to rig the clothed mesh is performed using the Mean Value Coordinates algorithm.
  • Mean value weights are computed for each joint of the skeleton, which are driven by the anatomical mesh deformation.
  • the skeleton joint position can be expressed as a linear function of semantic values (e.g. user ' s defined parameters).
  • the process 800 includes defining a template skeleton 802 for each clothing mesh model and adjusting the appropriate template skeleton based upon the shape of a clothed mesh created by generating a 3D anatomical mesh using the generative model and applying a clothing mesh.
  • the adjustments are performed using Mean Value Coordinates algorithm to develop mean value weights that are driven by the deformation of the clothing mesh template.
  • the process of generating the mesh for a 3D character model is constrained so that the same number of polygons is used to create the mesh of the 3D character irrespective of the shape of the 3D character. Therefore, skinning weights assigned for the clothed mesh of one 3D character can be applied to the clothed meshes of other 3D characters generated using the generative model.
  • an animator manually defines the skinning weights for a clothing mesh template for 3D characters generated using the generative model and then the same skinning weights are used to rig the clothed meshes of all 3D characters created using the generative model.
  • FIG. 9 A process for assigning skinning weights to the mesh of a 3D character model generated in accordance with an embodiment of the invention is illustrated in FIG. 9.
  • the process 900 includes manually defining (902) skinning weights for a clothing mesh template for 3D anatomical meshes generated using a generative model. Ideally, the skinning weights are defined with respect to the 3D clothed mesh template. However, the skinning weights can be defined with respect to the base 3D anatomical mesh generated by the generative model and transferred over to the clothed mesh template.
  • the character can be rigged by applying the defined skinning weights to the clothed mesh of the new 3D character.
  • Animation-ready 3D character models in accordance with the processes outlined above can be performed in real time and an interactive user interface used iteratively to display the 3D character and to receive inputs from the user for the generation of a new animation-ready 3D character model based upon modifications to the model parameters defining the previous animation-ready 3D character model.
  • the animation-ready 3D character model can be animated using motion data and/or downloaded or stored for use in a variety of formats.
  • FIG. 10 A workflow for generating an animation ready 3D character in accordance with an embodiment of the invention is illustrated in FIG. 10. As is illustrated in the workflow 1000, the user provides (1002) body shape or model parameters that enable the generation of a 3D anatomical mesh.
  • the 3D anatomical mesh is generated (1004) based upon a model created using a database 1006 representative of a continuous space of 3D character shapes.
  • the database also includes previously defined clothing mesh templates 1008 that include a clothing mesh and skinning weights defined with respect to a template skeleton.
  • the workflow adjusts (1012) the stored template skeleton for the selected clothes based upon the shape of the 3D anatomical mesh generated by the generative model.
  • the appropriate clothing mesh template corresponding to the selected clothing is applied to the 3D anatomical mesh, which is rigged (1014) using the adjusted template skeleton.
  • the template skeleton skinning weights are then applied (1016) to the rigged mesh to complete the animation-ready 3D character 1018.
  • the process can iterate by modifying the body shape parameters and clothing parameters until the 3D character is satisfactory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

On décrit des systèmes et des procédés destinés à générer automatiquement des modèles de personnages 3D prêts pour l'animation basés sur des paramètres de modèle et des vêtements, les procédés consistant à recevoir les paramètres de modèle définis par l'utilisateur et le choix de vêtements via une interface d'utilisateur. Les systèmes et procédés selon l'invention peuvent consister à générer un maillage anatomique 3D basé sur les paramètres de modèle définis par l'utilisateur en utilisant le modèle générateur, à appliquer le maillage des vêtements issu d'un gabarit de maillage des vêtements correspondant au choix de vêtements de l'utilisateur au maillage anatomique 3D généré pour créer un maillage habillé, à ajuster le squelette du gabarit de maillage des vêtements correspondant au choix de vêtements de l'utilisateur sur la base de la forme du maillage habillé, à générer des pondérations de formation de peau sur la base des pondérations de formation de peau du gabarit de maillage des vêtements correspondant au choix de vêtements de l'utilisateur, et à mémoriser un modèle de personnage 3D prêt pour l'animation comprenant le maillage habillé, le squelette ajusté et les pondérations de formation de peau générées.
PCT/US2009/065825 2008-11-24 2009-11-24 Génération en temps réel de modèles de personnages 3d prêts pour l' animation WO2010060113A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US20021208P 2008-11-24 2008-11-24
US61/200,212 2008-11-24
US58411709A 2009-08-31 2009-08-31
US12/584,117 2009-08-31

Publications (1)

Publication Number Publication Date
WO2010060113A1 true WO2010060113A1 (fr) 2010-05-27

Family

ID=42198557

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2009/065825 WO2010060113A1 (fr) 2008-11-24 2009-11-24 Génération en temps réel de modèles de personnages 3d prêts pour l' animation

Country Status (1)

Country Link
WO (1) WO2010060113A1 (fr)

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103310478A (zh) * 2013-06-04 2013-09-18 合肥工业大学 一种多样化虚拟人群生成的方法
EP2692399A1 (fr) * 2012-08-01 2014-02-05 Square Enix Co., Ltd. Dispositif d'affichage d'objet
US8659596B2 (en) 2008-11-24 2014-02-25 Mixamo, Inc. Real time generation of animation-ready 3D character models
US8704832B2 (en) 2008-09-20 2014-04-22 Mixamo, Inc. Interactive design, synthesis and delivery of 3D character motion data through the web
US8749556B2 (en) 2008-10-14 2014-06-10 Mixamo, Inc. Data compression for real-time streaming of deformable 3D models for 3D animation
US8797328B2 (en) 2010-07-23 2014-08-05 Mixamo, Inc. Automatic generation of 3D character animation from 3D meshes
US8928672B2 (en) 2010-04-28 2015-01-06 Mixamo, Inc. Real-time automatic concatenation of 3D animation sequences
US8982122B2 (en) 2008-11-24 2015-03-17 Mixamo, Inc. Real time concurrent design of shape, texture, and motion for 3D character animation
US9619914B2 (en) 2009-02-12 2017-04-11 Facebook, Inc. Web platform for interactive design, synthesis and delivery of 3D character motion data
US9626788B2 (en) 2012-03-06 2017-04-18 Adobe Systems Incorporated Systems and methods for creating animations using human faces
CN107067460A (zh) * 2016-01-07 2017-08-18 广东京腾科技有限公司 一种虚拟试衣方法、装置及系统
US9786084B1 (en) 2016-06-23 2017-10-10 LoomAi, Inc. Systems and methods for generating computer ready animation models of a human head from captured data images
US9984409B2 (en) 2014-12-22 2018-05-29 Ebay Inc. Systems and methods for generating virtual contexts
US10049482B2 (en) 2011-07-22 2018-08-14 Adobe Systems Incorporated Systems and methods for animation recommendations
US10068371B2 (en) 2013-11-14 2018-09-04 Ebay Inc. Extraction of body dimensions from planar garment photographs of fitting garments
US10198845B1 (en) 2018-05-29 2019-02-05 LoomAi, Inc. Methods and systems for animating facial expressions
US10204375B2 (en) 2014-12-01 2019-02-12 Ebay Inc. Digital wardrobe using simulated forces on garment models
US10310616B2 (en) 2015-03-31 2019-06-04 Ebay Inc. Modification of three-dimensional garments using gestures
US10366439B2 (en) 2013-12-27 2019-07-30 Ebay Inc. Regional item reccomendations
US10475113B2 (en) 2014-12-23 2019-11-12 Ebay Inc. Method system and medium for generating virtual contexts from three dimensional models
US10559111B2 (en) 2016-06-23 2020-02-11 LoomAi, Inc. Systems and methods for generating computer ready animation models of a human head from captured data images
US10748325B2 (en) 2011-11-17 2020-08-18 Adobe Inc. System and method for automatic rigging of three dimensional characters for facial animation
US11055758B2 (en) 2014-09-30 2021-07-06 Ebay Inc. Garment size mapping
US11100054B2 (en) 2018-10-09 2021-08-24 Ebay Inc. Digital image suitability determination to generate AR/VR digital content
CN115049769A (zh) * 2022-08-17 2022-09-13 深圳泽森软件技术有限责任公司 角色动画生成方法、装置、计算机设备和存储介质
US11551393B2 (en) 2019-07-23 2023-01-10 LoomAi, Inc. Systems and methods for animation generation

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040049309A1 (en) * 2001-01-19 2004-03-11 Gardner James Holden Patrick Production and visualisation of garments
US20040227752A1 (en) * 2003-05-12 2004-11-18 Mccartha Bland Apparatus, system, and method for generating a three-dimensional model to represent a user for fitting garments
US20050264572A1 (en) * 2004-03-05 2005-12-01 Anast John M Virtual prototyping system and method
US20060002631A1 (en) * 2004-06-30 2006-01-05 Accuray, Inc. ROI selection in image registration
US20070182736A1 (en) * 1999-06-11 2007-08-09 Weaver Christopher S Method and system for a computer-rendered three-dimensional mannequin
EP1884896A2 (fr) * 2006-07-31 2008-02-06 Avid Technology, Inc. Conversion de données de déformation de maillage vers des données d'animation pour un squelette, dénudage et ombrage dans un moteur d'animation de graphiques informatiques
US20080043021A1 (en) * 2006-08-15 2008-02-21 Microsoft Corporation Three Dimensional Polygon Mesh Deformation Using Subspace Energy Projection

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070182736A1 (en) * 1999-06-11 2007-08-09 Weaver Christopher S Method and system for a computer-rendered three-dimensional mannequin
US20040049309A1 (en) * 2001-01-19 2004-03-11 Gardner James Holden Patrick Production and visualisation of garments
US20040227752A1 (en) * 2003-05-12 2004-11-18 Mccartha Bland Apparatus, system, and method for generating a three-dimensional model to represent a user for fitting garments
US20050264572A1 (en) * 2004-03-05 2005-12-01 Anast John M Virtual prototyping system and method
US20060002631A1 (en) * 2004-06-30 2006-01-05 Accuray, Inc. ROI selection in image registration
EP1884896A2 (fr) * 2006-07-31 2008-02-06 Avid Technology, Inc. Conversion de données de déformation de maillage vers des données d'animation pour un squelette, dénudage et ombrage dans un moteur d'animation de graphiques informatiques
US20080043021A1 (en) * 2006-08-15 2008-02-21 Microsoft Corporation Three Dimensional Polygon Mesh Deformation Using Subspace Energy Projection

Cited By (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9373185B2 (en) 2008-09-20 2016-06-21 Adobe Systems Incorporated Interactive design, synthesis and delivery of 3D motion data through the web
US8704832B2 (en) 2008-09-20 2014-04-22 Mixamo, Inc. Interactive design, synthesis and delivery of 3D character motion data through the web
US9460539B2 (en) 2008-10-14 2016-10-04 Adobe Systems Incorporated Data compression for real-time streaming of deformable 3D models for 3D animation
US8749556B2 (en) 2008-10-14 2014-06-10 Mixamo, Inc. Data compression for real-time streaming of deformable 3D models for 3D animation
US8659596B2 (en) 2008-11-24 2014-02-25 Mixamo, Inc. Real time generation of animation-ready 3D character models
US9305387B2 (en) 2008-11-24 2016-04-05 Adobe Systems Incorporated Real time generation of animation-ready 3D character models
US9978175B2 (en) 2008-11-24 2018-05-22 Adobe Systems Incorporated Real time concurrent design of shape, texture, and motion for 3D character animation
US8982122B2 (en) 2008-11-24 2015-03-17 Mixamo, Inc. Real time concurrent design of shape, texture, and motion for 3D character animation
US9619914B2 (en) 2009-02-12 2017-04-11 Facebook, Inc. Web platform for interactive design, synthesis and delivery of 3D character motion data
US8928672B2 (en) 2010-04-28 2015-01-06 Mixamo, Inc. Real-time automatic concatenation of 3D animation sequences
US8797328B2 (en) 2010-07-23 2014-08-05 Mixamo, Inc. Automatic generation of 3D character animation from 3D meshes
US10565768B2 (en) 2011-07-22 2020-02-18 Adobe Inc. Generating smooth animation sequences
US10049482B2 (en) 2011-07-22 2018-08-14 Adobe Systems Incorporated Systems and methods for animation recommendations
US10748325B2 (en) 2011-11-17 2020-08-18 Adobe Inc. System and method for automatic rigging of three dimensional characters for facial animation
US11170558B2 (en) 2011-11-17 2021-11-09 Adobe Inc. Automatic rigging of three dimensional characters for animation
US9626788B2 (en) 2012-03-06 2017-04-18 Adobe Systems Incorporated Systems and methods for creating animations using human faces
US9747495B2 (en) 2012-03-06 2017-08-29 Adobe Systems Incorporated Systems and methods for creating and distributing modifiable animated video messages
US9675887B2 (en) 2012-08-01 2017-06-13 Kabushiki Kaisha Square Enix Object display device
EP2692399A1 (fr) * 2012-08-01 2014-02-05 Square Enix Co., Ltd. Dispositif d'affichage d'objet
CN103310478A (zh) * 2013-06-04 2013-09-18 合肥工业大学 一种多样化虚拟人群生成的方法
US11145118B2 (en) 2013-11-14 2021-10-12 Ebay Inc. Extraction of body dimensions from planar garment photographs of fitting garments
US10068371B2 (en) 2013-11-14 2018-09-04 Ebay Inc. Extraction of body dimensions from planar garment photographs of fitting garments
US10410414B2 (en) 2013-11-14 2019-09-10 Ebay Inc. Extraction of body dimensions from planar garment photographs of fitting garments
US11100564B2 (en) 2013-12-27 2021-08-24 Ebay Inc. Regional item recommendations
US10366439B2 (en) 2013-12-27 2019-07-30 Ebay Inc. Regional item reccomendations
US11055758B2 (en) 2014-09-30 2021-07-06 Ebay Inc. Garment size mapping
US11734740B2 (en) 2014-09-30 2023-08-22 Ebay Inc. Garment size mapping
US11599937B2 (en) 2014-12-01 2023-03-07 Ebay Inc. Digital wardrobe
US10204375B2 (en) 2014-12-01 2019-02-12 Ebay Inc. Digital wardrobe using simulated forces on garment models
US12125095B2 (en) 2014-12-01 2024-10-22 Ebay Inc. Digital wardrobe
US10977721B2 (en) 2014-12-01 2021-04-13 Ebay Inc. Digital wardrobe
US9984409B2 (en) 2014-12-22 2018-05-29 Ebay Inc. Systems and methods for generating virtual contexts
US10475113B2 (en) 2014-12-23 2019-11-12 Ebay Inc. Method system and medium for generating virtual contexts from three dimensional models
US11270373B2 (en) 2014-12-23 2022-03-08 Ebay Inc. Method system and medium for generating virtual contexts from three dimensional models
US11662829B2 (en) 2015-03-31 2023-05-30 Ebay Inc. Modification of three-dimensional garments using gestures
US11073915B2 (en) 2015-03-31 2021-07-27 Ebay Inc. Modification of three-dimensional garments using gestures
US10310616B2 (en) 2015-03-31 2019-06-04 Ebay Inc. Modification of three-dimensional garments using gestures
CN107067460A (zh) * 2016-01-07 2017-08-18 广东京腾科技有限公司 一种虚拟试衣方法、装置及系统
US9786084B1 (en) 2016-06-23 2017-10-10 LoomAi, Inc. Systems and methods for generating computer ready animation models of a human head from captured data images
US10062198B2 (en) 2016-06-23 2018-08-28 LoomAi, Inc. Systems and methods for generating computer ready animation models of a human head from captured data images
US10169905B2 (en) 2016-06-23 2019-01-01 LoomAi, Inc. Systems and methods for animating models from audio data
US10559111B2 (en) 2016-06-23 2020-02-11 LoomAi, Inc. Systems and methods for generating computer ready animation models of a human head from captured data images
US10198845B1 (en) 2018-05-29 2019-02-05 LoomAi, Inc. Methods and systems for animating facial expressions
US11100054B2 (en) 2018-10-09 2021-08-24 Ebay Inc. Digital image suitability determination to generate AR/VR digital content
US11487712B2 (en) 2018-10-09 2022-11-01 Ebay Inc. Digital image suitability determination to generate AR/VR digital content
US11551393B2 (en) 2019-07-23 2023-01-10 LoomAi, Inc. Systems and methods for animation generation
CN115049769A (zh) * 2022-08-17 2022-09-13 深圳泽森软件技术有限责任公司 角色动画生成方法、装置、计算机设备和存储介质

Similar Documents

Publication Publication Date Title
US9305387B2 (en) Real time generation of animation-ready 3D character models
US9978175B2 (en) Real time concurrent design of shape, texture, and motion for 3D character animation
WO2010060113A1 (fr) Génération en temps réel de modèles de personnages 3d prêts pour l' animation
US8797328B2 (en) Automatic generation of 3D character animation from 3D meshes
Hornung et al. Character animation from 2d pictures and 3d motion data
US6822653B2 (en) Methods and system for general skinning via hardware accelerators
CA2565600C (fr) Mappage dynamique de rides
US20080204457A1 (en) Rig Baking
EP1986158B1 (fr) Décoration d'un caractère généré par ordinateur avec des caractéristiques liées à la surface
Magnenat-Thalmann et al. Modeling of bodies and clothes for virtual environments
JP2023540434A (ja) 操作順序変形エンジンを用いたコンピュータアニメーションのためのシステム及び方法
EP1986157B1 (fr) Mise en place de caractéristiques liées à la peau sur un caractère généré par ordinateur
US20240331330A1 (en) System and Method for Dynamically Improving the Performance of Real-Time Rendering Systems via an Optimized Data Set
Orvalho et al. Transferring the rig and animations from a character to different face models
Tejera et al. Space-time editing of 3d video sequences
Di Giacomo et al. Towards Automatic Character Skeletonization and Interactive Skin Deformation.
Di Giacomo et al. Control structure and multi-resolution techniques for virtual human representation
WO2004104934A1 (fr) Procede et appareil de collisions dynamiques statistiques
Karunaratne et al. A new efficient expression generation and automatic cloning method for multimedia actors
WO2001037216A2 (fr) Systeme informatique de creation graphique tridimensionnel et intelligent, et procede correspondant
CN113658305A (zh) 三维模型的蒙皮方法、装置、设备及计算机可读存储介质
Maddock An investigation of the modelling-animation relationship in computer graphics
Orvalho A System to Reuse Facial Rigs and Animations

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09828399

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09828399

Country of ref document: EP

Kind code of ref document: A1