WO2021036665A1 - 一种基于人工智能的动画形象驱动方法和装置 - Google Patents

一种基于人工智能的动画形象驱动方法和装置 Download PDF

Info

Publication number
WO2021036665A1
WO2021036665A1 PCT/CN2020/105673 CN2020105673W WO2021036665A1 WO 2021036665 A1 WO2021036665 A1 WO 2021036665A1 CN 2020105673 W CN2020105673 W CN 2020105673W WO 2021036665 A1 WO2021036665 A1 WO 2021036665A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
expression
base
grid
parameter
Prior art date
Application number
PCT/CN2020/105673
Other languages
English (en)
French (fr)
Inventor
王盛
季兴
朱展图
林祥凯
暴林超
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP20857359.2A priority Critical patent/EP3923244A4/en
Priority to KR1020217029446A priority patent/KR102645506B1/ko
Priority to JP2021560989A priority patent/JP7307194B2/ja
Publication of WO2021036665A1 publication Critical patent/WO2021036665A1/zh
Priority to US17/486,641 priority patent/US11941737B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/80—2D [Two Dimensional] animation, e.g. using sprites
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/176—Dynamic expression
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2016—Rotation, translation, scaling

Definitions

  • This application relates to the field of data processing, in particular to the driving technology of animated images based on artificial intelligence.
  • the expression base is composed of deformable networks representing different expressions. Each deformable network is formed by changing the 3D model of the animated image under different expressions.
  • the pinching face base is composed of deformable networks representing different face shapes. The deformable networks are all faces with relatively large changes in the average face shape, and the faces in the pinched face base need to be related to the animated image.
  • the designer can manually design the 3D grid of the animation image. Since the expression base has strict semantic information, such as controlling closed eyes and mouth, etc., the expression base can be easily hand-crafted by the designer. Modeling is obtained. However, the basic body of the pinched face is obtained by decomposing a large amount of face data through principal component analysis, without clear semantic information, and inaccurate association with the mesh vertices of the model, which is difficult to obtain through manual design. As a result, the corresponding pinching base can only be determined through complex data processing, which delays the introduction of new animated images.
  • this application provides an artificial intelligence-based method and device for driving an animated image, which can directly drive a new animated image through the known expression parameters of other animated images, without processing to obtain the pinching base corresponding to the new animated image. , To speed up the introduction of new animated images.
  • an embodiment of the present application provides an artificial intelligence-based animation image driving method, which is executed by a processing device, and the method includes:
  • an embodiment of the present application provides an artificial intelligence-based animation image driving device, the device including an acquiring unit, a determining unit, and a driving unit:
  • the acquiring unit is configured to acquire an expression base corresponding to the image of the driver and an expression base corresponding to the image of the driven party, the image of the driver has a corresponding base of pinching faces, and the image of the driven party does not have a corresponding base of pinching faces ;
  • the determining unit is configured to determine the expression parameters corresponding to the driver image and the expression parameters corresponding to the driven party image according to the expression base corresponding to the driver image and the expression base corresponding to the driven party image Mapping relationship between;
  • the driving unit is configured to drive the driven party image according to the expression parameters corresponding to the driver's image and the mapping relationship.
  • an embodiment of the present application provides an artificial intelligence-based animation image driving method, which is executed by a processing device, and the method includes:
  • the image of the driving party has a corresponding structural base, and the image of the driven party does not have a corresponding structural base;
  • the structural base is used to identify the The structural feature of the corresponding image, and the deformation base is used to identify the deformation feature of the corresponding image;
  • an embodiment of the present application provides an artificial intelligence-based animation image driving device, the device including an acquiring unit, a determining unit, and a driving unit:
  • the acquiring unit is configured to acquire a deformation base corresponding to the image of the driving party and a deformation base corresponding to the image of the driven party, the image of the driving party has a corresponding structural base, and the image of the driven party does not have a corresponding structural base;
  • the structure base is used to identify the structural feature of the corresponding image, and the deformation base is used to identify the deformation feature of the corresponding image;
  • the determining unit is configured to determine the deformation parameter corresponding to the driver image and the deformation parameter corresponding to the driven image according to the deformation base corresponding to the driver image and the deformation base corresponding to the driven image Mapping relationship between;
  • the driving unit is configured to drive the driven party image according to the deformation parameter corresponding to the driving party image and the mapping relationship.
  • an embodiment of the present application provides a device, which includes a processor and a memory:
  • the memory is used to store program code and transmit the program code to the processor
  • the processor is configured to execute the method described in the first aspect or the third aspect according to instructions in the program code.
  • an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in the first aspect or the third aspect.
  • embodiments of the present application provide a computer program product, including instructions, which when run on a computer, cause the computer to execute the method described in the first aspect or the third aspect.
  • the animated image with the pinching base can be used as the driving party image, without the pinching base.
  • the animated image as the driven party image. Since the expression parameters corresponding to the image of the driver and the expression parameters corresponding to the image of the driven party should have a mapping relationship, after the mapping relationship is determined, the expression parameters corresponding to the image of the driver can be used to directly drive the image of the driven party, even if it is driven The square image does not have a pinch base.
  • the actual expression parameters corresponding to the image of the driver can drive the image of the driven party to make actual expressions
  • the actual expression parameters can reflect the degree of correlation between the actual expression and its expression base in different dimensions, that is, the actual expression parameters corresponding to the image of the driven party It can also reflect the degree of correlation between the actual expression of the image of the driven party and its expression base in different dimensions. Therefore, based on the relationship between the above expression parameters and the expression base, it can be based on the expression base corresponding to the image of the driver and the image corresponding to the driven party.
  • the expression base determines the mapping relationship between the expression parameters.
  • the driver image is used to drive the image of the driven party using the known expression parameters of the driver's image and the foregoing mapping relationship, so that the image of the driven party can make the known image
  • the actual expression identified by the expression parameter Therefore, without processing to obtain the pinch face base corresponding to the new animation image, the new animation image can be directly driven by the known expression parameters of other animation images, which speeds up the launch of the new animation image.
  • FIG. 1 is a schematic diagram of an application scenario of an artificial intelligence-based animation image driving method provided by an embodiment of the application
  • FIG. 2 is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application
  • FIG. 3 is an example diagram of the distribution and meaning of each dimension of the 3DMM library M provided by an embodiment of the application;
  • FIG. 4 is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • FIG. 5 is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • FIG. 6 is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • FIG. 7 is an effect diagram of a driving animation image provided by an embodiment of the application.
  • FIG. 8 is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • FIG. 9 is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • FIG. 10 is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • FIG. 11 is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • FIG. 12a is a flowchart of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • FIG. 12b is a structural diagram of an artificial intelligence-based animation image driving device provided by an embodiment of the application.
  • Fig. 13a is a structural diagram of an artificial intelligence-based animation image driving device provided by an embodiment of the application.
  • FIG. 13b is a structural diagram of a terminal device provided by an embodiment of this application.
  • FIG. 14 is a structural diagram of a server provided by an embodiment of the application.
  • the related technology can usually manually design the 3D grid of the animation image by the designer.
  • the designer can obtain the expression base through manual modeling.
  • the face pinch base has no clear semantic information and is not accurately associated with the mesh vertices of the model, it is difficult to obtain it through manual design.
  • the animated image with a pinch face base may be used under a specific expression base. It’s generally difficult to directly drive the expression base of another animated image. Therefore, in order to drive the expression base of another animated image and launch a new animated image, the corresponding pinching base can only be determined through complex data processing, which delays the introduction of the new animated image.
  • the embodiment of the present application provides an artificial intelligence-based animation image driving method, which can realize the transfer of expression parameters between the expression bases of different animation images, that is, when an animation image has a corresponding pinching base, an animation
  • the animated image with the pinching base is used as the driver image
  • the animated image without the pinching base is used as the driven party image
  • the expression parameters and driven party corresponding to the driver image are determined.
  • the mapping relationship between the facial expression parameters corresponding to the image does not need to be processed to obtain the pinch face base corresponding to the image of the driven party, and the image of the driven party can be driven by using the known facial expression parameters of the driver image and the foregoing mapping relationship.
  • the artificial intelligence-based animation image driving method provided by the embodiment of the present application can be applied to a processing device with an animation image creation capability, and the processing device can be a terminal device or a server.
  • the processing device may have the ability to implement computer vision technology.
  • the processing device implements the above-mentioned computer vision technology to determine the mapping relationship between the expression parameters according to the expression base corresponding to the image of the driver and the expression base corresponding to the image of the driven party, so that the mapping relationship between the expression parameters can be determined according to the mapping. Relationship, using the known expression parameters of the driver's image and the aforementioned mapping relationship to drive the driven party's image to achieve functions such as the rapid introduction of new animated images.
  • the processing device is a terminal device
  • the terminal device may be a smart terminal, a computer, a personal digital assistant (Personal Digital Assistant, PDA for short), a tablet computer, and the like.
  • PDA Personal Digital Assistant
  • the server may be an independent server or a cluster server.
  • the server can determine the mapping relationship between the expression parameters according to the expression base corresponding to the driver image and the expression base corresponding to the driven image, using the known image of the driver.
  • the expression parameters and the aforementioned mapping relationship drive the image of the driven party to obtain a new animation image, and display and launch the new animation image on the terminal device.
  • the animated image in the embodiment of the present application may include 2D animated image and 3D animated image; the animated image may specifically present only the face area or the head area, or may present the whole body area; in addition, the animated image specifically It can be expressed as a cartoon image, or as a virtual image constructed based on real characters or animals; this application does not limit the expression form of the animated image in any way.
  • the artificial intelligence-based animation image driving method provided in the embodiments of this application can be applied to application scenarios that need to drive hand-designed animation images, such as news broadcasts, weather forecasts, game commentary, and virtual game characters in game scenes.
  • the method provided in the embodiments of the present application can drive the animated image without processing to obtain the pinch face base corresponding to the animated image.
  • FIG. 1 is a schematic diagram of an application scenario of an artificial intelligence-based animation image driving method provided by an embodiment of the application.
  • This application scenario is introduced by taking the processing device as a terminal device as an example.
  • the application scenario includes the terminal device 101, and the terminal device 101 can obtain the expression base corresponding to the image of the driver and the expression base corresponding to the image of the driven party.
  • the driver image refers to an animated image with an expression base and a face pinching base
  • the driven party image refers to an animated image with an expression base but not a face pinching base.
  • the expression base is used to identify the facial expression features of the animated image. It is composed of deformable networks representing different expressions. Each deformable network is formed by changing the 3D model of the animated image under different expressions. .
  • the pinched face base is used to identify the basic features of the face of the animated image. It is composed of deformable networks representing different face shapes. Each deformable network is a face with a relatively large change in the average face shape. The face in the pinched face base needs to be in accordance with the animation. Image related.
  • an expression form of the expression parameter may be a coefficient, for example, a vector having a certain dimension; the mapping relationship may be a linear mapping relationship or a non-linear mapping relationship, which is not limited in this embodiment.
  • the actual expression parameters corresponding to the image of the driver can drive the image of the driven party to make actual expressions
  • the actual expression parameters can reflect the degree of correlation between the actual expression and its expression base in different dimensions.
  • the actual image corresponding to the image of the driven party The expression parameters can also reflect the degree of correlation between the actual expression of the driven party's image and its expression base in different dimensions. Therefore, based on the above-mentioned correlation between the expression parameters and the expression base, the terminal device 101 can be based on the expression base and the corresponding expression base of the driver's image.
  • the expression base corresponding to the image of the driver determines the mapping relationship between the expression parameters corresponding to the image of the driver and the expression parameters corresponding to the image of the driven party.
  • the terminal device 101 can calculate the expression parameters corresponding to the driven party image according to the mapping relationship and the expression parameters corresponding to the driver image, the calculated expression parameters of the driven party image are the same as the expression base dimension of the driven party image, Thus, the calculated expression parameters of the driven party image can be used to drive the driven party image to make an expression. Therefore, the terminal device 101 can directly drive the image of the driven party according to the expression parameters corresponding to the image of the driver and the mapping relationship.
  • the application scenario shown in FIG. 1 is only an example.
  • the artificial intelligence-based animation image driving method provided by the embodiment of the application can also be applied to other application scenarios. There are no restrictions on the applicable application scenarios of the artificial intelligence-based animation image driving method.
  • the method includes:
  • S201 Obtain an expression base corresponding to the image of the driver and an expression base corresponding to the image of the driven party.
  • some animated images are already launched animated images with an expression base and a face pinching base, while some animated images are new animated images that only have an expression base but not a face pinching base.
  • the driver's avatar is an animated avatar with a corresponding pinching base
  • the driven party's avatar is an animated avatar without a corresponding pinching base.
  • the animated image in the embodiment of the present application may be a model in a model library, or may be obtained through a linear combination of models in the model library.
  • the model library may be a human face 3D deformable model (3DMM) library or other model libraries, which is not limited in this implementation.
  • the animated image, such as the driving party image and the driven party image may be a 3D grid.
  • the 3DMM library is obtained from a large amount of high-precision face data through the Principal Component Analysis (PCA) method, which describes the main changes of high-dimensional face shape and expression relative to the average face, and can also describe texture information.
  • PCA Principal Component Analysis
  • the 3DMM library when the 3DMM library describes an expressionless face, it can be obtained by mu+ ⁇ (Pface i -mu)* ⁇ i .
  • mu is the average face under neutral expression
  • Pface i is the i-th face principal component component
  • ⁇ i is the weight of the i-th face principal component component, which is the face pinch parameter.
  • the grid corresponding to the animation image in the 3DMM library can be represented by M, that is, the relationship between the face shape, expression and vertices in the 3DMM library is represented by M, and M is a three-dimensional matrix of [m ⁇ n ⁇ d], where each One dimension is the vertex coordinates of the grid (m), the principal component of face shape (n), and the principal component of expression (d).
  • M is a three-dimensional matrix of [m ⁇ n ⁇ d], where each One dimension is the vertex coordinates of the grid (m), the principal component of face shape (n), and the principal component of expression (d).
  • the distribution and meaning of each dimension of the 3DMM library M is shown in Figure 3. Since m represents the value of the three coordinates of xyz, the number of vertices of the mesh is m/3, which is denoted as v. If the face shape or expression of the animated image is determined, then M can be a two-dimensional matrix.
  • the texture dimension in the 3DMM library is not considered, and assuming that the driving of the animated image is F, F can be determined by formula (1):
  • M is the grid of the animated image
  • ⁇ is the face pinching parameter
  • ⁇ is the expression parameter
  • d is the number of expression grids in the expression base
  • n is the number of face pinching grids in the face pinching base
  • M k, j,i is the k-th grid with the i-th expression grid and the j-th face pinching grid
  • ⁇ j is the j-th dimension in a set of face pinching parameters, which represents the weight of the j-th face principal component component
  • ⁇ i is the i-th dimension in a set of expression parameters, and represents the weight of the i-th expression principal component.
  • the process of determining the face pinching parameters is the face pinching algorithm
  • the process of determining the expression parameters is the face pinching algorithm.
  • the pinching parameters are used to make a linear combination with the pinching base to obtain the corresponding face shape.
  • a pinching base including 50 pinching grids (belonging to deformable grids, such as blendshape), and the pinching face corresponding to the pinching base
  • the parameter is a 50-dimensional vector, and each dimension can identify the degree of correlation between the face shape corresponding to the face pinching parameter and a face pinching grid.
  • the face pinching grids included in the face pinching base represent different face shapes, and each face pinching grid is a face image with a relatively large change from the average face, and is the main component of different dimensions of face shape obtained after a large number of faces are decomposed by PCA , And the number of vertices corresponding to different face-pinching meshes in the same face-pinching base remains the same.
  • the expression parameters are used to linearly combine with the expression base to obtain the corresponding expression.
  • an expression base including 50 (equivalent to 50) expression grids (belonging to deformable grids, such as blendshape), which corresponds to
  • the expression parameter of is a 50-dimensional vector, and each dimension can identify the degree of correlation between the expression corresponding to the expression parameter and an expression grid.
  • the expression grids included in the expression base represent different expressions. Each expression grid is formed by changing the same 3D model under different expressions. The number of vertices corresponding to different expression grids in the same expression base remains the same.
  • a single grid can be deformed through a predefined shape to obtain any number of grids.
  • S202 Determine the mapping relationship between the expression parameters corresponding to the driver image and the expression parameters corresponding to the driven party image according to the expression base corresponding to the image of the driver and the expression base corresponding to the image of the driven party.
  • the pinch face base remains unchanged, that is, the face shape is fixed, and only the expression base needs to be adjusted.
  • M k is the driving of the animated image of a fixed face
  • M k,i is the i-th expression grid
  • ⁇ i is the expression parameter corresponding to the i-th expression grid
  • n is the number of expression grids in the expression base .
  • the animated image a has an expression base and a pinch face base, and the animated image b has an expression base but not a face pinch base, the animated image a has already obtained some expression parameters through the pinch expression algorithm.
  • the animated image a can be used as a driver
  • the animated image b can be used as the driven party image.
  • the driving of the animated image a can be obtained:
  • n a is the number of expression grids included in the expression base of the animated image a.
  • n b is the number of expression grids in the expression base of the animated image b.
  • mapping relationship between the expression parameters corresponding to the image of the driver and the expression parameters corresponding to the image of the driven party can be expressed by the function f(), then
  • the formula (5) for calculating the expression parameters corresponding to the driven party image through the expression parameters corresponding to the driver's image is as follows:
  • ⁇ b is the expression parameter corresponding to the image of the driven party
  • ⁇ a is the expression parameter corresponding to the image of the driver
  • f() represents the mapping relationship between the expression parameters corresponding to the image of the driver and the expression parameters corresponding to the image of the driven party.
  • the expression parameters corresponding to the driver's image can be used to directly drive the driven party's image (animated image b).
  • the actual expression parameters corresponding to the image of the driver can drive the image of the driven party to make actual expressions
  • the actual expression parameters can reflect the degree of correlation between the actual expression and its expression base in different dimensions, that is, the actual expression parameters corresponding to the image of the driven party It can also reflect the degree of correlation between the actual expression of the image of the driven party and its expression base in different dimensions. Therefore, based on the relationship between the above expression parameters and the expression base, it can be based on the expression base corresponding to the image of the driver and the image corresponding to the driven party.
  • the expression base determines the mapping relationship between expression parameters.
  • S203 Drive the image of the driven party according to the expression parameters and the mapping relationship corresponding to the image of the driver.
  • ⁇ b is the expression parameter corresponding to the image of the driven party
  • ⁇ a is the expression parameter corresponding to the image of the driver
  • f represents the mapping relationship between the expression parameter corresponding to the image of the driver and the expression parameter corresponding to the image of the driven party.
  • the expression parameters ⁇ b of the animation image b can be calculated in combination with formula (6).
  • the dimension of ⁇ a is different from the expression base dimension of the animated image b, and the dimension of ⁇ b obtained through the mapping relationship is the same as the expression base dimension of the animated image b, which can drive the animated image b to make an expression, so use ⁇ b drives the animated image b through formula (6), that is, it is possible to directly drive the animated image b by using the expression parameters ⁇ a corresponding to the animated image a without determining the pinch base of the animated image b.
  • the animation image driving method provided in the embodiments of this application is not only applicable to the migration between the expression bases (for faces) corresponding to different animation images, that is, the expression parameters corresponding to the driver image are used to directly drive the image.
  • the driver's image makes facial expressions, such as mouth shape, smiling, crying, etc.
  • the animation image driving method provided by the embodiments of the present application may also be suitable for transferring other objects, such as body movements, etc., that is, using the action parameters corresponding to the image of the driver to directly drive the image of the driven party to make an action.
  • the animated image with the pinching base can be used as the driving party image, without the pinching base.
  • the animated image as the driven party image. Since the expression parameters corresponding to the image of the driver and the expression parameters corresponding to the image of the driven party have a mapping relationship, after the mapping relationship is determined, the expression parameters corresponding to the image of the driver can be used to directly drive the image of the driven party based on the mapping relationship. Even if the image of the driven party does not have a pinch base.
  • the actual expression parameters corresponding to the image of the driver can drive the image of the driven party to make actual expressions
  • the actual expression parameters can reflect the degree of correlation between the actual expression and its expression base in different dimensions, that is, the actual expression parameters corresponding to the image of the driven party It can also reflect the degree of correlation between the actual expression of the image of the driven party and its expression base in different dimensions. Therefore, based on the relationship between the above expression parameters and the expression base, it can be based on the expression base corresponding to the image of the driver and the image corresponding to the driven party.
  • the expression base determines the mapping relationship between the expression parameters.
  • the driver image is used to drive the image of the driven party using the known expression parameters and the mapping relationship, so that the image of the driven party makes the known image The actual expression identified by the expression parameter. Therefore, without processing to obtain the pinch face base corresponding to the new animation image, the new animation image can be directly driven by the known expression parameters of other animation images, which speeds up the introduction of the new animation image.
  • mapping relationship is mainly introduced as a linear mapping relationship.
  • This embodiment mainly provides two types of methods for determining the mapping relationship.
  • the idea of the first type of method is to determine another expression parameter, such as the second expression parameter, based on a known expression parameter, such as the first expression parameter.
  • the idea of the second type of method is to determine the mapping relationship based on analytical expressions. Because the expression base of the driver image and the expression base of the driven image can be converted to obtain exactly the same point cloud data, based on the point cloud data Solve the mapping relationship between the equation relationship.
  • the flow chart of the first type of method can be seen in Figure 4.
  • the first expression parameter is an expression parameter whose dimension used to drive the expression base is the first dimension, the dimension of the first expression base is the first dimension, and the first expression parameter is used to drive the expression base corresponding to the first image, thereby
  • the target grid is determined according to the first expression parameter and the first expression base corresponding to the first image.
  • the target grid has a target vertex topology and is used to identify the first image when the expression corresponding to the first expression parameter is made.
  • the target expression base corresponding to the first image with the target vertex topology is obtained, the dimension of the target expression base is the second dimension, and the target expression base is determined according to the second expression base corresponding to the second image.
  • a second expression parameter corresponding to the first image is determined, and the second expression parameter is used to reflect the degree of expression change of the first image relative to the target grid.
  • the mapping relationship between the expression parameter corresponding to the driver's image and the expression parameter corresponding to the driven party's image is determined.
  • the dotted box on the left indicates the process of determining the target grid
  • the dotted box on the right indicates the process of determining the target expression base.
  • the driving party image is the first image and the second image that have a corresponding pinching base.
  • the image, the driven party image is an image that does not have a corresponding pinching base in the first image and the second image.
  • the embodiment of the present application may provide a variety of different methods for determining the target grid. It is understandable that if the target vertex topology is the second vertex topology corresponding to the second expression base, since the first expression base corresponds to the first vertex topology, usually, the first expression parameter is used to drive the first expression base to obtain The mesh should have the first vertex topology. When the first vertex topology is different from the second vertex topology, in order to obtain the target mesh with the second vertex topology, the mesh with the first vertex topology needs to be converted into the target mesh with the second vertex topology.
  • the vertex topologies mentioned in the embodiments of the present application may be part of the vertex topology in the mesh, and the part of the vertex topology is the vertex topology involved in driving the animation image.
  • the vertex topology involved is the vertex topology corresponding to the face in the mesh representing the head.
  • the method of determining the target grid may be: determining the initial grid according to the first expression parameter and the first expression base corresponding to the first image.
  • the mesh has a first vertex topology corresponding to the first expression base, so the target mesh can be generated according to the correspondence between the first vertex topology and the second vertex topology.
  • one way to determine the target expression base may be: determine the expressionless grid corresponding to the first expressionless image from the first expression base, and determine the second expressionless expression base from the second expression base.
  • the expressionless grid corresponding to the image and then, according to the expressionless grid corresponding to the first image and the expressionless grid corresponding to the second image, determine the adjustment grid.
  • the adjusted grid has a second vertex topology to identify the non-expression grid.
  • the first image of the emoticon Since the grid deformation relationship in the adjustment grid and the second expression base is known, the target expression base can be generated according to the grid deformation relationship in the adjustment grid and the second expression base.
  • the grid deformation relationship can reflect the deformation relationship between the expression grid and the expressionless grid in the expression base.
  • the vertex topology and the animation image have a certain relationship, but the relationship between the two is not strongly related, that is, the animation image is the same, the vertex topology must be the same, but the vertex topology is the same, the animation image can be different (for example, the vertex topology is deformed).
  • the specific way to determine the adjustment grid can be: paste the expressionless grid corresponding to the second image to the expressionless grid corresponding to the first image through the face pinching algorithm.
  • the face pinching algorithm can use the nricp algorithm to obtain a new The grid is the adjustment grid.
  • other face pinching algorithms can also be used, which is not limited in this embodiment.
  • the mesh obtained by using the first expression parameter to drive the first expression base is the target mesh instead of the initial mesh, without the need to transform the initial mesh into the target Grid steps.
  • the method provided by the embodiment corresponding to FIG. 5 can be more accurately determined
  • the target grid and target expression base are obtained, and then the mapping relationship is accurately determined, so that the driven image can be better driven according to the mapping relationship and the expression parameters of the driver's image.
  • the second expression parameter corresponding to the first image determined by the embodiment corresponding to FIG. 5 and the large number of existing expression parameters corresponding to the first image may not have the same data distribution.
  • the data distribution of the image is different, and the mapping relationship cannot be accurately determined, and the expression is not in place when the image of the driven party is driven by the expression parameters corresponding to the image of the driver, and the correct expression cannot be mapped.
  • the embodiment of the application also provides Another way to determine the target grid.
  • the expressionless grid corresponding to the second image without expression is determined from the second expression base corresponding to the second image, and the expressionless grid corresponding to the second image and the first image are determined according to the expressionless grid corresponding to the second image.
  • the corresponding first expression base determines the adjusted expression base corresponding to the first image with the second vertex topology, and the dimension of the adjusted expression base is the first dimension.
  • the first expression parameter is used to drive and adjust the expression base, so that the target grid is determined according to the first expression parameter and the adjusted expression base.
  • the target expression base can be determined by determining the expressionless grid corresponding to the first expressionless image from the adjusted expression base, according to the expressionless grid corresponding to the first image and the expression in the second expression base. Grid deformation relationship to generate target expression base.
  • the method for determining and adjusting the expression base may be by using a pinch face algorithm to paste the expressionless grid corresponding to the second image to each expression grid in the first expression base, so as to obtain a new expression base such as an adjusted expression base.
  • a pinch face algorithm to paste the expressionless grid corresponding to the second image to each expression grid in the first expression base, so as to obtain a new expression base such as an adjusted expression base.
  • the method provided in the embodiment corresponding to FIG. 6 avoids the problem that the mapping relationship cannot be accurately determined due to different data distributions, so that the mapping quality is significantly improved.
  • FIG. 7 shows an effect diagram of driving an animated image based on the method provided by the corresponding embodiment of FIG. 5 and FIG. 6.
  • the left side is an effect diagram of driving the animated image based on the method provided in the embodiment corresponding to FIG. 5
  • the right side is an effect diagram of driving the animated image based on the method provided in the embodiment corresponding to FIG. 6. It can be seen that in the rendering on the right, there are fewer wrinkles around the lips and more like a normal speaking mouth shape.
  • the method of determining the mapping relationship between the expression parameters corresponding to the driver image and the expression parameters corresponding to the driven image may be: obtaining multiple pairs The first expression parameter and the second expression parameter, and then, the mapping relationship is determined according to a first matrix formed by a plurality of first expression parameters and a second matrix formed by a plurality of second expression parameters.
  • one of the first image and the second image has a corresponding face pinching base, and the other does not have a corresponding face pinching base.
  • the first avatar does not have a corresponding pinching base
  • the second avatar has a corresponding pinching base
  • the first expression parameter is a random expression parameter.
  • the animation image driving method is introduced.
  • the driver image is the second image
  • the driven party The image is the first image.
  • the driver image is an animated image a
  • the driven image is an animated image b
  • the first expression parameter is a random expression parameter Bb, whose appearance is Fb
  • the vertex topology is the first vertex topology Tb
  • the dimension is Nb.
  • the random The expression parameter Bb can directly drive the first expression base Eb
  • the first expression base Eb looks like Fb
  • the vertex topology is the first vertex topology Tb
  • the dimension is the first dimension Nb.
  • the second expression base Ea of the animation image a is like Fa
  • the vertex topology is the second vertex topology Ta
  • the dimension is the second dimension Na.
  • Ba used to drive the second expression base Ea. Among them, Fa is not equal to Fb, Ta is not equal to Tb, and Na is not equal to Nb.
  • the random expression parameter Bb is used to directly drive the first expression base Eb to obtain the initial mesh, which is counted as Mesh mid , whose appearance is Fb, and the vertex topology is the first vertex topology Tb. Subsequently, through the correspondence between the first vertex topology and the second vertex topology, Mesh mid is turned into a target mesh, whose vertex topology is the second vertex topology (target vertex topology) Ta, the appearance remains as Fb, and the target mesh is denoted as Mesh c .
  • the grid deformation relationship between the expressions in each dimension of the adjustment grid Newb and the second expression base Ea is known relative to the neutral expression (no expression), it can be adjusted according to the grid deformation relationship in the adjustment grid Newb and the second expression base Ea.
  • the grid deformation relationship deforms the target expression base Ea' from Newb.
  • the target expression base Ea' can be used to pinch out the target mesh Mesh c through the pinch expression algorithm, and the second expression parameter Ba' with dimension Na can be obtained at the same time.
  • the linear mapping relationship between the first expression parameter and the second expression parameter is satisfied.
  • the formula for determining the mapping relationship may be formula (8):
  • f is the mapping relationship
  • BB is the first matrix
  • BA' is the second matrix
  • inv is the matrix inversion operation.
  • the driver image is the first image
  • the driven image is the second image.
  • the driver image is the animated image a
  • the driven image is the animated image b
  • the first expression parameter is the expression parameter Ba, whose appearance is Fa
  • the vertex topology is the first vertex topology Ta
  • the dimension is Na
  • the expression parameter Ba can directly drive the first expression base Ea
  • the appearance of the first expression base Ea is Fa
  • the vertex topology is the first vertex topology Ta
  • the dimension is the first dimension Na.
  • the second expression base Eb of the animation image b has an appearance of Fb, the vertex topology is the second vertex topology Tb, and the dimension is the second dimension Nb.
  • the expression parameter Ba is used to directly drive the first expression base Ea to obtain the initial mesh, which is counted as Mesh mid , whose appearance is Fa, and the vertex topology is the first vertex topology Ta. Subsequently, through the correspondence between the first vertex topology and the second vertex topology, Mesh mid is turned into a target mesh, whose vertex topology is the second vertex topology (target vertex topology) Tb, the appearance remains Fa, and the target mesh is denoted as Mesh c .
  • the adjusted mesh is obtained, denoted as Newb, whose appearance is Fa, and the vertex topology is the second vertex topology Tb.
  • Newb whose appearance is Fa
  • the vertex topology is the second vertex topology Tb.
  • the target expression base can be deformed from Newb according to the mesh deformation relationship in the adjustment mesh Newb and the second expression base Eb Eb'.
  • the target expression base Eb' can be used to pinch out the target mesh Mesh c through the pinch expression algorithm, and the second expression parameter Bb with the dimension of Nb can be obtained at the same time.
  • the driver image is the first image and the driven image is the second image.
  • the driver image is the animated image a
  • the driven image is the animated image b
  • the first expression parameter is the expression parameter Ba, whose appearance is Fa
  • the vertex topology is the first vertex topology Ta
  • the dimension is Na
  • the expression parameter Ba can directly drive the first expression base Ea.
  • the second expression base Eb of the animation image b has an appearance of Fb
  • the vertex topology is the second vertex topology Tb
  • the dimension is the second dimension Nb.
  • the specific implementation of the embodiment corresponding to FIG. 6 can be referred to as shown in FIG. 10.
  • the target vertex topology is the second vertex topology Tb
  • first construct the adjusted expression base Ea' whose vertex topology is the second vertex topology Tb, and ensure that the adjusted expression base Ea' can be driven by the first expression parameter Ba.
  • the first way is to use a pinching algorithm such as nricp algorithm to paste the expressionless grid determined from the second expression base Eb to the first expression base Ea. On each expression grid, get the adjusted expression base Ea'.
  • the second way can be to paste the expressionless mesh determined from the second expression base Eb to the expressionless mesh in the first expression base Ea through the face pinching algorithm, and obtain a vertex topology that is the same as the mesh in Eb ,
  • the appearance is the expressionless mesh of Fa, and then according to the deformation of each expression mesh in Ea relative to the expressionless mesh, the expressionless mesh obtained above is changed to Fa and the vertex topology is Tb, so as to obtain Adjust the expression base Ea' to ensure that the corresponding relationship between the vertex pairs in Ea and Eb is unique when mapping. Adjust the appearance of the expression base Ea' to Fa, the vertex topology to the second vertex topology Tb, and the dimension to the first dimension Na.
  • the first expression parameter Ba can directly drive the first expression base Ea, and the adjusted expression base Ea' has the same dimension as the first expression base Ea, and the semantic information of each dimension is the same, so the first expression parameter can be used directly Ba drives to adjust the expression base Ea' to get the target grid.
  • the appearance of the target mesh is Fa
  • the vertex topology is the second vertex topology Tb
  • the target mesh is denoted as Mesh c .
  • the expressionless grid corresponding to the first expressionless image is determined from the adjusted expression base Ea', and the target expression base Eb is generated according to the expressionless grid corresponding to the first image and the grid deformation relationship in the second expression base.
  • the appearance of the target expression base Eb' is Fa
  • the vertex topology is the second vertex topology Tb
  • the dimension is Nb.
  • the driver image is the second image
  • the driven party The image is the first image.
  • the driver image is an animated image a
  • the driven image is an animated image b
  • the first expression parameter is a random expression parameter Bb, whose appearance is Fb
  • the vertex topology is the first vertex topology Tb
  • the dimension is Nb.
  • the random The expression parameter Bb can directly drive the first expression base Eb
  • the first expression base Eb looks like Fb
  • the vertex topology is the first vertex topology Tb
  • the dimension is the first dimension Nb.
  • the second expression base Ea of the animation image a is like Fa
  • the vertex topology is the second vertex topology Ta
  • the dimension is the second dimension Na.
  • Ba used to drive the second expression base Ea. Among them, Fa is not equal to Fb, Ta is not equal to Tb, and Na is not equal to Nb.
  • the specific implementation manner of the embodiment corresponding to FIG. 6 can be referred to as shown in FIG. 11.
  • the target vertex topology is the second vertex topology Ta
  • first construct the adjusted expression base Eb' whose vertex topology is the second vertex topology Ta, and ensure that the adjusted expression base Eb' can be driven by the first expression parameter Bb.
  • the first way is to use a pinching algorithm such as nricp algorithm to paste the expressionless grid determined from the second expression base Ea to the first expression base Eb. On each expression grid, get the adjusted expression base Eb'.
  • the second way can be to paste the expressionless mesh determined from the second expression base Ea to the expressionless mesh in the first expression base Eb through the face pinching algorithm, and obtain a vertex topology that is the same as the mesh in Ea.
  • the expression is Fb's expressionless grid, and then according to the deformation of each expression grid in Eb relative to the expressionless grid, the above-obtained expressionless grid is changed to Fb and the vertex topology is Ta, so as to obtain Adjust the expression base Eb' to ensure that the corresponding relationship between the vertex pairs in Ea and Eb is unique when mapping.
  • the appearance of the adjusted expression base Eb' is Fb
  • the vertex topology is the second vertex topology Ta
  • the dimension is the first dimension Nb.
  • the first expression parameter Bb can directly drive the first expression base Eb, and the adjusted expression base Eb' has the same dimension as the first expression base Eb, and the semantic information of each dimension is the same, so the first expression parameter can be used directly Bb drives and adjusts the expression base Eb' to get the target grid.
  • the appearance of the target mesh is Fb, the vertex topology is the second vertex topology Ta, and the target mesh is denoted as Mesh c .
  • the expressionless grid corresponding to the first expressionless image is determined from the adjusted expression base Eb', and the target expression base Ea is generated according to the expressionless grid corresponding to the first image and the grid deformation relationship in the second expression base.
  • the appearance of the target expression base Ea' is Fb
  • the vertex topology is the second vertex topology Ta
  • the dimension is Na.
  • the first type of method introduced above mainly determines the second expression parameter based on the existing first expression parameter obtained by sampling, and then determines the mapping relationship between the first expression parameter and the second expression parameter.
  • the embodiment of the present application also provides a second type of method.
  • the implementation of the second type of method may be: according to the expression base and expression corresponding to the driver object Parameters to determine the first point cloud data corresponding to the image of the driver; and, according to the expression base and expression parameters corresponding to the image of the driven party, determine the second point cloud data corresponding to the image of the driven party.
  • the first point cloud data can be converted to obtain the second point cloud data, or the second point cloud data can be converted to obtain the first point cloud data.
  • the mapping relationship between the expression parameters corresponding to the image of the driver and the expression parameters corresponding to the image of the driven party can be determined according to the first point cloud data, the second point cloud data and the conversion parameters.
  • the conversion may include rotation, translation, zooming, etc., for example.
  • the conversion parameter is used to identify the conversion relationship for converting the second point cloud data into the first point cloud data.
  • point cloud data for example, the first point cloud data and the second point cloud data
  • the expression base is E
  • the dimension of the expression base E is n
  • B is a vector of n*1
  • E is the expression base matrix
  • the emoticon base of the driver image is E a
  • the emoticon parameter of the driver image is B a
  • the emoticon driver image is mu a
  • the emoticon parameter of the driven image is B b
  • the second point cloud data R b mu b + E b * B b.
  • the conversion parameters can be determined by a variety of methods.
  • the conversion parameters can be calculated by the Iterative Closest Point (ICP) algorithm.
  • the conversion parameters can be expressed as formula (10):
  • trans is the conversion parameter
  • s means scaling
  • R means rotation
  • T means translation
  • the first point cloud data and the second point cloud data processed by the conversion parameters are exactly the same, and the first point cloud data and the second point cloud data processed by the conversion parameters have the relationship shown in the following formula (11):
  • the second type of method has smaller mathematical errors, and the obtained f is an analytical solution rather than a sampled solution, which avoids the problem of incomplete distribution caused by sampling.
  • the calculation result f in this method only depends on the expression base. Since the original point cloud data may be unevenly distributed, the down-sampling mesh can be used to obtain a uniform point inliner to form the expression base, and the effect can be obtained. At the same time, you can also control the parts used (such as only the mouth or the points of the eyes), so that you can accurately control the parts that need to be driven in the animated image, and avoid interference caused by unimportant parts (such as cheeks, etc.).
  • various structures of the animated character can also be driven.
  • the various structures that are driven are a deformable component of the animated image. Taking the animated image as a human being, for example, since human hands and feet can be deformed (for example, bent), the driven structure can be hands, feet, etc. Feet and so on.
  • an embodiment of the present application provides an animated character driving method.
  • the method includes:
  • the driver image has a corresponding structural base
  • the driven image does not have a corresponding structural base
  • the structural base is used to identify the structural features of the corresponding image
  • the deformation base is used to identify the deformation characteristics of the corresponding image.
  • the structural base can reflect structural features such as the length of the fingers, the thickness of the fingers, the width and thickness of the palm, and the position of the fingers
  • the deformable base can reflect the deformation characteristics such as the degree of bending of the fingers.
  • the corresponding image is a face
  • the structural base is the pinching base mentioned in the previous embodiment
  • the deformation base is the expression base.
  • the deformation parameter is used to identify the degree of change in the shape of the corresponding image. Taking the corresponding image as a hand as an example, the deformation parameter reflects the degree of bending of the finger. Of course, if the corresponding image is a face, the deformation parameter is the expression parameter mentioned in the foregoing embodiment.
  • S1203 Drive the image of the driven party according to the corresponding deformation parameters and the mapping relationship of the image of the driver.
  • the animated image is used for New Year's greetings.
  • There is an animated image a which has a pinch face base, and the expression base Ea of the animated image a is Fa, the vertex topology is Ta, and the dimension is Na;
  • the expression base Eb of the animated image b is Fb
  • the vertex topology is Tb
  • the dimension is Nb.
  • Fa is not equal to Fb
  • Ta is not equal to Tb
  • Na is not equal to Nb.
  • the animation image a with the known expression parameters Ba can be used as the driving party image, and the animation image b as the driven party.
  • the expression base Ea and the expression base Eb corresponding to the image of the driven party determine the mapping relationship between the expression parameter Ba corresponding to the image of the driver and the expression parameter Bb corresponding to the image of the driven party.
  • the expression parameter Bb corresponding to the driver's image can be determined through the known expression parameter Ba and the mapping relationship, thereby driving The new animation image speeds up the launch of the animation image b.
  • this embodiment also provides an artificial intelligence-based animation character driving device.
  • the device includes an acquiring unit 1201, a determining unit 1202, and a driving unit 1203:
  • the acquiring unit 1201 is configured to acquire an expression base corresponding to the image of the driver and an expression base corresponding to the image of the driven party, the image of the driver has a corresponding pinching base, and the image of the driven party does not have a corresponding pinching face base;
  • the determining unit 1202 is configured to determine the expression parameters corresponding to the driver image and the expression corresponding to the driven party image according to the expression base corresponding to the driver image and the expression base corresponding to the driven party image The mapping relationship between parameters;
  • the driving unit 1203 is configured to drive the image of the driven party according to the expression parameters corresponding to the image of the driver and the mapping relationship.
  • the driver image is an image with corresponding pinching bases in the first image and the second image
  • the driven party image is the first image and the second image without corresponding faces. Pinch the image of the face base; the determining unit 1202 is used to:
  • the target grid is determined according to the first expression parameter and the first expression base corresponding to the first image;
  • the first expression parameter is an expression parameter whose dimension used to drive the expression base is the first dimension, and the first The dimension of the expression base is the first dimension, and the target grid has a target vertex topology for identifying the first image when the expression corresponding to the first expression parameter is made;
  • the dimension of the target expression base is the second dimension; the target expression base is the second corresponding to the second image
  • the expression base is determined;
  • a second expression parameter corresponding to the first image is determined, and the second expression parameter is used to reflect the expression change of the first image relative to the target grid degree;
  • a mapping relationship between the expression parameter corresponding to the driver image and the expression parameter corresponding to the driven image is determined.
  • the target vertex topology is a second vertex topology corresponding to the second expression base
  • the determining unit 1202 is further configured to:
  • an adjustment grid is determined.
  • the adjustment grid has the second vertex topology and is used to identify when in an expressionless state.
  • the target expression base is generated.
  • the target vertex topology is a second vertex topology corresponding to the second expression base
  • the determining unit 1202 is further configured to:
  • the adjusted expression base corresponding to the first image with the second vertex topology is determined.
  • the dimension is the first dimension
  • the target expression base is generated according to the expressionless grid corresponding to the first image and the grid deformation relationship in the second expression base.
  • the first image does not have a corresponding face pinching base, and the second image has a corresponding face pinching base;
  • the first expression parameter is a random expression parameter;
  • the first image has a corresponding pinching base, and the second image does not have a corresponding pinching base.
  • the determining unit 1202 is further configured to:
  • the mapping relationship is determined according to a first matrix composed of a plurality of the first expression parameters and a second matrix composed of a plurality of the second expression parameters.
  • the determining unit 1202 is configured to:
  • a mapping relationship between the expression parameters corresponding to the driver image and the expression parameters corresponding to the driven image is determined.
  • the device includes an acquiring unit 1301, a determining unit 1302, and a driving unit 1303:
  • the acquiring unit 1301 is configured to acquire a deformation base corresponding to the driver's image and a deformation base corresponding to the driven party's image, where the driver's image has a corresponding structural base, and the driven-party's image does not have a corresponding structural base;
  • the structure base is used to identify the structural feature of the corresponding image, and the deformation base is used to identify the deformation feature of the corresponding image;
  • the determining unit 1302 is configured to determine the deformation parameter corresponding to the driver image and the deformation corresponding to the driven image according to the deformation base corresponding to the driver image and the deformation base corresponding to the driven image The mapping relationship between parameters;
  • the driving unit 1303 is configured to drive the driven party image according to the deformation parameter corresponding to the driving party image and the mapping relationship.
  • an embodiment of the present application also provides a device, which can drive an animated image based on artificial intelligence.
  • the equipment will be introduced below in conjunction with the drawings.
  • an embodiment of the present application provides a device 1300.
  • the device 1300 may also be a terminal device.
  • the terminal device may include a mobile phone, a tablet computer, and a personal digital assistant (PDA). , Point of Sales (POS), in-vehicle computers and other intelligent terminals. Take the terminal device as a mobile phone as an example:
  • FIG. 13b shows a block diagram of a part of the structure of a mobile phone related to a terminal device provided in an embodiment of the present application.
  • the mobile phone includes: a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (wireless fidelity, WiFi for short) module 1370, a processing 1380, and power supply 1390 and other components.
  • RF radio frequency
  • the memory 1320 may be used to store software programs and modules.
  • the processor 1380 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1320.
  • the memory 1320 may mainly include a storage program area and a storage data area.
  • the storage program area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; Data created by the use of mobile phones (such as audio data, phone book, etc.), etc.
  • the memory 1320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.
  • the processor 1380 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone, and executes by running or executing software programs and/or modules stored in the memory 1320, and calling data stored in the memory 1320. Various functions and processing data of the mobile phone can be used to monitor the mobile phone as a whole.
  • the processor 1380 may include one or more processing units; preferably, the processor 1380 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, application programs, etc. , The modem processor mainly deals with wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 1380.
  • the processor 1380 included in the terminal device also has the following functions:
  • the image of the driving party has a corresponding structural base, and the image of the driven party does not have a corresponding structural base;
  • the structural base is used to identify the The structural feature of the corresponding image, and the deformation base is used to identify the deformation feature of the corresponding image;
  • FIG. 14 is a structural diagram of the server 1400 provided by the embodiment of the present application.
  • the server 1400 may have relatively large differences due to different configurations or performances, and may include one or one The above central processing unit (Central Processing Units, CPU for short) 1422 (for example, one or more processors) and memory 1432, one or more storage media 1430 for storing application programs 1442 or data 1444 (for example, one or more storage equipment).
  • the memory 1432 and the storage medium 1430 may be short-term storage or permanent storage.
  • the program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of command operations on the server.
  • the central processing unit 1422 may be configured to communicate with the storage medium 1430, and execute a series of instruction operations in the storage medium 1430 on the server 1400.
  • the server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458, and/or one or more operating systems 1441, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • operating systems 1441 such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • the embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium is used to store program code, and the program code is used to execute the artificial intelligence-based animation image driving method described in each of the foregoing embodiments.
  • the embodiments of the present application also provide a computer program product including instructions, which when run on a computer, cause the computer to execute the artificial intelligence-based animation image driving method described in each of the foregoing embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Architecture (AREA)
  • Geometry (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本申请实施例公开了一种基于人工智能的动画形象驱动方法，当一个动画形象具有对应的捏脸基，一个动画形象不具有对应的捏脸基时，可以将具有捏脸基的作为驱动方形象，不具有捏脸基的作为被驱动方形象。获取驱动方形象对应的表情基和被驱动方形象对应的表情基，根据驱动方形象对应的表情基和被驱动方形象对应的表情基，确定出表情参数间的映射关系，根据该映射关系，使用驱动方形象已知的表情参数以及前述映射关系，对被驱动方形象进行驱动，以使得被驱动方形象做出该已知的表情参数所标识的实际表情。由此，不用处理得到新动画形象对应的捏脸基，就可以通过其他动画形象的已知表情参数直接来驱动该新动画形象，加快了新动画形象的推出速度。

Description

一种基于人工智能的动画形象驱动方法和装置
本申请要求于2019年08月30日提交中国专利局、申请号为201910816780.X、申请名称为“一种基于人工智能的动画形象驱动方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据处理领域，特别是涉及基于人工智能的动画形象驱动技术。
背景技术
目前，人机交互已经比较常见，用户可以与动画形象进行交互，交互时，用户可以输入一段任意的语音，相应地，驱动网络可以驱动一个动画形象做出该段语音对应的口型。在这一场景下，动画形象的存在能极大地增强真实感，提升表现力，带给用户更加沉浸式的体验。动画形象可以通过多种方式得到，例如由设计师手工设计。
在通过驱动网络驱动动画形象时，往往需要该动画形象具有对应的表情基和捏脸基。表情基由代表不同表情的可变形网络组成，每一个可变形网络均是由该动画形象的3D模型在不同表情下变化而成的，捏脸基由代表不同脸型的可变形网络组成，每一个可变形网络均为相对平均脸型变化较大的脸，捏脸基中的脸需要与该动画形象相关。
针对一些新创作出的动画形象，设计师可以手工设计动画形象的3D网格，由于表情基具有严格的语义信息，比如控制闭眼、闭嘴等，因此表情基可以很方便的通过设计师手工建模得到。然而，捏脸基本身由大量脸部数据通过主成分分析而分解得到，没有明确的语义信息，且与模型的网格顶点关联不准确，难以通过手工设计得到。导致只能通过复杂的数据处理确定出对应的捏脸基，延缓了新动画形象的推出。
发明内容
为了解决上述技术问题，本申请提供了一种基于人工智能的动画形象驱动方法和装置，可以通过其他动画形象的已知表情参数直接驱动新动画形象，无需处理得到新动画形象对应的捏脸基，加快了新动画形象的推出速度。
本申请实施例公开了如下技术方案：
第一方面，本申请实施例提供一种基于人工智能的动画形象驱动方法，由处理设备执行，所述方法包括：
获取驱动方形象对应的表情基和被驱动方形象对应的表情基，所述驱动方形象具有对应的捏脸基，所述被驱动方形象不具有对应的捏脸基；
根据所述驱动方形象对应的表情基和所述被驱动方形象对应的表情基，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数 间的映射关系；
根据所述驱动方形象对应的表情参数和所述映射关系，驱动所述被驱动方形象。
第二方面，本申请实施例提供一种基于人工智能的动画形象驱动装置，所述装置包括获取单元、确定单元和驱动单元：
所述获取单元，用于获取驱动方形象对应的表情基和被驱动方形象对应的表情基，所述驱动方形象具有对应的捏脸基，所述被驱动方形象不具有对应的捏脸基；
所述确定单元，用于根据所述驱动方形象对应的表情基和所述被驱动方形象对应的表情基，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系；
所述驱动单元，用于根据所述驱动方形象对应的表情参数和所述映射关系，驱动所述被驱动方形象。
第三方面，本申请实施例提供一种基于人工智能的动画形象驱动方法，由处理设备执行，所述方法包括：
获取驱动方形象对应的形变基和被驱动方形象对应的形变基，所述驱动方形象具有对应的结构基，所述被驱动方形象不具有对应的结构基；所述结构基用于标识所对应的形象的结构特征，所述形变基用于标识所对应的形象的形变特征；
根据所述驱动方形象对应的形变基和所述被驱动方形象对应的形变基，确定所述驱动方形象对应的形变参数和所述被驱动方形象对应的形变参数间的映射关系；
根据所述驱动方形象对应的形变参数和所述映射关系，驱动所述被驱动方形象。
第四方面，本申请实施例提供一种基于人工智能的动画形象驱动装置，所述装置包括获取单元、确定单元和驱动单元：
所述获取单元，用于获取驱动方形象对应的形变基和被驱动方形象对应的形变基，所述驱动方形象具有对应的结构基，所述被驱动方形象不具有对应的结构基；所述结构基用于标识所对应的形象的结构特征，所述形变基用于标识所对应的形象的形变特征；
所述确定单元，用于根据所述驱动方形象对应的形变基和所述被驱动方形象对应的形变基，确定所述驱动方形象对应的形变参数和所述被驱动方形象对应的形变参数间的映射关系；
所述驱动单元，用于根据所述驱动方形象对应的形变参数和所述映射关系，驱动所述被驱动方形象。
第五方面，本申请实施例提供一种设备，所述设备包括处理器以及存储器：
所述存储器用于存储程序代码，并将所述程序代码传输给所述处理器；
所述处理器用于根据所述程序代码中的指令执行第一方面或第三方面所述的方法。
第六方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质用于存储程序代码，所述程序代码用于执行第一方面或第三方面所述的方法。
第七方面，本申请实施例提供了一种计算机程序产品，包括指令，当其在计算机上运行时，使得计算机执行第一方面或第三方面所述的方法。
由上述技术方案可以看出，当一个动画形象具有对应的捏脸基，一个动画形象不具有对应的捏脸基时，可以将具有捏脸基的动画形象作为驱动方形象，不具有捏脸基的动画形象作为被驱动方形象。由于驱动方形象对应的表情参数与被驱动方形象对应的表情参数应具有映射关系，故在确定出该映射关系后，可以利用驱动方形象对应的表情参数直接驱动被驱动方形象，即使被驱动方形象不具有捏脸基。由于驱动方形象对应的实际表情参数可以驱动被驱动方形象做出实际表情，该实际表情参数可以体现该实际表情与其表情基在不同维度下的相关程度，即被驱动方形象对应的实际表情参数也可以体现被驱动方形象的实际表情与其表情基在不同维度下的相关程度，故基于上述表情参数与表情基间的关联关系，可以根据驱动方形象对应的表情基和被驱动方形象对应的表情基，确定出表情参数间的映射关系，根据该映射关系，使用驱动方形象已知的表情参数以及前述映射关系，对被驱动方形象进行驱动，以使得被驱动方形象做出该已知的表情参数所标识的实际表情。由此，无需处理得到新动画形象对应的捏脸基，就可以通过其他动画形象的已知表情参数直接来驱动该新动画形象，加快了新动画形象的推出速度。
附图说明
图1为本申请实施例提供的一种基于人工智能的动画形象驱动方法的应用场景示意图；
图2为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图3为本申请实施例提供的3DMM库M的各个维度分布和意义的示例图；
图4为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图5为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图6为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图7为本申请实施例提供的驱动动画形象的效果图；
图8为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图9为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图10为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图11为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图12a为本申请实施例提供的一种基于人工智能的动画形象驱动方法的流程图；
图12b为本申请实施例提供的一种基于人工智能的动画形象驱动装置的结构图；
图13a为本申请实施例提供的一种基于人工智能的动画形象驱动装置的结构图；
图13b为本申请实施例提供的一种终端设备的结构图；
图14为本申请实施例提供的一种服务器的结构图。
具体实施方式
下面结合附图，对本申请的实施例进行描述。
相关技术针对一些新创作出的动画形象，通常可以由设计师手工设计动画形象的3D网格，例如设计师可以通过手工建模得到表情基。然而，由于捏脸基没有明确的语义信息，且与模型的网格顶点关联不准确，因此，难以通过手工设计得到。
另外，由于不同动画形象的3D网格的顶点拓扑(例如包括顶点的数目和顶点间的三角拓扑)、表情基的维数可能有所不同，利用具有捏脸基的动画形象在特定表情基下的表情参数，一般难以直接驱动另一个动画形象的表情基。因此，为了驱动另一动画形象的表情基，推出新动画形象，只能通过复杂的数据处理确定出对应的捏脸基，延缓了新动画形象的推出。
为此，本申请实施例提供一种基于人工智能的动画形象驱动方法，该方法可以实现不同动画形象的表情基之间的表情参数迁移，即当一个动画形象具有对应的捏脸基，一个动画形象不具有对应的捏脸基时，将具有捏脸基的动画形象作为驱动方形象，不具有捏脸基的动画形象作为被驱动方形象，确定出驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系，从而无需处理得到被驱动方形象对应的捏脸基，使用驱动方形象已知的表情参数以及前述映射关系，就可以对被驱动方形象进行驱动。
本申请实施例提供的基于人工智能的动画形象驱动方法可以应用于具有动画形象建立能力的处理设备上，该处理设备可以是终端设备，也可以是服务器。
该处理设备可以具有实施计算机视觉技术的能力。在本申请实施例中，处理设备通过实施上述计算机视觉技术，可以实现根据驱动方形象对应的表情基和被驱动方形象对应的表情基，确定出表情参数间的映射关系，从而可以根据该映射关系，使用驱动方形象已知的表情参数以及前述映射关系，对被驱动方形象进行驱动，实现快速推出新动画形象等功能。
其中，若处理设备是终端设备，则终端设备可以是智能终端、计算机、个人数字助理(Personal Digital Assistant，简称PDA)、平板电脑等。
若该处理设备是服务器，则服务器可以为独立服务器，也可以为集群服务器。当服务器实施该基于人工智能的动画形象驱动方法时，服务器可以根据驱动方形象对应的表情基和被驱动方形象对应的表情基，确定出表情参数间的映射关系，使用驱动方形象已知的表情参数以及前述映射关系，对被驱动方形象进行驱动，得到新动画形象，并将新动画形象在终端设备上显示、推出。
需要说明的是，本申请实施例中的动画形象可以包括2D动画形象和3D动画形象；该动画形象具体可以仅呈现脸部区域或头部区域，也可以呈现全身区域；此外，该动画形象具体可以表现为卡通形象，也可以表现为基于现实人物或动物构建的虚拟形象；本申请在此不对动画形象的表现形式做任何限定。需要说明的是，本申请实施例提供的基于人工智能的动画形象驱动方法可以应用到需要驱动手工设计的动画形象的应用场景，例如新闻播报、天气预报、游戏解说以及游戏场景中虚拟游戏人物等，还可以应用到需要利用动画形象承担私人化服务的应用场景，例如心理医生，虚拟助手等面向个人的一对一服务。在这些应用场景下，利用本申请实施例提供的方法不用处理得到动画形象对应的捏脸基，便可以对动画形象进行驱动。
为了便于理解本申请的技术方案，下面结合实际应用场景对本申请实施例提供的基于人工智能的动画形象驱动方法进行介绍。
参见图1，图1为本申请实施例提供的基于人工智能的动画形象驱动方法的应用场景示意图。该应用场景以处理设备为终端设备为例进行介绍，该应用场景中包括终端设备101，终端设备101可以获取驱动方形象对应的表情基和被驱动方形象对应的表情基。其中，驱动方形象是指具有表情基和捏脸基的动画形象，被驱动方形象是指具有表情基但不具有捏脸基的动画形象。
需要说明的是，表情基用于标识动画形象脸部的表情特征，由代表不同表情的可变形网络组成，每一个可变形网络均是由该动画形象的3D模型在不同表情下变化而成的。捏脸基用于标识动画形象脸部的基本特征，由代表不同脸型的可变形网络组成，每一个可变形网络均为相对平均脸型变化较大的脸，捏脸基中的脸需要与该动画形象相关。
由于驱动方形象对应的表情参数与被驱动方形象对应的表情参数具有映射关系，若可以确定出该映射关系，便可以利用驱动方形象对应的表情参数 直接驱动被驱动方形象，即使被驱动方形象不具有捏脸基。其中，表情参数的一种表现形式可以是系数，例如可以是具有某一维数的向量；映射关系可以是线性映射关系，也可以是非线性映射关系，本实施例对此不做限定。
由于驱动方形象对应的实际表情参数可以驱动被驱动方形象做出实际表情，该实际表情参数可以体现该实际表情与其表情基在不同维度下的相关程度，同理，被驱动方形象对应的实际表情参数也可以体现被驱动方形象的实际表情与其表情基在不同维度下的相关程度，故基于上述表情参数与表情基间的关联关系，终端设备101可以根据驱动方形象对应的表情基和被驱动方形象对应的表情基，确定驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系。
由于终端设备101可以根据该映射关系和驱动方形象对应的表情参数计算得到被驱动方形象对应的表情参数，计算得到的被驱动方形象的表情参数与被驱动方形象的表情基维数相同，从而可以利用计算得到的被驱动方形象的表情参数驱动被驱动方形象做出表情。故，终端设备101根据驱动方形象对应的表情参数和该映射关系可以直接驱动被驱动方形象。
应理解，图1所示的应用场景仅为示例，在实际应用中，本申请实施例提供的基于人工智能的动画形象驱动方法还可以应用于其它应用场景，在此不对本申请实施例提供的基于人工智能的动画形象驱动方法适用的应用场景做任何限定。
接下来，将结合附图对本申请实施例提供的基于人工智能的动画形象驱动方法进行详细介绍。参见图2，所述方法包括：
S201、获取驱动方形象对应的表情基和被驱动方形象对应的表情基。
在一些动画形象中，有些动画形象为已经推出的动画形象，具有表情基和捏脸基，而有些动画形象是新动画形象，仅具有表情基而不具有捏脸基。在本实施例中，驱动方形象具为具有对应的捏脸基的动画形象，被驱动方形象为不具有对应的捏脸基的动画形象。
需要说明的是，本申请实施例中的动画形象可以为模型库中的模型，也可以是通过模型库中模型的线性组合得到的。该模型库可以是人脸3D可变形模型(3DMM)库，也可以是其他模型库，本实施对此不做限定。动画形象例如驱动方形象和被驱动方形象可以是一个3D网格。
以3DMM库为例，3DMM库由大量高精度脸部数据通过主成分分析方法(Principal Component Analysis，PCA)得到，描述了高维脸型和表情相对平均脸的主要变化，也可以描述纹理信息。
一般来说，3DMM库描述一个无表情的脸型时，可以通过mu+∑(Pface i-mu)*α i得到。其中，mu是中性表情下的平均脸，Pface i是第i个脸型主成分分量，α i是第i个脸型主成分分量的权重，也就是捏脸参数。
假设3DMM库中的动画形象对应的网格可以通过M表示，即通过M表示3DMM库中的脸型、表情和顶点之间的关系，M是一个[m×n×d]的三维矩阵，其中每一维分别为网格的顶点坐标(m)、脸型主成分(n)、表情主成分(d)。3DMM库M的各个维度分布和意义如图3所示。由于m表示xyz三个坐标的值，所以网格的顶点数为m/3，记作v。如果确定了动画形象的脸型或者表情，那么M可以是一个二维矩阵。
在本申请实施例中，不考虑3DMM库中的纹理维度，假设动画形象的驱动为F，则可以通过公式(1)确定F：
Figure PCTCN2020105673-appb-000001
其中，M为动画形象的网格，α为捏脸参数，β为表情参数；d为表情基中表情网格的个数，n为捏脸基中捏脸网格的个数，M k,j,i为具有第i个表情网格、第j个捏脸网格的第k个网格，α j为一组捏脸参数中的第j维，表示第j个脸型主成分分量的权重，β i为一组表情参数中的第i维，表示第i个表情主成分分量的权重。
其中，确定捏脸参数的过程为捏脸算法，确定表情参数的过程为捏表情算法。捏脸参数用于与捏脸基做线性组合得到对应的脸型，例如存在一个包括50个捏脸网格(属于可变形网格，例如blendshape)的捏脸基，该捏脸基对应的捏脸参数为一个50维的向量，每一维可以标识该捏脸参数所对应的脸型与一个捏脸网格的相关程度。捏脸基所包括的捏脸网格分别代表不同脸型，每一个捏脸网格均为相对平均脸变化较大的脸部形象，是大量的脸通过PCA分解之后得到的不同维度的脸型主成分，且同一个捏脸基中不同捏脸网格对应的顶点序号保持一致。
表情参数用于与表情基做线性组合得到对应的表情，例如存在一个包括50个(相当于维数为50)表情网格(属于可变形网格，例如blendshape)的表情基，该表情基对应的表情参数为一个50维的向量，每一维可以标识该表情参数所对应的表情与一个表情网格的相关程度。表情基所包括的表情网格分别代表不同表情，每一个表情网格均由同一个3D模型在不同表情下变化而成，同一个表情基中不同表情网格对应的顶点序号保持一致。
针对前述的可变形网格，单个网格可以通过预定义形状变形，得到任意数量网格。
S202、根据驱动方形象对应的表情基和被驱动方形象对应的表情基，确定驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系。
基于本实施例的方法，在通过表情参数间的映射关系，利用驱动方形象的表情参数驱动被驱动方形象的场景中，捏脸基保持不变，即脸型固定，只有表情基需要调节。
为此，基于上述公式(1)动画形象的驱动还可以表示为公式(2)：
Figure PCTCN2020105673-appb-000002
其中，M k为固定脸型的动画形象的驱动，M k,i为第i个表情网格，β i为第i个表情网格对应的表情参数，n为表情基中表情网格的个数。
若动画形象a具有表情基和捏脸基，动画形象b具有表情基但不具有捏脸基，动画形象a已经通过捏表情算法得到一些表情参数，在这种情况下，动画形象a可以作为驱动方形象，动画形象b可以作为被驱动方形象。通过表情参数和表情基的线性组合，基于公式(3)可以得到动画形象a的驱动：
Figure PCTCN2020105673-appb-000003
其中，
Figure PCTCN2020105673-appb-000004
为动画形象a的驱动，
Figure PCTCN2020105673-appb-000005
为动画形象a的第i个表情网格，
Figure PCTCN2020105673-appb-000006
为动画形象a的第i个表情网格对应的表情参数，n a为动画形象a的表情基中包括表情网格个数。
同理，动画形象b的驱动为公式(4)：
Figure PCTCN2020105673-appb-000007
其中，
Figure PCTCN2020105673-appb-000008
为动画形象b的驱动，
Figure PCTCN2020105673-appb-000009
为动画形象b的第i个表情网格，
Figure PCTCN2020105673-appb-000010
为动画形象b的第i个表情网格对应的表情参数，n b为动画形象b的表情基中表情网格个数。
由于驱动方形象对应的表情参数与被驱动方形象对应的表情参数具有映射关系，驱动方形象对应的表情参数与被驱动方形象对应的表情参数间的映射关系可以通过函数f()表示，则通过驱动方形象对应的表情参数计算被驱动方形象对应的表情参数的公式(5)如下：
β b＝f(β a)   (5)
其中，β b为被驱动方形象对应的表情参数，β a为驱动方形象对应的表情参数，f()表示驱动方形象对应的表情参数与被驱动方形象对应的表情参数间的映射关系。
故，若确定出该映射关系，结合公式(4)和(5)，便可以利用驱动方形象(动画形象a)对应的表情参数直接驱动被驱动方形象(动画形象b)。
由于驱动方形象对应的实际表情参数可以驱动被驱动方形象做出实际表情，该实际表情参数可以体现该实际表情与其表情基在不同维度下的相关程度，即被驱动方形象对应的实际表情参数也可以体现被驱动方形象的实际表情与其表情基在不同维度下的相关程度，故基于上述表情参数与表情基间的关联关系，可以根据驱动方形象对应的表情基和被驱动方形象对应的表情基，确定出表情参数间的映射关系。
S203、根据驱动方形象对应的表情参数和映射关系驱动被驱动方形象。
继续以上述动画形象a和动画形象b为例，若该映射关系为线性关系， 则上述公式(5)可以表示为公式(6)：
β b＝f*β a  (6)
其中，β b为被驱动方形象对应的表情参数，β a为驱动方形象对应的表情参数，f表示驱动方形象对应的表情参数与被驱动方形象对应的表情参数间的映射关系。
在确定出驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系f后，便可以结合公式(6)计算动画形象b的表情参数β b。β a的维数与动画形象b的表情基维数不同，而通过映射关系得到的β b的维数与动画形象b的表情基维数相同，可以驱动动画形象b做出表情，故，利用β b通过公式(6)驱动动画形象b，即实现无需确定动画形象b的捏脸基，便可以利用动画形象a对应的表情参数β a直接驱动动画形象b。
需要说明的是，本申请实施例提供的动画形象驱动方法，不仅可以适用于不同动画形象所对应的表情基(针对脸部)之间的迁移，即利用驱动方形象对应的表情参数直接驱动被驱动方形象做出表情，例如口型、微笑、哭等。本申请实施例提供的动画形象驱动方法还可以适用于迁移其他对象，例如肢体动作等，即利用驱动方形象对应的动作参数直接驱动被驱动方形象做出动作。
由上述技术方案可以看出，当一个动画形象具有对应的捏脸基，一个动画形象不具有对应的捏脸基时，可以将具有捏脸基的动画形象作为驱动方形象，不具有捏脸基的动画形象作为被驱动方形象。由于驱动方形象对应的表情参数与被驱动方形象对应的表情参数具有映射关系，故确定出该映射关系后，便可以基于该映射关系利用驱动方形象对应的表情参数直接驱动被驱动方形象，即使被驱动方形象不具有捏脸基。由于驱动方形象对应的实际表情参数可以驱动被驱动方形象做出实际表情，该实际表情参数可以体现该实际表情与其表情基在不同维度下的相关程度，即被驱动方形象对应的实际表情参数也可以体现被驱动方形象的实际表情与其表情基在不同维度下的相关程度，故基于上述表情参数与表情基间的关联关系，可以根据驱动方形象对应的表情基和被驱动方形象对应的表情基，确定出表情参数间的映射关系，根据该映射关系，使用驱动方形象已知的表情参数以及该映射关系，对被驱动方形象进行驱动，以使得被驱动方形象做出该已知的表情参数所标识的实际表情。由此，无需处理得到新动画形象对应的捏脸基，就可以通过其他动画形象的已知表情参数直接驱动该新动画形象，加快了新动画形象的推出速度。
接下来，将针对S202详细介绍如何确定驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系。其中，接下来的实施例中主要以映射关系为线性映射关系进行介绍。
本实施例主要提供确定映射关系的两类方法，第一类方法的思路是根据 一个已知的表情参数例如第一表情参数，确定出另一个表情参数例如第二表情参数，由于第一表情参数和第二表情参数之间具有映射关系，从而根据第一表情参数和第二表情参数求解映射关系。第二类方法的思路是基于解析表达式的映射关系确定方法，由于驱动方形象的表情基和被驱动方形象的表情基之间通过转换可以得到完全相同的点云数据，基于点云数据之间的等式关系求解映射关系。
第一类方法的流程图可以参见图4所示。第一表情参数为用于驱动表情基的维数为第一维数的表情参数，第一表情基的维数为第一维数，利用第一表情参数驱动第一形象对应的表情基，从而根据第一表情参数和第一形象对应的第一表情基确定目标网格。该目标网格具有目标顶点拓扑，用于标识做出第一表情参数所对应的表情时的第一形象。获取具有目标顶点拓扑的第一形象对应的目标表情基，目标表情基的维数为第二维数，目标表情基为根据第二形象所对应的第二表情基确定的。之后，根据目标网格和目标表情基，确定第一形象对应的第二表情参数，第二表情参数用于体现第一形象相对于目标网格的表情变化程度。根据第一表情参数和第二表情参数，确定驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系。其中，左边虚线框标注的为目标网格的确定过程，右边虚线框标注的是目标表情基的确定过程。
需要说明的是，第一形象和第二形象中一个具有对应的捏脸基，另一个不具有对应的捏脸基，驱动方形象为第一形象和第二形象中具有对应的捏脸基的形象，被驱动方形象为第一形象和第二形象中不具有对应的捏脸基的形象。
基于图4对应的实施例，本申请实施例可以提供多种不同的确定目标网格的方式。可以理解的是，若目标顶点拓扑为第二表情基对应的第二顶点拓扑，由于第一表情基对应的是第一顶点拓扑，那么通常情况下，利用第一表情参数驱动第一表情基得到的网格应该具有第一顶点拓扑。当第一顶点拓扑与第二顶点拓扑不相同时，为了得到具有第二顶点拓扑的目标网格，需要将具有第一顶点拓扑的网格转换成具有第二顶点拓扑的目标网格。
其中，本申请实施例所提到的顶点拓扑例如第一顶点拓扑、第二顶点拓扑、目标顶点拓扑可以为网格中的部分顶点拓扑，该部分顶点拓扑为驱动动画形象所涉及到的顶点拓扑。例如，动画形象为头部，驱动动画形象做出脸部表情时，所涉及的顶点拓扑为表示头部的网格中脸部所对应的顶点拓扑。
为此，在一种可能的实现方式中，参见图5所示，确定目标网格的方式可以是：根据第一表情参数和第一形象所对应的第一表情基确定初始网格，由于初始网格具有第一表情基对应的第一顶点拓扑，故，可以根据第一顶点拓扑和第二顶点拓扑的对应关系(correspondence)，生成目标网格。
在这种情况下，确定目标表情基的一种方式可以是：从第一表情基中确 定无表情的第一形象对应的无表情网格，并从第二表情基中确定无表情的第二形象对应的无表情网格，然后，根据第一形象对应的无表情网格和第二形象对应的无表情网格，确定调整网格，调整网格具有第二顶点拓扑，用于标识处于无表情时的第一形象。由于调整网格和第二表情基中的网格形变关系是已知的，故，根据调整网格和第二表情基中的网格形变关系，可以生成目标表情基。该网格形变关系可以体现表情基中的有表情网格相对于无表情网格的形变关系。
其中，顶点拓扑与动画形象具有一定关系，但是二者之间并非强相关的关系，即动画形象相同，顶点拓扑一定相同，但是顶点拓扑相同，动画形象可以不同(例如对顶点拓扑进行形变)。确定调整网格的具体方式可以是：将第二形象对应的无表情网格通过捏脸算法贴到第一形象对应的无表情网格上，捏脸算法例如可以采用nricp算法，从而得到一个新的网格即调整网格。当然，除了使用nricp算法，还可以采用其他捏脸算法，本实施例对此不做限定。
当第一顶点拓扑与第二顶点拓扑相同时，那么，利用第一表情参数驱动第一表情基得到的网格即目标网格，而非初始网格，无需经过上述将初始网格转换成目标网格的步骤。
需要说明的是，在第一形象对应的第二表情参数和第一形象对应的已有的大量表情参数具有相同的数据分布时，通过图5对应的实施例所提供的方法可以较为准确地确定出目标网格和目标表情基，进而准确地确定出映射关系，从而根据映射关系和驱动方形象的表情参数可以较好地驱动被驱动形象。
然而，在一些情况下，通过图5对应实施例确定出的第一形象对应的第二表情参数与第一形象对应的已有的大量表情参数不一定具有相同的数据分布，为了避免由于二者的数据分布不同，而无法准确地确定出映射关系，进而在利用驱动方形象对应的表情参数驱动被驱动方形象时造成表达不到位，无法映射出正确表情等等问题，本申请实施例还提供另一种确定目标网格的方式。
参见图6所示，在该方式中，从第二形象对应的第二表情基中确定无表情的第二形象对应的无表情网格，根据第二形象对应的无表情网格和第一形象所对应的第一表情基，确定具有第二顶点拓扑的第一形象对应的调整表情基，调整表情基的维数为第一维数。利用第一表情参数驱动调整表情基，从而根据第一表情参数和调整表情基，确定目标网格。在这种情况下，目标表情基的确定方式可以是从调整表情基中确定无表情的第一形象对应的无表情网格，根据第一形象对应的无表情网格和第二表情基中的网格形变关系，生成目标表情基。
其中，确定调整表情基的方式可以是通过捏脸算法，将第二形象对应的无表情网格贴到第一表情基中每一个表情网格上，从而得到新的表情基例如 调整表情基。这是一种非刚性配准方法，捏脸算法例如可以是nricp算法。
通过图6对应实施例所提供的方法，避免了由于数据分布不同而无法准确的确定出映射关系的问题，使得映射质量有明显提升。
图7示出了基于图5和图6对应实施例所提供的方法驱动动画形象的效果图。其中，左侧为基于图5对应实施例所提供的方法驱动动画形象的效果图，右侧为基于图6对应实施例所提供的方法驱动动画形象的效果图。可见，右侧的效果图中嘴唇周围皱纹更少，更像正常说话的口型。
需要说明的是，图5和图6中目标网格的确定方式可以相互替换，同理，图5和图6中目标表情基的确定方式也是可以相互替换的。
在使用第一类方法的情况下，根据第一表情参数和第二表情参数，确定驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系的方式可以是：获取多对第一表情参数和第二表情参数，然后，根据由多个第一表情参数构成的第一矩阵和由多个第二表情参数构成的第二矩阵，确定映射关系。
可以理解的是，在图5和图6对应的任一实施例中，第一形象和第二形象中一个具有对应的捏脸基，另一个不具有对应的捏脸基。其中，若第一形象不具有对应的捏脸基，则第二形象具有对应的捏脸基，第一表情参数为随机表情参数。或者，若第一形象具有对应的捏脸基，则第二形象不具有对应的捏脸基。基于图5和图6对应的实施例，针对第一形象和第二形象具有捏脸基的情况的不同，对动画形象驱动方法进行介绍。
若图5对应的实施例中第一形象不具有对应的捏脸基，第二形象具有对应的捏脸基，第一表情参数为随机表情参数，则驱动方形象为第二形象，被驱动方形象为第一形象。驱动方形象为动画形象a，被驱动方形象为动画形象b，第一表情参数为随机表情参数Bb，其样子为Fb，顶点拓扑为第一顶点拓扑Tb，维数为Nb，换言之，该随机表情参数Bb能直接驱动第一表情基Eb，第一表情基Eb的样子为Fb，顶点拓扑为第一顶点拓扑Tb，维数为第一维数Nb。动画形象a的第二表情基Ea，其样子为Fa，顶点拓扑为第二顶点拓扑Ta，维数为第二维数Na，已有大量用于驱动第二表情基Ea的表情参数Ba。其中，Fa不等于Fb，Ta不等于Tb，Na不等于Nb。
在这种情况下，图5对应的实施例的具体实现方式可以参见图8所示。利用随机表情参数Bb直接驱动第一表情基Eb得到初始网格，计作Mesh mid，其样子为Fb，顶点拓扑为第一顶点拓扑Tb。随后，通过第一顶点拓扑和第二顶点拓扑的对应关系，将Mesh mid变成目标网格，其顶点拓扑为第二顶点拓扑(目标顶点拓扑)Ta，样子保持为Fb，目标网格记作Mesh c。通过nricp算法将从第二表情基Ea中确定出的无表情网格贴到从第一表情基Eb中确定出的无表情网格上，通过寻找空间最近点的方法，得到点对的关系，从而得到调整网格，记作Newb，其样子为Fb，顶点网格为第二顶点拓扑Ta。再者，需 要得到一组新的表情基例如目标表情基Ea’，目标表情基Ea’的样子为Fb，顶点拓扑为第二顶点拓扑Ta，维数为Na。由于调整网格Newb和第二表情基Ea中各个维度的表情相对中性表情(无表情)的网格形变关系是已知的，故，可以根据调整网格Newb和第二表情基Ea中的网格形变关系从Newb中形变出目标表情基Ea’。由此，就可以使用目标表情基Ea’，通过捏表情算法捏出目标网格Mesh c，同时得到了维数为Na的第二表情参数Ba’。
由此，一组Bb和Ba’的映射关系就建立了。当随机生成大量第一表情参数Bb时，就可以产生大量对应的第二表情参数Ba’。假设第一表情参数Bb和第二表情参数Ba’的个数分别是L个，L个第一表情参数Bb构成第一矩阵，L个第二表情参数Ba’构成第二矩阵，分别记作BB和BA’。如公式(7)所示：
BA′＝[L×Na]，BB＝[L×Nb]  (7)
本方案第一表情参数和第二表情参数之间满足线性映射关系例如公式(6)所示，则映射关系的确定公式可以为公式(8)：
f＝BB*inv(BA′)   (8)
其中，f为映射关系，BB为第一矩阵，BA′为第二矩阵，inv为矩阵求逆运算。
在得到映射关系f后，由于已有大量的Ba，Ba和Ba’对应的表情基的维数都是Na，每一维数的语义信息一样，所以Ba和Ba’可以等价。由此，对于任意一组Ba，可以得到对应的Bb＝f*Ba，从而根据表情参数Ba得到表情参数Bb，以便驱动动画形象b。
若图5对应的实施例中第一形象具有对应的捏脸基，第二形象不具有对应的捏脸基，则驱动方形象为第一形象，被驱动方形象为第二形象。驱动方形象为动画形象a，被驱动方形象为动画形象b，第一表情参数为表情参数Ba，其样子为Fa，顶点拓扑为第一顶点拓扑Ta，维数为Na，换言之，该表情参数Ba能直接驱动第一表情基Ea，第一表情基Ea的样子为Fa，顶点拓扑为第一顶点拓扑Ta，维数为第一维数Na。动画形象b的第二表情基Eb，其样子为Fb，顶点拓扑为第二顶点拓扑Tb，维数为第二维数Nb，已有大量用于驱动第一表情基Ea的表情参数Ba。其中，Fa不等于Fb，Ta不等于Tb，Na不等于Nb。
在这种情况下，图5对应的实施例的具体实现方式可以参见图9所示。利用表情参数Ba直接驱动第一表情基Ea得到初始网格，计作Mesh mid，其样子为Fa，顶点拓扑为第一顶点拓扑Ta。随后，通过第一顶点拓扑和第二顶点拓扑的对应关系，将Mesh mid变成目标网格，其顶点拓扑为第二顶点拓扑(目标顶点拓扑)Tb，样子保持为Fa，目标网格记作Mesh c。通过nricp算法将从第二表情基Eb中确定出的无表情网格贴到从第一表情基Ea中确定出的无表情网格上，通过寻找空间最近点的方法，得到点对的关系，从而得到调整网 格，记作Newb，其样子为Fa，顶点拓扑为第二顶点拓扑Tb。再者，需要得到一组新的表情基例如目标表情基Eb’，目标表情基Eb’的样子为Fa，顶点拓扑为第二顶点拓扑Tb，维数为Nb。由于调整网格Newb和第二表情基Eb中的网格形变关系是已知的，故，可以根据调整网格Newb和第二表情基Eb中的网格形变关系从Newb中形变出目标表情基Eb’。由此，就可以使用目标表情基Eb’，通过捏表情算法捏出目标网格Mesh c，同时得到了维数为Nb的第二表情参数Bb。
利用已有的大量第一表情参数Ba就可以产生大量对应的第二表情参数Bb。同理可以利用上述公式(7)和(8)的方法确定第一表情参数和第二表情参数的映射关系。
由此，对于任意一组Ba，可以得到对应的Bb＝f*Ba，从而根据表情参数Ba得到表情参数Bb，以便驱动动画形象b。
若图6对应的实施例中第一形象具有对应的捏脸基，第二形象不具有对应的捏脸基，则驱动方形象为第一形象，被驱动方形象为第二形象。驱动方形象为动画形象a，被驱动方形象为动画形象b，第一表情参数为表情参数Ba，其样子为Fa，顶点拓扑为第一顶点拓扑Ta，维数为Na，换言之，该表情参数Ba能直接驱动第一表情基Ea。动画形象b的第二表情基Eb，其样子为Fb，顶点拓扑为第二顶点拓扑Tb，维数为第二维数Nb，已有大量用于驱动第一表情基Ea的表情参数Ba。其中，Fa不等于Fb，Ta不等于Tb，Na不等于Nb。
在这种情况下，图6对应的实施例的具体实现方式可以参见图10所示。若目标顶点拓扑为第二顶点拓扑Tb，首先构造顶点拓扑为第二顶点拓扑Tb的调整表情基Ea’，并且保证该调整表情基Ea’可以被第一表情参数Ba驱动。构造调整表情基Ea’的方式可以包括多种，第一种方式可以是通过捏脸算法例如nricp算法，将从第二表情基Eb中确定出的无表情网格贴到第一表情基Ea中每一个表情网格上，得到调整表情基Ea’。第二种方式可以是通过捏脸算法将从第二表情基Eb中确定出的无表情网格贴到第一表情基Ea中的无表情网格上，得到一个顶点拓扑和Eb中网格一样，样子为Fa的无表情网格，然后再根据Ea中各个表情网格相对于无表情网格的形变，将上述得到的样子为Fa、顶点拓扑为Tb的无表情网格做变化，从而得到调整表情基Ea’，保证了贴图时Ea和Eb中顶点对之间的对应关系是唯一的。调整表情基Ea’的样子为Fa，顶点拓扑为第二顶点拓扑Tb，维数为第一维数Na。
由于第一表情参数Ba可以直接驱动第一表情基Ea，而调整表情基Ea’与第一表情基Ea的维数相同，且每一维的语义信息相同，故，可以直接利用第一表情参数Ba驱动调整表情基Ea’，得到目标网格。目标网格的样子为Fa，顶点拓扑为第二顶点拓扑Tb，目标网格记作Mesh c。
为了根据目标网格和目标表情基确定出维数为第二维数Nb的第二表情 参数Bb，需要构造出样子为Fa，顶点拓扑为第二顶点拓扑Tb，维数为Nb的目标表情基。故，从调整表情基Ea’中确定无表情的第一形象对应的无表情网格，根据第一形象对应的无表情网格和第二表情基中的网格形变关系，生成目标表情基Eb’，目标表情基Eb’的样子为Fa，顶点拓扑为第二顶点拓扑Tb，维数为Nb。由此，就可以使用目标表情基Eb’，通过捏表情算法捏出目标网格Mesh c，同时得到了维数为Nb的第二表情参数Bb。
利用已有的大量第一表情参数Ba就可以产生大量对应的第二表情参数Bb。同理可以利用上述公式(7)和(8)的方法确定第一表情参数和第二表情参数的映射关系。
由此，对于任意一组Ba，可以得到对应的Bb＝f*Ba，从而根据表情参数Ba得到表情参数Bb，以便驱动动画形象b。
若图6对应的实施例中第一形象不具有对应的捏脸基，第二形象具有对应的捏脸基，第一表情参数为随机表情参数，则驱动方形象为第二形象，被驱动方形象为第一形象。驱动方形象为动画形象a，被驱动方形象为动画形象b，第一表情参数为随机表情参数Bb，其样子为Fb，顶点拓扑为第一顶点拓扑Tb，维数为Nb，换言之，该随机表情参数Bb能直接驱动第一表情基Eb，第一表情基Eb的样子为Fb，顶点拓扑为第一顶点拓扑Tb，维数为第一维数Nb。动画形象a的第二表情基Ea，其样子为Fa，顶点拓扑为第二顶点拓扑Ta，维数为第二维数Na，已有大量用于驱动第二表情基Ea的表情参数Ba。其中，Fa不等于Fb，Ta不等于Tb，Na不等于Nb。
在这种情况下，图6对应的实施例的具体实现方式可以参见图11所示。若目标顶点拓扑为第二顶点拓扑Ta，首先构造顶点拓扑为第二顶点拓扑Ta的调整表情基Eb’，并且保证该调整表情基Eb’可以被第一表情参数Bb驱动。构造调整表情基Eb’的方式可以包括多种，第一种方式可以是通过捏脸算法例如nricp算法，将从第二表情基Ea中确定出的无表情网格贴到第一表情基Eb中每一个表情网格上，得到调整表情基Eb’。第二种方式可以是通过捏脸算法将从第二表情基Ea中确定出的无表情网格贴到第一表情基Eb中的无表情网格上，得到一个顶点拓扑和Ea中网格一样，样子为Fb的无表情网格，然后再根据Eb中各个表情网格相对于无表情网格的形变，将上述得到的样子为Fb、顶点拓扑为Ta的无表情网格做变化，从而得到调整表情基Eb’，保证了贴图时Ea和Eb中顶点对之间的对应关系是唯一的。调整表情基Eb’的样子为Fb，顶点拓扑为第二顶点拓扑Ta，维数为第一维数Nb。
由于第一表情参数Bb可以直接驱动第一表情基Eb，而调整表情基Eb’与第一表情基Eb的维数相同，且每一维的语义信息相同，故，可以直接利用第一表情参数Bb驱动调整表情基Eb’，得到目标网格。目标网格的样子为Fb，顶点拓扑为第二顶点拓扑Ta，目标网格记作Mesh c。
为了根据目标网格和目标表情基确定出维数为第二维数Na的第二表情 参数Ba’，需要构造出样子为Fb，顶点拓扑为第二顶点拓扑Ta，维数为Na的目标表情基。故，从调整表情基Eb’中确定无表情的第一形象对应的无表情网格，根据第一形象对应的无表情网格和第二表情基中的网格形变关系，生成目标表情基Ea’，目标表情基Ea’的样子为Fb，顶点拓扑为第二顶点拓扑Ta，维数为Na。由此，就可以使用目标表情基Ea’，通过捏表情算法捏出目标网格Mesh c，同时得到了维数为Na的第二表情参数Ba’。
当随机生成大量第一表情参数Bb时，就可以产生大量对应的第二表情参数Ba’。同理可以利用上述公式(7)和(8)的方法确定第一表情参数和第二表情参数的映射关系。
在得到映射关系f后，由于已有大量的Ba，Ba和Ba’对应的表情基的维数都是Na，每一维数的语义信息一样，所以Ba和Ba’可以等价。由此，对于任意一组Ba，可以得到对应的Bb＝f*Ba，从而根据表情参数Ba得到表情参数Bb，以便驱动动画形象b。
前述介绍的第一类方法主要是基于采样得到的已有第一表情参数确定出第二表情参数，进而确定第一表情参数和第二表情参数的映射关系。为了避免由于采样不均而造成第二表情参数分布补全的问题，本申请实施例还提供了第二类方法，第二类方法的实现方式可以是：根据驱动方对象对应的表情基和表情参数，确定驱动方形象对应的第一点云数据；以及，根据被驱动方形象对应的表情基和表情参数，确定被驱动方形象对应的第二点云数据。第一点云数据经过转换可以得到第二点云数据，或者第二点云数据经过转换可以得到第一点云数据，若确定出第一点云数据和第二点云数据的转换参数，便可以根据第一点云数据、第二点云数据和转换参数，确定驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系。其中，转换例如可以包括旋转、平移、缩放等。转换参数用于标识将第二点云数据转换为第一点云数据的转换关系。
接下来，对确定点云数据(例如第一点云数据和第二点云数据)的原理进行介绍。
若无表情的动画形象为mu，表情基为E，表情基E的维数为n，则B是n*1的向量，E是表情基矩阵，则对于给定的n维参数B可以通过公式(9)得到点云数据R：
R＝mu+E*B  (9)
若驱动方形象的表情基E a，驱动方形象的表情参数为B a，无表情的驱动方形象为mu a，被驱动方形象的表情基E b，被驱动方形象的表情参数为B b，无表情的驱动方形象为mu b，则第一点云数据R a＝mu a+E a*B a，第二点云数据R b＝mu b+E b*B b。
可以理解的是，在本实施例可以通过多种方法确定转换参数，例如可以通过最近点迭代算法(Iterative Closest Point，ICP)计算出转换参数，转换参数 可以表示为公式(10)：
trans＝|sR sT|   (10)
其中，trans为转换参数，s表示缩放，R表示旋转，T表示平移。
以利用转换参数将第一点云数据转换成第二点云数据为例，介绍根据第一点云数据、第二点云数据和转换参数，确定驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系。
利用转换参数处理的第一点云数据与第二点云数据完全相同，则利用转换参数处理的第一点云数据与第二点云数据具有如下公式(11)所示的关系：
mu b+E b*B b＝trans*(mu a+E a*B a)   (11)
由于trans主要对表情基起作用，故，公式(11)可以变化为公式(12)：
mu b+E b*B b＝mu a+E c*B c   (12)
其中，Ec为利用trans对表情基E a处理后得到的新表情基。由于trans主要对表情基起作用，并不会影响表情参数，所以Bc＝Ba。同时，在公式(12)的基础上，由于两套表情基长得一摸一样，无表情的形象也一摸一样，故mu b＝mu a。由此公式(12)可以进一步化简为公式(13)：
E b*B b＝E c*B a   (13)
由于本实施例的目的是确定驱动方形象对应的表情参数Ba和被驱动方形象对应的表情参数Bb间的映射关系f，目的是使B b＝f*B a，则结合公式(13)可以得到f＝E b -1*E c。
第二类方法在数学上误差更小，得到的f是一个解析解，而非采样的解，避免了采样造成的分布不全的问题。
另外，该方法中计算结果f只取决于表情基，由于原始的点云数据可能分布不均匀，可以通过降采样mesh，得到均匀的点inliner构成表情基，可以得到的效果。同时也可以控制使用的部分(比如只用嘴巴或者眼睛的点)，这样可以准确的控制动画形象中所需驱动的部分，避免不重要部分造成的干扰(比如脸颊等等)。
在本实施例中，除了可以驱动动画形象的脸部外，还可以驱动动画形象的各种结构。应理解，被驱动的各种结构是动画形象的一个可形变组成部分，以动画形象是人为例，由于人的手、脚等可以发生形变(例如弯曲)，则被驱动的结构可以是手、脚等。
为此，本申请实施例提供一种动画形象驱动方法，参见图12a，所述方法包括：
S1201、获取驱动方形象对应的形变基和被驱动方形象对应的形变基。
其中，驱动方形象具有对应的结构基，被驱动方形象不具有对应的结构基；结构基用于标识所对应的形象的结构特征，形变基用于标识所对应的形象的形变特征。以所对应形象是手为例，结构基可以体现手指长度、手指粗 细、手掌宽度和厚度、手指的位置等结构特征；形变基可以体现手指的弯曲程度等形变特征。当然，若所对应形象是脸部，结构基即为前述实施例提到的捏脸基，形变基即表情基。
S1202、根据驱动方形象对应的形变基和被驱动方形象对应的形变基，确定驱动方形象对应的形变参数和被驱动方形象对应的形变参数间的映射关系。
其中，形变参数用于标识所对应形象的外形变化程度。以所对应形象是手为例，形变参数体现手指的弯曲程度等。当然，若所对应形象是脸部，形变参数即为前述实施例提到的表情参数。
S1203、根据驱动方形象对应的形变参数和映射关系驱动被驱动方形象。
本实施例各个步骤的具体实现方式可以参照图2对应实施例的实现方式，此处不再赘述。
接下来，将结合实际应用场景对本申请实施例提供的基于人工智能的动画形象驱动方法进行介绍。
在该应用场景中，假设动画形象用于拜年，已有动画形象a，动画形象a具有捏脸基，动画形象a的表情基Ea，其样子为Fa，顶点拓扑为Ta，维数为Na；若希望推出新的动画形象b用于拜年，动画形象b的表情基Eb，其样子为Fb，顶点拓扑为Tb，维数为Nb。其中，Fa不等于Fb，Ta不等于Tb，Na不等于Nb。
若动画形象a的表情参数Ba已知，为了加快动画形象b的推出速度，可以将已知表情参数Ba的动画形象a作为驱动方形象，动画形象b作为被驱动方，根据驱动方形象对应的表情基Ea和被驱动方形象对应的表情基Eb，确定驱动方形象对应的表情参数Ba和被驱动方形象对应的表情参数Bb间的映射关系。这样，当已知驱动方形象的表情参数Ba时，不用处理得到动画形象b对应的捏脸基，就可以通过已知表情参数Ba和映射关系确定被驱动方形象对应的表情参数Bb，从而驱动该新动画形象，加快了动画形象b的推出速度。
基于前述实施例提供的方法，本实施例还提供一种基于人工智能的动画形象驱动装置。参见图12b，所述装置包括获取单元1201、确定单元1202和驱动单元1203：
所述获取单元1201，用于获取驱动方形象对应的表情基和被驱动方形象对应的表情基，所述驱动方形象具有对应的捏脸基，所述被驱动方形象不具有对应的捏脸基；
所述确定单元1202，用于根据所述驱动方形象对应的表情基和所述被驱动方形象对应的表情基，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系；
所述驱动单元1203，用于根据所述驱动方形象对应的表情参数和所述映射关系，驱动所述被驱动方形象。
在一种可能的实现方式中，所述驱动方形象为第一形象和第二形象中具有对应的捏脸基的形象，所述被驱动方形象为第一形象和第二形象中不具有对应的捏脸基的形象；所述确定单元1202，用于：
根据第一表情参数和所述第一形象对应的第一表情基确定目标网格；所述第一表情参数为用于驱动表情基的维数为第一维数的表情参数，所述第一表情基的维数为所述第一维数，所述目标网格具有目标顶点拓扑，用于标识做出所述第一表情参数所对应的表情时的所述第一形象；
获取具有所述目标顶点拓扑的所述第一形象对应的目标表情基，所述目标表情基的维数为第二维数；所述目标表情基为根据所述第二形象所对应的第二表情基确定的；
根据所述目标网格和所述目标表情基，确定所述第一形象对应的第二表情参数，所述第二表情参数用于体现所述第一形象相对于所述目标网格的表情变化程度；
根据所述第一表情参数和第二表情参数，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系。
在一种可能的实现方式中，所述目标顶点拓扑为所述第二表情基对应的第二顶点拓扑，所述确定单元1202，还用于：
根据第一表情参数和所述第一形象所对应的第一表情基确定初始网格，所述初始网格具有所述第一表情基对应的第一顶点拓扑；
根据第一顶点拓扑和第二顶点拓扑的对应关系，生成所述目标网格；
以及，用于：
从所述第一表情基中确定无表情的所述第一形象对应的无表情网格，并从所述第二表情基中确定无表情的所述第二形象对应的无表情网格；
根据所述第一形象对应的无表情网格和所述第二形象对应的无表情网格，确定调整网格，所述调整网格具有所述第二顶点拓扑，用于标识处于无表情时的第一形象；
根据所述调整网格和所述第二表情基中的网格形变关系，生成所述目标表情基。
在一种可能的实现方式中，所述目标顶点拓扑为所述第二表情基对应的第二顶点拓扑，所述确定单元1202，还用于：
从第二形象对应的第二表情中基确定无表情的所述第二形象对应的无表情网格；
根据所述第二形象对应的无表情网格和所述第一形象所对应的第一表情基，确定具有第二顶点拓扑的所述第一形象对应的调整表情基，所述调整表情基的维数为第一维数；
根据所述第一表情参数和所述调整表情基，确定所述目标网格；
以及，用于：
从所述调整表情基中确定无表情的所述第一形象对应的无表情网格；
根据所述第一形象对应的无表情网格和所述第二表情基中的网格形变关系，生成所述目标表情基。
在一种可能的实现方式中，所述第一形象不具有对应的捏脸基，所述第二形象具有对应的捏脸基；所述第一表情参数为随机表情参数；或者，
所述第一形象具有对应的捏脸基，所述第二形象不具有对应的捏脸基。
在一种可能的实现方式中，所述确定单元1202，还用于：
获取多对所述第一表情参数和所述第二表情参数；
根据由多个所述第一表情参数构成的第一矩阵和由多个所述第二表情参数构成的第二矩阵，确定所述映射关系。
在一种可能的实现方式中，所述确定单元1202，用于：
根据所述驱动方对象对应的表情基和表情参数，确定所述驱动方形象对应的第一点云数据；
根据所述被驱动方形象对应的表情基和表情参数，确定所述被驱动方形象对应的第二点云数据；
确定所述第一点云数据和第二点云数据的转换参数，所述转换参数用于标识所述第二点云数据与所述第一点云数据之间的转换关系；
根据所述第一点云数据、所述第二点云数据和所述转换参数，确定所述驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系。
本实施例还提供一种基于人工智能的动画形象驱动装置。参见图13a，所述装置包括获取单元1301、确定单元1302和驱动单元1303：
所述获取单元1301，用于获取驱动方形象对应的形变基和被驱动方形象对应的形变基，所述驱动方形象具有对应的结构基，所述被驱动方形象不具有对应的结构基；所述结构基用于标识所对应的形象的结构特征，所述形变基用于标识所对应的形象的形变特征；
所述确定单元1302，用于根据所述驱动方形象对应的形变基和所述被驱动方形象对应的形变基，确定所述驱动方形象对应的形变参数和所述被驱动方形象对应的形变参数间的映射关系；
所述驱动单元1303，用于根据所述驱动方形象对应的形变参数和所述映射关系驱动所述被驱动方形象。
本申请实施例还提供了一种设备，该设备可以基于人工智能驱动动画形象。下面结合附图对该设备进行介绍。请参见图13b所示，本申请实施例提供了一种的设备1300，该设备1300还可以是终端设备，该终端设备可以为包括手机、平板电脑、个人数字助理(Personal Digital Assistant，简称PDA)、销售终端(Point of Sales，简称POS)、车载电脑等任意智能终端，以终端设 备为手机为例：
图13b示出的是与本申请实施例提供的终端设备相关的手机的部分结构的框图。参考图13b，手机包括：射频(Radio Frequency，简称RF)电路1310、存储器1320、输入单元1330、显示单元1340、传感器1350、音频电路1360、无线保真(wireless fidelity，简称WiFi)模块1370、处理器1380、以及电源1390等部件。本领域技术人员可以理解，图13b中示出的手机结构并不构成对手机的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
存储器1320可用于存储软件程序以及模块，处理器1380通过运行存储在存储器1320的软件程序以及模块，从而执行手机的各种功能应用以及数据处理。存储器1320可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器1320可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
处理器1380是手机的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在存储器1320内的软件程序和/或模块，以及调用存储在存储器1320内的数据，执行手机的各种功能和处理数据，从而对手机进行整体监控。可选的，处理器1380可包括一个或多个处理单元；优选的，处理器1380可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器1380中。
在本实施例中，该终端设备所包括的处理器1380还具有以下功能：
获取驱动方形象对应的表情基和被驱动方形象对应的表情基，所述驱动方形象具有对应的捏脸基，所述被驱动方形象不具有对应的捏脸基；
根据所述驱动方形象对应的表情基和所述被驱动方形象对应的表情基，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系；
根据所述驱动方形象对应的表情参数和所述映射关系，驱动所述被驱动方形象。
或，
获取驱动方形象对应的形变基和被驱动方形象对应的形变基，所述驱动方形象具有对应的结构基，所述被驱动方形象不具有对应的结构基；所述结构基用于标识所对应的形象的结构特征，所述形变基用于标识所对应的形象的形变特征；
根据所述驱动方形象对应的形变基和所述被驱动方形象对应的形变基， 确定所述驱动方形象对应的形变参数和所述被驱动方形象对应的形变参数间的映射关系；
根据所述驱动方形象对应的形变参数和所述映射关系，驱动所述被驱动方形象。
本申请实施例还提供服务器，请参见图14所示，图14为本申请实施例提供的服务器1400的结构图，服务器1400可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上中央处理器(Central Processing Units，简称CPU)1422(例如，一个或一个以上处理器)和存储器1432，一个或一个以上存储应用程序1442或数据1444的存储介质1430(例如一个或一个以上海量存储设备)。其中，存储器1432和存储介质1430可以是短暂存储或持久存储。存储在存储介质1430的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对服务器中的一系列指令操作。更进一步地，中央处理器1422可以设置为与存储介质1430通信，在服务器1400上执行存储介质1430中的一系列指令操作。
服务器1400还可以包括一个或一个以上电源1426，一个或一个以上有线或无线网络接口1450，一个或一个以上输入输出接口1458，和/或，一个或一个以上操作系统1441，例如Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTM等等。
上述实施例中由服务器所执行的步骤可以基于该图14所示的服务器结构实现。
本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质用于存储程序代码，所述程序代码用于执行前述各个实施例所述的基于人工智能的动画形象驱动方法。
本申请实施例还提供一种包括指令的计算机程序产品，当其在计算机上运行时，使得计算机执行前述各个实施例所述的基于人工智能的动画形象驱动方法。
以上所述，以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (16)

  1. 一种动画形象驱动方法，由处理设备执行，所述方法包括：
    获取驱动方形象对应的表情基和被驱动方形象对应的表情基，所述驱动方形象具有对应的捏脸基，所述被驱动方形象不具有对应的捏脸基；
    根据所述驱动方形象对应的表情基和所述被驱动方形象对应的表情基，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系；
    根据所述驱动方形象对应的表情参数和所述映射关系，驱动所述被驱动方形象。
  2. 根据权利要求1所述的方法，所述驱动方形象为第一形象和第二形象中具有对应的捏脸基的形象，所述被驱动方形象为所述第一形象和所述第二形象中不具有对应的捏脸基的形象；
    所述根据所述驱动方形象对应的表情基和所述被驱动方形象对应的表情基，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系，包括：
    根据第一表情参数和所述第一形象对应的第一表情基确定目标网格；所述第一表情参数为用于驱动表情基的维数为第一维数的表情参数，所述第一表情基的维数为所述第一维数，所述目标网格具有目标顶点拓扑，用于标识做出所述第一表情参数所对应的表情时的所述第一形象；
    获取具有所述目标顶点拓扑的所述第一形象对应的目标表情基，所述目标表情基的维数为第二维数；所述目标表情基为根据所述第二形象所对应的第二表情基确定的；
    根据所述目标网格和所述目标表情基，确定所述第一形象对应的第二表情参数，所述第二表情参数用于体现所述第一形象相对于所述目标网格的表情变化程度；
    根据所述第一表情参数和所述第二表情参数，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系。
  3. 根据权利要求2所述的方法，所述目标顶点拓扑为所述第二表情基对应的第二顶点拓扑，所述根据第一表情参数和所述第一形象对应的第一表情基确定目标网格，包括：
    根据第一表情参数和所述第一形象所对应的第一表情基确定初始网格，所述初始网格具有所述第一表情基对应的第一顶点拓扑；
    根据所述第一顶点拓扑和所述第二顶点拓扑的对应关系，生成所述目标网格；
    所述获取具有所述目标顶点拓扑的所述第一形象对应的目标表情基，包括：
    从所述第一表情基中确定无表情的所述第一形象对应的无表情网格，并 从所述第二表情基中确定无表情的所述第二形象对应的无表情网格；
    根据所述第一形象对应的无表情网格和所述第二形象对应的无表情网格，确定调整网格，所述调整网格具有所述第二顶点拓扑，用于标识处于无表情时的第一形象；
    根据所述调整网格和所述第二表情基中的网格形变关系，生成所述目标表情基。
  4. 根据权利要求2所述的方法，所述目标顶点拓扑为所述第二表情基对应的第二顶点拓扑，所述根据第一表情参数和所述第一形象对应的第一表情基确定目标网格，包括：
    从所述第二形象对应的第二表情基中确定无表情的所述第二形象对应的无表情网格；
    根据所述第二形象对应的无表情网格和所述第一形象所对应的第一表情基，确定具有所述第二顶点拓扑的所述第一形象对应的调整表情基，所述调整表情基的维数为第一维数；
    根据所述第一表情参数和所述调整表情基，确定所述目标网格；
    所述获取具有所述目标顶点拓扑的所述第一形象对应的目标表情基，包括：
    从所述调整表情基中确定无表情的所述第一形象对应的无表情网格；
    根据所述第一形象对应的无表情网格和所述第二表情基中的网格形变关系，生成所述目标表情基。
  5. 根据权利要求3或4所述的方法，所述第一形象不具有对应的捏脸基，所述第二形象具有对应的捏脸基；所述第一表情参数为随机表情参数；或者，
    所述第一形象具有对应的捏脸基，所述第二形象不具有对应的捏脸基。
  6. 根据权利要求2-4任意一项所述的方法，所述根据所述第一表情参数和第二表情参数，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系，包括：
    获取多对所述第一表情参数和所述第二表情参数；
    根据由多个所述第一表情参数构成的第一矩阵和由多个所述第二表情参数构成的第二矩阵，确定所述映射关系。
  7. 根据权利要求1所述的方法，所述根据所述驱动方形象对应的表情基和所述被驱动方形象对应的表情基，确定所述驱动方形象对应的表情参数和被驱动方形象对应的表情参数间的映射关系，包括：
    根据所述驱动方对象对应的表情基和表情参数，确定所述驱动方形象对应的第一点云数据；
    根据所述被驱动方形象对应的表情基和表情参数，确定所述被驱动方形象对应的第二点云数据；
    确定所述第一点云数据和所述第二点云数据的转换参数，所述转换参数用于标识所述第二点云数据与所述第一点云数据之间的转换关系；
    根据所述第一点云数据、所述第二点云数据和所述转换参数，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系。
  8. 一种动画形象驱动方法，由处理设备执行，所述方法包括：
    获取驱动方形象对应的形变基和被驱动方形象对应的形变基，所述驱动方形象具有对应的结构基，所述被驱动方形象不具有对应的结构基；所述结构基用于标识所对应的形象的结构特征，所述形变基用于标识所对应的形象的形变特征；
    根据所述驱动方形象对应的形变基和所述被驱动方形象对应的形变基，确定所述驱动方形象对应的形变参数和所述被驱动方形象对应的形变参数间的映射关系；
    根据所述驱动方形象对应的形变参数和所述映射关系，驱动所述被驱动方形象。
  9. 一种动画形象驱动装置，所述装置包括获取单元、确定单元和驱动单元：
    所述获取单元，用于获取驱动方形象对应的表情基和被驱动方形象对应的表情基，所述驱动方形象具有对应的捏脸基，所述被驱动方形象不具有对应的捏脸基；
    所述确定单元，用于根据所述驱动方形象对应的表情基和所述被驱动方形象对应的表情基，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系；
    所述驱动单元，用于根据所述驱动方形象对应的表情参数和所述映射关系，驱动所述被驱动方形象。
  10. 根据权利要求9所述的装置，所述驱动方形象为第一形象和第二形象中具有对应的捏脸基的形象，所述被驱动方形象为所述第一形象和所述第二形象中不具有对应的捏脸基的形象；所述确定单元，用于：
    根据第一表情参数和所述第一形象对应的第一表情基确定目标网格；所述第一表情参数为用于驱动表情基的维数为第一维数的表情参数，所述第一表情基的维数为所述第一维数，所述目标网格具有目标顶点拓扑，用于标识做出所述第一表情参数所对应的表情时的所述第一形象；
    获取具有所述目标顶点拓扑的所述第一形象对应的目标表情基，所述目标表情基的维数为第二维数；所述目标表情基为根据所述第二形象所对应的第二表情基确定的；
    根据所述目标网格和所述目标表情基，确定所述第一形象对应的第二表情参数，所述第二表情参数用于体现所述第一形象相对于所述目标网格的表 情变化程度；
    根据所述第一表情参数和所述第二表情参数，确定所述驱动方形象对应的表情参数和所述被驱动方形象对应的表情参数间的映射关系。
  11. 根据权利要求10所述的装置，所述目标顶点拓扑为所述第二表情基对应的第二顶点拓扑，所述确定单元，还用于：
    根据第一表情参数和所述第一形象所对应的第一表情基确定初始网格，所述初始网格具有所述第一表情基对应的第一顶点拓扑；
    根据所述第一顶点拓扑和所述第二顶点拓扑的对应关系，生成所述目标网格；
    以及，用于：
    从所述第一表情基中确定无表情的所述第一形象对应的无表情网格，并从所述第二表情基中确定无表情的所述第二形象对应的无表情网格；
    根据所述第一形象对应的无表情网格和所述第二形象对应的无表情网格，确定调整网格，所述调整网格具有所述第二顶点拓扑，用于标识处于无表情时的第一形象；
    根据所述调整网格和所述第二表情基中的网格形变关系，生成所述目标表情基。
  12. 根据权利要求10所述的装置，所述目标顶点拓扑为所述第二表情基对应的第二顶点拓扑，所述确定单元，还用于：
    从所述第二形象对应的第二表情中基确定无表情的所述第二形象对应的无表情网格；
    根据所述第二形象对应的无表情网格和所述第一形象所对应的第一表情基，确定具有所述第二顶点拓扑的所述第一形象对应的调整表情基，所述调整表情基的维数为第一维数；
    根据所述第一表情参数和所述调整表情基，确定所述目标网格；
    以及，用于：
    从所述调整表情基中确定无表情的所述第一形象对应的无表情网格；
    根据所述第一形象对应的无表情网格和所述第二表情基中的网格形变关系，生成所述目标表情基。
  13. 一种动画形象驱动装置，所述装置包括获取单元、确定单元和驱动单元：
    所述获取单元，用于获取驱动方形象对应的形变基和被驱动方形象对应的形变基，所述驱动方形象具有对应的结构基，所述被驱动方形象不具有对应的结构基；所述结构基用于标识所对应的形象的结构特征，所述形变基用于标识所对应的形象的形变特征；
    所述确定单元，用于根据所述驱动方形象对应的形变基和所述被驱动方形象对应的形变基，确定所述驱动方形象对应的形变参数和所述被驱动方形 象对应的形变参数间的映射关系；
    所述驱动单元，用于根据所述驱动方形象对应的形变参数和所述映射关系，驱动所述被驱动方形象。
  14. 一种设备，所述设备包括处理器以及存储器：
    所述存储器用于存储程序代码，并将所述程序代码传输给所述处理器；
    所述处理器用于根据所述程序代码中的指令执行权利要求1-8任一项所述的方法。
  15. 一种计算机可读存储介质，所述计算机可读存储介质用于存储程序代码，所述程序代码用于执行权利要求1-8任一项所述的方法。
  16. 一种计算机程序产品，包括指令，当其在计算机上运行时，使得计算机执行如权利要求1-8中任一项所述的方法。
PCT/CN2020/105673 2019-08-30 2020-07-30 一种基于人工智能的动画形象驱动方法和装置 WO2021036665A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP20857359.2A EP3923244A4 (en) 2019-08-30 2020-07-30 METHOD AND DEVICE FOR CONTROLLING ANIMATED IMAGES BASED ON ARTIFICIAL INTELLIGENCE
KR1020217029446A KR102645506B1 (ko) 2019-08-30 2020-07-30 인공 지능에 기반한 애니메이션 이미지 구동 방법 및 장치
JP2021560989A JP7307194B2 (ja) 2019-08-30 2020-07-30 動画キャラクターの駆動方法並びにその、装置、機器及びコンピュータプログラム
US17/486,641 US11941737B2 (en) 2019-08-30 2021-09-27 Artificial intelligence-based animation character control and drive method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910816780.XA CN110517339B (zh) 2019-08-30 2019-08-30 一种基于人工智能的动画形象驱动方法和装置
CN201910816780.X 2019-08-30

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/486,641 Continuation US11941737B2 (en) 2019-08-30 2021-09-27 Artificial intelligence-based animation character control and drive method and apparatus

Publications (1)

Publication Number Publication Date
WO2021036665A1 true WO2021036665A1 (zh) 2021-03-04

Family

ID=68629598

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/105673 WO2021036665A1 (zh) 2019-08-30 2020-07-30 一种基于人工智能的动画形象驱动方法和装置

Country Status (6)

Country Link
US (1) US11941737B2 (zh)
EP (1) EP3923244A4 (zh)
JP (1) JP7307194B2 (zh)
KR (1) KR102645506B1 (zh)
CN (1) CN110517339B (zh)
WO (1) WO2021036665A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110517339B (zh) * 2019-08-30 2021-05-25 腾讯科技（深圳）有限公司 一种基于人工智能的动画形象驱动方法和装置
CN113792705B (zh) * 2021-09-30 2024-04-23 北京跳悦智能科技有限公司 一种视频表情迁移方法及系统、计算机设备
CN114708636A (zh) * 2022-04-01 2022-07-05 成都市谛视科技有限公司 一种密集人脸网格表情驱动方法、装置及介质
CN114693845A (zh) * 2022-04-08 2022-07-01 Oppo广东移动通信有限公司 风格化对象的驱动方法及装置、介质和电子设备
CN115393488B (zh) * 2022-10-28 2023-03-03 北京百度网讯科技有限公司 虚拟人物表情的驱动方法、装置、电子设备和存储介质
KR102636217B1 (ko) * 2023-04-14 2024-02-14 고려대학교산학협력단 가중 국소변환을 이용한 3차원 데이터 증강 방법 및 이를 위한 장치
CN116485959B (zh) * 2023-04-17 2024-07-23 北京优酷科技有限公司 动画模型的控制方法、表情的添加方法及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105528805A (zh) * 2015-12-25 2016-04-27 苏州丽多数字科技有限公司 一种虚拟人脸动画合成方法
CN109961496A (zh) * 2019-02-22 2019-07-02 厦门美图之家科技有限公司 表情驱动方法及表情驱动装置
CN110517339A (zh) * 2019-08-30 2019-11-29 腾讯科技（深圳）有限公司 一种基于人工智能的动画形象驱动方法和装置

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001043345A (ja) * 1999-07-28 2001-02-16 Mitsubishi Electric Corp 表情認識装置、およびそれを用いた投薬制御システム、覚醒レベル評価システム、回復評価システム
US6950104B1 (en) 2000-08-30 2005-09-27 Microsoft Corporation Methods and systems for animating facial features, and methods and systems for expression transformation
JP2002298155A (ja) 2001-03-29 2002-10-11 Hic:Kk 感情による３ｄコンピュータグラフィックス表情モデル形成システム
US9747495B2 (en) * 2012-03-06 2017-08-29 Adobe Systems Incorporated Systems and methods for creating and distributing modifiable animated video messages
US9799096B1 (en) * 2014-07-08 2017-10-24 Carnegie Mellon University System and method for processing video to provide facial de-identification
CN104217454B (zh) * 2014-08-21 2017-11-03 中国科学院计算技术研究所 一种视频驱动的人脸动画生成方法
FR3053509B1 (fr) * 2016-06-30 2019-08-16 Fittingbox Procede d’occultation d’un objet dans une image ou une video et procede de realite augmentee associe
CN107633542A (zh) * 2016-07-19 2018-01-26 珠海金山网络游戏科技有限公司 一种捏脸编辑和动画附加融合方法和系统
CN106709975B (zh) * 2017-01-11 2017-12-22 山东财经大学 一种交互式三维人脸表情动画编辑方法、系统及扩展方法
CN108874114B (zh) * 2017-05-08 2021-08-03 腾讯科技（深圳）有限公司 实现虚拟对象情绪表达的方法、装置、计算机设备及存储介质
CN108876879B (zh) * 2017-05-12 2022-06-14 腾讯科技（深圳）有限公司 人脸动画实现的方法、装置、计算机设备及存储介质
US10860838B1 (en) * 2018-01-16 2020-12-08 Electronic Arts Inc. Universal facial expression translation and character rendering system
JP7342008B2 (ja) * 2018-02-27 2023-09-11 マジック リープ， インコーポレイテッド 仮想アバタのためのメッシュの合致
US10198845B1 (en) * 2018-05-29 2019-02-05 LoomAi, Inc. Methods and systems for animating facial expressions
US10896535B2 (en) * 2018-08-13 2021-01-19 Pinscreen, Inc. Real-time avatars using dynamic textures
US11151362B2 (en) * 2018-08-30 2021-10-19 FaceValue B.V. System and method for first impression analysis and face morphing by adjusting facial landmarks using faces scored for plural perceptive traits
CN109493403A (zh) * 2018-11-13 2019-03-19 北京中科嘉宁科技有限公司 一种基于运动单元表情映射实现人脸动画的方法
US11475608B2 (en) * 2019-09-26 2022-10-18 Apple Inc. Face image generation with pose and expression control
US11393149B2 (en) * 2020-07-02 2022-07-19 Unity Technologies Sf Generating an animation rig for use in animating a computer-generated character based on facial scans of an actor and a muscle model
US11977979B2 (en) * 2021-07-23 2024-05-07 Qualcomm Incorporated Adaptive bounding for three-dimensional morphable models

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105528805A (zh) * 2015-12-25 2016-04-27 苏州丽多数字科技有限公司 一种虚拟人脸动画合成方法
CN109961496A (zh) * 2019-02-22 2019-07-02 厦门美图之家科技有限公司 表情驱动方法及表情驱动装置
CN110517339A (zh) * 2019-08-30 2019-11-29 腾讯科技（深圳）有限公司 一种基于人工智能的动画形象驱动方法和装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3923244A4

Also Published As

Publication number Publication date
KR20210126697A (ko) 2021-10-20
KR102645506B1 (ko) 2024-03-07
CN110517339A (zh) 2019-11-29
CN110517339B (zh) 2021-05-25
US11941737B2 (en) 2024-03-26
JP2022528999A (ja) 2022-06-16
EP3923244A1 (en) 2021-12-15
US20220012930A1 (en) 2022-01-13
EP3923244A4 (en) 2022-08-24
JP7307194B2 (ja) 2023-07-11

Similar Documents

Publication Publication Date Title
WO2021036665A1 (zh) 一种基于人工智能的动画形象驱动方法和装置
WO2020192568A1 (zh) 人脸图像生成方法、装置、设备及存储介质
KR102616010B1 (ko) 포토리얼리스틱 실시간 인물 애니메이션을 위한 시스템 및 방법
CN114820905B (zh) 虚拟形象生成方法、装置、电子设备及可读存储介质
CN111047509B (zh) 一种图像特效处理方法、装置及终端
WO2023035531A1 (zh) 文本图像超分辨率重建方法及其相关设备
KR20210113948A (ko) 가상 아바타 생성 방법 및 장치
CN115049799A (zh) 3d模型和虚拟形象的生成方法和装置
JP7418370B2 (ja) 髪型を変換するための方法、装置、デバイス及び記憶媒体
US20180276870A1 (en) System and method for mass-animating characters in animated sequences
CN114202615A (zh) 人脸表情的重建方法、装置、设备和存储介质
CN110211017A (zh) 图像处理方法、装置及电子设备
CN111507259B (zh) 脸部特征提取方法、装置、电子设备
US20230316591A1 (en) Identity Preserved Controllable Facial Image Manipulation
CN117974890A (zh) 人脸图像的处理方法、装置、直播系统、电子设备及介质
WO2023130819A9 (zh) 图像处理方法、装置、设备、存储介质及计算机程序
WO2022253094A1 (zh) 图像生成方法、装置、设备和介质
US20220198828A1 (en) Method and apparatus for generating image
CN113327311B (zh) 基于虚拟角色的显示方法、装置、设备、存储介质
CN114638919A (zh) 虚拟形象的生成方法、电子设备、程序产品及用户终端
CN111311712B (zh) 视频帧处理方法和装置
CN116030150B (zh) 虚拟形象生成方法、装置、电子设备和介质
JPWO2019224947A1 (ja) 学習装置、画像生成装置、学習方法、画像生成方法及びプログラム
US20240212282A1 (en) Image rendering method and apparatus, device, and medium
CN115830219A (zh) 一种捏脸参数生成、模型训练方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20857359

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20217029446

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2020857359

Country of ref document: EP

Effective date: 20210908

ENP Entry into the national phase

Ref document number: 2021560989

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE