WO2020134436A1 - 生成动画表情的方法和电子设备 - Google Patents

生成动画表情的方法和电子设备 Download PDF

Info

Publication number
WO2020134436A1
WO2020134436A1 PCT/CN2019/113041 CN2019113041W WO2020134436A1 WO 2020134436 A1 WO2020134436 A1 WO 2020134436A1 CN 2019113041 W CN2019113041 W CN 2019113041W WO 2020134436 A1 WO2020134436 A1 WO 2020134436A1
Authority
WO
WIPO (PCT)
Prior art keywords
expression
expression base
grid
dimensional grid
personalized
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/113041
Other languages
English (en)
French (fr)
Inventor
魏源
白蔚
吴日辉
张举勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
University of Science and Technology of China USTC
Original Assignee
Huawei Technologies Co Ltd
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd, University of Science and Technology of China USTC filed Critical Huawei Technologies Co Ltd
Publication of WO2020134436A1 publication Critical patent/WO2020134436A1/zh
Priority to US17/360,866 priority Critical patent/US11410369B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/40—Three-dimensional [3D] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition

Definitions

  • the present application relates to the field of image processing technology, and more specifically, to a method and electronic device for generating animated expressions.
  • an electronic device When generating an animated expression, an electronic device generally shoots a face video through a camera and extracts the motion information of the dense (or sparse) feature points of the face, and then migrates the motion information of the extracted dense (or sparse) feature points to the virtual Characters (or anthropomorphic animals) faces, to get animated expressions.
  • the specific process of generating animated expressions in the traditional scheme can be as follows: first, extract feature points or point clouds (a series of points containing facial features) representing facial expressions; second, according to the extracted feature points or point clouds, A variety of pre-stored expression bases of different identity characteristics are combined into a personalized expression base that matches the identity characteristics of the human face, and finally an animated expression is generated according to the personalized expression base.
  • the present application provides a method and electronic device for generating an animated expression, which can enhance the expression effect of the animated expression.
  • a method for generating an animated expression includes: obtaining an initial three-dimensional grid; transforming the initial three-dimensional grid to obtain a target three-dimensional grid; and determining a personalized match to a human face based on a basic expression base Expression base; determine the personalized expression base coefficient according to the target three-dimensional grid and personalized expression base, where the superposition of the personalized expression base coefficient and the personalized expression base is used to represent the facial expression characteristics; according to the personalized expression base coefficient Generate animated emoticons.
  • the vertices in the initial three-dimensional grid are used to express facial expression features.
  • the topological structure of the target three-dimensional grid is the same as the topological structure of the basic expression base.
  • a target three-dimensional network with the same topological structure as the basic expression base can be obtained, which realizes the unification of the topological structure and facilitates subsequent animation expressions based on the target three-dimensional grid.
  • the personalized expression base matching the human face may be an expression base containing the identity characteristics of the human face.
  • the topology structure of the personalized expression base is the same as the topology structure of the basic expression base.
  • the personalized expression base may include expression bases corresponding to various expressions.
  • the personalized expression base may include expression bases that are common to human faces (for example, 47 common expressions).
  • the animation expression may specifically be a continuous animation expression corresponding to an animation video, or a static animation expression corresponding to a frame of video image.
  • the animation expression finally generated is the animation expression corresponding to one frame of image.
  • generating the animated expression based on the personalized expression base coefficient includes: generating the animated expression based on the personalized expression base coefficient and the anthropomorphic character expression base.
  • the above-mentioned generating the animated expression based on the personalized expression base coefficient and the anthropomorphic character expression base may refer to transferring the personalized expression base coefficient to the anthropomorphic character expression base to generate the animated expression.
  • the anthropomorphic character expression base may be a virtual character expression base or a anthropomorphic animal expression base.
  • the three-dimensional grid can better reflect the facial expression characteristics.
  • the converted three-dimensional grid and the basic The topological structure of the expression base remains the same, so that finally, an animated expression closer to the expression characteristics of the human face can be generated according to the transformed three-dimensional grid.
  • the animation expression can be made more realistic, and the display effect of the animation expression can be improved.
  • the personalized expression base includes multiple expression bases, which correspond to different expressions, and are determined according to the target three-dimensional grid and the personalized expression base.
  • the personalized expression base coefficient includes: determining the personalized expression base coefficient according to the difference between the linear combination of the coordinates of the reference vertices in various expression bases and the coordinates of the vertices in the target three-dimensional grid.
  • the reference vertex of each expression base in the above-mentioned multiple expression bases is a point in each expression base that is at a position corresponding to the vertex in the target three-dimensional grid.
  • the personalized expression base coefficient is determined according to the difference between the linear combination of the coordinates of the reference vertices in multiple expression bases and the coordinates of the vertices in the target three-dimensional grid.
  • the method includes: determining a plurality of first difference values; and determining a personalized expression base coefficient according to a sum of products of the plurality of first difference values and corresponding weight values.
  • the multiple first difference values are linear combination differences of coordinates of multiple vertices in the target three-dimensional grid and corresponding reference vertices in multiple expression bases, respectively.
  • determining the personalized expression base coefficient according to the sum of the products of the plurality of first differences and the corresponding weight values respectively includes: when the sum of the products of the plurality of first differences and the corresponding weight values takes the minimum value, The linear combination coefficient when the coordinates of the reference vertices in multiple expression bases are linearly combined is determined as the personalized expression base coefficient.
  • the personalized expression base coefficient can be made to reflect the personalized expression base coefficient as realistically as possible
  • the facial expression features facilitate subsequent generation of animated expressions that are closer to the facial expression features.
  • multiple first difference values respectively correspond to multiple different weight values.
  • the multiple first difference values respectively correspond to multiple vertices in the target three-dimensional grid, wherein the weight value of the first difference value corresponding to the vertices located in the first preset area in the target three-dimensional grid is greater than Or it is equal to the first preset weight value, and the weight of the first difference value corresponding to the vertex outside the first preset region in the target three-dimensional grid is less than the first preset weight value.
  • the above-mentioned pre-set area may be an important area in the target grid, and the first preset area may be an area where some important organs of the human face are located.
  • the first preset area may be an area where the human eye is located, or the first preset area may be an area where the mouth is located, or the first preset area may be an area where the human eye and mouth are located.
  • the first preset area may be determined according to simulation or simulation results, or the first preset area may also be directly determined based on experience.
  • multiple first difference values correspond to different weight values, which can fully consider the importance of the first difference value corresponding to each vertex when calculating the personalized expression base coefficients, so as to solve the personalized expression base more accurately coefficient.
  • the personalized expression base includes multiple expression bases, and the multiple expression bases correspond to different expressions, and the personality is determined according to the target three-dimensional grid and the personalized expression base.
  • the expression base coefficients include: determining the vertex distance of the target 3D mesh; determining the reference distance corresponding to each expression base in multiple expression bases to obtain multiple reference distances; according to the linear combination of multiple reference distances and the vertex distance Poor, determine the personalized expression base coefficient.
  • vertex distance is the distance between two vertices in the target 3D mesh
  • the reference distance corresponding to each expression base is the distance between two reference vertices in each expression base. Point corresponding to position
  • determining the personalized expression base coefficient according to the difference between the linear combination of multiple reference distances and the vertex distance includes: determining multiple second difference values; The sum of the products of the second difference values and the corresponding weight values determines the personalized expression base coefficient.
  • the plurality of second difference values are difference values of linear combinations of the distances of the vertices in the target three-dimensional grid and the corresponding reference distances in the various expression bases.
  • determining the personalized expression base coefficient according to the sum of the products of the plurality of second difference values and the corresponding weight values includes: when the sum of the products of the plurality of second difference values and the corresponding weight values takes the minimum value, The linear combination coefficient when multiple reference distances are linearly combined is determined as the personalized expression base coefficient.
  • the personalized expression base coefficient can be made to reflect as realistically as possible
  • the facial expression features facilitate subsequent generation of animated expressions that are closer to the facial expression features.
  • the personalized expression base coefficient may be solved according to the first difference or the second difference alone, or the first difference and the second difference may be combined to jointly solve the personalized expression base coefficient .
  • multiple second difference values respectively correspond to multiple different weight values.
  • the plurality of second difference values respectively correspond to a plurality of vertex distances in the target three-dimensional grid, wherein the vertex distance between the vertices located in the second preset area in the target three-dimensional grid corresponds to the second
  • the weight value of the difference value is greater than or equal to the second preset weight value, and the weight of the second difference value corresponding to the vertex distance between the vertices outside the second preset region in the target three-dimensional mesh is less than the second preset weight value.
  • the above second preset area may be an important area in the target grid, and the second preset area may be an area where some important organs of the human face are located.
  • the above-mentioned second preset area may be the area where the human eye is located.
  • the weight value of the second difference value corresponding to the vertex distance between the vertices corresponding to the upper eyelid and the lower eyelid may be greater than the second preset weight value .
  • the second preset area may be an area where the mouth is located. In this case, the weight value of the second difference value corresponding to the vertex distance between the vertices corresponding to the upper and lower lips may be greater than the second preset weight value.
  • the second preset area may be determined according to actual simulation or simulation results, or the second preset area may also be determined based on experience.
  • multiple second difference values correspond to different weight values, which can fully take into account the importance of the corresponding second difference values of different vertex distances when calculating the personalized expression base coefficient, so as to solve the personality more accurately Expression base coefficient.
  • the initial three-dimensional grid is transformed to obtain the target three-dimensional grid, including: determining a topological reference grid; performing rigid deformation on the topological reference grid, Obtain the topological reference grid after rigid deformation; the topological reference grid after rigid deformation and the initial three-dimensional mesh are processed until the degree of bonding between the topological reference grid after rigid deformation and the initial three-dimensional grid meets the preset Fitting degree; replace the coordinates of the vertices in the topological reference grid after rigid deformation with the coordinates of the vertices in the initial three-dimensional grid that match the vertices in the rigid deformed grid to obtain the target three-dimensional grid.
  • the topology structure of the above-mentioned topological reference grid is the same as the topology structure of the basic expression base.
  • the topological reference grid can be used to transform the initial three-dimensional grid; the size of the topological reference grid after rigid deformation and the initial three-dimensional grid The size of the grid is the same; the target 3D grid finally obtained by the above process has the topology structure of the topological reference grid and the shape of the initial 3D grid.
  • rigidly deforming the topological reference grid includes: rotating, translating, or scaling the topological reference grid.
  • the orientation of the topological reference grid after rigid deformation may also be the same as the orientation of the initial three-dimensional grid.
  • the degree of fit between the topological reference grid after rigid deformation and the initial three-dimensional grid meets the preset degree of fit, which may refer to the corresponding reference point in the initial three-dimensional grid and the topological reference grid after rigid deformation
  • the distance between vertices is less than a preset distance.
  • the topological reference grid after rigid deformation and the initial three-dimensional grid are subjected to lamination processing until the topological reference grid after rigid deformation and the initial three-dimensional grid
  • the fitting degree meets the preset fitting degree, including: repeating steps A and B until the non-rigid deformed topological reference mesh and initial three-dimensional mesh fit the preset fitting degree ;
  • step A and step B are:
  • Step A Perform non-rigid deformation on the topological reference grid after rigid deformation according to the radial basis function RBF to obtain a topological reference grid after non-rigid change;
  • Step B Fit the non-rigid deformed topological reference grid to the initial three-dimensional grid.
  • non-rigid deformation may mean that the entire grid cannot be uniformly deformed by a single rotation, translation, or scaling, but by different rotations, translations, or scaling of different areas inside the grid (grid Different areas of the grid have different deformation methods.) During non-rigid deformation, different areas within the grid may also undergo relative motion.
  • the degree of fitting between the non-rigid deformed topological reference grid and the initial three-dimensional grid can meet the fitting requirements, so that the target three-dimensional obtained after fitting
  • the grid has both the topology structure of the topological reference grid and the shape of the initial three-dimensional grid, which facilitates the subsequent generation of animated expressions that are closer to the facial expression characteristics.
  • the above-mentioned basic expression base is composed of multiple sets of expression bases with different identity characteristics
  • the above-mentioned determining the personalized expression base matching the human face based on the basic expression base includes: determining personalized expressions based on multiple sets of expression bases with different identity characteristics base.
  • each identity feature corresponds to a group of expression bases, and each group of expression bases includes expression bases of multiple expressions.
  • determining the personalized expression base according to multiple sets of expression bases with different identity characteristics includes: superimposing multiple sets of expression bases with different identity characteristics according to the identity characteristic information of the human face to obtain a personalized expression base.
  • the personalized expression base when the basic expression base is composed of multiple sets of expression bases with different identity characteristics, the personalized expression base can be derived more accurately according to the multiple sets of expression bases.
  • the above-mentioned basic expression base is an average expression base
  • the above-mentioned determining the personalized expression base matching the human face according to the basic expression base includes: determining the personalized expression base according to the average expression base.
  • the above average expression base can be obtained by processing multiple sets of expression bases with different identity characteristics, and the average expression base can reflect an average identity characteristic.
  • determining the personalized expression base based on the average expression base includes: transforming the average expression base according to the identity feature information of the human face to obtain the personalized expression base.
  • the storage overhead when storing the basic expression base can be reduced, and storage resources can be saved.
  • a method for generating an animated expression includes: acquiring an initial three-dimensional grid sequence; transforming the initial three-dimensional grid sequence to obtain a target three-dimensional grid sequence; and determining to match a human face according to a basic expression base Personalized expression base; generate a personalized expression base coefficient sequence based on the target three-dimensional grid sequence and personalized expression base; generate an animated expression based on the personalized expression base coefficient sequence.
  • the initial three-dimensional grid sequence includes multiple initial three-dimensional grids, and the multiple initial three-dimensional grids are respectively used to represent facial expression characteristics at multiple different moments;
  • the target three-dimensional grid sequence includes multiple target three-dimensional grids, The multiple target three-dimensional grids have the same topological structure as the basic expression base;
  • the personalized expression base matching the human face may be an expression base containing the identity characteristics of the human face;
  • the personalized expression base sequence includes multiple sets of personalized expression bases Coefficient, the superposition of the personalized expression base coefficient and the personalized expression base in the personalized expression base coefficient sequence is used to express the facial expression characteristics.
  • the initial three-dimensional grid sequence in the above-mentioned second aspect may be composed of a plurality of initial three-dimensional grids in the above-mentioned first aspect; the target three-dimensional grid sequence in the above-mentioned second aspect may also be composed of the above-mentioned first aspect A plurality of target three-dimensional grids; the personalized expression base sequence in the second aspect may be composed of multiple sets of personalized expression base sequences in the first aspect.
  • the definition and interpretation of the initial three-dimensional grid, the target three-dimensional grid and the personalized expression base coefficients in the first aspect described above are also applicable to the initial three-dimensional grid sequence in the second aspect of the initial three-dimensional grid sequence, the target three-dimensional grid The target three-dimensional grid in the sequence and the personalized expression base coefficients in the personalized expression base sequence.
  • the three-dimensional grid can better reflect the facial expression characteristics.
  • the converted three-dimensional grid and the basic The topological structure of the expression base remains the same, so that finally, an animated expression closer to the expression characteristics of the human face can be generated according to the transformed three-dimensional grid.
  • the animation expression can be made more realistic, and the display effect of the animation expression can be improved.
  • an electronic device in a third aspect, includes a module for performing the method in the first aspect and any one of the implementation manners of the first aspect.
  • an electronic device includes a module for performing the method in the second aspect and any one of the implementation manners of the second aspect.
  • an electronic device including a memory and a processor, where the memory is used to store a program, and the processor is used to execute the program stored in the memory, when the program stored in the memory is executed by the processor
  • the processor is configured to execute the method in the first aspect and any one of the implementation manners of the first aspect.
  • an electronic device including a memory and a processor, where the memory is used to store a program, and the processor is used to execute the program stored in the memory, when the program stored in the memory is executed by the processor
  • the processor is configured to execute the method in the second aspect and any implementation manner of the second aspect.
  • the above memory is a non-volatile memory.
  • the aforementioned memory and processor are coupled to each other.
  • a computer-readable storage medium is provided.
  • the computer-readable storage medium is used to store program code.
  • the program code is executed by a computer, the computer is used to perform the first aspect and the first Any one of the methods in one aspect.
  • a computer-readable storage medium is provided.
  • the computer-readable storage medium is used to store program code.
  • the program code is executed by a computer, the computer is used to perform the second aspect and the first The method in any one of the two aspects.
  • the above computer-readable storage medium may be located inside the electronic device, and the program code stored in the computer-readable storage medium may be executed by the electronic device.
  • the electronic device can execute the method in the implementation manner of any one of the first aspect or the second aspect.
  • a chip includes a processor, and the processor is configured to execute the method in the first aspect and any one of the implementation manners of the first aspect.
  • a chip is provided.
  • the chip includes a processor, and the processor is configured to execute the method in the second aspect and any one of the implementation manners of the second aspect.
  • the above chip is installed inside the electronic device.
  • a computer program for causing a computer or an electronic device to execute the method in the first aspect and any one of the implementation manners in the first aspect is provided.
  • a computer program for causing a computer or an electronic device to execute the method in any of the second aspect and any implementation manner of the second aspect.
  • the above computer program may be stored in the electronic device, and the computer program may be executed by the electronic device.
  • the electronic device executes the computer program
  • the electronic device can execute the method in the implementation manner of any one of the first aspect or the second aspect.
  • the electronic device may be a mobile terminal (for example, a smart phone), a computer, a personal digital assistant, a wearable device, a vehicle-mounted device, an Internet of Things device, an augmented reality (AR) device, and virtual reality (virtual reality) , VR) equipment and so on.
  • a mobile terminal for example, a smart phone
  • a computer for example, a personal digital assistant
  • a wearable device for example, a smart phone
  • vehicle-mounted device for example, a vehicle-mounted device, an Internet of Things device, an augmented reality (AR) device, and virtual reality (virtual reality) , VR) equipment and so on.
  • AR augmented reality
  • VR virtual reality
  • the above electronic device may also be other devices capable of displaying video pictures or displaying pictures.
  • FIG. 1 is a schematic flowchart of a method for generating an animated expression according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of the initial three-dimensional grid and the three-dimensional grid after rigid deformation
  • Figure 3 is a schematic diagram of matching points marked in the initial three-dimensional grid and the three-dimensional grid after rigid deformation
  • Figure 4 is a schematic diagram of the basic expression base
  • FIG. 5 is a schematic diagram of a method for generating an animated expression according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of vertices in a target three-dimensional grid and corresponding reference vertices in multiple expression bases;
  • FIG. 7 is a schematic diagram of a vertex distance between vertices in a target three-dimensional mesh and a reference distance between corresponding reference vertices in multiple expression bases;
  • FIG. 8 is a schematic flowchart of a method for generating an animated expression according to an embodiment of the present application.
  • FIG. 9 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • FIG. 10 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • FIG. 11 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • FIG. 12 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • the method for generating an animated expression in the embodiment of the present application may be executed by an electronic device.
  • the above electronic device may be a mobile terminal (for example, a smart phone), a computer, a personal digital assistant, a wearable device, a vehicle-mounted device, an Internet of Things device, an AR device, a VR device, and so on.
  • the above electronic device may also be other devices capable of displaying video pictures or displaying pictures.
  • the above electronic device may be a device running various operating systems.
  • the above-mentioned electronic device may be a device running an Android system, a device running an IOS system, or a device running a windows system.
  • FIG. 1 is a schematic flowchart of a method for generating an animated expression according to an embodiment of the present application.
  • the method shown in FIG. 1 may be performed by an electronic device.
  • the method shown in FIG. 1 includes steps 101 to 105, and these steps are described in detail below.
  • an initial three-dimensional grid may be extracted from the input video or image through a neural network, and the initial three-dimensional grid is used to represent facial expression features.
  • the above initial three-dimensional grid can contain a large number of vertices (vertex), which correspond to the positions of the face.
  • the positions of these vertices and the positional relationship between them can be used to express the facial expressions .
  • the topological structure of the target three-dimensional grid is the same as the topological structure of the basic expression base.
  • step 102 by transforming the initial three-dimensional grid, a target three-dimensional network with the same topology structure as the basic expression base can be obtained, which realizes the unification of the topological structure and facilitates subsequent animation expressions based on the target three-dimensional grid.
  • Transforming the initial three-dimensional grid in step 102 above to obtain the target three-dimensional grid may specifically include the following steps:
  • topological reference grid after rigid deformation perform rigid deformation on the topological reference grid to obtain a topological reference grid after rigid deformation.
  • the size of the topological reference grid after rigid deformation is the same as the size of the initial three-dimensional grid;
  • the topological reference grid may be the same reference grid as the topological structure of the basic expression base or the personalized table base.
  • the topological reference grid is used to transform the initial three-dimensional grid.
  • the rigid deformation of the topological reference grid in step 202 may refer to operations such as rotation, translation, and scaling on the topological reference grid, so that the size and orientation of the topological reference grid are the same as the initial three-dimensional grid.
  • the rigidly deformed topological reference grid obtained by rigidly deforming the topological reference grid has the same size and orientation as the initial three-dimensional grid.
  • the matching points of the rigidly deformed topological reference grid and the initial three-dimensional grid can be selected manually, and then the rigidly deformed topological reference grid and the initial three-dimensional grid are performed according to these matching points
  • the fitting process makes the topological reference grid after rigid deformation coincide with the corresponding matching points of the initial three-dimensional grid to achieve initial bonding.
  • the topological reference grid after rigid deformation and the initial three-dimensional grid can also be continued Fit so that the topological reference grid after rigid deformation coincides with as many points as possible in the initial three-dimensional grid.
  • the four vertices in the initial three-dimensional grid can be manually selected, and the topological reference grid after rigid deformation matches the four vertices 4 reference vertices (U1, U2, U3, U4).
  • V1, V2, V3 and V4 are the vertices of the positions of the left eye, right eye, upper lip and right ear in the initial three-dimensional grid
  • U1, U2, U3 and U4 are also the topological references after rigid deformation The vertex of the position of the left eye, right eye, upper lip, and right ear in the grid.
  • the initial three-dimensional mesh and the topological reference mesh after rigid deformation can be matched for the first time, so that the four vertices (V1, V2, V3, V4) in the initial three-dimensional mesh ) Respectively coincide with the four reference vertices (U1, U2, U3, U4) in the topological reference grid after rigid deformation.
  • the degree of fitting between the topological reference grid after rigid deformation and the initial three-dimensional grid meets the preset degree of fitting, which may refer to the vertex in the initial three-dimensional grid and the topological reference grid after rigid deformation
  • the distance between the corresponding reference vertices is less than a preset distance.
  • performing coordinate replacement is equivalent to stretching the rigidly deformed topological reference grid so that the reference vertices in the rigidly deformed topological reference grid completely fit the corresponding vertices in the initial three-dimensional grid,
  • the stretched grid is the target 3D grid.
  • the rigidly deformed topological reference grid and the initial three-dimensional mesh are bonded in step 203
  • the rigidly deformed topological reference grid may be deformed while the rigidly deformed topological reference is referred to
  • the grid is bonded to the initial three-dimensional grid so that the degree of bonding meets the preset degree of bonding.
  • the laminating process in step 203 above specifically includes steps:
  • Step A Perform non-rigid deformation on the topological reference grid after rigid deformation according to radial basis function (RBF) to obtain a topological reference grid after non-rigid change;
  • RBF radial basis function
  • Step B Fit the non-rigid deformed topological reference grid to the initial three-dimensional grid
  • the degree of fitting between the non-rigid deformed topological reference grid and the initial three-dimensional grid can meet the fitting requirements, so that the target three-dimensional obtained after fitting
  • the grid has both the topology structure of the topological reference grid and the shape of the initial three-dimensional grid, which facilitates the subsequent generation of animated expressions that are closer to the facial expression characteristics.
  • step A When performing non-rigid deformation on the topological reference grid after rigid deformation in step A above, you can manually select several matching points in the topological reference grid after rigid deformation and the initial three-dimensional grid. Next, you can use this Based on several matching points, continue to determine the vertices of the topological reference mesh after rigid deformation matching the vertices in the initial three-dimensional mesh.
  • the vertex in the topological reference mesh after rigid deformation that matches the vertex of the initial three-dimensional mesh can be determined according to formula (1).
  • u i is the vertex in the topological reference grid after rigid deformation
  • v i is the vertex in the initial three-dimensional grid that matches u i
  • S f (u i ) is the radial basis function
  • the coefficients in S f (u i ) can pass through the existing matching points (for example, the vertices V1, V2 and V3 in the initial three-dimensional grid and the matching vertices U1, U2 and U3 in the topological reference grid after rigid deformation ) To construct the linear equation.
  • the personalized expression base matching the human face may be an expression base containing the identity characteristics of the human face.
  • the topological structure of the personalized expression base is the same as that of the basic expression base, and the basic expression base may be an expression base preset according to an application scenario.
  • the identity feature of the human face may refer to the shape or characteristics of the facial organs.
  • the identity feature of the human face may be large eyes, small eyes, a large mouth, a high nose bridge, and so on.
  • facial expression features refer to the movement of certain organs of the human face.
  • facial expression features can include blinking, grinning, frowning, and cheeks.
  • just looking at the shape of the face may not be able to distinguish between the facial expression characteristics and the identity characteristics of the face. For example, for a large mouth, it may not be possible to distinguish whether the mouth is relatively large or is due to a grinning expression. Therefore, in the process of generating animated expressions corresponding to human faces, it is generally necessary to distinguish the identity characteristics and facial expression characteristics of human faces,
  • the personalized expression base may include multiple expression bases, and each expression base corresponds to a partial expression.
  • the personalized expression base may include 47 common local expressions (the 47 expressions can cover facial expressions such as eyebrows, eyes, nose, mouth, chin, and cheeks) corresponding to 47 types of facial expressions.
  • the 47 partial expressions mentioned above may include some common expressions on the human face, for example, blinking, opening mouth, frowning, raising eyebrows, etc.
  • the above expressions may also include expressions obtained by subdividing some common expressions on the human face.
  • the 47 partial expressions may include expressions such as moving the inside of the left eyebrow upward, lifting the lower eyelid of the right eye, and eversion of the upper lip.
  • the topology structure of the personalized expression base is also the same as that of the basic expression base. Therefore, in this application, by transforming the initial three-dimensional grid, the target three-dimensional grid with the same topology as the basic expression base can be obtained, which is convenient for subsequent generation based on the target three-dimensional grid with the same topology and the personalized expression base.
  • the base coefficient of personalized expression can improve the display effect of the final animation expression.
  • step 103 when the personalized expression base matching the human face is determined according to the basic expression base, different expression bases can be used to derive the personalized expression base according to different constitutions of the basic expression base.
  • the first case the basic expression base consists of multiple sets of expression bases with different identity characteristics.
  • each identity feature corresponds to a group of expression bases, and each group of expression bases contains multiple expression bases.
  • the basic expression base is composed of multiple sets of expression bases with different identity characteristics, wherein the expression base of each identity includes multiple expression bases, and the multiple expression bases correspond to different local expressions.
  • the basic expression base may be composed of common expression bases of 50 identity features, where each identity feature corresponds to a set of expression bases, and a set of expression bases corresponding to each identity feature may include common expressions of 47 expressions base.
  • determining the personalized expression base matching the human face based on the basic expression base includes: determining the personalized expression base based on multiple sets of expression bases with different identity characteristics.
  • the generated personalized expression base is an expression base containing the facial identity characteristics.
  • determining the personalized expression base based on multiple sets of expression bases with different identity characteristics includes: superimposing multiple sets of expression bases with different identity characteristics according to the identity characteristic information of the human face to obtain a personalized expression base.
  • the personalized expression base can be derived more accurately according to the multiple sets of expression bases.
  • the second case the basic expression base is the average expression base.
  • determining the personalized expression base matching the human face according to the basic expression base includes: determining the personalized expression base based on the expression base of the average identity feature.
  • the above average expression base can be obtained by processing multiple sets of expression bases with different identity characteristics, and the average expression base can reflect an average identity characteristic.
  • the above average expression base may be obtained by superposing 50 sets of expression bases corresponding to the 50 identity characteristics in the first case.
  • identity characteristic information of the human face is also needed, so that the generated personalized expression base is an expression base containing the identity characteristics of the face.
  • determining the personalized expression base according to the average expression base includes: transforming the average expression base according to the identity feature information of the human face to obtain the personalized expression base.
  • the storage overhead when storing the basic expression base can be reduced, and storage resources can be saved.
  • the above-mentioned basic expression base may also be composed of a group of expression bases corresponding to any identity feature.
  • Each list emotion base corresponds to an identity feature) as a basic expression base. In this case, it is also possible to reduce the storage overhead when storing the basic expression and save storage resources.
  • the superposition of the personalized expression base coefficient and the personalized expression base is used to represent the facial expression characteristics of the human face.
  • the superposition of the personalized expression base coefficient and the personalized expression base may be a linear combination or a non-linear combination of the personalized expression base coefficient and the personalized expression base.
  • the target three-dimensional grid is obtained by transforming the initial three-dimensional grid.
  • the target three-dimensional grid can represent the facial expression characteristics, and the superposition of the personalized expression base and the personalized expression base coefficients can also represent the face.
  • the facial expression feature therefore, in step 104, the personalized facial expression base coefficient can be inferred based on the target three-dimensional grid and the personalized facial expression base parameter.
  • the three-dimensional grid can better reflect the facial expression characteristics.
  • the converted three-dimensional grid and the basic The topological structure of the expression base remains the same, so that finally, an animated expression closer to the expression characteristics of the human face can be generated according to the transformed three-dimensional grid.
  • the animation expression can be made more realistic, and the display effect of the animation expression can be improved.
  • an animated expression may be generated according to the personalized expression base coefficient and the anthropomorphic character expression base.
  • the anthropomorphic character expression base here may specifically be a virtual character expression base or a anthropomorphic animal expression base.
  • the types of partial expressions included in the anthropomorphic character expression base are the same as the types of expressions included in the personalized expression base, and the expression semantics of each expression in the anthropomorphic character expression base are the same as the semantics of the individual expressions included in the personalized expression base, respectively. .
  • the partial expressions corresponding to the personalized expression base are shown in the first column of Table 1, and the partial expressions corresponding to the anthropomorphic character expression base are shown in the second column of Table 1.
  • the partial expressions corresponding to the personalized expression base include expressions such as blinking, opening mouth, raising eyebrows, moving inside of the left eyebrow and lifting the lower eyelid of the right eye
  • the expression base of the anthropomorphic character also corresponds to the same partial expressions.
  • FIG. 5 is a schematic diagram of a method for generating an animated expression according to an embodiment of the present application.
  • the target three-dimensional grid can be obtained by deforming the initial three-dimensional grid, and the personalized expression base can be obtained through the basic expression base.
  • the personalized expression base coefficient can be determined according to the target 3D grid and personalized expression base, and then the personalized expression base coefficient and the anthropomorphic character expression base can be collected to obtain the animated expression .
  • the first method determine the personalized expression base coefficient according to the coordinates of the reference vertices in various expression bases and the coordinates of the vertices in the target three-dimensional grid.
  • the personalized expression base coefficient is determined according to the difference between the linear combination of the coordinates of the reference vertices in multiple expression bases and the coordinates of the vertices in the target three-dimensional grid.
  • the personalized expression base includes multiple expression bases, which correspond to different expressions, and the reference vertex of each expression base in the multiple expression bases is each expression base The point in the corresponding position of the vertex in the target three-dimensional grid.
  • Expression base 1 expression base 2 and expression base 3 include reference vertices A1, A2 and A3, respectively, and reference vertices A1, A2 and A3 are expression base 1 respectively.
  • the expression base 2 and the expression base 3 are points corresponding to the vertex A in the target three-dimensional grid. Then, the difference between the linear combination of the coordinates of the reference vertices in the three expression bases and the coordinates of the vertices in the target three-dimensional grid can be expressed by equation (2).
  • P(A) represents the coordinate value of vertex A
  • P(A1), P(A2) and P(A3) represent the coordinate values of reference vertices A1, A2 and A3, respectively
  • (x, y, z) is a linear combination coefficient
  • the personalized expression base coefficient is determined according to the difference between the linear combination of the coordinates of the reference vertices in the multiple expression bases and the coordinates of the vertices in the target three-dimensional grid, which specifically includes: determining multiple first differences Value; determine the personalized expression base coefficient according to the sum of the products of the multiple first difference values and the corresponding weight values respectively.
  • the multiple first difference values are linear combination differences of coordinates of multiple vertices in the target three-dimensional grid and corresponding reference vertices in multiple expression bases, respectively.
  • determining the personalized expression base coefficient according to the sum of the products of the plurality of first differences and the corresponding weight values respectively includes: when the sum of the products of the plurality of first differences and the corresponding weight values takes the minimum value, The linear combination coefficient when the coordinates of the reference vertices in multiple expression bases are linearly combined is determined as the personalized expression base coefficient.
  • each first difference value is respectively the linearity of the coordinates of different vertices in the target three-dimensional grid and the corresponding reference points in various expression bases The combined difference.
  • the target 3D mesh further includes vertex B and vertex C
  • expression base 1 includes reference vertices B1 and C1
  • expression base 2 includes reference vertices B2 and C2
  • expression base 3 includes reference vertices B3 and C3, where the reference vertices B1, B2, and B3 are the points of the expression base 1, expression base 2, and expression base 3 that are at positions corresponding to the vertex B in the target three-dimensional grid, and reference vertices C1, C2, and C3
  • These are the points in the expression base 1, the expression base 2 and the expression base 3 that are in the corresponding positions to the vertex C in the target three-dimensional grid.
  • the sum of the above formula (2), formula (3) and formula (4) can be obtained, and the coefficient corresponding to the smallest sum of the three formulas (x, The value of y, z) is determined as the personalized expression base coefficient.
  • the personalized expression base coefficient can be made to reflect the personalized expression base coefficient as realistically as possible
  • the facial expression features facilitate subsequent generation of animated expressions that are closer to the facial expression features.
  • the second way according to the difference between the linear combination of the reference distances in multiple expression bases and the distance of the vertices in the target three-dimensional grid, determine the personalized expression base coefficient.
  • the personalized expression base includes multiple expression bases, which correspond to different expressions.
  • the above-mentioned determination of the personalized expression base coefficients according to the target three-dimensional grid and the personalized expression base includes:
  • the personalized expression base coefficient is determined.
  • the above vertex distance is the distance between two vertices in the target 3D mesh
  • the reference distance corresponding to each of the above expression bases is the distance between two reference vertices in each expression base
  • the two reference vertices It is a point in each expression base that is in a corresponding position to two vertices in the target three-dimensional grid.
  • the target 3D mesh includes vertex M and vertex N, the distance between vertex M and vertex N is D, and expression base 1 includes vertex M1 and N1, and M1 and N1 are expression base 1 respectively.
  • the expression base 2 includes the vertices M2 and N2, M2 and N2 are the expression base 2 and the vertex M and the vertex The point where N is at the corresponding position, the distance between vertex M2 and vertex N2 is D2;
  • expression base 3 includes vertices M3 and N3, M3 and N3 are the corresponding positions of vertex M and vertex N in expression base 3, respectively Point, the distance between vertex M3 and vertex N3 is D3.
  • D is the reference distance in the expression base 1
  • D2 is the reference distance in the expression base 2
  • D3 is the reference distance in the expression base 3.
  • the difference between the linear combination of the reference distances of the three expression bases and the apex distance can be as shown in equation (5).
  • the personalized expression base coefficient is determined according to the difference between the linear combination of multiple reference distances and the vertex distance, which specifically includes: determining multiple second difference values; according to multiple second difference values and corresponding weights, respectively The sum of the products of the values determines the personalized expression base coefficient.
  • the plurality of second difference values are difference values of linear combinations of the distances of the vertices in the target three-dimensional grid and the corresponding reference distances in the various expression bases.
  • determining the personalized expression base coefficient according to the sum of the products of the plurality of second difference values and the corresponding weight values includes: when the sum of the products of the plurality of second difference values and the corresponding weight values takes the minimum value, The linear combination coefficient when multiple reference distances are linearly combined is determined as the personalized expression base coefficient.
  • each second difference value is respectively a linearity of a distance between different vertices in the target three-dimensional grid and a corresponding reference distance in multiple expression bases The combined difference.
  • the vertices included in the target 3D mesh and the vertex distance between the vertices, and the reference vertices and reference vertices included in the expression base 1, expression base 2 and expression base 3 are as follows:
  • the target 3D mesh also includes vertices M, N, R and S, where the distance between vertices M and N is d(M,N) and the distance between vertices R and S is d(R,S );
  • the expression base 1 includes reference vertices M1, N1, R1, and S1. These reference vertices are points corresponding to vertices M, N, R, and S, respectively.
  • the distance between vertices M1 and N1 is d(M1, N1), the distance between vertices R1 and S1 is d(R1, S1);
  • the expression base 2 includes reference vertices M2, N2, R2, and S2. These reference vertices are points corresponding to vertices M, N, R, and S, respectively.
  • the distance between vertices M2 and N2 is d(M2, N2), the distance between vertices R2 and S2 is d(R2, S2);
  • the expression base 3 includes reference vertices M3, N3, R3, and S3. These reference vertices are points corresponding to vertices M, N, R, and S, respectively.
  • the distance between vertices M3 and N3 is d(M3, N3), the vertex distance between vertices R3 and S3 is d(R3, S3).
  • the personalized expression base coefficient can be made to reflect as realistically as possible
  • the facial expression features facilitate subsequent generation of animated expressions that are closer to the facial expression features.
  • the first method and the second method described above are to determine the personalized expression base coefficient according to the first difference value and the second difference value, respectively.
  • the first difference value and the second difference value can also be combined to determine the expression base coefficient, and this method (third method) will be described in detail below.
  • the third way according to the difference between the linear combination of the coordinates of the reference vertices in various expression bases and the coordinates of the vertices in the target 3D grid, and the linear combination of the reference distances in various expression bases and the target 3D grid
  • the difference of the apex distance comprehensively determines the personalized expression base coefficient.
  • the third method is equivalent to the combination of the first method and the second method described above.
  • the personalized expression base coefficient is determined according to the target three-dimensional grid and the personalized expression base, including:
  • the personalized expression base coefficient is determined according to the sum of the first value and the second value.
  • the linear combination coefficient corresponding to the smallest sum of the first value and the second value may be determined as the personalized expression base coefficient.
  • three first difference values and three second difference values can be obtained, where the three first difference values can be expressed by formula (8) to formula (10), 3
  • the second difference can be expressed by formula (11) to formula (13).
  • the values of the corresponding coefficients (x, y, z) in the formula (8) to the formula (13) and the minimum value can be determined as the personalized expression base coefficients.
  • the personalized expression base parameters are comprehensively determined by the two differences, so that the personalized expression base coefficient can be determined more accurately.
  • the fourth way can also be used to determine the personalized expression base coefficient.
  • the fourth way determine the personalized expression base coefficient according to formula (14).
  • E point represents the square of the sum of the difference between the linear combination of the coordinates of the reference vertices in the various expression bases and the coordinates of the vertices in the target three-dimensional grid
  • E edge represents the The square of the sum of the difference between the linear combination of the reference distance and the vertex distance in the target 3D mesh
  • Elasso is a constraint function used to sparse the final linearized expression base coefficients (using as few local expressions as possible) Represents facial expressions).
  • E point , E edge and Elasso can be obtained by formulas (15) to (17).
  • B represents the number of expression bases
  • b represents the b-th expression base
  • N represents the number of vertices in the target three-dimensional grid (the number of vertices in each expression base is also N)
  • i represents The ith vertex
  • x b represents The combination coefficient of
  • p i represents the three-dimensional coordinates of the ith vertex in the target three-dimensional grid
  • w i is Weight value
  • B represents the number of expression bases
  • b represents the bth expression base
  • i and j represent the ith vertex and jth vertex, respectively
  • K represents the number of edges
  • e ij represents the distance between the i-th vertex and the j-th vertex in the target 3D mesh
  • x b represents The combination coefficient of w ij is 'S weight value
  • x b represents The combination coefficient of Represents the sum of absolute values of all expression base coefficients.
  • the linear combination coefficient corresponding to the smallest value of E may be determined as the personalized expression base coefficient.
  • the personalized expression base coefficient finally obtained can be relatively sparse (there are more 0s in the personalized expression base coefficient), so that the final utilization is more
  • the expression base with less expression can express animated expressions.
  • FIG. 8 is a schematic flowchart of a method for generating animation according to an embodiment of the present application.
  • the method shown in FIG. 8 may be performed by an electronic device.
  • the method shown in FIG. 8 includes steps 301 to 305. These steps will be described below.
  • the initial three-dimensional grid sequence in step 301 includes multiple initial three-dimensional grids, and the multiple initial three-dimensional grids are respectively used to represent facial expression features at multiple different moments.
  • the initial three-dimensional grid in the above initial three-dimensional grid sequence is the same as the initial three-dimensional grid obtained in step 101, and the above definition and interpretation of the initial three-dimensional grid also apply to the initial three-dimensional grid sequence in step 301. In order to avoid repetition, the three-dimensional grid will not be described in detail here.
  • the target three-dimensional grid sequence includes multiple target three-dimensional grids, and the multiple target three-dimensional grids have the same topological structure as the basic expression base.
  • each initial three-dimensional grid in the initial three-dimensional grid sequence can be transformed to obtain multiple target three-dimensional grids, and then the target three-dimensional grid sequence can be obtained, wherein, for each initial three-dimensional grid
  • the process of transforming the grid to obtain the target three-dimensional grid can refer to the relevant content of step 102 above. In order to avoid unnecessary repetition, it will not be described in detail here.
  • the personalized expression base matching the human face may be an expression base containing the identity characteristics of the human face.
  • step 303 reference may be made to the specific implementation manner in step 103 above to obtain a personalized expression base, which is not described in detail here.
  • the personalized expression base sequence includes multiple sets of personalized expression base coefficients, and the superposition of the personalized expression base coefficients and the personalized expression bases in the personalized expression base coefficient sequence is used to represent the facial expression characteristics;
  • a personalized expression base coefficient corresponding to each target 3D grid can be generated according to each target 3D grid in the target 3D grid sequence and a personalized expression base, thereby obtaining multiple personalized expression base coefficients Composed of personalized expression base coefficient sequences.
  • the specific method in step 104 may be adopted to obtain the personalized expression base coefficient.
  • the animation expression generated in step 305 may be a continuous animation expression.
  • the process of generating an animated expression for each personalized expression base coefficient sequence in the personalized expression base coefficient sequence may refer to the related content of step 105 above, and will not be described in detail here.
  • an animated expression can be generated according to the personalized expression base coefficient sequence and the anthropomorphic character expression base.
  • the anthropomorphic character expression base here may specifically be a virtual character expression base or a anthropomorphic animal expression base.
  • the three-dimensional grid can better reflect the facial expression characteristics.
  • the converted three-dimensional grid and the basic The topological structure of the expression base remains the same, so that finally, an animated expression closer to the expression characteristics of the human face can be generated according to the transformed three-dimensional grid.
  • the animation expression can be made more realistic, and the display effect of the animation expression can be improved.
  • FIG. 9 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • the electronic device 1000 shown in FIG. 9 can perform various steps in the method for generating an animated expression shown in FIG. 1, including:
  • the obtaining module 1001 is used to obtain an initial three-dimensional grid, and the vertices in the initial three-dimensional grid are used to represent facial expression features;
  • the transformation module 1002 is configured to transform the initial three-dimensional grid to obtain a target three-dimensional grid, and the target three-dimensional grid has the same topological structure as the basic expression base;
  • a processing module 1003, the processing module 1003 is used for:
  • the personalized expression base is an expression base containing the identity characteristics of the human face
  • the three-dimensional grid can better reflect the facial expression characteristics.
  • the converted three-dimensional grid and the basic The topological structure of the expression base remains the same, so that finally, an animated expression closer to the expression characteristics of the human face can be generated according to the transformed three-dimensional grid.
  • the personalized expression base includes multiple expression bases, the multiple expression bases respectively correspond to different expressions
  • the processing module 1003 is configured to: according to the multiple expression bases
  • the coordinates of the reference vertices of each expression base and the coordinates of the vertices in the target three-dimensional grid determine the personalized expression base coefficient, wherein the reference vertices of each expression base in the multiple expression bases are A point in each expression base corresponding to a vertex in the target three-dimensional grid.
  • processing module 1003 is used to:
  • the plurality of first difference values being a difference value of a linear combination of coordinates of a plurality of vertices in the target three-dimensional grid and corresponding reference vertices in the plurality of expression bases;
  • the personalized expression base coefficient is determined according to the sum of products of the plurality of first difference values and corresponding weight values, respectively.
  • the multiple first difference values respectively correspond to multiple different weight values.
  • the personalized expression base includes multiple expression bases, the multiple expression bases respectively correspond to different expressions, and the processing module is used to:
  • the personalized expression base coefficient is determined according to the difference between the linear combination of the plurality of reference distances and the distance between the vertices.
  • processing module 1003 is used to:
  • the plurality of second difference values being difference values of a linear combination of a plurality of vertex distances in the target three-dimensional grid and corresponding reference distances in the plurality of expression bases;
  • the personalized expression base coefficient is determined according to the sum of products of the plurality of second difference values and corresponding weight values, respectively.
  • the multiple second difference values respectively correspond to multiple different weight values.
  • the transformation module 1002 is used to:
  • the transformation module 1002 is used to repeatedly perform steps A and B until the degree of fit between the non-rigid deformed topological reference grid and the initial three-dimensional grid meets a preset Fit degree
  • the steps A and B are:
  • Step A Perform non-rigid deformation on the topological reference grid after rigid deformation according to RBF to obtain a topological reference grid after non-rigid change;
  • Step B Fit the non-rigid deformed topological reference grid to the initial three-dimensional grid.
  • FIG. 10 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • the electronic device 2000 shown in FIG. 10 can perform various steps in the method for generating an animated expression shown in FIG. 8, including:
  • the obtaining module 2001 is used to obtain an initial three-dimensional grid sequence, the initial three-dimensional grid sequence includes a plurality of initial three-dimensional grids, and the multiple initial three-dimensional grids are respectively used to represent facial expression features at multiple different moments ;
  • the transformation module 2002 is used to transform the initial three-dimensional grid sequence to obtain a target three-dimensional grid sequence.
  • the target three-dimensional grid sequence includes multiple target three-dimensional grids, the multiple target three-dimensional grids and the basic expression
  • the topological structure of the base is the same;
  • Processing module 2003 the processing module 2003 is used for:
  • the personalized expression base is an expression base containing the identity characteristics of the human face
  • the personalized expression base coefficient sequence including multiple sets of personalized expression base coefficients, and the personalized expression base coefficient sequence
  • the superimposition of the personalized expression base coefficient in and the personalized expression base is used to express the facial expression characteristics of the human face
  • Animated expressions are generated according to the personalized expression base coefficient sequence.
  • the three-dimensional grid can better reflect the facial expression characteristics.
  • the converted three-dimensional grid and the basic The topological structure of the expression base remains the same, so that finally, an animated expression closer to the expression characteristics of the human face can be generated according to the transformed three-dimensional grid.
  • FIG. 11 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • the electronic device 3000 shown in FIG. 11 includes:
  • the memory 3001 is used to store programs
  • the processor 3002 is used to execute the program stored in the memory 3001. When the program stored in the memory 3001 is executed by the processor 3002, the processor 3002 is used to execute each of the methods for generating animated expressions shown in FIG. 1 or FIG. step.
  • the acquisition module 1001, the conversion module 1002, and the processing module 1003 in the electronic device 1000 may be equivalent to the processor 3002 in the electronic device 3000.
  • the acquisition module 2001, the conversion module 2002, and the processing module 2003 in the electronic device 2000 may be equivalent to the processor 3002 in the electronic device 3000.
  • the processor 3002 may specifically be a central processing unit (CPU), a field programmable gate array (FPGA), a graphics processing unit (GPU), or other chips with data processing functions and many more.
  • the processor 3002 may also be at least two types of CPU, FPGA, GPU, and data processing chip.
  • FIG. 12 is a schematic block diagram of an electronic device according to an embodiment of the present application.
  • the electronic device 4000 in FIG. 12 includes a communication module 4010, a sensor 4020, a user input module 4030, an output module 4040, a processor 4050, an audio and video input module 4060, a memory 4070, and a power supply 4080.
  • the above-mentioned electronic device 4000 may execute the steps of the method for generating an animated expression in the embodiment of the present application.
  • the processor 4050 in the electronic device 4000 may execute the steps in the method for generating an animated expression in the embodiment of the present application.
  • the communication module 4010 may include at least one module that enables communication between the electronic device and other electronic devices.
  • the communication module 4010 may include one or more of a wired network interface, a broadcast receiving module, a mobile communication module, a wireless Internet module, a local area communication module, and a location (or positioning) information module.
  • the sensor 4020 can sense some operations of the user, and the sensor 4020 can include a distance sensor, a touch sensor, and so on.
  • the sensor 4020 can sense the user's operation such as touching the screen or approaching the screen.
  • the user input module 4030 is used to receive input digital information, character information, or contact touch operation/contactless gestures, and receive signal input related to user settings and function control of the system.
  • the user input module 4030 includes a touch panel and/or other input devices.
  • the output module 4040 includes a display panel for displaying information input by the user, information provided to the user, various menu interfaces of the system, and the like.
  • the display panel may be configured in the form of a liquid crystal display (liquid crystal) (LCD) or an organic light-emitting diode (OLED).
  • the touch panel may cover the display panel to form a touch display screen.
  • the output module 4040 may also include an audio output module, an alarm, and a haptic module.
  • the audio and video input module 4060 is used to input audio signals or video signals.
  • the audio and video input module 4060 may include a camera and a microphone.
  • the power supply 4080 can receive external power and internal power under the control of the processor 4050, and provide power required for the operation of each module of the entire electronic device.
  • the processor 4050 may refer to one or more processors.
  • the processor 4050 may include one or more central processors, or include a central processor and a graphics processor, or include an application processor and a coprocessor (Eg micro control unit or neural network processor).
  • the processor 4050 includes multiple processors, the multiple processors may be integrated on the same chip, or may be independent chips.
  • a processor may include one or more physical cores, where the physical core is the smallest processing module.
  • the memory 4070 stores computer programs including an operating system program 4071, application programs 4072, and the like.
  • Typical operating systems such as Microsoft’s Windows, Apple’s MacOS, etc. are used for desktop or notebook systems, and are based on Google’s Android Systems such as systems for mobile terminals.
  • Google’s Android Systems such as systems for mobile terminals.
  • the memory 4070 may be one or more of the following types: flash memory, hard disk type memory, micro multimedia card type memory, card memory (such as SD or XD memory), random access memory (random access memory) , RAM), static random access memory (static RAM, SRAM), read-only memory (read only memory, ROM), electrically erasable programmable read-only memory (electrically erasable programmable-read-only memory (EEPROM), programmable Read only memory (programmable ROM, PROM), magnetic memory, magnetic disk or optical disk.
  • the memory 4070 may also be a network storage device on the Internet, and the system may perform operations such as updating or reading the memory 4070 on the Internet.
  • the processor 4050 is used to read the computer program in the memory 4070, and then execute the electronic device defined by the computer program, for example, the processor 4050 reads the operating system program 4072 to run the operating system on the system and implement various functions of the operating system, or One or more application programs 4071 are read to run applications on the system.
  • the above-mentioned memory 4070 may store a computer program (the computer program is a program corresponding to the resource scheduling electronic device of the embodiment of the present application).
  • the processor 4050 executes the program and the program, the processor 4050 can execute the embodiment of the present application Resource scheduling electronic device.
  • the memory 4070 also stores other data 4073 other than the computer program.
  • the memory 4070 may store the load characteristics of the frame drawing thread involved in the resource scheduling electronic device of the present application, the load prediction value of the frame drawing thread, and so on.
  • connection relationship of each module or unit in the electronic device 4000 shown in FIG. 12 is only an example.
  • the electronic device in the embodiment of the present application may also be an electronic device obtained by connecting each module or unit in FIG. 12 with other connection relationships (for example, all modules or units are connected by a bus).
  • the memory 3001 in the electronic device 3000 corresponds to the memory 4070 in the electronic device 4000
  • the processor 3002 in the electronic device 3000 corresponds to the memory 4050 in the electronic device 4000.
  • the foregoing electronic device 1000, electronic device 2000, electronic device 3000, and electronic device 4000 may specifically be mobile terminals (for example, smart phones), computers, personal digital assistants, wearable devices, vehicle-mounted devices, Internet of Things devices, AR devices, and VR devices and many more.
  • mobile terminals for example, smart phones
  • computers personal digital assistants
  • wearable devices for example, smart phones
  • vehicle-mounted devices for example, Internet of Things devices, AR devices, and VR devices and many more.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical, or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present application essentially or part of the contribution to the existing technology or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种生成动画表情的方法和电子设备，该生成动画表情的方法包括：获取初始三维网格（101），该初始三维网格中的顶点用于表示人脸的表情特征；对初始三维网格进行变换，得到目标三维网格（102），该目标三维网格与基础表情基的拓扑结构相同；根据基础表情基确定与人脸匹配的个性化表情基（103）；根据目标三维网格和个性化表情基，确定个性化表情基系数（104），其中，该个性化表情基系数与个性化表情基的叠加用于表示人脸的表情特征；根据个性化表情基系数生成动画表情（105）。该方法能够生成显示效果更好的动画表情。

Description

生成动画表情的方法和电子设备
本申请要求于2018年12月29日提交中国专利局、申请号为201811636045.2、申请名称为“生成动画表情的方法和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，并且更具体地，涉及一种生成动画表情的方法和电子设备。
背景技术
电子设备在生成动画表情时一般是先通过摄像头拍摄人脸视频，并提取面部稠密(或者稀疏)特征点的运动信息，然后再将提取的面部稠密(或者稀疏)特征点的运动信息迁移到虚拟人物(或拟人动物)面部，从而得到动画表情。
传统方案生成动画表情的具体过程可以如下所示：首先，提取表示人脸表情的特征点或者点云(包含人脸特征的一系列的点)；其次，根据提取到的特征点或者点云将预先存储的多种不同身份特征的表情基组合成与人脸的身份特征相匹配的个性化表情基，最后根据个性化表情基生成动画表情。
但是由于特征点或者点云不能很好的反映人脸的表情特征，因此，传统方案根据点云或者特征点生成的动画表情的显示效果不太理想。
发明内容
本申请提供一种生成动画表情的方法和电子设备，能够增强动画表情的表示效果。
第一方面，提供了一种生成动画表情的方法，该方法包括：获取初始三维网格；对初始三维网格进行变换，得到目标三维网格；根据基础表情基确定与人脸匹配的个性化表情基；根据目标三维网格和个性化表情基，确定个性化表情基系数，其中，个性化表情基系数与个性化表情基的叠加用于表示人脸的表情特征；根据个性化表情基系数生成动画表情。
其中，上述初始三维网格中的顶点用于表示人脸的表情特征。上述目标三维网格的拓扑结构与基础表情基的拓扑结构相同。
通过对初始三维网格进行变换，能够得到与基础表情基的拓扑结构相同的目标三维网络，实现了拓扑结构的统一，便于后续根据目标三维网格最终得到动画表情。
另外，上述与人脸匹配的个性化表情基可以是包含人脸的身份特征的表情基，上述个性化表情基的拓扑结构与基础表情基的拓扑结构相同。
上述个性化表情基可以包含多种表情对应的表情基，例如，上述个性化表情基可以包含人脸常见的一些表情(例如，常见的47种表情)的表情基。
上述动画表情具体可以是一段动画视频对应的一段连续的动画表情，也可以是一帧视频图像对应的静止的动画表情。
例如，当上述初始三维网格包含一帧图像对应的人脸表情时，那么最终生成的动画表情就是一帧图像对应的动画表情。
可选地，上述根据个性化表情基系数生成动画表情，包括：根据个性化表情基系数和拟人角色表情基生成动画表情。
上述根据个性化表情基系数和拟人角色表情基生成动画表情，可以是指将个性化表情基系数迁移到拟人角色表情基中，以生成动画表情。
上述拟人角色表情基可以是虚拟人物表情基，也可以是拟人动物表情基。
本申请中，由于三维网格包含的顶点信息更加丰富，使得三维网格能够更好地反映人脸的表情特征，另外，通过对三维网格进行变换，能够使得变换后的三维网格与基础表情基的拓扑结构保持一致，从而使得最终根据变换后的三维网格能够生成与人脸的表情特征更加接近的动画表情。
具体地，由于生成的动画表情反映的表情特征与人脸表情的表情特征更加接近，能够使得动画表情更加逼真，可以提高动画表情的显示效果。
结合第一方面，在第一方面的某些实现方式中，上述个性化表情基包含多种表情基，该多种表情基分别对应不同的表情，根据目标三维网格和个性化表情基，确定个性化表情基系数，包括：根据多种表情基中的参考顶点的坐标的线性组合与目标三维网格中的顶点的坐标的差，确定个性化表情基系数。
其中，上述多种表情基中的每种表情基的参考顶点是每种表情基中与目标三维网格中的顶点处于相对应位置的点。
结合第一方面，在第一方面的某些实现方式中，根据多种表情基中的参考顶点的坐标的线性组合与目标三维网格中的顶点的坐标的差，确定个性化表情基系数，包括：确定多个第一差值；根据多个第一差值分别与相应权重值的乘积的和确定个性化表情基系数。
其中，上述多个第一差值为目标三维网格中的多个顶点的坐标分别与多种表情基中的相应参考顶点的线性组合的差值。
可选地，根据多个第一差值分别与相应权重值的乘积的和确定个性化表情基系数，包括：在多个第一差值分别与相应权重值的乘积的和取最小值时，将多种表情基中的参考顶点的坐标线性组合时的线性组合系数确定为个性化表情基系数。
本申请中，通过将多个第一差值分别与相应权重值的乘积的和取最小值时对应的线性组合系数确定为个性化表情基系数，能够使得个性化表情基系数尽可能真实的反映人脸的表情特征，便于后续生成与人脸的表情特征更加接近的动画表情。
结合第一方面，在第一方面的某些实现方式中，多个第一差值分别对应多个不同的权重值。
可选地，上述多个第一差值分别对应目标三维网格中的多个顶点，其中，位于目标三维网格中的第一预设区域内的顶点对应的第一差值的权重值大于或者等于第一预设权重值，位于目标三维网格中的第一预设区域之外的顶点对应的第一差值的权重小于第一预设权重值。
上述预第一设区域可以是目标网格中的重点区域，该第一预设区域可以是人脸的一些 重要器官所在的区域。例如，上述第一预设区域可以是人眼所在的区域，或者，上述第一预设区域可以是嘴巴所在的区域，或者，上述第一预设区域可以是人眼和嘴巴所在的区域。
上述第一预设区域可以根据仿真或者模拟结果来确定，或者，上述第一预设区域也可以根据经验来直接确定。
本申请中，多个第一差值对应不同的权重值，能够在计算个性化表情基系数时充分考虑到各个顶点对应的第一差值的重要程度，便于更准确的求解出个性化表情基系数。
结合第一方面，在第一方面的某些实现方式中，个性化表情基包含多种表情基，多种表情基分别对应不同的表情，上述根据目标三维网格和个性化表情基，确定个性化表情基系数，包括：确定目标三维网格的顶点距离；确定多种表情基中的每种表情基对应的参考距离，得到多个参考距离；根据多个参考距离的线性组合与顶点距离的差，确定个性化表情基系数。
其中，上述顶点距离为目标三维网格中的两个顶点之间的距离
另外，上述每种表情基对应的参考距离为每种表情基中的两个参考顶点之间的距离，该两个参考顶点是每种表情基中分别与目标三维网格中的两个顶点处于相对应位置的点
结合第一方面，在第一方面的某些实现方式中，上述根据多个参考距离的线性组合与顶点距离的差，确定个性化表情基系数，包括：确定多个第二差值；根据多个第二差值分别与相应权重值的乘积的和确定个性化表情基系数。
其中，上述多个第二差值为目标三维网格中的多个顶点距离分别与多种表情基中对应的参考距离的线性组合的差值。
可选地，根据多个第二差值分别与相应权重值的乘积的和确定个性化表情基系数，包括：在多个第二差值分别与相应权重值的乘积的和取最小值时，将多个参考距离线性组合时的线性组合系数确定为个性化表情基系数。
本申请中，通过将多个第二差值分别与相应权重值的乘积的和取最小值时对应的线性组合系数确定为个性化表情基系数，能够使得个性化表情基系数尽可能真实的反映人脸的表情特征，便于后续生成与人脸的表情特征更加接近的动画表情。
可选地，在本申请中，可以根据单独根据第一差值或者第二差值来求解个性化表情基系数，也可以综合第一差值和第二差值来共同求解个性化表情基系数。
结合第一方面，在第一方面的某些实现方式中，多个第二差值分别对应多个不同的权重值。
可选地，上述多个第二差值分别对应目标三维网格中的多个顶点距离，其中，位于目标三维网格中的第二预设区域内的顶点之间的顶点距离对应的第二差值的权重值大于或者等于第二预设权重值，位于目标三维网格中的第二预设区域之外的顶点之间的顶点距离对应的第二差值的权重小于第二预设权重值。
上述第二预设区域可以是目标网格中的重点区域，该第二预设区域可以是人脸的一些重要器官所在的区域。例如，上述第二预设区域可以是人眼所在的区域，此时，上眼皮与下眼皮对应的顶点之间的顶点距离所对应的第二差值的权重值可以大于第二预设权重值。再如，上述第二预设区域可以是嘴巴所在的区域，此时，上嘴唇和下嘴唇对应的顶点之间的顶点距离对应的第二差值的权重值可以大于第二预设权重值。
上述第二预设区域可以根据实际仿真或者模拟结果来确定，或者，上述第二预设区域 也可以根据经验来确定。
本申请中，多个第二差值对应不同的权重值，能够在计算个性化表情基系数时充分考虑到不同的顶点距离的对应的第二差值的重要程度，便于更准确的求解出个性化表情基系数。
结合第一方面，在第一方面的某些实现方式中，对初始三维网格进行变换，得到目标三维网格，包括：确定拓扑参照网格；对拓扑参照网格进行刚性(rigid)变形，得到刚性变形后的拓扑参照网格；对刚性变形后的拓扑参照网格和初始三维网格进行贴合处理，直到刚性变形后的拓扑参照网格与初始三维网格的贴合程度满足预设贴合程度；将刚性变形后的拓扑参照网格中的顶点的坐标替换为初始三维网格中与刚性变形后的网格中的顶点相匹配的顶点的坐标，得到目标三维网格。
其中，上述拓扑参照网格的拓扑结构与基础表情基的拓扑结构相同，该拓扑参照网格可以用于对初始三维网格进行变换；上述刚性变形后的拓扑参照网格的大小与初始三维网格的大小相同；上述过程最终得到的目标三维网格具有拓扑参照网格的拓扑结构和初始三维网格的外形。
可选地，对拓扑参照网格进行刚性变形，包括：对拓扑参照网格进行旋转、平移或者缩放。
上述刚性变形后的拓扑参照网格除了大小与初始三维网格的大小相同之外，刚性变形后的拓扑参照网格的朝向还可以与初始三维网格的朝向相同。
另外，刚性变形后的拓扑参照网格与初始三维网格的贴合程度满足预设贴合程度，可以是指初始三维网格中的顶点与刚性变形后的拓扑参照网格中相对应的参考顶点之间的距离小于某个预设距离。
结合第一方面，在第一方面的某些实现方式中，对刚性变形后的拓扑参照网格和初始三维网格进行贴合处理，直到刚性变形后的拓扑参照网格与初始三维网格的贴合程度满足预设贴合程度，包括：重复执行步骤A和步骤B，直到非刚性(non-rigid)变形后的拓扑参照网格与初始三维网格的贴合程度满足预设贴合程度；
其中，步骤A和步骤B分别为：
步骤A：根据径向基函数RBF对刚性变形后的拓扑参照网格进行非刚性变形，得到非刚性变化后的拓扑参照网格；
步骤B：将非刚性变形后的拓扑参照网格与初始三维网格贴合。
应理解，非刚性变形可以是指不能采用单一的旋转、平移或者缩放对网格整体统一进行变形，而是通过对网格内部的不同区域分别采用旋转、平移或者缩放等方式进行变形(网格中的不同区域的变形方式不同)，在非刚性变形过程中，网格内部的不同区域也可能会发生相对运动。
本申请中，通过对上述步骤A和步骤B的迭代，能够使得非刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足贴合要求，使得贴合之后得到的目标三维网格既具有拓扑参照网格的拓扑结构又具有初始三维网格的外形，便于后续生成与人脸的表情特征更加接近的动画表情。
可选地，上述基础表情基由多组不同身份特征的表情基构成，上述根据基础表情基确定与人脸匹配的个性化表情基，包括：根据多组不同身份特征的表情基确定个性化表情基。
在上述基础表情基中，每种身份特征对应一组表情基，每组表情基中包含多个表情的表情基。
可选地，根据多组不同身份特征的表情基确定个性化表情基，包括：根据人脸的身份特征信息，对多组不同身份特征的表情基进行叠加，以得到个性化表情基。
本申请中，当基础表情基是由多组不同身份特征的表情基构成时，根据该多组表情基能够更准确地推导出个性化表情基。
可选地，上述基础表情基为平均表情基，上述根据基础表情基确定与人脸匹配的个性化表情基，包括：根据平均表情基确定个性化表情基。
其中，上述平均表情基可以是通过对多组不同身份特征的表情基进行处理得到的，该平均表情基能够反映一种平均身份特征。
可选地，根据平均表情基确定个性化表情基，包括：根据人脸的身份特征信息，对平均表情基进行变换，以得到个性化表情基。
本申请中，当基础表情基为平均表情基时，能够减少存储基础表情基时的存储开销，节省存储资源。
第二方面，提供了一种生成动画表情的方法，该方法包括：获取初始三维网格序列；对初始三维网格序列进行变换，得到目标三维网格序列；根据基础表情基确定与人脸匹配的个性化表情基；根据目标三维网格序列和个性化表情基，生成个性化表情基系数序列；根据个性化表情基系数序列生成动画表情。
其中，初始三维网格序列包括多个初始三维网格，该多个初始三维网格分别用于表示人脸在多个不同时刻的表情特征；目标三维网格序列包括多个目标三维网格，该多个目标三维网格与基础表情基的拓扑结构相同；上述与人脸匹配的个性化表情基可以是包含人脸的身份特征的表情基；个性化表情基序列包括多组个性化表情基系数，个性化表情基系数序列中的个性化表情基系数与个性化表情基的叠加用于表示人脸的表情特征。
应理解，上述第二方面中的初始三维网格序列可以由上述第一方面中的多个初始三维网格组成；上述第二方面中的目标三维网格序列也可以由上述第一方面中的多个目标三维网格组成；上述第二方面中的个性化表情基序列可以由上述第一方面中的多组个性化表情基序列组成。上述第一方面中对初始三维网格，目标三维网格以及个性化表情基系数的限定和解释同样适用于第二方面中的初始三维网格序列中的初始三维网格序列，目标三维网格序列中的目标三维网格以及个性化表情基序列中的个性化表情基系数。
本申请中，由于三维网格包含的顶点信息更加丰富，使得三维网格能够更好地反映人脸的表情特征，另外，通过对三维网格进行变换，能够使得变换后的三维网格与基础表情基的拓扑结构保持一致，从而使得最终根据变换后的三维网格能够生成与人脸的表情特征更加接近的动画表情。
具体地，由于生成的动画表情反映的表情特征与人脸表情的表情特征更加接近，能够使得动画表情更加逼真，可以提高动画表情的显示效果。
第三方面，提供了一种电子设备，所述电子设备包括用于执行上述第一方面及第一方面中的任意一种实现方式中的方法的模块。
第四方面，提供了一种电子设备，所述电子设备包括用于执行上述第二方面及第二方面中的任意一种实现方式中的方法的模块。
第五方面，提供了一种电子设备，包括存储器和处理器，所述存储器用于存储程序，所述处理器用于执行所述存储器存储的程序，当所述存储器存储的程序被处理器执行时，所述处理器用于执行上述第一方面及第一方面中的任意一种实现方式中的方法。
第六方面，提供了一种电子设备，包括存储器和处理器，所述存储器用于存储程序，所述处理器用于执行所述存储器存储的程序，当所述存储器存储的程序被处理器执行时，所述处理器用于执行上述第二方面及第二方面中的任意一种实现方式中的方法。
可选地，上述存储器为非易失性存储器。
可选地，上述存储器与处理器互相耦合在一起。
第七方面，提供了一种计算机可读存储介质，所述计算机可读介质存储介质用于存储程序代码，当所述程序代码被计算机执行时，所述计算机用于执行上述第一方面及第一方面中的任意一种实现方式中的方法。
第八方面，提供了一种计算机可读存储介质，所述计算机可读介质存储介质用于存储程序代码，当所述程序代码被计算机执行时，所述计算机用于执行上述第二方面及第二方面中的任意一种实现方式中的方法。
可选地，上述计算机可读存储介质可以位于电子设备内部，该计算机可读存储介质存储的程序代码可以被电子设备执行。
当该计算机可读存储介质存储的程序代码被电子设备执行时，该电子设备能够执行上述第一方面或者第二方面中任意一种方面的实现方式中的方法。
第九方面，提供了一种芯片，所述芯片包括处理器，所述处理器用于执行上述第一方面及第一方面中的任意一种实现方式中的方法。
第十方面，提供了一种芯片，所述芯片包括处理器，所述处理器用于执行上述第二方面及第二方面中的任意一种实现方式中的方法。
可选地，上述芯片安装在电子设备内部。
第十一方面，提供了一种用于使得计算机或者电子设备执行上述第一方面及第一方面中的任意一种实现方式中的方法的计算机程序(或称计算机程序产品)。
第十二方面，提供了一种用于使得计算机或者电子设备执行上述第二方面及第二方面中的任意一种实现方式中的方法的计算机程序(或称计算机程序产品)。
可选地，上述计算机程序可以存储在电子设备内，该计算机程序可以被电子设备执行。
当电子设备执行上述计算机程序时，该电子设备能够执行上述第一方面或者第二方面中任意一种方面的实现方式中的方法。
可选地，上述电子设备可以是移动终端(例如，智能手机)，电脑，个人数字助理，可穿戴设备，车载设备，物联网设备，增强现实(augmented reality，AR)设备和虚拟现实(virtual reality，VR)设备等等。
另外，上述电子设备还可以是其它能够显示视频画面或者显示图片的设备。
附图说明
图1是本申请实施例的生成动画表情的方法的示意性流程图；
图2是初始三维网格和刚性变形后的三维网格的示意图；
图3初始三维网格和刚性变形后的三维网格中标注的匹配点的示意图；
图4是基础表情基的示意图；
图5是本申请实施例的生成动画表情的方法的示意图；
图6是目标三维网格中的顶点以及多个表情基中相对应的参考顶点的示意图；
图7是目标三维网格中的顶点之间的顶点距离以及多个表情基中相对应的参考顶点之间的参考距离的示意图；
图8是本申请实施例的生成动画表情的方法的示意性流程图；
图9是本申请实施例的电子设备的示意性框图；
图10是本申请实施例的电子设备的示意性框图；
图11是本申请实施例的电子设备的示意性框图；
图12是本申请实施例的电子设备的示意性框图。
具体实施方式
下面将结合附图，对本申请中的技术方案进行描述。
本申请实施例中的生成动画表情的方法可以由电子设备执行。
上述电子设备可以是移动终端(例如，智能手机)，电脑，个人数字助理，可穿戴设备，车载设备，物联网设备，AR设备和VR设备等等。
另外，上述电子设备还可以是其它能够显示视频画面或者显示图片的设备。
上述电子设备可以是运行各种操作系统的设备。例如，上述电子设备可以是运行安卓系统的设备，也可以是运行IOS系统的设备，也可以是运行windows系统的设备。
图1是本申请实施例的生成动画表情的方法的示意性流程图。图1所示的方法可以由电子设备来执行，图1所示的方法包括步骤101至105，下面对这些步骤进行详细的介绍。
101、获取初始三维网格。
在上述步骤101之前，可以通过神经网络从输入视频或图像中提取初始三维网格，该初始三维网格用于表示人脸的表情特征。
上述初始三维网格中可以包含大量的顶点(vertex)，这些顶点与人脸的各个位置是相对应的，这些顶点所处的位置以及相互之间的位置关系可以用于表示人脸的特征表情。
102、对初始三维网格进行变换，得到目标三维网格。
上述目标三维网格的拓扑结构与基础表情基的拓扑结构相同。
在步骤102中，通过对初始三维网格进行变换，能够得到与基础表情基的拓扑结构相同的目标三维网络，实现了拓扑结构的统一，便于后续根据目标三维网格最终得到动画表情。
上述步骤102中对初始三维网格进行变换，得到目标三维网格具体可以包括以下步骤：
201、确定拓扑参照网格；
202、对拓扑参照网格进行刚性变形，得到刚性变形后的拓扑参照网格，该刚性变形后的拓扑参照网格的大小与初始三维网格的大小相同；
203、对刚性变形后的拓扑参照网格和初始三维网格进行贴合处理，直到刚性变形后的拓扑参照网格与初始三维网格的贴合程度满足预设贴合程度；
204、将刚性变形后的拓扑参照网格中的顶点的坐标替换为初始三维网格中与刚性变 形后的网格中的顶点相匹配的顶点的坐标，得到目标三维网格，其中，该目标三维网格具有拓扑参照网格的拓扑结构和初始三维网格的外形。
上述拓扑参照网格可以是与基础表情基或者个性化表基的拓扑结构相同的参照网格，该拓扑参照网格用于对初始三维网格进行变换。
上述步骤202中对拓扑参照网格进行刚性变形，可以是指对拓扑参照网格进行旋转、平移、缩放等操作，使得拓扑参照网格的大小以及朝向与初始三维网格相同。
例如，如图2所示，通过对拓扑参照网格进行刚性变形后得到的刚性变形后的拓扑参照网格的大小和朝向与初始三维网格相同。
在上述步骤203中，可以通过人工的方式来选取刚性变形后的拓扑参照网格与初始三维网格的匹配点，然后根据这些匹配点对刚性变形后的拓扑参照网格和初始三维网格进行贴合处理，使得刚性变形后的拓扑参照网格与初始三维网格的相应的匹配点重合，实现初步贴合，接下来，还可以对刚性变形后的拓扑参照网格与初始三维网格继续贴合，以使得刚性变形后的拓扑参照网格与初始三维网格中的点尽可能多的重合。
例如，如图3所示，可以通过人工的方式选择初始三维网格中的4个顶点(V1,V2,V3,V4)，以及刚性变形后的拓扑参照网格中与该4个顶点相匹配的4个参照顶点(U1,U2,U3,U4)。其中，V1、V2、V3和V4分别是位于初始三维网格中左眼、右眼、上嘴唇和右耳朵的位置的顶点，而U1、U2、U3和U4也分别是刚性变形后的拓扑参照网格中左眼、右眼、上嘴唇和右耳朵的位置的顶点。在人工标注出图3所示的匹配点之后，可以对初始三维网格和刚性变形后的拓扑参照网格进行初次匹配，使得初始三维网格中的4个顶点(V1,V2,V3,V4)分别与刚性变形后的拓扑参照网格中的4个参照顶点(U1,U2,U3,U4)重合。
在上述步骤203中，刚性变形后的拓扑参照网格与初始三维网格的贴合程度满足预设贴合程度，可以是指初始三维网格中的顶点与刚性变形后的拓扑参照网格中相对应的参考顶点之间的距离小于某个预设距离。
在上述步骤204中，进行坐标替换相当于是对刚性变形后的拓扑参照网格进行拉伸，使得刚性变形后的拓扑参照网格中的参照顶点与初始三维网格中相应的顶点完全贴合，拉伸后的网格就是目标三维网格。
另外，在上述步骤203中对刚性变形后的拓扑参照网格和初始三维网格进行贴合处理时，可以是一边对刚性变形后的拓扑参照网格进行变形，一边将刚性变形后的拓扑参照网格与初始三维网格进行贴合，以使得贴合程度满足预设贴合程度。
具体地，上述步骤203中的贴合过程具体包括步骤：
步骤A：根据径向基函数(radial basis function，RBF)对所述刚性变形后的拓扑参照网格进行非刚性变形，得到非刚性变化后的拓扑参照网格；
步骤B：将非刚性变形后的拓扑参照网格与所述初始三维网格贴合；
重复上述步骤A和步骤B，直到非刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足预设贴合程度。
本申请中，通过对上述步骤A和步骤B的迭代，能够使得非刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足贴合要求，使得贴合之后得到的目标三维网格既具有拓扑参照网格的拓扑结构又具有初始三维网格的外形，便于后续生成与人脸的表情特征更加接近的动画表情。
在上述步骤A中对刚性变形后的拓扑参照网格进行非刚性变形时，可以先人工选择刚性变形后的拓扑参照网格与初始三维网格中的几个匹配点，接下来，可以以这几个匹配点为基础继续确定刚性变形后的拓扑参照网格与初始三维网格中的顶点相匹配的顶点。
具体地，可以根据公式(1)确定刚性变形后的拓扑参照网格中与初始三维网格的顶点相匹配的顶点。
u i+S f(u i)＝v i          (1)
其中，u i为刚性变形后的拓扑参照网格中的顶点，v i为初始三维网格中与u i相匹配的顶点，S f(u i)为径向基函数。
S f(u i)中的系数可以通过已有的匹配点(例如，初始三维网格中的顶点V1,V2和V3以及刚性变形后的拓扑参照网格中相匹配的顶点U1,U2和U3)构建的线性方程来获取。
103、根据基础表情基确定与人脸匹配的个性化表情基。
其中，上述与人脸匹配的个性化表情基可以是包含人脸的身份特征的表情基。另外，上述个性化表情基的拓扑结构与基础表情基的拓扑结构相同，上述基础表情基可以是根据应用场景预先设置好的表情基。
上述人脸的身份特征可以是指人脸器官的形状或者特点，例如，人脸的身份特征可以是大眼睛、小眼睛、大嘴巴以及高鼻梁等等。
而人脸的表情特征是指人脸的某些器官的动作情况，例如，人脸的表情特征可以包括眨眼睛，咧嘴，皱眉以及鼓腮等等。
在某些情况下，只看人脸的外形可能无法区分人脸的表情特征和人脸的身份特征，例如，对于大嘴来说，可能无法区分是本身嘴巴比较大还是由于咧嘴表情产生的。因此，在生成人脸对应的动画表情的过程中，一般需要区分开人脸的身份特征和表情特征，
应理解，上述个性化表情基可以包含多种表情基，每种表情基对应一种局部表情。例如，上述个性化表情基可以包含常见的47种局部表情(该47种表情能够覆盖人脸的眉毛、眼睛、鼻子、嘴、下巴以及面颊等部位的表情)对应的47种表情基。
上述47种局部表情可以包含人脸常见的一些表情，例如，眨眼，张嘴巴，皱眉，抬眉等等。另外，上述表情还可以包括对人脸常见的一些表情进行细分之后得到的表情，例如，上述47种局部表情可以包括左眉内侧上移，右眼下眼睑提升以及上嘴唇外翻等表情。
由于个性化表情基是根据基础表情基生成的，因此，个性化表情基的拓扑结构也与基础表情基的拓扑结构相同。因此，本申请中，通过对初始三维网格进行变换，能够得到与基础表情基拓扑结构相同的目标三维网格，便于后续根据具有相同拓扑结构的目标三维网格和个性化表情基生成更准确的个性化表情基系数，能够提高最终生成的动画表情的显示效果。
在步骤103中根据基础表情基确定与人脸匹配的个性化表情基时，根据基础表情基的构成情况不同，可以采用不同的表情基来推导得到个性化表情基。
第一种情况：基础表情基由多组不同身份特征的表情基构成。
在第一种情况下，每种身份特征对应一组表情基，每组表情基中包含多个表情的表情基。
例如，如图4所示，基础表情基由多组不同身份特征的表情基组成，其中，每组身份的表情基包含多个表情基，该多个表情基对应不同的局部表情。
再如，基础表情基可以由常见的50种身份特征的表情基构成，其中，每种身份特征对应一组表情基，每种身份特征对应的一组表情基可以包含常见的47种表情的表情基。
假设基础表情基由常见的50种身份特征的表情基构成，每种身份特征对应的一组表情基包含47种表情的表情基，那么，基础表情基一共由47*50＝2350个表情基构成。
在第一种情况下，根据基础表情基确定与人脸匹配的个性化表情基，包括：根据多组不同身份特征的表情基确定个性化表情基。
在根据多组不同身份特征的表情基确定个性化表情基还需要用到人脸的身份特征信息，以使得生成的个性化表情基是包含人脸身份特征的表情基。
具体地，根据多组不同身份特征的表情基确定个性化表情基，包括：根据人脸的身份特征信息，对多组不同身份特征的表情基进行叠加，以得到个性化表情基。
在第一种情况下，当基础表情基是由多组不同身份特征的表情基构成时，根据该多组表情基能够更准确地推导出个性化表情基。
第二种情况：基础表情基为平均表情基。
在第二种情况下，根据基础表情基确定与人脸匹配的个性化表情基，包括：根据平均身份特征的表情基确定个性化表情基。
其中，上述平均表情基可以是通过对多组不同身份特征的表情基进行处理得到的，该平均表情基能够反映一种平均身份特征。例如，上述平均表情基可以对上述第一种情况中的50种身份特征对应的50组表情基进行叠加得到的。
在根据平均表情基确定个性化表情基还需要用到人脸的身份特征信息，以使得生成的个性化表情基是包含人脸身份特征的表情基。
具体地，根据平均表情基确定个性化表情基，包括：根据人脸的身份特征信息，对平均表情基进行变换，以得到个性化表情基。
在第二种情况下，当基础表情基为平均表情基时，能够减少存储基础表情基时的存储开销，节省存储资源。
另外，上述基础表情基还可以由任意一种身份特征对应的一组表情基构成。例如，可以选择第一种情况中50种身份特征中的任意一种身份特征对应的一组表情基作为基础表情基，或者，也可以选择图4中的任意一列表情基(图4中的每一列表情基对应一种身份特征)作为基础表情基。在这种情况下，也能够减少存储基础表情基时的存储开销，节省存储资源。
104、根据目标三维网格和个性化表情基，确定个性化表情基系数。
其中，个性化表情基系数与个性化表情基的叠加用于表示人脸的表情特征。在这里，个性化表情基系数与个性化表情基的叠加可以是个性化表情基系数与个性化表情基的线性组合或者非线性组合。
在步骤104中，目标三维网格是初始三维网格变换得到的，该目标三维网格能够表示人脸的表情特征，而个性化表情基与个性化表情基系数的叠加也能够表示人脸的表情特征，因此，在步骤104中，可以根据目标三维网格和个性化表情基参数来反推个性化表情基系数。
105、根据个性化表情基系数生成动画表情。
本申请中，由于三维网格包含的顶点信息更加丰富，使得三维网格能够更好地反映人 脸的表情特征，另外，通过对三维网格进行变换，能够使得变换后的三维网格与基础表情基的拓扑结构保持一致，从而使得最终根据变换后的三维网格能够生成与人脸的表情特征更加接近的动画表情。
具体地，由于生成的动画表情反映的表情特征与人脸表情的表情特征更加接近，能够使得动画表情更加逼真，可以提高动画表情的显示效果。
具体地，在步骤105中，可以根据个性化表情基系数和拟人角色表情基来生成动画表情。这里的拟人角色表情基具体可以是虚拟人物表情基或者拟人动物表情基。
上述拟人角色表情基包含的局部表情的种类与个性化表情基包含的表情的种类相同，并且，拟人角色表情基中的各个表情的表情语义分别与个性化表情基中包含的各个表情的语义相同。
例如，个性化表情基对应的局部表情如表1的第一列所示，拟人角色表情基对应的局部表情如表1的第二列所示。如表1所示，个性化表情基对应的局部表情中包含眨眼、张嘴巴、抬眉、左眉内侧上移和右眼下眼睑提升等表情时，拟人角色表情基也对应相同的局部表情。
表1
个性化表情基对应的局部表情 拟人角色表情基对应的局部表情
眨眼 眨眼
张嘴巴 张嘴巴
抬眉 抬眉
左眉内侧上移 左眉内侧上移
右眼下眼睑提升 右眼下眼睑提升
… …
在根据个性化表情基系数和拟人角色表情基来生成动画表情时，相当于是采用一系列的系数对拟人角色表情基对应的多种局部表情进行叠加，最终得到与人脸的表情特征更加接近的动画表情。
为了更形象的理解本申请实施例的生成动画表情的方法的整个过程，下面结合图5对本申请实施例的生成动画表情的方法的流程进行介绍。
图5是本申请实施例的生成动画表情的方法的示意图。
如图5所示，可以通过对初始三维网格进行变形得到目标三维网格，通过基础表情基获得个性化表情基，其中，在根据基础表情基确定与人脸匹配的个性化表情基的时候还可以参照初始三维网格或者目标三维网格，以使得个性化表情基包含人脸的身份特征。在获取到目标三维网格和个性化表情基之后，就可以根据目标三维网格和个性化表情基确定个性化表情基系数，进而再集合个性化表情基系数和拟人角色表情基来得到动画表情。
应理解，图5所示的各个过程已在上文图1所示的方法中进行了详细的介绍，为了避免不必要的重复，这里不再详细描述。
可选地，上述步骤104中根据目标三维网格和个性化表情基，确定个性化表情基系数的具体实现方式有多种，下面分别对这些的具体实现方式进行详细的介绍。
第一种方式：根据多种表情基中的参考顶点的坐标与目标三维网格中的顶点的坐标，确定个性化表情基系数。
具体地，根据多种表情基中的参考顶点的坐标的线性组合与目标三维网格中的顶点的坐标的差，确定个性化表情基系数。
具体地，在第一种方式下，个性化表情基包含多种表情基，该多种表情基分别对应不同的表情，该多种表情基中的每种表情基的参考顶点是每种表情基中与所述目标三维网格中的顶点处于相对应位置的点。
例如，如图6所示，目标三维网格中存在顶点A，表情基1、表情基2和表情基3分别包括参考顶点A1、A2和A3，参考顶点A1、A2和A3分别是表情基1、表情基2和表情基3中与目标三维网格中的顶点A处于相对应位置的点。那么，这三种表情基中的参考顶点的坐标的线性组合与目标三维网格中的顶点的坐标的差可以用算式(2)来表示。
P(A)-[x*P(A1)+y*P(A2)+z*P(A3)]        (2)
其中，P(A)表示顶点A的坐标值，P(A1)、P(A2)和P(A3)分别表示参考顶点A1、A2和A3的坐标值，(x,y,z)为线性组合系数。
应理解，图6中仅仅是以3个表情基为例进行举例，本申请并不限定表情基的具体数目。
在第一种方式下，根据多种表情基中的参考顶点的坐标的线性组合与目标三维网格中的顶点的坐标的差，确定个性化表情基系数，具体包括：确定多个第一差值；根据多个第一差值分别与相应权重值的乘积的和确定个性化表情基系数。
其中，上述多个第一差值为目标三维网格中的多个顶点的坐标分别与多种表情基中的相应参考顶点的线性组合的差值。
可选地，根据多个第一差值分别与相应权重值的乘积的和确定个性化表情基系数，包括：在多个第一差值分别与相应权重值的乘积的和取最小值时，将多种表情基中的参考顶点的坐标线性组合时的线性组合系数确定为个性化表情基系数。
应理解，可以通过上述算式(2)得到多个第一差值，其中，每个第一差值分别为目标三维网格中的不同顶点的坐标与多种表情基中的相应参考点的线性组合的差值。
例如，如图6所示，目标三维网格中还包括顶点B和顶点C，表情基1中包括参考顶点B1和C1，表情基2中包括参考顶点B2和C2，表情基3中包括参考顶点B3和C3，其中，参考顶点B1、B2和B3分别是表情基1、表情基2和表情基3中与目标三维网格中的顶点B处于相对应位置的点，参考顶点C1、C2和C3分别是表情基1、表情基2和表情基3中与目标三维网格中的顶点C处于相对应位置的点。那么，可以根据图6得到3个第一差值，该三个第一差值分别如算式(2)至算式(4)所示。
P(A)-[x*P(A1)+y*P(A2)+z*P(A3)]       (2)
P(B)-[x*P(B1)+y*P(B2)+z*P(B3)]          (3)
P(C)-[x*P(C1)+y*P(C2)+z*P(C3)]        (4)
在获取到上述算式(2)至算式(4)之后，可以求上述算式(2)、算式(3)和算式(4)的和，将该3个算式的和最小时对应的系数(x,y,z)的取值确定为个性化表情基系数。
本申请中，通过将多个第一差值分别与相应权重值的乘积的和取最小值时对应的线性组合系数确定为个性化表情基系数，能够使得个性化表情基系数尽可能真实的反映人脸的表情特征，便于后续生成与人脸的表情特征更加接近的动画表情。
第二种方式：根据多种表情基中的参考距离的线性组合与目标三维网格中的顶点距离的差，确定个性化表情基系数。
在第二种方式下，个性化表情基包含多种表情基，该多种表情基分别对应不同的表情，上述根据目标三维网格和个性化表情基，确定个性化表情基系数，具体包括：
确定目标三维网格的顶点距离；确定多种表情基中的每种表情基对应的参考距离，得到多个参考距离；
根据多个参考距离的线性组合与顶点距离的差，确定个性化表情基系数。
其中，上述顶点距离为目标三维网格中的两个顶点之间的距离，上述每种表情基对应的参考距离为每种表情基中的两个参考顶点之间的距离，该两个参考顶点是每种表情基中分别与目标三维网格中的两个顶点处于相对应位置的点。
例如，如图7所示，目标三维网格中包括顶点M和顶点N，顶点M和顶点N之间的距离为D，表情基1中包括顶点M1和N1，M1和N1分别是表情基1中与顶点M和顶点N处于相对应位置的点，顶点M1和顶点N1之间的距离为D1；表情基2中包括顶点M2和N2，M2和N2分别是表情基2中与顶点M和顶点N处于相对应位置的点，顶点M2和顶点N2之间的距离为D2；表情基3中包括顶点M3和N3，M3和N3分别是表情基3中与顶点M和顶点N处于相对应位置的点，顶点M3和顶点N3之间的距离为D3。其中，D为目标三维网格中的顶点距离，D1为表情基1中的参考距离，D2为表情基2中的参考距离，D3为表情基3中的参考距离。
以图7为例，三种表情基的参考距离的线性组合与顶点距离的差可以如算式(5)所示。
d(M,N)-[u*d(M1,N1)+v*d(M2,N2)+w*d(M3,N3)]        (5)
应理解，图7中仅仅是以3个表情基为例进行举例，本申请并不限定表情基的具体数目。
在第二种方式下，根据多个参考距离的线性组合与顶点距离的差，确定个性化表情基系数，具体包括：确定多个第二差值；根据多个第二差值分别与相应权重值的乘积的和确定个性化表情基系数。
其中，上述多个第二差值为目标三维网格中的多个顶点距离分别与多种表情基中对应的参考距离的线性组合的差值。
可选地，根据多个第二差值分别与相应权重值的乘积的和确定个性化表情基系数，包括：在多个第二差值分别与相应权重值的乘积的和取最小值时，将多个参考距离线性组合时的线性组合系数确定为个性化表情基系数。
应理解，可以通过上述算式(5)得到多个第二差值，其中，每个第二差值分别为目标三维网格中的不同顶点距离与多种表情基中的相应的参考距离的线性组合的差值。
例如，如图7所示，目标三维网格包含的顶点和顶点之间的顶点距离，以及表情基1、表情基2和表情基3包含的参考顶点以及参考顶点距离如下：
目标三维网格中还包括顶点M、N、R和S，其中，顶点M和N之间的顶点距离为d(M,N)，顶点R和S之间的顶点距离为d(R,S)；
表情基1中包括参考顶点M1、N1、R1和S1，这些参考顶点分别是与顶点M、N、R和S处于相对应位置的点，顶点M1和N1之间的顶点距离为d(M1,N1)，顶点R1和S1 之间的顶点距离为d(R1,S1)；
表情基2中包括参考顶点M2、N2、R2和S2，这些参考顶点分别是与顶点M、N、R和S处于相对应位置的点，顶点M2和N2之间的顶点距离为d(M2,N2)，顶点R2和S2之间的顶点距离为d(R2,S2)；
表情基3中包括参考顶点M3、N3、R3和S3，这些参考顶点分别是与顶点M、N、R和S处于相对应位置的点，顶点M3和N3之间的顶点距离为d(M3,N3)，顶点R3和S3之间的顶点距离为d(R3,S3)。
那么，可以根据图7得到3个第二差值，该3个第二差值分别如算式(5)至算式(7)所示。
d(M,N)-[u*d(M1,N1)+v*d(M2,N2)+w*d(M3,N3)]       (5)
d(E,F)-[u*d(E1,F1)+v*d(E2,F2)+w*d(E3,F3)]        (6)
d(R,S)-[u*d(R1,S1)+v*d(R2,S2)+w*d(R3,S3)]        (7)
在获取到上述算式(5)至算式(7)之后，可以求上述算式(5)、算式(6)和算式(7)的和，将该3个算式的和最小时对应的系数(u,v,w)的取值确定为个性化表情基系数。
本申请中，通过将多个第二差值分别与相应权重值的乘积的和取最小值时对应的线性组合系数确定为个性化表情基系数，能够使得个性化表情基系数尽可能真实的反映人脸的表情特征，便于后续生成与人脸的表情特征更加接近的动画表情。
上述第一种方式和第二种方式是分别依据第一差值和第二差值来确定个性化表情基系数。实际上也可以综合第一差值和第二差值来共同确定表情基系数，下面对这种方式(第三种方式)进行详细的介绍。
第三种方式：根据多种表情基中的参考顶点的坐标的线性组合与目标三维网格中的顶点的坐标的差，以及多种表情基中的参考距离的线性组合与目标三维网格中的顶点距离的差综合确定个性化表情基系数。
第三种方式相当于上述第一方式与第二方式的结合，在第三种方式中，根据目标三维网格和个性化表情基，确定个性化表情基系数，包括：
确定多个第一差值；
确定多个第一差值分别与相应权重值的乘积的和，得到第一取值；
确定多个第二差值；
确定多个第二差值分别与相应权重值的乘积的和，得到第二取值；
根据第一取值与第二取值的总和确定个性化表情基系数。
在根据第一取值与第二取值的总和确定个性化表情基系数时，可以将第一取值和第二取值的和最小时对应的线性组合系数确定为个性化表情基系数。
例如，以图6和图7为例，可以得到3个第一差值和3个第二差值，其中，3个第一差值可以用算式(8)至算式(10)来表示，3个第二差值可以用算式(11)至算式(13)来表示。
P(A)-[x*P(A1)+y*P(A2)+z*P(A3)]         (8)
P(B)-[x*P(B1)+y*P(B2)+z*P(B3)]         (9)
P(C)-[x*P(C1)+y*P(C2)+z*P(C3)]         (10)
d(M,N)-[x*d(M1,N1)+y*d(M2,N2)+z*d(M3,N3)]      (11)
d(E,F)-[x*d(E1,F1)+y*d(E2,F2)+z*d(E3,F3)]        (12)
d(R,S)-[x*d(R1,S1)+y*d(R2,S2)+z*d(R3,S3)]       (13)
在求解个性化表情基系数时，可以将算式(8)至算式(13)的和取最小值时对应的系数(x,y,z)的取值确定为个性化表情基系数。
在第三种方式中，通过两种差值来来综合确定个性化表情基参数，能够更准确地确定个性化表情基系数。
可选地，还可以采用第四种方式来确定个性化表情基系数。
第四种方式：根据公式(14)来确定个性化表情基系数。
E＝E point+E edge+E lasso          (14)
在上述公式(14)中，E point表示多种表情基中的参考顶点的坐标的线性组合与目标三维网格中的顶点的坐标的差的和的平方，E edge表示多种表情基中的参考距离的线性组合与目标三维网格中的顶点距离的差的和的平方，E lasso是一个约束函数，用于使得最终求得的线性化表情基系数稀疏化(用尽量少的局部表情来表示人脸表情)。E point、E edge和E lasso可以通过公式(15)至(17)获得。
Figure PCTCN2019113041-appb-000001
Figure PCTCN2019113041-appb-000002
Figure PCTCN2019113041-appb-000003
在上述公式(15)中，B表示表情基的数目，b表示第b个表情基，N表示目标三维网格中的顶点的数目(每个表情基中的顶点的数目也是N)，i表示第i个顶点，
Figure PCTCN2019113041-appb-000004
表示第b个表情基中第i个顶点的三维坐标，x b表示
Figure PCTCN2019113041-appb-000005
的组合系数，p i表示目标三维网格中的第i个顶点的三维坐标，w i为
Figure PCTCN2019113041-appb-000006
的权重值；
在上述公式(16)中，B表示表情基的数目，b表示第b个表情基，i和j分别表示第i个顶点和第j个顶点，K表示边的数目，
Figure PCTCN2019113041-appb-000007
表示第b个表情基中的第i个顶点和第j个顶点之间的距离，e ij表示目标三维网格中的第i个顶点和第j个顶点之间的距离，x b表示
Figure PCTCN2019113041-appb-000008
的组合系数，w ij为
Figure PCTCN2019113041-appb-000009
的权重值；
在上述公式(17)中，x b表示
Figure PCTCN2019113041-appb-000010
的组合系数，
Figure PCTCN2019113041-appb-000011
表示所有的表情基系数的绝对值的和。
在根据上述公式(14)确定个性化表情基系数时，可以将E的数值最小时对应的线性组合系数确定为个性化表情基系数。
另外，本申请中，通过采用E lasso对个性化表情基系数进行约束，能够使得最终获取得到的个性化表情基系数比较稀疏(个性化表情基系数中存在较多的0)，使得最终利用较少表情的表情基就能够表达出动画表情。
上文结合图1至图7对本申请实施例的生成动画表情的方法进行了详细介绍。实际上，在图1所示的方法中还可以获取多个初始三维网格，以得到初始三维网格序列，然后 再根据该初始三维网格序列最终生成动画表情，下面结合图8对根据初始三维网格序列最终生成动画表情的过程进行介绍。
图8是本申请实施例的生成动画的方法的示意性流程图。图8所示的方法可以由电子设备来执行，图8所示的方法包括步骤301至305，下面对这些步骤进行介绍。
301、获取初始三维网格序列。
上述步骤301中的初始三维网格序列包括多个初始三维网格，该多个初始三维网格分别用于表示人脸在多个不同时刻的表情特征。
上述初始三维网格序列中的初始三维网格与步骤101中获得的初始三维网格相同，上文中对初始三维网格的限定和解释同样适用于步骤301中的初始三维网格序列中的初始三维网格，为避免重复，这里不再详细描述。
302、对初始三维网格序列进行变换，得到目标三维网格序列。
其中，上述目标三维网格序列包括多个目标三维网格，多个目标三维网格与基础表情基的拓扑结构相同。
在步骤302中，可以通过对初始三维网格序列中的每个初始三维网格分别进行变换，以得到多个目标三维网格，进而得到目标三维网格序列，其中，对每个初始三维网格进行变换得到目标三维网格的过程可以参照上文步骤102的相关内容，为避免不必要的重复，这里不再详细描述。
303、根据基础表情基确定与人脸匹配的个性化表情基。
其中，上述与人脸匹配的个性化表情基可以是包含人脸的身份特征的表情基。
在步骤303中，可以参照上文步骤103中的具体实现方式来获取个性化表情基，这里不再详细描述。
304、根据目标三维网格序列和个性化表情基，确定个性化表情基系数序列。
其中，个性化表情基序列包括多组个性化表情基系数，个性化表情基系数序列中的个性化表情基系数与个性化表情基的叠加用于表示人脸的表情特征；
上述步骤304中，可以根据目标三维网格序列中的每个目标三维网格和个性化表情基，生成每个目标三维网格对应的个性化表情基系数，进而得到多个个性化表情基系数组成的个性化表情基系数序列。
其中，在根据每个目标三维网格和个性化表情基生成个性化表情基系数时，可以采用步骤104中的具体方式来获取个性化表情基系数。
305、根据个性化表情基系数序列生成动画表情。
步骤305中生成的动画表情可以是一段连续的动画表情。其中，个性化表情基系数序列中的每个个性化表情基系数生成动画表情过程可以参照上文步骤105的相关内容，这里不再详细描述。
在步骤305中，可以根据个性化表情基系数序列和拟人角色表情基来生成动画表情。这里的拟人角色表情基具体可以是虚拟人物表情基或者拟人动物表情基。
上文中对拟人角色表情基的限定和解释同样适用于步骤305中涉及的拟人角色表情基。
在根据个性化表情基系数序列和拟人角色表情基来生成动画表情时，可以先根据每个个性化表情基系数和拟人角色表情基生成动画画面，从而得到多个动画画面，接下来把 这些动画画面拼接起来就能得到最终的动画表情。
本申请中，由于三维网格包含的顶点信息更加丰富，使得三维网格能够更好地反映人脸的表情特征，另外，通过对三维网格进行变换，能够使得变换后的三维网格与基础表情基的拓扑结构保持一致，从而使得最终根据变换后的三维网格能够生成与人脸的表情特征更加接近的动画表情。
具体地，由于生成的动画表情反映的表情特征与人脸表情的表情特征更加接近，能够使得动画表情更加逼真，可以提高动画表情的显示效果。
图9是本申请实施例的电子设备的示意性框图。图9所示的电子设备1000，可以执行图1所示的生成动画表情的方法中的各个步骤，包括：
获取模块1001，用于获取初始三维网格，所述初始三维网格中的顶点用于表示人脸的表情特征；
变换模块1002，用于对所述初始三维网格进行变换，得到目标三维网格，所述目标三维网格与基础表情基的拓扑结构相同；
处理模块1003，所述处理模块1003用于：
根据所述基础表情基确定与所述人脸匹配的个性化表情基，其中，所述个性化表情基是包含所述人脸的身份特征的表情基；
根据所述目标三维网格和所述个性化表情基，确定个性化表情基系数，其中，所述个性化表情基系数与所述个性化表情基的叠加用于表示所述人脸的表情特征；
根据所述个性化表情基系数生成动画表情。
本申请中，由于三维网格包含的顶点信息更加丰富，使得三维网格能够更好地反映人脸的表情特征，另外，通过对三维网格进行变换，能够使得变换后的三维网格与基础表情基的拓扑结构保持一致，从而使得最终根据变换后的三维网格能够生成与人脸的表情特征更加接近的动画表情。
可选地，作为一个实施例，所述个性化表情基包含多种表情基，所述多种表情基分别对应不同的表情，所述处理模块1003用于：根据所述多种表情基中的每种表情基的参考顶点的坐标和所述目标三维网格中的顶点的坐标，确定所述个性化表情基系数，其中，所述多种表情基中的每种表情基的参考顶点是所述每种表情基中与所述目标三维网格中的顶点处于相对应位置的点。
可选地，作为一个实施例，所述处理模块1003用于：
确定多个第一差值，所述多个第一差值为所述目标三维网格中的多个顶点的坐标分别与所述多种表情基中的相应参考顶点的线性组合的差值；
根据所述多个第一差值分别与相应权重值的乘积的和确定所述个性化表情基系数。
可选地，作为一个实施例，所述多个第一差值分别对应多个不同的权重值。
可选地，作为一个实施例，所述个性化表情基包含多种表情基，所述多种表情基分别对应不同的表情，所述处理模块用于：
确定所述目标三维网格的顶点距离，所述顶点距离为所述目标三维网格中的两个顶点之间的距离；
确定所述多种表情基中的每种表情基对应的参考距离，得到多个参考距离，其中，所述每种表情基对应的参考距离为所述每种表情基中的两个参考顶点之间的距离，所述两个 参考顶点是所述每种表情基中分别与所述目标三维网格中的两个顶点处于相对应位置的点；
根据所述多个参考距离的线性组合与所述顶点距离的差，确定所述个性化表情基系数。
可选地，作为一个实施例，所述处理模块1003用于：
确定多个第二差值，所述多个第二差值为所述目标三维网格中的多个顶点距离分别与所述多种表情基中对应的参考距离的线性组合的差值；
根据所述多个第二差值分别与相应权重值的乘积的和确定所述个性化表情基系数。
可选地，作为一个实施例，所述多个第二差值分别对应多个不同的权重值。
可选地，作为一个实施例，所述变换模块1002用于：
确定拓扑参照网格，所述拓扑参照网格与所述基础表情基的拓扑结构相同；
对所述拓扑参照网格进行刚性变形，得到刚性变形后的拓扑参照网格，所述刚性变形后的拓扑参照网格的大小与所述初始三维网格的大小相同；
对所述刚性变形后的拓扑参照网格和所述初始三维网格进行贴合处理，直到所述刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足预设贴合程度；
将所述刚性变形后的拓扑参照网格中的顶点的坐标替换为所述初始三维网格中与所述刚性变形后的网格中的顶点相匹配的顶点的坐标，得到所述目标三维网格，其中，所述目标三维网格具有所述拓扑参照网格的拓扑结构和所述初始三维网格的外形。
可选地，作为一个实施例，所述变换模块1002用于重复执行步骤A和步骤B，直到所述非刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足预设贴合程度；
其中，所述步骤A和步骤B分别为：
步骤A：根据RBF对所述刚性变形后的拓扑参照网格进行非刚性变形，得到非刚性变化后的拓扑参照网格；
步骤B：将所述非刚性变形后的拓扑参照网格与所述初始三维网格贴合。
图10是本申请实施例的电子设备的示意性框图。图10所示的电子设备2000，可以执行图8所示的生成动画表情的方法中的各个步骤，包括：
获取模块2001，用于获取初始三维网格序列，所述初始三维网格序列包括多个初始三维网格，所述多个初始三维网格分别用于表示人脸在多个不同时刻的表情特征；
变换模块2002，用于对所述初始三维网格序列进行变换，得到目标三维网格序列，所述目标三维网格序列包括多个目标三维网格，所述多个目标三维网格与基础表情基的拓扑结构相同；
处理模块2003，所述处理模块2003用于：
根据所述基础表情基确定与所述人脸匹配的个性化表情基，其中，所述个性化表情基是包含所述人脸的身份特征的表情基；
根据所述目标三维网格序列和所述个性化表情基，生成所述个性化表情基系数序列，所述个性化表情基序列包括多组个性化表情基系数，所述个性化表情基系数序列中的个性化表情基系数与所述个性化表情基的叠加用于表示所述人脸的表情特征；
根据所述个性化表情基系数序列生成动画表情。
本申请中，由于三维网格包含的顶点信息更加丰富，使得三维网格能够更好地反映人 脸的表情特征，另外，通过对三维网格进行变换，能够使得变换后的三维网格与基础表情基的拓扑结构保持一致，从而使得最终根据变换后的三维网格能够生成与人脸的表情特征更加接近的动画表情。
图11是本申请实施例的电子设备的示意性框图。图11所示的电子设备3000包括：
存储器3001，用于存储程序；
处理器3002，用于执行存储器3001中存储的程序，当存储器3001中存储的程序被处理器3002执行时，处理器3002用于执行图1或者图8所示的生成动画表情的方法中的各个步骤。
应理解，上述电子设备1000中的获取模块1001、变换模块1002以及处理模块1003可以相当于电子设备3000中的处理器3002。上述电子设备2000中的获取模块2001、变换模块2002以及处理模块2003可以相当于电子设备3000中的处理器3002。
上述处理器3002具体可以是中央处理器(central processing unit，CPU)、现场可编程门阵(field programmable gate array，FPGA)、图形处理器(graphics processing unit，GPU)或者其他具有数据处理功能的芯片等等。上述处理器3002还可以是CPU、FPGA、GPU以及数据处理芯片中的至少两种组成。
图12是本申请实施例的电子设备的示意性框图。
图12中的电子设备4000包括通信模块4010、传感器4020、用户输入模块4030、输出模块4040、处理器4050、音视频输入模块4060、存储器4070以及电源4080。
上述电子设备4000可以执行上述本申请实施例的生成动画表情的方法的各个步骤，具体地，电子设备4000中的处理器4050可以执行本申请实施例的生成动画表情的方法的各个步骤。
下面对电子设备4000中的各个模块或者单元进行详细的介绍。
通信模块4010可以包括至少一个能使该电子设备与其他电子设备之间进行通信的模块。例如，通信模块4010可以包括有线网络接口、广播接收模块、移动通信模块、无线因特网模块、局域通信模块和位置(或定位)信息模块等其中的一个或多个。
传感器4020可以感知用户的一些操作，传感器4020可以包括距离传感器，触摸传感器等等。传感器4020可以感知用户触摸屏幕或者靠近屏幕等操作。
用户输入模块4030，用于接收输入的数字信息、字符信息或接触式触摸操作/非接触式手势，以及接收与系统的用户设置以及功能控制有关的信号输入等。用户输入模块4030包括触控面板和/或其他输入设备。
输出模块4040包括显示面板，用于显示由用户输入的信息、提供给用户的信息或系统的各种菜单界面等。可选的，可以采用液晶显示器(liquid crystal display，LCD)或有机发光二极管(organic light-emitting diode,OLED)等形式来配置显示面板。在其他一些实施例中，触控面板可覆盖显示面板上，形成触摸显示屏。另外，输出模块4040还可以包括音频输出模块、告警器以及触觉模块等。
音视频输入模块4060，用于输入音频信号或视频信号。音视频输入模块4060可以包括摄像头和麦克风。
电源4080可以在处理器4050的控制下接收外部电力和内部电力，并且提供整个电子设备各个模块运行时需要的电力。
处理器4050可以指一个或多个处理器，例如，处理器4050可以包括一个或多个中央处理器，或者包括一个中央处理器和一个图形处理器，或者包括一个应用处理器和一个协处理器(例如微控制单元或神经网络处理器)。当处理器4050包括多个处理器时，这多个处理器可以集成在同一块芯片上，也可以各自为独立的芯片。一个处理器可以包括一个或多个物理核，其中物理核为最小的处理模块。
存储器4070存储计算机程序，该计算机程序包括操作系统程序4071和应用程序4072等。典型的操作系统如微软公司的Windows，苹果公司的MacOS等用于台式机或笔记本的系统，又如谷歌公司开发的基于
Figure PCTCN2019113041-appb-000012
的安卓
Figure PCTCN2019113041-appb-000013
系统等用于移动终端的系统。当本申请实施例的资源调度电子设备通过软件的方式实现时，可以认为是通过应用程序4071来具体实现的。
存储器4070可以是以下类型中的一种或多种：闪速(flash)存储器、硬盘类型存储器、微型多媒体卡型存储器、卡式存储器(例如SD或XD存储器)、随机存取存储器(random access memory,RAM)、静态随机存取存储器(static RAM,SRAM)、只读存储器(read only memory,ROM)、电可擦除可编程只读存储器(electrically erasable programmable read-only memory，EEPROM)、可编程只读存储器(programmable ROM,PROM)、磁存储器、磁盘或光盘。在其他一些实施例中，存储器4070也可以是因特网上的网络存储设备，系统可以对在因特网上的存储器4070执行更新或读取等操作。
处理器4050用于读取存储器4070中的计算机程序，然后执行计算机程序定义的电子设备，例如处理器4050读取操作系统程序4072从而在该系统运行操作系统以及实现操作系统的各种功能，或读取一种或多种应用程序4071，从而在该系统上运行应用。
例如，上述存储器4070可以存储一种计算机程序(该计算机程序是本申请实施例的资源调度电子设备对应的程序)，当处理器4050执行该极端及程序时，处理器4050能够执行本申请实施例的资源调度电子设备。
存储器4070还存储有除计算机程序之外的其他数据4073，例如，存储器4070可以存储本申请的资源调度电子设备中涉及的绘帧线程的负载特征，绘帧线程的负载预测值等等。
图12所示的电子设备4000中的各个模块或者单元的连接关系仅为一种示例。本申请实施例中的电子设备还可以是图12中的各个模块或者单元采用其他连接关系(例如，所有模块或者单元通过总线连接)连接得到的电子设备。
上述电子设备3000中的存储器3001相当于电子设备4000中的存储器4070，电子设备3000中的处理器3002相当于电子设备4000中的存储器4050。
上述电子设备1000、电子设备2000、电子设备3000以及电子设备4000具体可以是移动终端(例如，智能手机)，电脑，个人数字助理，可穿戴设备，车载设备，物联网设备，AR设备和VR设备等等。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (20)

  1. 一种生成动画表情的方法，其特征在于，包括：
    获取初始三维网格，所述初始三维网格中的顶点用于表示人脸的表情特征；
    对所述初始三维网格进行变换，得到目标三维网格，所述目标三维网格与基础表情基的拓扑结构相同；
    根据所述基础表情基确定与所述人脸匹配的个性化表情基；
    根据所述目标三维网格和所述个性化表情基，确定个性化表情基系数，其中，所述个性化表情基系数与所述个性化表情基的叠加用于表示所述人脸的表情特征；
    根据所述个性化表情基系数生成动画表情。
  2. 如权利要求1所述的方法，其特征在于，所述个性化表情基包含多种表情基，所述多种表情基分别对应不同的表情，所述根据所述目标三维网格和所述个性化表情基，确定个性化表情基系数，包括：
    根据所述多种表情基中的每种表情基的参考顶点的坐标和所述目标三维网格中的顶点的坐标，确定所述个性化表情基系数，其中，所述多种表情基中的每种表情基的参考顶点是所述每种表情基中与所述目标三维网格中的顶点处于相对应位置的点。
  3. 如权利要求2所述的方法，其特征在于，所述根据所述多种表情基中的每种表情基的参考顶点的坐标和所述目标三维网格中的顶点的坐标，确定所述个性化表情基系数，包括：
    确定多个第一差值，所述多个第一差值为所述目标三维网格中的多个顶点的坐标分别与所述多种表情基中的相应参考顶点的线性组合的差值；
    根据所述多个第一差值分别与相应权重值的乘积的和确定所述个性化表情基系数。
  4. 如权利要求3所述的方法，其特征在于，所述多个第一差值分别对应多个不同的权重值。
  5. 如权利要求1-4中任一项所述的方法，其特征在于，所述个性化表情基包含多种表情基，所述多种表情基分别对应不同的表情，所述根据所述目标三维网格和所述个性化表情基，确定个性化表情基系数，包括：
    确定所述目标三维网格的顶点距离，所述顶点距离为所述目标三维网格中的两个顶点之间的距离；
    确定所述多种表情基中的每种表情基对应的参考距离，得到多个参考距离，其中，所述每种表情基对应的参考距离为所述每种表情基中的两个参考顶点之间的距离，所述两个参考顶点是所述每种表情基中分别与所述目标三维网格中的两个顶点处于相对应位置的点；
    根据所述多个参考距离的线性组合与所述顶点距离的差，确定所述个性化表情基系数。
  6. 如权利要求5所述的方法，其特征在于，所述根据所述多个参考距离的线性组合与所述顶点距离的差，确定所述个性化表情基系数，包括：
    确定多个第二差值，所述多个第二差值为所述目标三维网格中的多个顶点距离分别与 所述多种表情基中对应的参考距离的线性组合的差值；
    根据所述多个第二差值分别与相应权重值的乘积的和确定所述个性化表情基系数。
  7. 如权利要求6所述的方法，其特征在于，所述多个第二差值分别对应多个不同的权重值。
  8. 如权利要求1-7中任一项所述的方法，其特征在于，所述对所述初始三维网格进行变换，得到目标三维网格，包括：
    确定拓扑参照网格，所述拓扑参照网格与所述基础表情基的拓扑结构相同；
    对所述拓扑参照网格进行刚性变形，得到刚性变形后的拓扑参照网格，所述刚性变形后的拓扑参照网格的大小与所述初始三维网格的大小相同；
    对所述刚性变形后的拓扑参照网格和所述初始三维网格进行贴合处理，直到所述刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足预设贴合程度；
    将所述刚性变形后的拓扑参照网格中的顶点的坐标替换为所述初始三维网格中与所述刚性变形后的网格中的顶点相匹配的顶点的坐标，得到所述目标三维网格，其中，所述目标三维网格具有所述拓扑参照网格的拓扑结构和所述初始三维网格的外形。
  9. 如权利要求8所述的方法，其特征在于，所述对所述刚性变形后的拓扑参照网格和所述初始三维网格进行贴合处理，直到所述刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足预设贴合程度，包括：
    重复执行步骤A和步骤B，直到所述非刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足预设贴合程度；
    其中，所述步骤A和步骤B分别为：
    步骤A：根据径向基函数RBF对所述刚性变形后的拓扑参照网格进行非刚性变形，得到非刚性变化后的拓扑参照网格；
    步骤B：将所述非刚性变形后的拓扑参照网格与所述初始三维网格贴合。
  10. 一种生成动画表情的方法，其特征在于，包括：
    获取初始三维网格序列，所述初始三维网格序列包括多个初始三维网格，所述多个初始三维网格分别用于表示人脸在多个不同时刻的表情特征；
    对所述初始三维网格序列进行变换，得到目标三维网格序列，所述目标三维网格序列包括多个目标三维网格，所述多个目标三维网格与基础表情基的拓扑结构相同；
    根据所述基础表情基确定与所述人脸匹配的个性化表情基；
    根据所述目标三维网格序列和所述个性化表情基，生成个性化表情基系数序列，所述个性化表情基序列包括多组个性化表情基系数，所述个性化表情基系数序列中的个性化表情基系数与所述个性化表情基的叠加用于表示所述人脸的表情特征；
    根据所述个性化表情基系数序列生成动画表情。
  11. 一种电子设备，其特征在于，包括：
    获取模块，用于获取初始三维网格，所述初始三维网格中的顶点用于表示人脸的表情特征；
    变换模块，用于对所述初始三维网格进行变换，得到目标三维网格，所述目标三维网格与基础表情基的拓扑结构相同；
    处理模块，所述处理模块用于：
    根据所述基础表情基确定与所述人脸匹配的个性化表情基；
    根据所述目标三维网格和所述个性化表情基，确定个性化表情基系数，其中，所述个性化表情基系数与所述个性化表情基的叠加用于表示所述人脸的表情特征；
    根据所述个性化表情基系数生成动画表情。
  12. 如权利要求11所述的电子设备，其特征在于，所述个性化表情基包含多种表情基，所述多种表情基分别对应不同的表情，所述处理模块用于：
    根据所述多种表情基中的每种表情基的参考顶点的坐标和所述目标三维网格中的顶点的坐标，确定所述个性化表情基系数，其中，所述多种表情基中的每种表情基的参考顶点是所述每种表情基中与所述目标三维网格中的顶点处于相对应位置的点。
  13. 如权利要求12所述的电子设备，其特征在于，所述处理模块用于：
    确定多个第一差值，所述多个第一差值为所述目标三维网格中的多个顶点的坐标分别与所述多种表情基中的相应参考顶点的线性组合的差值；
    根据所述多个第一差值分别与相应权重值的乘积的和确定所述个性化表情基系数。
  14. 如权利要求13所述的电子设备，其特征在于，所述多个第一差值分别对应多个不同的权重值。
  15. 如权利要求11-14中任一项所述的电子设备，其特征在于，所述个性化表情基包含多种表情基，所述多种表情基分别对应不同的表情，所述处理模块用于：
    确定所述目标三维网格的顶点距离，所述顶点距离为所述目标三维网格中的两个顶点之间的距离；
    确定所述多种表情基中的每种表情基对应的参考距离，得到多个参考距离，其中，所述每种表情基对应的参考距离为所述每种表情基中的两个参考顶点之间的距离，所述两个参考顶点是所述每种表情基中分别与所述目标三维网格中的两个顶点处于相对应位置的点；
    根据所述多个参考距离的线性组合与所述顶点距离的差，确定所述个性化表情基系数。
  16. 如权利要求15所述的电子设备，其特征在于，所述处理模块用于：
    确定多个第二差值，所述多个第二差值为所述目标三维网格中的多个顶点距离分别与所述多种表情基中对应的参考距离的线性组合的差值；
    根据所述多个第二差值分别与相应权重值的乘积的和确定所述个性化表情基系数。
  17. 如权利要求16所述的电子设备，其特征在于，所述多个第二差值分别对应多个不同的权重值。
  18. 如权利要求11-17中任一项所述的电子设备，其特征在于，所述变换模块用于：
    确定拓扑参照网格，所述拓扑参照网格与所述基础表情基的拓扑结构相同；
    对所述拓扑参照网格进行刚性变形，得到刚性变形后的拓扑参照网格，所述刚性变形后的拓扑参照网格的大小与所述初始三维网格的大小相同；
    对所述刚性变形后的拓扑参照网格和所述初始三维网格进行贴合处理，直到所述刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足预设贴合程度；
    将所述刚性变形后的拓扑参照网格中的顶点的坐标替换为所述初始三维网格中与所述刚性变形后的网格中的顶点相匹配的顶点的坐标，得到所述目标三维网格，其中，所述 目标三维网格具有所述拓扑参照网格的拓扑结构和所述初始三维网格的外形。
  19. 如权利要求18所述的电子设备，其特征在于，所述变换模块用于重复执行步骤A和步骤B，直到所述非刚性变形后的拓扑参照网格与所述初始三维网格的贴合程度满足预设贴合程度；
    其中，所述步骤A和步骤B分别为：
    步骤A：根据径向基函数RBF对所述刚性变形后的拓扑参照网格进行非刚性变形，得到非刚性变化后的拓扑参照网格；
    步骤B：将所述非刚性变形后的拓扑参照网格与所述初始三维网格贴合。
  20. 一种电子设备，其特征在于，包括：
    获取模块，用于获取初始三维网格序列，所述初始三维网格序列包括多个初始三维网格，所述多个初始三维网格分别用于表示人脸在多个不同时刻的表情特征；
    变换模块，用于对所述初始三维网格序列进行变换，得到目标三维网格序列，所述目标三维网格序列包括多个目标三维网格，所述多个目标三维网格与基础表情基的拓扑结构相同；
    处理模块，所述处理模块用于：
    根据所述基础表情基确定与所述人脸匹配的个性化表情基；
    根据所述目标三维网格序列和所述个性化表情基，生成个性化表情基系数序列，所述个性化表情基序列包括多组个性化表情基系数，所述个性化表情基系数序列中的个性化表情基系数与所述个性化表情基的叠加用于表示所述人脸的表情特征；
    根据所述个性化表情基系数序列生成动画表情。
PCT/CN2019/113041 2018-12-29 2019-10-24 生成动画表情的方法和电子设备 Ceased WO2020134436A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/360,866 US11410369B2 (en) 2018-12-29 2021-06-28 Method for generating animated expression and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811636045.2A CN111383308B (zh) 2018-12-29 2018-12-29 生成动画表情的方法和电子设备
CN201811636045.2 2018-12-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/360,866 Continuation US11410369B2 (en) 2018-12-29 2021-06-28 Method for generating animated expression and electronic device

Publications (1)

Publication Number Publication Date
WO2020134436A1 true WO2020134436A1 (zh) 2020-07-02

Family

ID=71126932

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/113041 Ceased WO2020134436A1 (zh) 2018-12-29 2019-10-24 生成动画表情的方法和电子设备

Country Status (3)

Country Link
US (1) US11410369B2 (zh)
CN (2) CN111383308B (zh)
WO (1) WO2020134436A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112927328A (zh) * 2020-12-28 2021-06-08 北京百度网讯科技有限公司 表情迁移方法、装置、电子设备及存储介质
CN113744375A (zh) * 2021-09-10 2021-12-03 北京爱奇艺科技有限公司 数据的迁移方法及装置、电子设备、可读存储介质

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112530016B (zh) * 2020-10-30 2022-11-11 北京字跳网络技术有限公司 一种道具吸附的方法、装置、设备及存储介质
US20220172431A1 (en) * 2020-11-30 2022-06-02 Sony Interactive Entertainment LLC Simulated face generation for rendering 3-d models of people that do not exist
US11756334B2 (en) * 2021-02-25 2023-09-12 Qualcomm Incorporated Facial expression recognition
CN113192165B (zh) * 2021-05-12 2025-03-07 广州虎牙科技有限公司 基于个性化表情基的控制信息生成方法、装置、电子设备和可读存储介质
CN115546361A (zh) * 2021-06-30 2022-12-30 腾讯科技（深圳）有限公司 三维卡通形象处理方法、装置、计算机设备和存储介质
CN113609960B (zh) * 2021-08-03 2023-07-28 北京奇艺世纪科技有限公司 一种目标图片的人脸驱动方法及装置
CN114581572A (zh) * 2022-03-21 2022-06-03 中国科学院深圳先进技术研究院 基于二维的三维表情拟合方法、装置、设备及存储介质
CN114708636A (zh) * 2022-04-01 2022-07-05 成都市谛视科技有限公司 一种密集人脸网格表情驱动方法、装置及介质
CN115170817B (zh) * 2022-07-21 2023-04-28 广州大学 基于三维人-物网格拓扑增强的人物交互检测方法
CN115393482A (zh) * 2022-08-08 2022-11-25 网易（杭州）网络有限公司 一种表情动画重定向方法、装置及电子设备
CN115374330B (zh) * 2022-08-12 2025-10-17 功夫动漫股份有限公司 一种三维动漫制作的表情动作控制方法及装置
CN115512193A (zh) * 2022-08-22 2022-12-23 网易（杭州）网络有限公司 面部表情生成方法和装置
US12293466B2 (en) * 2023-03-24 2025-05-06 Electronic Arts Inc. Systems and methods for generating a model database with blendshape representation
CN117274528B (zh) * 2023-08-31 2025-01-21 北京百度网讯科技有限公司 三维网格数据的获取方法、装置、电子设备与可读存储介质
CN117132713A (zh) * 2023-09-07 2023-11-28 北京百度网讯科技有限公司 模型训练方法、数字人驱动方法及相关装置
CN117237487A (zh) * 2023-11-15 2023-12-15 北京蔚领时代科技有限公司 创建与编辑BlendShape节点的方法、装置、存储介质及电子设备
US20250252639A1 (en) * 2024-02-06 2025-08-07 Zoom Video Communications, Inc. 3d digital virtual character generation for video conferencing
CN120088837B (zh) * 2025-04-30 2025-09-05 中国电子科技集团公司信息科学研究院 基于形状空间的人脸连续表情迁移方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101149840A (zh) * 2006-09-20 2008-03-26 清华大学 一种复杂表情仿真系统及其实现方法
US20140240324A1 (en) * 2008-12-04 2014-08-28 Intific, Inc. Training system and methods for dynamically injecting expression information into an animated facial mesh
CN104217454A (zh) * 2014-08-21 2014-12-17 中国科学院计算技术研究所 一种视频驱动的人脸动画生成方法
CN106204698A (zh) * 2015-05-06 2016-12-07 北京蓝犀时空科技有限公司 为自由组合创作的虚拟形象生成及使用表情的方法和系统
CN108986221A (zh) * 2018-07-27 2018-12-11 河海大学常州校区 一种基于模板人脸逼近的不规范三维人脸网格纹理方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100416611C (zh) * 2006-09-14 2008-09-03 浙江大学 一种基于网格拓扑建模的三维人脸动画制作方法
CN100562895C (zh) * 2008-01-14 2009-11-25 浙江大学 一种基于区域分割和分段学习的三维人脸动画制作的方法
CN103854306A (zh) * 2012-12-07 2014-06-11 山东财经大学 一种高真实感的动态表情建模方法
CN103198508A (zh) * 2013-04-07 2013-07-10 河北工业大学 人脸表情动画生成方法
CN104978764B (zh) * 2014-04-10 2017-11-17 华为技术有限公司 三维人脸网格模型处理方法和设备
US10586380B2 (en) * 2016-07-29 2020-03-10 Activision Publishing, Inc. Systems and methods for automating the animation of blendshape rigs
CN108305309B (zh) * 2018-04-13 2021-07-20 腾讯科技(成都)有限公司 基于立体动画的人脸表情生成方法和装置
CN108629294A (zh) * 2018-04-17 2018-10-09 华南理工大学 基于变形图的人体与人脸网格模板拟合方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101149840A (zh) * 2006-09-20 2008-03-26 清华大学 一种复杂表情仿真系统及其实现方法
US20140240324A1 (en) * 2008-12-04 2014-08-28 Intific, Inc. Training system and methods for dynamically injecting expression information into an animated facial mesh
CN104217454A (zh) * 2014-08-21 2014-12-17 中国科学院计算技术研究所 一种视频驱动的人脸动画生成方法
CN106204698A (zh) * 2015-05-06 2016-12-07 北京蓝犀时空科技有限公司 为自由组合创作的虚拟形象生成及使用表情的方法和系统
CN108986221A (zh) * 2018-07-27 2018-12-11 河海大学常州校区 一种基于模板人脸逼近的不规范三维人脸网格纹理方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112927328A (zh) * 2020-12-28 2021-06-08 北京百度网讯科技有限公司 表情迁移方法、装置、电子设备及存储介质
CN112927328B (zh) * 2020-12-28 2023-09-01 北京百度网讯科技有限公司 表情迁移方法、装置、电子设备及存储介质
CN113744375A (zh) * 2021-09-10 2021-12-03 北京爱奇艺科技有限公司 数据的迁移方法及装置、电子设备、可读存储介质

Also Published As

Publication number Publication date
CN116843804A (zh) 2023-10-03
US11410369B2 (en) 2022-08-09
CN111383308A (zh) 2020-07-07
CN111383308B (zh) 2023-06-23
US20210327116A1 (en) 2021-10-21

Similar Documents

Publication Publication Date Title
WO2020134436A1 (zh) 生成动画表情的方法和电子设备
US11587300B2 (en) Method and apparatus for generating three-dimensional virtual image, and storage medium
US20230169571A1 (en) Image-to-image translation using unpaired data for supervised learning
US10599914B2 (en) Method and apparatus for human face image processing
CN112819947A (zh) 三维人脸的重建方法、装置、电子设备以及存储介质
WO2021120834A1 (zh) 基于生物识别的手势识别方法、装置、计算机设备及介质
CN108197547A (zh) 人脸姿态估计方法、装置、终端及存储介质
RU2671990C1 (ru) Способ отображения трехмерного лица объекта и устройство для него
CN111047509A (zh) 一种图像特效处理方法、装置及终端
CN115049799A (zh) 3d模型和虚拟形象的生成方法和装置
CN116309983B (zh) 虚拟人物模型的训练方法、生成方法、装置和电子设备
CN111507259B (zh) 脸部特征提取方法、装置、电子设备
CN115147265A (zh) 虚拟形象生成方法、装置、电子设备和存储介质
JP2021086462A (ja) データ生成方法、データ生成装置、モデル生成方法、モデル生成装置及びプログラム
CN114549728A (zh) 图像处理模型的训练方法、图像处理方法、装置及介质
US20240386661A1 (en) Method and system of generating customized three-dimensional images
CN116740788A (zh) 虚拟人说话视频生成方法、服务器、设备及存储介质
CN113380269B (zh) 视频图像生成方法、装置、设备、介质和计算机程序产品
CN110096145A (zh) 基于混合现实和神经网络的心理状态显示方法及装置
CN110059739B (zh) 图像合成方法、装置、电子设备和计算机可读存储介质
CN114742951B (zh) 素材生成、图像处理方法、装置、电子设备及存储介质
CN114332365B (zh) 虚拟人物形象生成方法、装置、电子设备及存储介质
CN107194980A (zh) 人脸模型构建方法、装置及电子设备
KR20230098063A (ko) 객체 기반의 이미지 생성 방법 및 장치
US12205209B1 (en) Virtual try-on based on predetermined cloth patterns driven by generative ai

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19904914

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19904914

Country of ref document: EP

Kind code of ref document: A1