JPWO2021225868A5

JPWO2021225868A5 -

Info

Publication number: JPWO2021225868A5
Application number: JP2023513265A
Authority: JP
Publication date: 2024-05-14

Claims

1. A method for generating a mesh deformation of a face model, comprising:
generating a first plurality of deformation maps by applying a first plurality of neural network training models;
generating a second plurality of deformation maps by applying a second plurality of neural network training models, the second plurality of deformation maps having a greater resolution than the first plurality of deformation maps;
extracting a first plurality of vertex offsets based on the first plurality of deformation maps;
extracting a second plurality of vertex offsets based on the second plurality of deformation maps;
applying the first plurality of vertex offsets and the second plurality of vertex offsets to a mesh of the facial model to generate a mesh deformation of the facial model;
A method comprising:

The method of claim 1, wherein extracting the first plurality of vertex offsets includes performing an interpolation of at least one of the first plurality of deformation maps at a plurality of vertex positions.

The method of claim 2, wherein the interpolation includes bilinear or cubic interpolation.

The method of claim 1, wherein applying the first and second vertex offsets to a mesh of the face model includes adding the first and second vertex offsets to values of vertices of the mesh.

The method of claim 1, wherein each of the first plurality of neural network training models includes a convolutional neural network (CNN) model.

The step of applying the first plurality of vertex offsets and the second plurality of vertex offsets to the mesh comprises:
applying the first plurality of vertex offsets to values of all vertices of the mesh;
applying the second plurality of vertex offsets to values of at most a subset of the vertices of the mesh to generate the mesh deformation;
The method of claim 1 , comprising:

The method of claim 6, wherein the subset of vertices of the mesh corresponds to one or more regions of the mesh that exhibit a level of approximation error that exceeds a certain threshold.

The step of applying the first plurality of vertex offsets and the second plurality of vertex offsets to the face model mesh comprises:
adding the first plurality of vertex offsets to values of a plurality of vertices of the mesh to generate a plurality of intermediate vertex values;
adding the second plurality of vertex offsets to at most a subset of the plurality of intermediate vertex values to generate the mesh deformation;
The method of claim 1 , comprising:

identifying one or more rigid mesh segments of the face model that move rigidly during deformation based on the first plurality of deformation maps and the second plurality of deformation maps;
performing a separate approximation of each of the one or more rigid mesh segments;
The method of claim 1 further comprising:

The method of claim 9, wherein performing a separate approximation of each of the one or more rigid mesh segments includes performing rigid rotations and translations on the vertices of the rigid mesh segments.

1. A system for generating a mesh deformation of a face model, comprising:
The system includes one or more processors, the one or more processors:
generating a first plurality of deformation maps by applying a first plurality of neural network training models;
configured to generate a second plurality of deformation maps by applying a second plurality of neural network training models, the second plurality of deformation maps having a greater resolution than the first plurality of deformation maps;
The one or more processors:
Extracting a first plurality of vertex offsets based on the first plurality of deformation maps;
Extracting a second plurality of vertex offsets based on the second plurality of deformation maps;
applying the first plurality of vertex offsets and the second plurality of vertex offsets to a mesh of the facial model to generate a mesh deformation of the facial model.
It is configured as follows:
system.

The system of claim 11, wherein the one or more processors are further configured to extract the first plurality of vertex offsets by performing an interpolation of at least one of the first plurality of deformation maps at a plurality of vertex positions.

The system of claim 12, wherein the interpolation includes bilinear or cubic interpolation.

The system of claim 11, wherein the one or more processors are further configured to apply the first and second vertex offsets to a mesh of the face model by adding the first vertex offsets to values of vertices of the mesh.

The system of claim 11, wherein each of the first plurality of neural network training models includes a convolutional neural network (CNN) model.

The one or more processors:
applying the first plurality of vertex offsets to values of all vertices of the mesh; and
and applying the first and second plurality of vertex offsets to the mesh by applying the second plurality of vertex offsets to values of at most a subset of the vertices of the mesh to generate the mesh deformation.
The system of claim 11.

The system of claim 16, wherein the subset of vertices of the mesh corresponds to one or more regions of the mesh that exhibit a level of approximation error above a certain threshold.

1. A machine-readable, non-transitory medium having machine-executable instructions stored thereon for generating a mesh deformation of a face model, the medium comprising:
The instruction:
generating a first plurality of deformation maps by applying a first plurality of neural network training models;
generating a second plurality of deformation maps by applying a second plurality of neural network training models, the second plurality of deformation maps having a greater resolution than the first plurality of deformation maps;
extracting a first plurality of vertex offsets based on the first plurality of deformation maps;
extracting a second plurality of vertex offsets based on the second plurality of deformation maps;
applying the first plurality of vertex offsets and the second plurality of vertex offsets to a mesh of the facial model to generate a mesh deformation of the facial model;
(c) A machine-readable, non-transitory medium, including