WO2020108304A1 - 人脸网格模型的重建方法、装置、设备和存储介质 - Google Patents

人脸网格模型的重建方法、装置、设备和存储介质 Download PDF

Info

Publication number
WO2020108304A1
WO2020108304A1 PCT/CN2019/117944 CN2019117944W WO2020108304A1 WO 2020108304 A1 WO2020108304 A1 WO 2020108304A1 CN 2019117944 W CN2019117944 W CN 2019117944W WO 2020108304 A1 WO2020108304 A1 WO 2020108304A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
template
feature points
model
grid
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/117944
Other languages
English (en)
French (fr)
Inventor
刘更代
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Baiguoyuan Information Technology Co Ltd
Original Assignee
Guangzhou Baiguoyuan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Baiguoyuan Information Technology Co Ltd filed Critical Guangzhou Baiguoyuan Information Technology Co Ltd
Priority to US17/297,342 priority Critical patent/US11972527B2/en
Publication of WO2020108304A1 publication Critical patent/WO2020108304A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/30—Polynomial surface description
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G06T17/205—Re-meshing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2021—Shape modification

Definitions

  • the embodiments of the present application relate to the field of image processing, for example, to a method, device, device and storage medium for reconstruction of a face grid model.
  • a three-dimensional grid registration algorithm is generally used to deform a grid template model until it matches the shape of the scanned data.
  • the face mesh template is transformed by sparse data interpolation to obtain the corresponding three-dimensional face mesh model; or a small number of points are collected in the face mesh template and matched with the scan data to establish a deformed graph network, and The corresponding post-processing process is performed to obtain the corresponding three-dimensional face grid model.
  • the resolution of the face grid model is usually not high, but when the deformation is gradually matched with the scanned data, the details of the scanned data still need to be retained At this time, the acquisition speed of the accurate corresponding points in the face scan data and the face grid template is slow, the overall process operation is cumbersome, and there may be problems of slow convergence and instability;
  • the post-processing of the deformed graph network also has the problem of slow speed and poor smoothness in the obtained 3D face mesh model.
  • Embodiments of the present application provide a method, device, equipment, and storage medium for reconstruction of a face grid model, to solve the problems of slow reconstruction of the face grid model and poor smoothness in the related art, and to realize the face network
  • the three-dimensional reconstruction of the lattice model improves the reconstruction rate and reconstruction effect of the face mesh model.
  • An embodiment of the present application provides a method for reconstructing a face mesh model.
  • the method includes:
  • An embodiment of the present application provides a face mesh model reconstruction device.
  • the device includes:
  • the data acquisition module is set to acquire the face scan data to be reconstructed and the three-dimensional face grid template
  • the hierarchical matching module is set to extract key feature points in the three-dimensional face grid template in layers, and in turn, according to the posture matching positions of the key feature points extracted in layers in the face scan data, the three-dimensional person
  • the face mesh template is transformed to obtain the target face mesh model
  • the model reconstruction module is set to obtain global feature points in the target face mesh model, and according to the posture matching position of the global feature points in the face scan data, to the target face mesh model Perform deformation to obtain the reconstructed face mesh model.
  • An embodiment of the present application provides a device, which includes:
  • One or more processors are One or more processors;
  • Storage device set to store one or more programs
  • the one or more processors implement the reconstruction method of the face grid model described in any embodiment of the present application.
  • An embodiment of the present application provides a computer-readable storage medium that stores a computer program, and when the program is executed by a processor, a method for reconstructing a face mesh model described in any embodiment of the present application is implemented.
  • FIG. 1A is a flowchart of a method for reconstructing a face mesh model provided in Embodiment 1 of the present application;
  • FIG. 1B is a schematic diagram of a low-precision three-dimensional face grid template provided by Embodiment 1 of the present application;
  • FIG. 1C is a schematic diagram of a high-precision three-dimensional face grid template provided in Embodiment 1 of the present application;
  • FIG. 1D is a schematic diagram of face scanning data provided by Embodiment 1 of the present application.
  • FIG. 1E is a schematic diagram of a result of registering a three-dimensional face grid template directly according to some feature points according to Embodiment 1 of the present application;
  • FIG. 1F is a schematic diagram of a result of post-processing the deformed three-dimensional face mesh template provided by Embodiment 1 of the present application;
  • FIG. 1F is a schematic diagram of a result of post-processing the deformed three-dimensional face mesh template provided by Embodiment 1 of the present application;
  • FIG. 2A is a flowchart of another method for reconstructing a face mesh model provided in Embodiment 2 of the present application;
  • FIG. 2B is a schematic diagram of a result of registering a subdivided three-dimensional face grid template provided by Embodiment 2 of the present application;
  • FIG. 2C is a schematic diagram of a result of performing accuracy restoration on a reconstructed high-precision face mesh model provided by Embodiment 2 of the present application;
  • FIG. 2C is a schematic diagram of a result of performing accuracy restoration on a reconstructed high-precision face mesh model provided by Embodiment 2 of the present application;
  • FIG. 3 is a schematic structural diagram of a face mesh model reconstruction device according to Embodiment 3 of the present application.
  • FIG. 4 is a schematic structural diagram of a device provided in Embodiment 4 of the present application.
  • the embodiment of the present application mainly adopts a hierarchical 3D mesh registration algorithm to match the deformation of the 3D face mesh template with the face scan data, and again performs global feature points in the deformed 3D face mesh template The details are matched to obtain the reconstructed face mesh model, which solves the problems of the slow reconstruction speed and poor smoothness of the face mesh model in the related art, and improves the smoothness of the reconstructed face mesh model.
  • FIG. 1A is a flowchart of a method for reconstructing a face mesh model according to Embodiment 1 of the present application.
  • This embodiment can be applied to any kind of 3D model processing for creating a realistic face mesh model from face scan data In the device.
  • the solution of the embodiment of the present application may be applied in the process of how to create a corresponding realistic face grid model according to face scan data.
  • the reconstruction method of the face grid model provided by this embodiment may be performed by the apparatus for reconstructing the face grid model provided by the embodiment of the present application.
  • the apparatus may be implemented by software and/or hardware, and integrated in Among the devices performing the method, the device may be any three-dimensional model processing device with image processing capabilities.
  • the method may include S110-S130.
  • S110 Obtain face scan data to be reconstructed and a three-dimensional face grid template.
  • the three-dimensional face grid template is a standard face grid model.
  • the three-dimensional face grid template is a three-dimensional face model with a grid distribution schematic created in advance according to the standard face shape.
  • the selected The pixel resolution of the model is different, and the creation of a three-dimensional face grid template with different accuracy is to make the number and density of grids in the created three-dimensional face grid template different, as shown in Figure 1B.
  • the three-dimensional face grid template is the high-precision three-dimensional face grid template shown in FIG. 1C.
  • a pre-created 3D face mesh template is stored in the 3D model processing device that executes the reconstruction method of the face mesh model in this embodiment.
  • the face scan data can be matched with the three-dimensional face grid template to obtain a reconstructed face grid model.
  • the face scan data is three-dimensional scan data obtained by scanning the same type of face for different types of face mesh models to be reconstructed, as shown in FIG. 1D.
  • the reconstructed facial mesh model in this embodiment is mainly used for video screen display containing facial expressions in entertainment applications or game websites, in video products containing facial expressions, the facial expression animation technology is mostly data Driven, that is, relying on a parametric model of a face grid model with multiple types of expressions established on a face database, so it is necessary to obtain a large amount of lifelikeness from the original multi-face scan data containing different expressions The face grid model, thereby creating the face database.
  • the face scan data can be directly stored in advance in the face of this embodiment before the face grid model is reconstructed.
  • it can be directly obtained when reconstructing the face mesh model; or, when reconstructing the face mesh model, the face scan corresponding to the expression is scanned by the three-dimensional scanning technology in real time
  • the data directly performs reconstruction of the face grid model, which is not limited in this embodiment.
  • the face scan data corresponding to the expression can be selected, which can be directly stored in advance
  • Corresponding face scan data can be found in a large number of face scan data; it can also be based on the expression information to scan the face of the corresponding expression in real time through three-dimensional scanning technology to obtain the corresponding face scan data in this reconstruction process.
  • it is also necessary to obtain a pre-stored three-dimensional face mesh template that is, a standard face mesh model, through the same three-dimensional face mesh template and multiple different Face scan data is matched to obtain reconstructed different face grid models.
  • the same three-dimensional face grid template is processed, which reduces the noise and defect interference in the process of converting different face scan data to the corresponding face grid model, and solves the problem of different face scans.
  • the grid model directly generated by the data does not have the problem of consistent topology.
  • Feature points are points in the model that have distinctive characteristics and can effectively reflect the essential characteristics of the model and can identify the target part in the model.
  • feature points refer to pixel points that can represent the facial features in the three-dimensional face grid template. That is, multiple mesh vertices distributed in the three-dimensional face mesh template.
  • the key feature points are the local feature points in the 3D face grid template.
  • the key feature points are the feature points that can clearly represent the positions of the five senses in the three-dimensional face mesh template, such as the eyes, nose, mouth, and chin, or the detailed positions in the face such as the eye socket, nose, etc. .
  • the posture matching position is the position of the face target part represented by each key feature point in the three-dimensional face grid template, and the same target part determined corresponds to the matching position in the face scan data, for example, represents three-dimensional
  • the pose matching position of the nose feature points in the face grid template in the face scan model is the position of the nose in the face scan data.
  • the feature points in the three-dimensional face grid template need to be determined, and according to the matching positions of the feature points in the face scan data, control
  • the three-dimensional face grid template is deformed so that the position of each feature point in the deformed three-dimensional face grid template matches the position of the feature point in the face scan data one by one.
  • the face is obtained by Scan the data and all the feature points accurately corresponding to the 3D face grid template, use sparse data interpolation to deform the 3D face grid template, and reconstruct the corresponding face grid model, but use all the feature points to
  • the deformation method makes the reconstruction speed of the face mesh model very low, and some accurately corresponding feature points are difficult to obtain in advance; and if the reconstruction of the face mesh model is obtained by deforming some feature points Because there are fewer feature points, it will lead to unclear details in the reconstructed face grid model, which makes the reconstructed face grid model smoother, and directly matches part of the feature points with the face scan data The results are shown in Figure 1E. After matching, the details of the face mesh model are not clear, and the overall smoothness is poor.
  • the key feature points in the three-dimensional face grid template are extracted hierarchically and matched with the face scan data in order to solve the problem that the reconstruction speed of the face grid model is very low when all feature points are extracted, and The problem that the detailed features of the reconstructed face grid model when extracting some feature points is not clear.
  • the three-dimensional face grid template can be hierarchically classified through algorithms such as scale-invariant feature transform (SIFT) algorithm and accelerated segmented test feature (features from accelerated segment test (FAST) algorithm).
  • SIFT scale-invariant feature transform
  • FAST accelerated segmented test feature
  • the key feature points in are extracted, and the posture matching positions of the multiple key feature points extracted in layers in the face scan data are determined in turn, and the corresponding three-dimensional face grid template is controlled to deform according to the corresponding posture matching positions, The target face grid model matching the face scan data is obtained.
  • the corresponding face scan data and the original three-dimensional face mesh template are obtained.
  • the original three-dimensional face mesh template firstly, a part of the mesh vertices are extracted as key feature points, and the extracted The posture matching position of multiple key feature points in the face scan data, so that the original three-dimensional face grid template is deformed according to the posture matching position, so that each key feature point extracted this time after the deformation
  • the position in the 3D face grid template corresponds to the posture matching position of the key feature point in the face scan data.
  • the original 3D face grid template will also drive other feature points not extracted this time during the deformation process.
  • the deformed 3D face mesh template is less similar to the face scan data; therefore, it is possible to continue to extract part of the net except for the first extracted key feature points in the first deformed 3D face mesh template
  • the lattice vertex serves as the key feature point extracted at the second level, and determines the posture matching position of the multiple key feature points extracted by the second level in the face scan data, and continues to pull the first deformed position according to the posture matching position
  • the three-dimensional face grid template is deformed again, and according to the key feature points extracted in layers, the corresponding three-dimensional face grid template is circularly deformed according to the above process until the accurate matching with the face scan data is achieved, thereby obtaining the corresponding Target face mesh model.
  • the key feature points are obtained in layers, and the three-dimensional face grid template is deformed in sequence according to the posture matching position in the face scan data, so that the sequentially deformed three-dimensional face grid template can gradually approach the human face Scanning data gradually reduces the amount of calculation in the subsequent deformation process.
  • the operation in the deformation process is gradually reduced. Volume, which improves the reconstruction rate of the face grid model.
  • deforming the three-dimensional face grid template may include S1210-S1220.
  • the Iterated Closest Points (ICP) algorithm is used to determine the pose matching position of the key feature points extracted in layers in the face scan data.
  • the ICP algorithm is a point registration method based on contour features, which is mainly to determine the transformation relationship between two piles of point cloud data, that is, multiple keys in the three-dimensional face grid template in this embodiment.
  • the transformation relationship between the feature points and the corresponding feature points in the face scan data can be calculated and used
  • the minimum root mean square algorithm calculates the registration of the key feature point and the nearest point, obtains the corresponding coordinate transformation, and determines the transformed three-dimensional face grid template.
  • the transformation relationship between the three-dimensional face grid template determined by the ICP algorithm and the face scan data can be used to sequentially determine the key feature points extracted in layers on the human face The attitude matching position in the scan data.
  • the ICP algorithm is used to determine the posture matching position of the key feature points extracted in the first level in the face scan data, and the original three-dimensional face grid template is deformed, and then the first three-dimensional face grid after the first deformation
  • the key feature points of the second level are extracted from the template, and the posture matching position of the key feature points extracted by the second level in the face scan data is determined by the ICP algorithm, and the three-dimensional face grid template after the first deformation is deformed again , And so on, to determine the posture matching position of the key feature points extracted in layers in the face scan data, so as to deform the corresponding three-dimensional face grid template.
  • the three-dimensional grid registration algorithm is a model matching algorithm, which can gradually deform a three-dimensional face grid template until it matches the three-dimensional shape of the face scan data.
  • the key feature points extracted at different levels are equivalent to re-establishing a low-resolution mesh model for the three-dimensional face mesh template, and then the corresponding three-dimensional is pulled through the deformation of the low-resolution mesh model.
  • the face grid template is projected onto the face scan data, that is, the three-dimensional face grid template is deformed.
  • the deformation process of the three-dimensional face grid template can be achieved by minimizing the energy function, which can be optimized using the Gauss-Newton iterative algorithm.
  • the energy function can determine the constraint conditions corresponding to the other feature points during the deformation of the corresponding three-dimensional face grid template to ensure the smoothness of the reconstructed face grid model.
  • the point-to-point distance and the point-to-face distance are used in this embodiment, that is, the distance from the key feature point in the 3D face grid template to the closest point in the face scan data, and multiple keys Feature point to face scan data distance:
  • n i is the normal vector at the closest point c i of the key feature point v i on the face scan data in the three-dimensional face grid template. Doing so allows multiple key feature points to be scanned along the face The data slides tangentially to increase the robustness during the deformation process
  • i is the index (or number) of the vertices of the mesh
  • ⁇ is the weighting coefficient.
  • the above formula can be used according to the posture matching position of the key feature points extracted in layers in the face scan data and the three-dimensional grid registration algorithm in turn, for the corresponding three-dimensional face net
  • the lattice template is deformed, and the movement of other feature points in the deformation process is constrained to ensure the consistency of the movement of the feature points and improve the smoothness of the reconstructed face grid model.
  • the three-dimensional deformation of the three-dimensional face grid template is three layers, that is, three times of deformation matching processing, which can ensure that the target face grid model obtained after the deformation is similar to the face scan data.
  • the number of hierarchical deformations is not limited, and can be determined according to the similarity between the deformed three-dimensional face grid template and face scan data.
  • the target face mesh model is obtained by matching the face scan data with the three-dimensional face mesh template multiple times and deformed, the target face mesh model is also a mesh model.
  • the global feature points are all the mesh vertices distributed in the target face mesh model.
  • the target face mesh model is a three-dimensional face mesh template that is matched with face scan data multiple times and deformed multiple times, the target face mesh model and face scan data can be guaranteed The degree of similarity is high. At this time, in order to reconstruct the detailed features in the target face mesh model, a face mesh model that more closely matches the face scan data is obtained.
  • the target face mesh can be obtained by The global feature points in the model, and according to the posture matching positions of the multiple global feature points in the face scan data, referring to the deformation process mentioned in S120, deform the target face mesh model to ensure the reconstructed face While the mesh model is smooth enough, it can also reconstruct more detailed features as much as possible, and optimize to obtain a face mesh model that is more similar to the face scan data, thereby obtaining a reconstructed face mesh model. Since the target face mesh model at this time is highly similar to the face scan data, in this post-processing, the deformation process according to the global feature points in the pose matching position of the face scan data is relatively simple, and the calculation The amount is lower, it will not waste more time of developers, and can guarantee a higher reconstruction effect.
  • the post-processing of the three-dimensional face grid template is to reconstruct the detailed features in the target face grid model by projecting the face scan data only once along the normal direction of the three-dimensional face grid template.
  • the post-processing results can be shown in FIG. 1F, and it can be seen that the smoothness of the face mesh model reconstructed by the post-processing method in the related art is poor.
  • the post-processing in this embodiment can be used to continuously modify the weight ⁇ of the regular term obtained by iterative solution in the linear equations shown below to reconstruct the target face mesh model in detail:
  • M is a regularization matrix that constrains the consistency of movement between global feature points
  • D and W are the displacement of global feature points into face scan data and the weight of each global feature point
  • T I is the actual displacement of each global feature point during the deformation process
  • I is the identity matrix.
  • the technical solution provided in this embodiment extracts the key feature points in the three-dimensional face grid template by layers, matches the face scan data in layers according to the key feature points, and uses the global feature points to perform the matching on the model Detail reconstruction to obtain the reconstructed face mesh model, which solves the problems of slow reconstruction and poor smoothness of the face mesh model in related technologies, realizes the three-dimensional reconstruction of the face mesh model, and improves the face mesh model
  • the reconstruction rate and reconstruction effect improve the smoothness of the reconstructed face mesh model.
  • FIG. 2A is a flowchart of another method for reconstructing a face grid model according to Embodiment 2 of the present application. This embodiment is based on the above embodiment. Referring to FIG. 2A, this embodiment may include steps S210-S270.
  • S210 Obtain face scan data to be reconstructed and a three-dimensional face grid template.
  • S220 Obtain the original resolution of the three-dimensional face grid template, and determine the grid accuracy of the three-dimensional face grid template according to the original resolution.
  • the medium-precision 3D face net In order to improve the reconstruction effect of the face grid model, the grid template first needs to deal with the accuracy of the three-dimensional face grid template, that is, the pixel resolution of the model.
  • the original resolution of the three-dimensional face grid template is first determined, and the three-dimensional is determined according to the original resolution The mesh accuracy of the face mesh template when it is created, so that the high-precision 3D face mesh template can be processed later.
  • the grid accuracy of the three-dimensional face grid template after determining the grid accuracy of the three-dimensional face grid template, it is necessary to determine whether the grid accuracy is lower than a preset accuracy threshold, if the grid accuracy of the three-dimensional face grid template is lower than the preset accuracy Threshold value, indicating that the original resolution of the obtained 3D face grid template is low.
  • the corresponding high-precision 3D face grid template needs to be obtained to improve the reconstruction effect of the face grid model;
  • the grid accuracy of the face grid template is not lower than the preset precision threshold, indicating that the original resolution of the 3D face grid template obtained this time is already high, which can achieve the reconstruction effect of the reconstructed face grid model It is required that the three-dimensional face grid template with grid precision can be directly processed at this time.
  • S240 Perform subdivision on the three-dimensional face grid template to obtain the subdivided three-dimensional face grid template.
  • the tessellation is to automatically embed new vertices between the vertices of the low-precision 3D face mesh template and the vertices.
  • the 3D face mesh template After automatically inserting a large number of new vertices, the 3D face mesh template
  • the surface will be divided very delicately, and it will look smoother and denser.
  • Surface subdivision is a technology that can automatically create vertices inside the model chip and refine the model to obtain a better picture effect.
  • the tessellation can automatically create vertices that are hundreds of times similar to the vertices of the original model. These vertices are not virtual vertices, but real vertices.
  • the vertices created by tessellation are directly designed when modeling.
  • the low-precision 3D face mesh template is subdivided to obtain a high-precision 3D face mesh template, which has the same effect as directly creating a high-precision 3D face mesh template during modeling , And the use of surface subdivision can reduce a lot of time and cost designers spend to achieve the same effect.
  • the low-precision three-dimensional face mesh template when it is determined that the mesh accuracy of the three-dimensional face mesh template is lower than the preset accuracy threshold, the low-precision three-dimensional face mesh template can be subjected to corresponding surface subdivision through the surface subdivision algorithm Operation, get the subdivided 3D face grid template, and perform the corresponding subsequent operations on the subdivided 3D face grid template to obtain the reconstructed face grid model, improve the reconstruction of the face grid model Smoothness.
  • S260 Obtain the global feature points in the target face mesh model, and deform the target face mesh model according to the posture matching position of the global feature points in the face scan data to obtain the reconstructed face mesh model.
  • the subsequent execution of the hierarchical extraction of key feature points, and in turn according to the position of the key feature points in the face scan data matching position to The deformation operation of the three-dimensional face grid template is also the corresponding operation performed on the subdivided three-dimensional face grid template, and the reconstructed face grid model obtained at this time is also a high-precision person as shown in FIG. 2B
  • the face grid model according to the results shown in FIG. 1E obtained by directly matching partial feature points with face scan data in the related art, it can be seen that the subdivided three-dimensional person in this embodiment
  • the method of hierarchical registration of the face grid template can significantly improve the reconstruction effect of the face grid model.
  • the technical solution provided in this embodiment obtains a high-precision 3D face grid template by subdividing a low-precision 3D face grid template, and performs a human face according to the high-precision 3D face grid template
  • the reconstruction of the mesh model improves the reconstruction rate and reconstruction effect of the face mesh model, and improves the smoothness of the reconstructed face mesh model.
  • FIG. 3 is a schematic structural diagram of an apparatus for reconstructing a face mesh model provided in Embodiment 3 of the present application.
  • the apparatus may include: a data acquisition module 310 configured to acquire face scan data to be reconstructed And three-dimensional face grid template; hierarchical matching module 320, set to extract key feature points in the three-dimensional face grid template hierarchically, and then sequentially match the posture extracted from the hierarchical key feature points in the face scan data Position, deform the 3D face mesh template to obtain the target face mesh model; the model reconstruction module 330 is set to obtain the global feature points in the target face mesh model, and scan the data on the face according to the global feature points The posture matching position in the image is transformed into the target face mesh model to obtain the reconstructed face mesh model.
  • the technical solution provided in this embodiment extracts the key feature points in the three-dimensional face grid template by layers, matches the face scan data in layers according to the key feature points, and uses the global feature points to perform the matching on the model Detail reconstruction to obtain the reconstructed face mesh model, which solves the problem of slow reconstruction and poor smoothness of the face mesh model in related technologies, realizes the reconstruction of the face mesh model, and improves the face mesh model
  • the reconstruction rate and reconstruction effect improve the smoothness of the reconstructed face mesh model.
  • the above-mentioned hierarchical matching module 320 may include: a position matching unit configured to determine the posture matching position of the hierarchically extracted key feature points in the face scan data using the ICP algorithm; a hierarchical registration unit configured to In turn, the three-dimensional face grid template is deformed according to the posture matching position of the key feature points extracted in layers in the face scan data and the three-dimensional grid registration algorithm.
  • the reconstruction device of the face grid model described above may further include: an accuracy determination module configured to acquire the three-dimensional face before extracting key feature points in the three-dimensional face grid template in layers The original resolution of the grid template, and the grid accuracy of the three-dimensional face grid template is determined according to the original resolution; the surface subdivision module is set to set the grid accuracy of the three-dimensional face grid template below the preset precision threshold In this case, the 3D face mesh template is subdivided to obtain the subdivided 3D face mesh template.
  • the reconstruction device of the above face grid model may further include:
  • the precision restoration module is set to perform precision restoration on the reconstructed face grid model according to the original resolution after obtaining the reconstructed face grid model to obtain the reconstructed face grid model at the original resolution.
  • the reconstruction device of the face grid model provided in this embodiment can be applied to the reconstruction method of the face grid model provided in any of the foregoing embodiments, and has corresponding functions and beneficial effects.
  • the device includes a processor 40, a storage device 41, and a communication device 42; the number of processor 40 in the device may be one or more
  • a processor 40 is taken as an example; the processor 40, the storage device 41, and the communication device 42 in the device may be connected by a bus or other means.
  • a connection by a bus is used as an example.
  • the storage device 41 may be configured to store software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the reconstruction method of the face grid model provided in the embodiments of the present application.
  • the processor 40 executes one or more functional applications of the device and data processing by running the software programs, instructions, and modules stored in the storage device 41, that is, the reconstruction method of the face grid model described above.
  • the storage device 41 may mainly include a storage program area and a storage data area.
  • the storage program area may store an operating system and at least one function required application program; the storage data area may store data created according to the use of the terminal, etc. .
  • the storage device 41 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • the storage device 41 may include memories remotely provided with respect to the processor 40, and these remote memories may be connected to the device through a network. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.
  • the communication device 42 may be configured to implement network connection or mobile data connection between devices.
  • a device provided in this embodiment may be configured to perform the reconstruction method of a face mesh model provided in any of the above embodiments, and has corresponding functions and beneficial effects.
  • Embodiment 5 of the present application further provides a computer-readable storage medium that stores a computer program, and when the program is executed by a processor, a method for reconstructing a face mesh model in any of the foregoing embodiments may be implemented.
  • the method may include:
  • the key feature points in the 3D face grid template are extracted in layers, and the 3D face grid template is deformed according to the posture matching position of the key feature points extracted in layers in the face scan data to obtain the target face Grid model.
  • a storage medium containing computer-executable instructions provided by an embodiment of the present application the computer-executable instructions are not limited to the method operations described above, and may also perform reconstruction of the face grid model provided by any embodiment of the present application Related operations in the method.
  • the present application can be implemented by software and necessary general hardware, or by hardware.
  • the technical solution of this application can be embodied in the form of software products in essence or part of contributions to related technologies.
  • the computer software products can be stored in computer-readable storage media, such as computer floppy disks, read-only memory (Read -Only Memory (ROM), Random Access Memory (RAM), Flash (FLASH), hard disk or CD-ROM, etc., including multiple instructions to make a computer device (which can be a personal computer, server, or network) Equipment, etc.) to execute the method described in the embodiments of the present application.
  • the included units and modules are only divided according to the function logic, but it is not limited to the above division, as long as the corresponding function can be realized; in addition, each The name of the functional unit is only for the purpose of distinguishing each other, and is not used to limit the protection scope of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Architecture (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Algebra (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Image Generation (AREA)

Abstract

本文公开了一种人脸网格模型的重建方法、装置、设备和存储介质。该方法包括：获取待重建的人脸扫描数据以及三维人脸网格模板；分层次提取三维人脸网格模板中的关键特征点，并依次根据分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形，得到目标人脸网格模型；获取目标人脸网格模型中的全局特征点，并根据全局特征点在人脸扫描数据中的姿态匹配位置，对目标人脸网格模型进行变形，得到重建的人脸网格模型。

Description

人脸网格模型的重建方法、装置、设备和存储介质
本申请要求在2018年11月29日提交中国专利局、申请号为201811446646.7的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及图像处理领域，例如涉及一种人脸网格模型的重建方法、装置、设备和存储介质。
背景技术
随着视频技术的发展，有人脸画面显示需求的娱乐应用中对于逼真的人脸模型的创建需求也越来越大。创建一个逼真的三维人脸模型是一个非常困难的工作，如果完全手工建造，需要耗费大量的人力物力。而在游戏和电影工业中，使用三维扫描技术来获取逼真人脸已经成为主流，随着三维扫描硬件价格的降低，这项技术将会越来越普及。然而，当我们需要批量创建一系列逼真人脸的时候，由于使用三维扫描技术获取的扫描数据普遍存在噪声和缺损，而且通过获取的扫描数据直接生成的网格模型不具备一致的拓扑结构，此时为了获取精确的人脸网格模型，设计人员仍然要承担从三维扫描数据转换到精确网格模型的繁重工作。
针对这些问题，普遍采用一种三维网格注册算法，将一个网格模板模型变形直到与扫描数据形状匹配为止，此时需要获取人脸扫描数据与人脸网格模板中一系列精确的对应点，采用稀疏数据插值的方法对人脸网格模板进行变形，得到对应的三维人脸网格模型；或者通过在人脸网格模板中采集少量点，与扫描数据匹配而建立变形图网络，并进行对应的后处理过程，得到对应的三维人脸网格模型。
而由于在娱乐应用如手机应用程序(Application，APP)或者网络游戏中，人脸网格模型的分辨率通常不高，但在逐步变形与扫描数据匹配时，仍然需要保留扫描数据中的细节特征，此时对于人脸扫描数据与人脸网格模板中的精确对应点的获取速度较慢，整体过程操作较为繁琐，可能会存在收敛速度慢、不稳定的问题；而对采用少量点建立的变形图网络进行后处理，也会存在速度慢且得到的三维人脸网格模型中平滑度较差的问题。
发明内容
本申请实施例提供了一种人脸网格模型的重建方法、装置、设备和存储介 质，以解决相关技术中人脸网格模型重建速率慢、且平滑性较差的问题，实现人脸网格模型的三维重建，提高人脸网格模型的重建速率和重建效果。
本申请实施例提供了一种人脸网格模型的重建方法，该方法包括：
获取待重建的人脸扫描数据以及三维人脸网格模板；
分层次提取三维人脸网格模板中的关键特征点，并依次根据分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形，得到目标人脸网格模型；
获取所述目标人脸网格模型中的全局特征点，并根据所述全局特征点在所述人脸扫描数据中的姿态匹配位置，对所述目标人脸网格模型进行变形，得到重建的人脸网格模型。
本申请实施例提供了一种人脸网格模型的重建装置，该装置包括：
数据获取模块，设置为获取待重建的人脸扫描数据以及三维人脸网格模板；
分层匹配模块，设置为分层次提取三维人脸网格模板中的关键特征点，并依次根据分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形，得到目标人脸网格模型；
模型重建模块，设置为获取所述目标人脸网格模型中的全局特征点，并根据所述全局特征点在所述人脸扫描数据中的姿态匹配位置，对所述目标人脸网格模型进行变形，得到重建的人脸网格模型。
本申请实施例提供了一种设备，该设备包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现本申请任意实施例中所述的人脸网格模型的重建方法。
本申请实施例提供了一种计算机可读存储介质，存储有计算机程序，该程序被处理器执行时实现本申请任意实施例中所述的人脸网格模型的重建方法。
附图说明
图1A为本申请实施例一提供的一种人脸网格模型的重建方法的流程图；
图1B为本申请实施例一提供的一种低精度的三维人脸网格模板的示意图；
图1C为本申请实施例一提供的一种高精度的三维人脸网格模板的示意图；
图1D为本申请实施例一提供的一种人脸扫描数据的示意图；
图1E为本申请实施例一提供的一种直接根据部分特征点对三维人脸网格模板进行注册的结果示意图；
图1F为本申请实施例一提供的一种对变形后的三维人脸网格模板进行后处理的结果示意图；
图2A为本申请实施例二提供的另一种人脸网格模型的重建方法的流程图；
图2B为本申请实施例二提供的一种对细分后的三维人脸网格模板进行注册的结果示意图；
图2C为本申请实施例二提供的一种对重建的高精度的人脸网格模型进行精度还原的结果示意图；
图3为本申请实施例三提供的一种人脸网格模型的重建装置的结构示意图；
图4为本申请实施例四提供的一种设备的结构示意图。
具体实施方式
下面结合附图和实施例对本申请进行说明。本文所描述的具体实施例仅仅用于解释本申请，而非对本申请的限定。为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。
本申请实施例主要通过分层次的采用三维网格注册算法，将三维人脸网格模板进行变形与人脸扫描数据匹配，并再次对变形后的三维人脸网格模板中的全局特征点进行细节匹配，得到重建的人脸网格模型，解决了相关技术中人脸网格模型重建速率慢、且平滑性较差的问题，提高了重建后的人脸网格模型的平滑性。
实施例一
图1A为本申请实施例一提供的一种人脸网格模型的重建方法的流程图，本实施例可适用于任意一种通过人脸扫描数据来创建逼真人脸网格模型的三维模型处理设备中。本申请实施例的方案可以适用于如何根据人脸扫描数据创建对应的逼真人脸网格模型的过程中。本实施例提供的一种人脸网格模型的重建方法可以由本申请实施例提供的人脸网格模型的重建装置来执行，该装置可以通过软件和/或硬件的方式来实现，并集成在执行本方法的设备中，该设备可以是具备图像处理能力的任意一种三维模型处理设备。
一实施例中，参考图1A，该方法可以包括S110-S130。
S110，获取待重建的人脸扫描数据以及三维人脸网格模板。
本实施例中，该三维人脸网格模板为标准人脸的网格模型。一实施例中，三维人脸网格模板是根据标准的人脸形状预先创建的添加有网格分布示意的三维人脸模型，在三维人脸网格模板的创建过程中，可以根据选定的模型像素分辨率不同，而创建不同精度的三维人脸网格模板，也就是使创建的三维人脸网格模板中的网格分布数量和密集程度不同，如图1B中所示的低精度的三维人脸网格模板，图1C中所示的高精度的三维人脸网格模板。本实施例在人脸网格模型的重建时，会在执行本实施例中的人脸网格模型的重建方法的三维模型处理设备中存储一个预先创建的三维人脸网格模板，针对不同的人脸扫描数据，均可以通过该三维人脸网格模板与其匹配，得到重建的人脸网格模型。
此外，人脸扫描数据是针对所需要重建的人脸网格模型的类型不同，对同一类型的人脸进行扫描得到的三维扫描数据，如图1D中所示。由于本实施例中重建的人脸网格模型主要用于娱乐应用或游戏网站的包含人脸表情的视频画面显示中，而在包含人脸表情的视频产品中，人脸表情动画技术多是数据驱动的，也就是依托于一个人脸数据库上建立的拥有多类表情的人脸网格模型的参数化模型，因此需要通过在原始的包含不同表情的多类人脸扫描数据中获取大量逼真的人脸网格模型，从而创建该人脸数据库。此时针对所需要的不同表情，通过三维扫描技术可以得到大量不同的人脸扫描数据，该人脸扫描数据可以在重建人脸网格模型之前，直接预先存储在执行本实施例中的人脸网格模型的重建方法的三维模型处理设备中，以在重建人脸网格模型时可以直接获取；或者，在重建人脸网格模型时，实时通过三维扫描技术扫描得到对应表情的人脸扫描数据，直接执行人脸网格模型的重建，本实施例中对此不作限定。
一实施例中，执行人脸网格模型的重建时，首先可以针对本次需要重建的人脸网格模型中包含的表情信息，选定对应表情的人脸扫描数据，可以是直接在预先存储的大量人脸扫描数据中查找到对应的人脸扫描数据；也可以是直接根据该表情信息通过三维扫描技术实时扫描对应表情的人脸，得到本次重建过程中对应的人脸扫描数据。此外，人脸网格模型在重建过程中，还需要获取预先存储的三维人脸网格模板，也就是标准人脸的网格模型，通过同一个三维人脸网格模板分别与多个不同的人脸扫描数据匹配，得到重建的不同人脸网格模型。本实施例中针对同一个三维人脸网格模板进行处理，降低了将不同人脸扫描数据转换到对应人脸网格模型过程中存在的噪声和缺损干扰，而且解决了通过不同的人脸扫描数据直接生成的网格模型不具备一致拓扑结构的问题。
S120，分层次提取三维人脸网格模板中的关键特征点，并依次根据分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形，得到目标人脸网格模型。
特征点为模型中具有鲜明特性并能够有效反映模型本质特征、能够标识模型中目标部位的点，本实施例中，特征点是指能够表示出三维人脸网格模板中五官部位的像素点，也就是三维人脸网格模板中所分布的多个网格顶点。关键特征点是三维人脸网格模板中的局部特征点。一实施例中，关键特征点是在全部网格顶点中可以明确表示三维人脸网格模板中的眼睛、鼻子、嘴巴、下巴等五官位置或者眼窝、鼻翼等人脸中的细节位置的特征点。同时，姿态匹配位置为根据每个关键特征点所表示的人脸目标部位在三维人脸网格模板中的位置，确定的相同目标部位对应位于人脸扫描数据中的匹配位置，例如，表示三维人脸网格模板中的鼻子的特征点在人脸扫描模型中的姿态匹配位置为人脸扫描数据中鼻子所在的位置。
本实施例中为了将三维人脸网格模板与人脸扫描数据匹配，首先需要确定出三维人脸网格模板中的特征点，并根据该特征点在人脸扫描数据中的匹配位置，控制三维人脸网格模板进行变形，使得每个特征点在变形后的三维人脸网格模板中的位置与该特征点在人脸扫描数据中的位置一一匹配，相关技术中通过获取人脸扫描数据与三维人脸网格模板上精确对应的全部特征点，使用稀疏数据插值的方式来使三维人脸网格模板进行变形，进行对应的人脸网格模型重建，但是采用全部特征点来进行变形的方式使得人脸网格模型的重建速度非常低，而且一些精确对应的特征点是很难提前获取的；而若采用通过部分特征点进行变形得到重建后的人脸网格模型的方式，由于特征点较少，又会导致重建的人脸网格模型中的细节特征不明确，使得重建的人脸网格模型的平滑度较低，直接根据部分特征点与人脸扫描数据进行匹配的结果如图1E中所示，匹配后的人脸网格模型细节特征不明确，整体平滑性较差。因此，本实施例中通过分层次的提取三维人脸网格模板中的关键特征点，依次与人脸扫描数据进行匹配，解决提取全部特征点时人脸网格模型的重建速度非常低，以及提取部分特征点时重建的人脸网格模型的细节特征不明确的问题。一实施例中，可以通过尺度不变特征转换(Scale-invariant feature transform，SIFT)算法、加速分段测试特征(features from accelerated segment test，FAST)算法等算法，分层次对三维人脸网格模板中的关键特征点进行提取，并依次确定分层次提取的多个关键特征点在人脸扫描数据中的姿态匹配位置，根据对应的姿态匹配位置依次控制相应的三维人脸网格模板进行变形，得到与人脸扫描数据匹配的目标人脸网格模型。
示例性的，获取对应的人脸扫描数据与原始的三维人脸网格模板，在原始的三维人脸网格模板中首先提取出部分网格顶点，作为关键特征点，并确定本次提取的多个关键特征点在人脸扫描数据中的姿态匹配位置，从而根据该姿态匹配位置拉动原始的三维人脸网格模板进行变形，使本次提取的每个关键特征点在本次变形后的三维人脸网格模板中的位置与该关键特征点在人脸扫描数据 中的姿态匹配位置对应，原始的三维人脸网格模板在变形过程中也会带动本次未提取的其他特征点进行移动，但是未提取的特征点在本次变形后的三维人脸网格模板中的位置与该未提取的特征点在人脸扫描数据中的姿态匹配位置的匹配程度较低，也就是本次变形后的三维人脸网格模板与人脸扫描数据的相似程度较低；因此可以在第一次变形后的三维人脸网格模板中继续提取除首次提取的关键特征点之外的部分网格顶点，作为第二层次提取的关键特征点，并确定该第二层次提取的多个关键特征点在人脸扫描数据中的姿态匹配位置，继续根据该姿态匹配位置拉动第一次变形后的三维人脸网格模板再次进行变形，根据分层次提取的关键特征点，参照上述过程依次循环对相应的三维人脸网格模板进行变形，直至实现与人脸扫描数据的精确匹配，从而得到对应的目标人脸网格模型。本实施例中分层次获取关键特征点，并依次根据在人脸扫描数据中的姿态匹配位置对三维人脸网格模板进行变形，可以使依次变形后的三维人脸网格模板逐渐接近人脸扫描数据，使得后续的变形过程中的运算量逐渐降低，相比采用全部特征点对三维人脸网格模板进行变形，一次性与人脸扫描数据匹配而言，逐次降低了变形过程中的运算量，提高了人脸网格模型的重建速率。
一实施例中，在三维人脸网格模板的变形过程中，除关键特征点之外的其他特征点在随着三维人脸网格模板的变形进行移动时，会满足一定的约束条件，保证三维人脸网格模板在变形过程中多个网格顶点移动的一致性。据此，在本实施例中，依次根据分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形可以包括S1210-S1220。
S1210，采用迭代最近点(IteratedClosest Points，ICP)算法确定分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置。
本实施例中，ICP算法是一种基于轮廓特征的点配准方法，主要是确定两堆点云数据之间的变换关系，也就是本实施例中三维人脸网格模板中的多个关键特征点与人脸扫描数据中对应的特征点之间的变换关系，一实施例中，可以通过计算三维人脸网格模板中每个关键特征点在人脸扫描数据中的最近点，并采用最小均方根算法计算该关键特征点与最近点的配准，得到对应的坐标变换，并确定变换后的三维人脸网格模板，此时通过计算变换后的三维人脸网格模板与人脸扫描数据之间的均方根误差，并据此依次迭代运算，从而确定最终匹配的变换关系。一实施例中，在分层次提取关键特征点后，可以采用ICP算法中确定的三维人脸网格模板与人脸扫描数据之间的变换关系，依次确定分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置。也就是采用ICP算法确定第一层次提取的关键特征点在人脸扫描数据中的姿态匹配位置，对原始的三维人脸网格模板进行变形，再次在第一次变形后的三维人脸网格模板中提取第二层次的关键特征点，采用ICP算法确定第二层次提取的关键特征点在人脸扫描 数据中的姿态匹配位置，对第一次变形后的三维人脸网格模板再次进行变形，依次类推，分别确定分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置，以便对相应的三维人脸网格模板进行变形。
S1220，依次根据分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置以及三维网格注册算法，对三维人脸网格模板进行变形。
本实施例中，三维网格注册算法是一种模型匹配算法，可以将一个三维人脸网格模板逐步变形直到与人脸扫描数据的三维形状匹配为止。本实施例中首先针对分层次提取的关键特征点，相当于针对三维人脸网格模板重新建立一个低分辨率的网格模型，然后通过该低分辨率的网格模型变形来拉动对应的三维人脸网格模板向人脸扫描数据投影，也就是对三维人脸网格模板进行变形。此时三维人脸网格模板的变形过程可以通过最小化能量函数来实现，该能量函数可以使用高斯牛顿迭代算法来进行优化，此时该能量函数可以为：E＝E fit+αE rigid+βE smooth；该能量函数中，E fit可以保证每次变形后的三维人脸网格模板中的关键特征点与在人脸扫描数据的姿态匹配位置相匹配；αE rigid可以保证低分辨率的网格模型中的多个特征点，也就是三维人脸网格模板中的关键特征点的运动只包括旋转与平移，α是E rigid的权重系数；βE smooth可以用来约束多个特征点(关键特征点与其他特征点)之间的运动尽量保持一致，避免不平滑的结果出现，β是E smooth的权重系数。该能量函数可以确定其他特征点在相应的三维人脸网格模板进行变形的过程中对应满足的约束条件，以保证重建的人脸网格模型的平滑度。
对于E fit，本实施例中使用点到点的距离与点到面的距离，也就是三维人脸网格模板中的关键特征点到人脸扫描数据中的最近点的距离，以及多个关键特征点到人脸扫描数据的距离：
Figure PCTCN2019117944-appb-000001
上述公式中，n i为三维人脸网格模板中的关键特征点v i在人脸扫描数据上的最近点c i处的法线向量，这样做可以允许多个关键特征点沿人脸扫描数据进行切向适度滑动，增加变形过程中的鲁棒性，i是网格顶点的索引(或序号)，γ是权重系数。
一实施例中，本实施例中通过上述公式可以依次根据分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置以及三维网格注册算法，对相应的三维人脸网格模板进行变形，并对变形过程中的其他特征点的移动进行约束，保证特征点运动的一致性，提高重建的人脸网格模型的平滑度。
本实施例中对三维人脸网格模板分层次的变形为三层，也就是进行三次变形匹配处理，便可以保证变形后得到的目标人脸网格模型与人脸扫描数据的相似程度较高，本实施例中对分层次变形的次数不作限定，可以根据变形后的三 维人脸网格模板与人脸扫描数据的相似程度来确定。
S130，获取目标人脸网格模型中的全局特征点，并根据全局特征点在人脸扫描数据中的姿态匹配位置，对目标人脸网格模型进行变形，得到重建的人脸网格模型。
本实施例中，由于目标人脸网格模型是由三维人脸网格模板经过多次与人脸扫描数据匹配，进行变形得到，因此目标人脸网格模型也为一种网格模型，此时全局特征点为目标人脸网格模型中所分布的全部网格顶点。
一实施例中，由于目标人脸网格模型为三维人脸网格模板经过多次与人脸扫描数据匹配，进行多次变形后得到，因此可以保证目标人脸网格模型与人脸扫描数据的相似程度较高，此时为了重建目标人脸网格模型中的细节特征，得到与人脸扫描数据更为相似匹配的人脸网格模型，本实施例中可以通过获取目标人脸网格模型中的全局特征点，并根据多个全局特征点在人脸扫描数据中的姿态匹配位置，参照S120中提及的变形过程，对目标人脸网格模型进行变形，以保证重建的人脸网格模型足够平滑的同时，也能够尽量重建出更多的细节特征，优化得到与人脸扫描数据更加相似的人脸网格模型，从而得到重建的人脸网格模型。由于此时的目标人脸网格模型与人脸扫描数据的相似程度较高，因此本次后处理中根据全局特征点在人脸扫描数据的姿态匹配位置，进行的变形过程也较为简单，运算量较低，不会浪费开发人员较多的时间，而且可以保证较高的重建效果。
相关技术中对三维人脸网格模板进行后处理是通过沿三维人脸网格模板的法线方向来向人脸扫描数据仅投影一次来重建目标人脸网格模型中的细节特征，得到的后处理结果可以如图1F中所示，可以看出通过相关技术中的后处理方法重建的人脸网格模型的平滑度较差。而本实施例中的后处理可以通过不断修改如下所示的线性方程组中通过迭代求解得到的正则项的权重α来做目标人脸网格模型的细节重建：
Figure PCTCN2019117944-appb-000002
该线性方程组中，M是正则化矩阵，约束全局特征点之间移动的一致性，而D和W分别是全局特征点到人脸扫描数据中的位移以及每个全局特征点的权重，T是在变形过程中每个全局特征点的实际位移，I是单位矩阵。
本实施例提供的技术方案，通过分层次提取三维人脸网格模板中的关键特征点，根据该关键特征点分层次与人脸扫描数据进行匹配，并采用全局特征点对匹配后的模型进行细节重建，得到重建的人脸网格模型，解决了相关技术中人脸网格模型重建速率慢、且平滑性较差的问题，实现人脸网格模型的三维重建，提高人脸网格模型的重建速率和重建效果，提高了重建后的人脸网格模型的平滑性。
实施例二
图2A为本申请实施例二提供的另一种人脸网格模型的重建方法的流程图。本实施例以上述实施例为基础。参照图2A，本实施例可以包括步骤S210-S270。
S210，获取待重建的人脸扫描数据以及三维人脸网格模板。
S220，获取三维人脸网格模板的原始分辨率，并根据原始分辨率确定三维人脸网格模板的网格精度。
一实施例中，由于三维人脸网格模板在创建过程中，选定的模型像素分辨率越高，也就是三维人脸网格模板中网格分布数量越多，此时要求设计人员花费的精力和成本也就越高，因此在人脸网格模型重建时，设计人员一般会选用中等精度的三维人脸网格模板对人脸扫描数据进行匹配，此时针对中等精度的三维人脸网格模板，为了提高人脸网格模型的重建效果，首先需要对三维人脸网格模板的精度，也就是模型像素分辨率进行对应处理。
本实施例中，在获取到三维人脸网格模板后，为了提高人脸网格模型的重建效果，首先会判断该三维人脸网格模板的原始分辨率，并根据该原始分辨率确定三维人脸网格模板在创建时的网格精度，以便后续对高精度的三维人脸网格模板进行处理。
S230，判断三维人脸网格模板的网格精度是否低于预设精度阈值，若三维人脸网格模板的网格精度低于预设精度阈值，执行S240；若三维人脸网格模板的网格精度不低于预设精度阈值，执行S250。
一实施例中，在确定三维人脸网格模板的网格精度后，需要判断该网格精度是否低于预设精度阈值，若三维人脸网格模板的网格精度低于该预设精度阈值，说明本次获取的三维人脸网格模板的原始分辨率较低，此时需要得到对应的高精度的三维人脸网格模板，以提高人脸网格模型的重建效果；若三维人脸网格模板的网格精度不低于该预设精度阈值，说明本次获取的三维人脸网格模板的原始分辨率已经较高，能够达到对重建的人脸网格模型的重建效果的要求，此时可以直接对该网格精度的三维人脸网格模板进行处理。
S240，对三维人脸网格模板进行曲面细分，得到细分后的三维人脸网格模板。
一实施例中，曲面细分是在低精度的三维人脸网格模板中的网格顶点与顶点之间自动嵌入新的顶点，在自动插入大量新的顶点之后，三维人脸网格模板的曲面会被分得非常细腻，看上去更加平滑致密。曲面细分是一种能够在模型芯片内部自动创造顶点，使模型细化，从而获得更好画面效果的技术。曲面细 分能自动创造出数百倍与原始模型的顶点相似的顶点，这些顶点不是虚拟的顶点，而是实实在在的顶点，曲面细分创造出的顶点等同于建模的时候直接设计出来的顶点，因此对低精度的三维人脸网格模板采用曲面细分，从而得到高精度的三维人脸网格模板，与在建模时直接创建高精度的三维人脸网格模板的效果相同，而采用曲面细分可以减少设计人员花费的大量时间和成本，实现同样的效果。
因此，本实施例中在确定三维人脸网格模板的网格精度低于预设精度阈值的情况下，可以通过曲面细分算法对低精度的三维人脸网格模板进行对应的曲面细分操作，得到细分后的三维人脸网格模板，并对细分后的三维人脸网格模板进行相应的后续操作，得到重建的人脸网格模型，提高重建的人脸网格模型的平滑度。
S250，分层次提取三维人脸网格模板中的关键特征点，并依次根据分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形，得到目标人脸网格模型。
S260，获取目标人脸网格模型中的全局特征点，并根据全局特征点在人脸扫描数据中的姿态匹配位置，对目标人脸网格模型进行变形，得到重建的人脸网格模型。
S270，在确定三维人脸网格模板的网格精度低于预设精度阈值的情况下，根据原始分辨率对重建的人脸网格模型进行精度还原，得到原始分辨率下的重建的人脸网格模型。
一实施例中，在对原始的三维人脸网格模板进行曲面细分操作后，后续执行的分层次提取关键特征点，并依次根据关键特征点在人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形的操作也是针对细分后的三维人脸网格模板执行的对应操作，此时得到的重建的人脸网格模型也是如图2B中所示的高精度的人脸网格模型，据此与相关技术中直接根据部分特征点与人脸扫描数据进行匹配得到的如图1E中所示的结果相比，可以看出本实施例中对细分后的三维人脸网格模板进行分层注册的方法，可以明显提高人脸网格模型的重建效果。此时，为了得到原始分辨率的人脸网格模型，还需要根据三维人脸网格模板的原始分辨率对重建后的高精度的人脸网格模型进行精度还原，也就是提取高精度的人脸网格模型中位于原始分辨率下的网格顶点，去掉多余的网格顶点，从而得到如图2C中所示的原始分辨率下的人脸网格模型，作为最终重建的人脸网格模型，与采用相关技术中的后处理技术得到的如图1F中所示的结果相比，重建后的人脸网格模型中的细节特征更加完善，有效地提高了人脸网格模型的重建效果。
本实施例提供的技术方案，通过对低精度的三维人脸网格模板进行曲面细分，得到高精度的三维人脸网格模板，并根据该高精度的三维人脸网格模板进行人脸网格模型的重建，提高了人脸网格模型的重建速率和重建效果，提高了重建后的人脸网格模型的平滑性。
实施例三
图3为本申请实施例三提供的一种人脸网格模型的重建装置的结构示意图，如图3所示，该装置可以包括：数据获取模块310，设置为获取待重建的人脸扫描数据以及三维人脸网格模板；分层匹配模块320，设置为分层次提取三维人脸网格模板中的关键特征点，并依次根据分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形，得到目标人脸网格模型；模型重建模块330，设置为获取目标人脸网格模型中的全局特征点，并根据全局特征点在人脸扫描数据中的姿态匹配位置，对目标人脸网格模型进行变形，得到重建的人脸网格模型。
本实施例提供的技术方案，通过分层次提取三维人脸网格模板中的关键特征点，根据该关键特征点分层次与人脸扫描数据进行匹配，并采用全局特征点对匹配后的模型进行细节重建，得到重建的人脸网格模型，解决了相关技术中人脸网格模型重建速率慢、且平滑性较差的问题，实现人脸网格模型的重建，提高人脸网格模型的重建速率和重建效果，提高了重建后的人脸网格模型的平滑性。
一实施例中，上述分层匹配模块320，可以包括：位置匹配单元，设置为采用ICP算法确定分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置；分层注册单元，设置为依次根据分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置以及三维网格注册算法，对三维人脸网格模板进行变形。
一实施例中，上述人脸网格模型的重建装置，还可以包括：精度确定模块，设置为在分层次提取所述三维人脸网格模板中的关键特征点之前，获取所述三维人脸网格模板的原始分辨率，并根据原始分辨率确定三维人脸网格模板的网格精度；曲面细分模块，设置为在三维人脸网格模板的网格精度低于预设精度阈值的情况下，对三维人脸网格模板进行曲面细分，得到细分后的三维人脸网格模板。
一实施例中，上述人脸网格模型的重建装置，还可以包括：
精度还原模块，设置为在得到重建的人脸网格模型之后，根据原始分辨率对重建的人脸网格模型进行精度还原，得到原始分辨率下的重建的人脸网格模 型。
本实施例提供的人脸网格模型的重建装置可适用于上述任意实施例提供的人脸网格模型的重建方法，具备相应的功能和有益效果。
实施例四
图4为本申请实施例四提供的一种设备的结构示意图，如图4所示，该设备包括处理器40、存储装置41和通信装置42；设备中处理器40的数量可以是一个或多个，图4中以一个处理器40为例；设备中的处理器40、存储装置41和通信装置42可以通过总线或其他方式连接，图4中以通过总线连接为例。
存储装置41作为一种计算机可读存储介质，可设置为存储软件程序、计算机可执行程序以及模块，如本申请实施例中提供的人脸网格模型的重建方法对应的程序指令/模块。处理器40通过运行存储在存储装置41中的软件程序、指令以及模块，从而执行设备的一种或多种功能应用以及数据处理，即实现上述人脸网格模型的重建方法。
存储装置41可主要包括存储程序区和存储数据区，存储装置41中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据终端的使用所创建的数据等。此外，存储装置41可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储装置41可包括相对于处理器40远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
通信装置42可设置为实现设备间的网络连接或者移动数据连接。
本实施例提供的一种设备可设置为执行上述任意实施例提供的人脸网格模型的重建方法，具备相应的功能和有益效果。
实施例五
本申请实施例五还提供了一种计算机可读存储介质，存储有计算机程序，该程序被处理器执行时可实现上述任意实施例中的人脸网格模型的重建方法。该方法可以包括：
获取待重建的人脸扫描数据以及三维人脸网格模板。
分层次提取三维人脸网格模板中的关键特征点，并依次根据分层次提取的关键特征点在人脸扫描数据中的姿态匹配位置，对三维人脸网格模板进行变形， 得到目标人脸网格模型。
获取目标人脸网格模型中的全局特征点，并根据全局特征点在人脸扫描数据中的姿态匹配位置，对目标人脸网格模型进行变形，得到重建的人脸网格模型。
本申请实施例所提供的一种包含计算机可执行指令的存储介质，该计算机可执行指令不限于如上所述的方法操作，还可以执行本申请任意实施例所提供的人脸网格模型的重建方法中的相关操作。
通过以上关于实施方式的描述，所属领域的技术人员可以了解到，本申请可借助软件及必需的通用硬件来实现，也可以通过硬件实现，。本申请的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如计算机的软盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、闪存(FLASH)、硬盘或光盘等，包括多个指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请实施例所述的方法。
上述人脸网格模型的重建装置的实施例中，所包括的单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，每个功能单元的名称也只是为了便于相互区分，并不用于限制本申请的保护范围。

Claims (10)

  1. 一种人脸网格模型的重建方法，包括：
    获取待重建的人脸扫描数据以及三维人脸网格模板；
    分层次提取所述三维人脸网格模板中的关键特征点，并依次根据分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置，对所述三维人脸网格模板进行变形，得到目标人脸网格模型；
    获取所述目标人脸网格模型中的全局特征点，并根据所述全局特征点在所述人脸扫描数据中的姿态匹配位置，对所述目标人脸网格模型进行变形，得到重建的人脸网格模型。
  2. 根据权利要求1所述的方法，其中，所述依次根据分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置，对所述三维人脸网格模板进行变形，包括：
    采用迭代最近点ICP算法确定分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置；
    依次根据分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置以及三维网格注册算法，对所述三维人脸网格模板进行变形。
  3. 根据权利要求1或2所述的方法，在所述分层次提取所述三维人脸网格模板中的关键特征点之前，还包括：
    获取所述三维人脸网格模板的原始分辨率，并根据所述原始分辨率确定所述三维人脸网格模板的网格精度；
    在所述三维人脸网格模板的网格精度低于预设精度阈值的情况下，对所述三维人脸网格模板进行曲面细分，得到细分后的三维人脸网格模板。
  4. 根据权利要求3所述的方法，在所述得到重建的人脸网格模型之后，还包括：
    根据所述原始分辨率对所述重建的人脸网格模型进行精度还原，得到所述原始分辨率下的重建的人脸网格模型。
  5. 一种人脸网格模型的重建装置，包括：
    数据获取模块，设置为获取待重建的人脸扫描数据以及三维人脸网格模板；
    分层匹配模块，设置为分层次提取所述三维人脸网格模板中的关键特征点，并依次根据分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置，对所述三维人脸网格模板进行变形，得到目标人脸网格模型；
    模型重建模块，设置为获取所述目标人脸网格模型中的全局特征点，并根 据所述全局特征点在所述人脸扫描数据中的姿态匹配位置，对所述目标人脸网格模型进行变形，得到重建的人脸网格模型。
  6. 根据权利要求5所述的装置，其中，所述分层匹配模块包括：
    位置匹配单元，设置为采用迭代最近点ICP算法确定分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置；
    分层注册单元，设置为依次采用三维网格注册算法根据分层次提取的所述关键特征点在所述人脸扫描数据中的姿态匹配位置，对所述三维人脸网格模板进行变形。
  7. 根据权利要求5所述的装置，还包括：
    精度确定模块，设置为在所述分层次提取所述三维人脸网格模板中的关键特征点之前，获取所述三维人脸网格模板的原始分辨率，并根据所述原始分辨率确定所述三维人脸网格模板的网格精度；
    曲面细分模块，设置为在所述三维人脸网格模板的网格精度低于预设精度阈值的情况下，对所述三维人脸网格模板进行曲面细分，得到细分后的三维人脸网格模板。
  8. 根据权利要求7所述的装置，还包括：
    精度还原模块，设置为在所述得到重建的人脸网格模型之后，根据所述原始分辨率对所述重建的人脸网格模型进行精度还原，得到所述原始分辨率下的重建的人脸网格模型。
  9. 一种设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序；
    当所述至少一个程序被所述至少一个处理器执行，使得所述至少一个处理器实现如权利要求1-4中任一所述的人脸网格模型的重建方法。
  10. 一种计算机可读存储介质，存储有计算机程序，所述程序被处理器执行时实现如权利要求1-4中任一所述的人脸网格模型的重建方法。
PCT/CN2019/117944 2018-11-29 2019-11-13 人脸网格模型的重建方法、装置、设备和存储介质 Ceased WO2020108304A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/297,342 US11972527B2 (en) 2018-11-29 2019-11-13 Method and apparatus for reconstructing face mesh model, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811446646.7 2018-11-29
CN201811446646.7A CN109377563A (zh) 2018-11-29 2018-11-29 一种人脸网格模型的重建方法、装置、设备和存储介质

Publications (1)

Publication Number Publication Date
WO2020108304A1 true WO2020108304A1 (zh) 2020-06-04

Family

ID=65374708

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/117944 Ceased WO2020108304A1 (zh) 2018-11-29 2019-11-13 人脸网格模型的重建方法、装置、设备和存储介质

Country Status (3)

Country Link
US (1) US11972527B2 (zh)
CN (1) CN109377563A (zh)
WO (1) WO2020108304A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115908712A (zh) * 2022-11-28 2023-04-04 阿里巴巴（中国）有限公司 基于图像的三维重建及模型训练方法、设备及存储介质

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109377563A (zh) * 2018-11-29 2019-02-22 广州市百果园信息技术有限公司 一种人脸网格模型的重建方法、装置、设备和存储介质
CN111382618B (zh) * 2018-12-28 2021-02-05 广州市百果园信息技术有限公司 一种人脸图像的光照检测方法、装置、设备和存储介质
CN111028343B (zh) * 2019-12-16 2020-12-11 腾讯科技（深圳）有限公司 三维人脸模型的生成方法、装置、设备及介质
CN111325846B (zh) * 2020-02-13 2023-01-20 腾讯科技（深圳）有限公司 表情基确定方法、虚拟形象驱动方法、装置及介质
CN111559081A (zh) * 2020-05-08 2020-08-21 上海交通大学医学院附属第九人民医院 赝复体制作方法、系统、终端以及介质
CN111768488B (zh) * 2020-07-07 2023-12-29 网易（杭州）网络有限公司 一种虚拟角色脸部模型的处理方法和装置
CN112489216B (zh) * 2020-11-27 2023-07-28 北京百度网讯科技有限公司 面部重建模型的评测方法、装置、设备及可读存储介质
CN113593001B (zh) * 2021-02-07 2025-05-16 大连理工大学 目标对象三维重建方法、装置、计算机设备和存储介质
CN113076918B (zh) * 2021-04-15 2022-09-06 河北工业大学 基于视频的人脸表情克隆方法
CN113658233B (zh) * 2021-07-02 2025-08-08 广州虎牙科技有限公司 三维人脸模型的非刚性配准方法、装置、设备及存储介质
CN114943814B (zh) * 2022-06-08 2024-07-26 长春理工大学 一种三维扫描辅助面部建模分析系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100214289A1 (en) * 2009-02-25 2010-08-26 Jing Xiao Subdivision Weighting for Robust Object Model Fitting
CN105427385A (zh) * 2015-12-07 2016-03-23 华中科技大学 一种基于多层形变模型的高保真人脸三维重建方法
CN106469465A (zh) * 2016-08-31 2017-03-01 深圳市唯特视科技有限公司 一种基于灰度和深度信息的三维人脸重建方法
CN109377563A (zh) * 2018-11-29 2019-02-22 广州市百果园信息技术有限公司 一种人脸网格模型的重建方法、装置、设备和存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5886702A (en) * 1996-10-16 1999-03-23 Real-Time Geometry Corporation System and method for computer modeling of 3D objects or surfaces by mesh constructions having optimal quality characteristics and dynamic resolution capabilities
AU3639699A (en) * 1998-04-13 1999-11-01 Eyematic Interfaces, Inc. Wavelet-based facial motion capture for avatar animation
FR2955409B1 (fr) * 2010-01-18 2015-07-03 Fittingbox Procede d'integration d'un objet virtuel dans des photographies ou video en temps reel
CN102426712B (zh) * 2011-11-03 2014-07-16 中国科学院自动化研究所 一种基于两幅图像的三维头部建模方法
CN102663810B (zh) * 2012-03-09 2014-07-16 北京航空航天大学 一种基于相位偏移扫描的三维人脸全自动建模方法
CN103606190B (zh) * 2013-12-06 2017-05-10 上海明穆电子科技有限公司 单张人脸正面照片自动转换为三维人脸模型的方法
US9639737B2 (en) * 2015-09-29 2017-05-02 Eth Zürich (Eidgenöessische Technische Hochschule Zürich) Methods and systems of performing performance capture using an anatomically-constrained local model
US10930086B2 (en) * 2016-11-01 2021-02-23 Dg Holdings, Inc. Comparative virtual asset adjustment systems and methods
CN107818578B (zh) * 2017-10-11 2020-09-18 上海北冕信息科技有限公司 一种基于注册方法的快速人脸模型重建算法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100214289A1 (en) * 2009-02-25 2010-08-26 Jing Xiao Subdivision Weighting for Robust Object Model Fitting
CN105427385A (zh) * 2015-12-07 2016-03-23 华中科技大学 一种基于多层形变模型的高保真人脸三维重建方法
CN106469465A (zh) * 2016-08-31 2017-03-01 深圳市唯特视科技有限公司 一种基于灰度和深度信息的三维人脸重建方法
CN109377563A (zh) * 2018-11-29 2019-02-22 广州市百果园信息技术有限公司 一种人脸网格模型的重建方法、装置、设备和存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SUN, XI: "Non-official translation: Face 3D Reconstruction and Recognition Based on Binocular Stereo Vision", INFORMATION & TECHNOLOGY, CHINA DOCTORAL DISSERTATIONS FULL-TEXT DATABASE, 15 September 2011 (2011-09-15), ISSN: 1674-022X, DOI: 20200121142655X *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115908712A (zh) * 2022-11-28 2023-04-04 阿里巴巴（中国）有限公司 基于图像的三维重建及模型训练方法、设备及存储介质

Also Published As

Publication number Publication date
US11972527B2 (en) 2024-04-30
CN109377563A (zh) 2019-02-22
US20220292776A1 (en) 2022-09-15

Similar Documents

Publication Publication Date Title
WO2020108304A1 (zh) 人脸网格模型的重建方法、装置、设备和存储介质
CN107832807B (zh) 一种图像处理方法和系统
Tagliasacchi et al. Curve skeleton extraction from incomplete point cloud
CN111583408B (zh) 基于手绘草图的人体三维建模系统
US8411081B2 (en) Systems and methods for enhancing symmetry in 2D and 3D objects
US20130124148A1 (en) System and Method for Generating Editable Constraints for Image-based Models
WO2021063271A1 (zh) 人体模型重建方法、重建系统及存储介质
CN114842136B (zh) 一种基于可微分渲染器的单图三维人脸重建方法
WO2024103890A1 (zh) 模型构建方法、重建方法、装置、电子设备及非易失性可读存储介质
US12361663B2 (en) Dynamic facial hair capture of a subject
CN116912433B (zh) 三维模型骨骼绑定方法、装置、设备及存储介质
CN111951368A (zh) 一种点云、体素和多视图融合的深度学习方法
CN116433812B (zh) 利用2d人脸图片生成虚拟人物的方法和装置
CN117876609B (zh) 一种多特征三维人脸重建方法、系统、设备及存储介质
Guo et al. Tetsphere splatting: Representing high-quality geometry with lagrangian volumetric meshes
CN112862949A (zh) 基于多视图的物体3d形状重建方法
Chen et al. Sketch2NeRF: multi-view sketch-guided text-to-3D generation
CN116580170A (zh) 基于二维图像特征引导的三维模型变形方法及装置
CN114494576A (zh) 一种基于隐函数的快速高精度多视点人脸三维重建方法
Lyu et al. Laplacian-based 3D mesh simplification with feature preservation
Zhang et al. Norest-net: normal estimation neural network for 3-d noisy point clouds
CN118521699B (zh) 一种虚拟人三维头发丝发型的生成方法及系统
He et al. Data-driven 3D human head reconstruction
CN120543786A (zh) 基于稀疏视图的三维图像重建方法、装置及计算机设备
CN118397016A (zh) 基于结构先验的室内场景布局估计方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19889175

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19889175

Country of ref document: EP

Kind code of ref document: A1