CN117557721A - A single image detail three-dimensional face reconstruction method, system, equipment and medium - Google Patents

A single image detail three-dimensional face reconstruction method, system, equipment and medium Download PDF

Info

Publication number
CN117557721A
CN117557721A CN202311463711.8A CN202311463711A CN117557721A CN 117557721 A CN117557721 A CN 117557721A CN 202311463711 A CN202311463711 A CN 202311463711A CN 117557721 A CN117557721 A CN 117557721A
Authority
CN
China
Prior art keywords
dimensional face
face
image
features
detailed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311463711.8A
Other languages
Chinese (zh)
Inventor
李慧斌
王静婷
余璀璨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Jiaotong University
Original Assignee
Xian Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Jiaotong University filed Critical Xian Jiaotong University
Priority to CN202311463711.8A priority Critical patent/CN117557721A/en
Publication of CN117557721A publication Critical patent/CN117557721A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本发明公开了一种单张图像细节三维人脸重建方法、系统、设备和介质，对二维人脸图像进行三维人脸重建、渲染与提取特征，得到全局特征；对二维人脸图像进行三维人脸重建、渲染后的图片进行光线匹配，并计算人脸顶点的符号距离值以及局部位置特征；根据全局特征以及局部位置特征，得到隐式双向反射函数；优化SDF‑Net以及隐式双向反射函数的各个分量网络；计算细节的三维人脸几何模型以及每个顶点对应的颜色值，组合构成带有高保真纹理的细节三维人脸模型。本发明仅使用单张输入图片，可以实现单张图像的三维人脸重建，省去了数据集构建及预处理等步骤；本方法使用隐式双向反射函数来表示人脸纹理，能够得到高保真的三维人脸纹理。

The invention discloses a single image detail three-dimensional face reconstruction method, system, equipment and medium, which performs three-dimensional face reconstruction, rendering and feature extraction on two-dimensional face images to obtain global features; The 3D face reconstruction and rendered pictures are subjected to light matching, and the symbolic distance values of the face vertices and local position features are calculated; based on the global features and local position features, the implicit bidirectional reflection function is obtained; SDF‑Net and implicit bidirectional reflection are optimized Each component network of the reflection function; calculates the detailed 3D face geometric model and the color value corresponding to each vertex, which is combined to form a detailed 3D face model with high-fidelity texture. This invention only uses a single input image to achieve three-dimensional face reconstruction of a single image, eliminating steps such as data set construction and preprocessing; this method uses an implicit bidirectional reflection function to represent face texture, and can obtain high fidelity 3D face texture.

Description

Method, system, equipment and medium for reconstructing detail three-dimensional face of single image
Technical Field
The invention belongs to the technical field of three-dimensional face recognition, and particularly relates to a method, a system, equipment and a medium for reconstructing a single-image detail three-dimensional face.
Background
In recent years, face images are widely used in the life of people, and applications such as face recognition, face beautifying and face editing are ubiquitous. The concepts of "universe", "digital person" and the like also enter the field of view of the masses, and bring brand new experience to the life and entertainment of people. Three-dimensional face reconstruction has also received widespread attention as an important component of "digital man" technology. The existing methods are mainly divided into two main categories: methods based on implicit spatial coding and methods based on explicit spatial regression.
However, some existing methods have certain defects in geometric details and face textures. For three-dimensional face geometry, most methods are based on a linear three-dimensional face deformable model (3 DMM), the result is very linear, the reconstruction result lacks personalized geometric details, and different individuals have the problem of small visual difference. For three-dimensional face geometry, most methods construct the texture of a three-dimensional face by regressing vertex-by-vertex RGB values, the texture lacks reality, and the problems of illumination and the like are not considered.
In addition, most methods for reconstructing three-dimensional faces are trained based on data set construction, however, three-dimensional face data is difficult to acquire, open source data sets are fewer and different in quality, data preprocessing and registration are needed for different data sets, a large amount of manpower and time are consumed, and the reconstruction result has a great correlation with preprocessing quality. Therefore, exploring a three-dimensional face reconstruction method with personalized face geometry and high-fidelity texture is an urgent and challenging problem.
Disclosure of Invention
In order to overcome the problems in the prior art, the invention aims to provide a single image detail three-dimensional face reconstruction method, a system, equipment and a medium.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a single image detail three-dimensional face reconstruction method comprises the following steps:
acquiring a two-dimensional face image;
carrying out three-dimensional face reconstruction, rendering and feature extraction on the two-dimensional face image to obtain global features;
performing three-dimensional face reconstruction on the two-dimensional face image, performing light matching on the rendered image, and calculating a symbol distance value and local position characteristics of the face vertex;
obtaining an implicit bidirectional reflection function according to the global features and the local position features;
optimizing each component network of the SDF-Net and the implicit bidirectional reflection function by adopting a self-supervision training method;
calculating a detailed three-dimensional face geometric model according to the optimized SDF-Net and the symbol distance value of the face vertex;
according to each component network of the optimized bidirectional reflection function, calculating to obtain a color value corresponding to each vertex;
and combining the three-dimensional face geometric model containing the details and the color values corresponding to each vertex to form the detail three-dimensional face model with high-fidelity textures.
Further, three-dimensional face reconstruction, rendering and feature extraction are performed on the two-dimensional face image to obtain global image features and global depth features, and the method comprises the following steps:
reconstructing and rendering a three-dimensional face image to obtain a multi-view image set, and calculating a multi-view depth map set through a monocular depth estimation algorithm according to the multi-view image set; and respectively extracting features from the multi-view image set and the multi-view depth map set to obtain global image features and global depth features.
Further, performing three-dimensional face reconstruction on a two-dimensional face image, performing light matching on the rendered image, and calculating a symbol distance value and local position characteristics of a face vertex, wherein the method comprises the following steps of:
performing three-dimensional face reconstruction on the two-dimensional face image, and performing ray matching on the rendered picture to obtain an intersection point of the ray and the hidden surface; and inputting the three-dimensional coordinates of the intersection points of the light rays and the hidden surface into the SDF-Net to calculate the symbol distance value and the local position characteristic of the vertex of the human face.
Further, a set of multi-view depth maps is calculated from the set of multi-view images by a monocular depth estimation algorithm.
Further, the SDF-Net and each component network of the implicit bidirectional reflection function are optimized through the convergence of the loss function;
loss functionThe method comprises the following steps:
wherein alpha is 1 As a first coefficient, alpha 2 Is the second coefficient, alpha 3 Is a third coefficient, alpha 4 Is a fourth coefficient;
pixel level loss
Wherein P represents the pixel where the sampling point is located, P is the combination of the pixels,for the pixel where the sampling point is located, +.>Mask at pixel p for image i, +.>For the pixel value of image i at pixel p, c p (i) For the pixel value calculated by adopting the bidirectional reflection function, i is the image sequence number;
mask loss
Wherein M represents a mask;s is a set of colored images i,α (p) is an activation functionAlpha is a super parameter;
symbol distance field loss
In the method, in the process of the invention,representing the desire;
for point p * Is the nearest neighbor of (2), registration loss->Can be expressed as:
loss in normal direction
Further, implicit bidirectional reflectance functionThe method comprises the following steps:
wherein,representing diffuse reflection albedo, x p Representing the intersection of a ray with a hidden surface, +.>Representing light rays and primaryIntersection points of the initial coarse mesh->Representing local position features corresponding to the jth picture,/->Representing global image features corresponding to the jth picture,/->Representing diffuse reflection shadows, < >>Representing the intersection of a ray with an initial coarse mesh +.>Is arranged in the normal direction of (a),representing global depth features corresponding to the jth picture, a s Representing specular reflection albedo, < >>Represents specular reflection shadows, n p Representing the intersection point x of a ray with a hidden surface p V denotes the direction of light.
Further, the detailed three-dimensional face geometric model is as follows:
wherein G is d A three-dimensional face geometry model representing details,represents the face vertex set, n represents the normal direction of the face vertex,/->Symbol distance value representing face vertex, G c Representing an initial coarse mesh.
A low quality three-dimensional face recognition system, comprising:
the two-dimensional face image acquisition module is used for acquiring a two-dimensional face image;
the global feature acquisition module is used for carrying out three-dimensional face reconstruction, rendering and feature extraction on the two-dimensional face image to obtain global features;
the matching and calculating module is used for carrying out three-dimensional face reconstruction on the two-dimensional face image, carrying out light matching on the rendered image, and calculating the symbol distance value and the local position characteristic of the face vertex;
the implicit bidirectional reflection function calculation module is used for calculating and obtaining an implicit bidirectional reflection function according to the global characteristics and the local position characteristics;
an optimization module for optimizing each component network of the SDF-Net and the implicit bidirectional reflection function;
the detailed three-dimensional face geometric model calculation module is used for calculating a detailed three-dimensional face geometric model according to the optimized SDF-Net and the symbol distance value of the face vertex;
the color value calculation module is used for calculating and obtaining a color value corresponding to each vertex according to each component network of the optimized bidirectional reflection function;
and the combination module is used for combining the three-dimensional face geometric model containing details and the color value corresponding to each vertex to form the detail three-dimensional face model with high-fidelity textures.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the single image detail three-dimensional face reconstruction method as described above.
A computer readable storage medium storing a computer program which when executed by a processor implements the steps of a single image detail three-dimensional face reconstruction method as described above.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, three-dimensional face reconstruction is realized through the two-dimensional face image, no additional training data is needed, and as a self-supervision training method is adopted in the SDF-Net and bidirectional reflection function optimization process, only a single input picture is used, and the three-dimensional face reconstruction of a single image can be realized through the image set of three-dimensional face reconstruction of the two-dimensional face image by the method, and the steps of data set construction, preprocessing and the like are omitted; the method uses the implicit bidirectional reflection function to represent the face texture, and can obtain the high-fidelity three-dimensional face texture.
Furthermore, the invention fuses the local position features with the global image and depth features, and guides and calculates the three-dimensional face with high fidelity by using different fusion features aiming at each component network of different implicit bidirectional reflection functions.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a graph showing the reconstruction geometry results of the present invention;
FIG. 3 is a graph of the reconstruction geometry and texture results of the present invention;
FIG. 4 is a flow chart of the present invention;
FIG. 5 is a schematic diagram of the system of the present invention.
Detailed Description
In order that the invention may be readily understood, a more complete description of the invention will be rendered by reference to the appended drawings. The drawings illustrate preferred embodiments of the invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
The invention is divided into texture and geometry parts: firstly, for an input two-dimensional face image, styleGAN in a 3DMM method is used for generating and complementing texture information of an invisible viewing angle on the basis of a reference image. The StyleGAN is trained based on the data set, and high-fidelity and clear face textures can be generated. By means of a traditional 3DMM model, a rough geometrical three-dimensional face with high-fidelity textures is obtained as an initial three-dimensional face. And rendering the initial three-dimensional face to obtain a multi-view two-dimensional face image, and generating a corresponding depth map by using a monocular depth estimation algorithm.
To fully utilize the two-dimensional image information, a cross-regional sensitive feature extractor is used to extract the corresponding depth map features. A local-global implicit differentiable rendering framework based on the symbol distance field is then used to optimize the SDF-Net and the respective component networks of the bi-directional reflection function. And calculating a detailed three-dimensional face geometric model according to the optimized SDF-Net. According to each component network of the optimized bidirectional reflection function, obtaining a bidirectional reflection function calculated value (namely a color value) corresponding to each vertex, and finally, obtaining a three-dimensional face geometric model G containing details d And registering the calculated values of the bidirectional reflection functions corresponding to each vertex, and realizing the restoration of geometric details. Based on the method, the invention has high-fidelity face textures and fine three-dimensional face geometry, and the three-dimensional face with personalized geometric details and high-fidelity textures can be obtained.
Specifically, referring to fig. 1 and fig. 4, the method for reconstructing a detail three-dimensional face of a single image according to the present invention is divided into two stages of coarse reconstruction and detail reconstruction, and specifically comprises the following steps:
(1) Coarse reconstruction stage
Step 1.1: firstly, for an input two-dimensional face image, a 3 DMM-based method is used for reconstructing a three-dimensional face, and a coarse-geometry three-dimensional face with high-fidelity textures is obtained as an initial three-dimensional face.
Step 1.2: rendering the initial three-dimensional face to obtain a multi-view image set I * ＝{I 1 ,I 2 ,I 3 ,…,I N Computing a set D of multi-view depth maps from the set of multi-view images by a monocular depth estimation algorithm * ＝{D 1 ,D 2 ,D 3 ,…,D N }，I 1 For a first web at multiple viewing anglesImage, I 2 For the second multiview image, I 3 For the third multiview image, I N For the nth multi-view image, D 1 For the first multi-view depth map, D 2 For the second multi-view depth map, D 3 For the third multi-view depth map, D N And N is the number of multi-view depth maps for the fourth multi-view depth map.
(2) Detail reconstruction stage
Step one: respectively extracting features from the multi-view image set and the multi-view depth map set to obtain global image features and global depth features;
step two: performing light matching on each pixel point on each picture in the multi-view image set, and obtaining an intersection point x of light and the hidden surface based on a light ray tracing algorithm p Intersection point of light ray and coarse grid
Intersection point x of ray and hidden surface p As sampling points, three-dimensional coordinates of the sampling points are input into SDF-Net to calculate the symbol distance value of the face vertex and the local position characteristics of the sampling points;
step three: the method uses a bi-directional reflection function Seen as being composed of four components +.>a s ，Constructing;
wherein,representing diffuse reflection albedo, x p Representing lightIntersection of line with hidden surface->Representing the intersection of a ray with the initial coarse mesh, +.>Representing local position features corresponding to the jth picture,/->Representing global image features corresponding to the jth picture,/->Representing diffuse reflection shadows, < >>Representing the intersection of a ray with an initial coarse mesh +.>Is arranged in the normal direction of (a),representing global depth features corresponding to the jth picture, a s Representing the specular albedo, being constant, +.>Represents specular reflection shadows, n p Representing the intersection point x of a ray with a hidden surface p V denotes the direction of light.
Thus, an implicit bidirectional reflection functionCan be written as:
thus, for different components, different features are input into the fully connected network to calculate component values:
specifically, diffuse reflection albedo is calculated according to the global image characteristics and the local position characteristics
Calculating diffuse reflection shadows from global depth features and local position featuresSpecular reflection shadow->
Step four: the loss function is calculated to converge to optimize the SDF-Net and the respective component networks of the bi-directional reflection function.
Loss functionCan be expressed as:
wherein alpha is 1 As a first coefficient, alpha 2 Is the second coefficient, alpha 3 Is a third coefficient, alpha 4 Is a fourth coefficient;
pixel level loss
Wherein P represents the pixel where the sampling point is located, P is the combination of the pixels,for the pixel where the sampling point is located, +.>Mask at pixel p for image i, +.>For the pixel value of image i at pixel p, c p (i) For the pixel value calculated using the bi-directional reflection function, i is the image sequence number.
Mask loss
Wherein M represents a mask;s is a set of colored pixels i，α (p) is an activation function and α is a hyper-parameter.
Symbol distance field loss
In the method, in the process of the invention,indicating the desire.
In addition, provision is made forFor point p * Is the nearest neighbor of (2), registration loss->Can be expressed as: />
Loss in normal direction
Step five: and calculating a detailed three-dimensional face geometric model according to the optimized SDF-Net.
Wherein G is d A three-dimensional face geometry model representing details,represents the face vertex set, n represents the normal direction of the face vertex,/->Symbol distance value representing face vertex, G c Representing an initial coarse mesh.
Three-dimensional face geometry model G containing details d And the calculated value (namely the color value) of the bi-directional reflection function corresponding to each vertex, thus forming the detail three-dimensional face model with high-fidelity textures.
Fig. 2 is a graph of the results of reconstruction for 5 input face pictures, each method. As can be seen from fig. 2, the three-dimensional face reconstructed by the method (MMFG method) has more personalized details than other similar methods (FaceScape method, pixiv2Vertex method, PBIDR method).
Fig. 3 shows the reconstruction geometry and texture results of the present invention, and fig. 3 shows four sets of sample results, three each, input image as input picture, generated 3d mesh as reconstruction geometry, texture 3d mesh as textured reconstruction results. As can be seen from fig. 3, the reconstruction result of the method contains both personalized geometric details and high-fidelity face textures.
Referring to fig. 5, another embodiment of the present invention provides a low-quality three-dimensional face recognition system, comprising:
the two-dimensional face image acquisition module is used for acquiring a two-dimensional face image;
the global feature acquisition module is used for carrying out three-dimensional face reconstruction, rendering and feature extraction on the two-dimensional face image to obtain global features;
the matching and calculating module is used for carrying out three-dimensional face reconstruction on the two-dimensional face image, carrying out light matching on the rendered image, and calculating the symbol distance value and the local position characteristic of the face vertex;
the implicit bidirectional reflection function calculation module is used for calculating and obtaining an implicit bidirectional reflection function according to the global characteristics and the local position characteristics;
an optimization module for optimizing each component network of the SDF-Net and the implicit bidirectional reflection function;
the detailed three-dimensional face geometric model calculation module is used for calculating a detailed three-dimensional face geometric model according to the optimized SDF-Net and the symbol distance value of the face vertex;
the color value calculation module is used for calculating and obtaining a color value corresponding to each vertex according to each component network of the optimized bidirectional reflection function;
and the combination module is used for combining the three-dimensional face geometric model containing details and the color value corresponding to each vertex to form the detail three-dimensional face model with high-fidelity textures.
Another embodiment of the invention provides a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the single image detail three-dimensional face reconstruction method as described above when the processor executes the computer program.
Another embodiment of the present invention provides a computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the single image detail three-dimensional face reconstruction method as described above.
The invention has the following advantages:
first: the method is a reconstruction method specific to the input picture, does not need additional training data, and can realize three-dimensional face reconstruction of a single image by using a small data set of the method by only using the single input picture due to the adoption of a self-supervision training method, and the steps of data set construction, preprocessing and the like are omitted;
second,: the method uses a bi-directional reflection function to represent the face texture so as to obtain the high-fidelity three-dimensional face texture. According to the invention, the local position features are fused with the global image and depth features, and different fusion features are used for guiding calculation aiming at different bidirectional reflection function components, so that the high-fidelity three-dimensional face can be better reconstructed.
The foregoing is illustrative of the preferred embodiments of the present invention and is not to be construed as limiting the claims. The present invention is not limited to the above embodiments, and the specific structure thereof is allowed to vary. It is intended that all such variations as fall within the scope of the appended claims.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.

Claims (10)

1.一种单张图像细节三维人脸重建方法，其特征在于，包括以下步骤：1. A single image detail three-dimensional face reconstruction method, which is characterized by including the following steps: 获取二维人脸图像；Obtain two-dimensional face images; 对二维人脸图像进行三维人脸重建、渲染与提取特征，得到全局特征；Perform 3D face reconstruction, rendering and feature extraction on 2D face images to obtain global features; 对二维人脸图像进行三维人脸重建、渲染后的图片进行光线匹配，并计算人脸顶点的符号距离值以及局部位置特征；Perform three-dimensional face reconstruction on two-dimensional face images, perform light matching on the rendered images, and calculate the signed distance values and local position features of the face vertices; 根据全局特征以及局部位置特征，得到隐式双向反射函数；Based on the global features and local location features, the implicit bidirectional reflection function is obtained; 采用自监督训练方法优化SDF-Net以及隐式双向反射函数的各个分量网络；Self-supervised training method is used to optimize each component network of SDF-Net and implicit bidirectional reflection function; 根据优化后的SDF-Net以及人脸顶点的符号距离值，计算细节的三维人脸几何模型；Based on the optimized SDF-Net and the signed distance values of the face vertices, calculate the detailed three-dimensional face geometric model; 根据优化后的双向反射函数的各个分量网络，计算得到每个顶点对应的颜色值；According to each component network of the optimized bidirectional reflection function, the color value corresponding to each vertex is calculated; 将包含细节的三维人脸几何模型以及每个顶点对应的颜色值组合，构成带有高保真纹理的细节三维人脸模型。A detailed 3D face geometric model containing details and the color value corresponding to each vertex are combined to form a detailed 3D face model with high-fidelity texture. 2.根据权利要求1所述的单张图像细节三维人脸重建方法，其特征在于，对二维人脸图像进行三维人脸重建、渲染与提取特征，得到全局图像特征以及全局深度特征，包括以下步骤：2. The single image detail three-dimensional face reconstruction method according to claim 1, characterized in that three-dimensional face reconstruction, rendering and feature extraction are performed on the two-dimensional face image to obtain global image features and global depth features, including Following steps: 对二维人脸图像进行三维人脸重建、渲染，得到多视角图像集合，根据多视角图像集合通过单目深度估计算法计算多视角深度图集合；对多视角图像集合和多视角深度图集合分别提取特征，得到全局图像特征以及全局深度特征。Perform three-dimensional face reconstruction and rendering on the two-dimensional face image to obtain a multi-view image set, and calculate the multi-view depth map set through the monocular depth estimation algorithm based on the multi-view image set; the multi-view image set and the multi-view depth map set are separately Extract features to obtain global image features and global depth features. 3.根据权利要求1所述的单张图像细节三维人脸重建方法，其特征在于，对二维人脸图像进行三维人脸重建、渲染后的图片进行光线匹配，并计算人脸顶点的符号距离值以及局部位置特征，包括以下步骤：3. The three-dimensional face reconstruction method of single image details according to claim 1, characterized in that three-dimensional face reconstruction is performed on the two-dimensional face image, the rendered image is subjected to light matching, and the symbols of the face vertices are calculated. Distance values and local location features include the following steps: 对二维人脸图像进行三维人脸重建、渲染后的图片进行光线匹配，获得光线与隐表面的交点；将光线与隐表面的交点三维坐标输入到SDF-Net中计算人脸顶点的符号距离值以及局部位置特征。Perform 3D face reconstruction on the 2D face image and perform ray matching on the rendered image to obtain the intersection point of the ray and the hidden surface; input the 3D coordinates of the intersection point of the ray and the hidden surface into SDF-Net to calculate the signed distance of the face vertices values and local location features. 4.根据权利要求1所述的单张图像细节三维人脸重建方法，其特征在于，根据多视角图像集合通过单目深度估计算法计算多视角深度图集合。4. The single image detailed three-dimensional face reconstruction method according to claim 1, characterized in that the multi-view depth map set is calculated through a monocular depth estimation algorithm based on the multi-view image set. 5.根据权利要求1所述的单张图像细节三维人脸重建方法，其特征在于，通过损失函数收敛，优化SDF-Net以及隐式双向反射函数的各个分量网络；5. The single image detail three-dimensional face reconstruction method according to claim 1, characterized in that, through the convergence of the loss function, each component network of SDF-Net and the implicit bidirectional reflection function is optimized; 损失函数为：loss function for: 其中，α1为第一系数，α2为第二系数，α3为第三系数，α4为第四系数；Among them, α 1 is the first coefficient, α 2 is the second coefficient, α 3 is the third coefficient, and α 4 is the fourth coefficient; 像素级别损失 Pixel level loss 式中，p表示采样点所在像素，P为像素的结合，为采样点所在的像素，/>为图像i在像素p的掩码，/>为图像i在像素p的像素值，cp(i)为采用双向反射函数计算得到的像素值，i为图像序号；In the formula, p represents the pixel where the sampling point is located, and P is the combination of pixels. is the pixel where the sampling point is located,/> is the mask of image i at pixel p, /> is the pixel value of image i at pixel p, c p (i) is the pixel value calculated using the bidirectional reflection function, and i is the image serial number; 掩码损失 mask loss 式中，M表示掩码；为有颜色的像集合，si,α(p)为激活函数，α为超参数；In the formula, M represents the mask; is the set of colored images, s i, α (p) is the activation function, and α is the hyperparameter; 符号距离场损失 Signed distance field loss 式中，表示期望；In the formula, express expectations; 为点p的最邻近点，则配准损失/>可以表示为：/> is the nearest neighbor point of point p, then the registration loss/> It can be expressed as:/> 法向损失 normal loss 6.根据权利要求1所述的单张图像细节三维人脸重建方法，其特征在于，隐式双向反射函数为：6. The single image detailed three-dimensional face reconstruction method according to claim 1, characterized in that the implicit bidirectional reflection function for: 其中，表示漫反射反照率，xp表示光线与隐表面的交点，/>表示光线与初始粗糙网格的交点，/>表示第j张图片对应的局部位置特征，/>表示第j张图片对应的全局图像特征，/>表示漫反射阴影，/>表示光线与初始粗糙网格的交点/>的法向，/>表示第j张图片对应的全局深度特征，as表示镜面反射反照率，/>表示镜面反射阴影，np表示光线与隐表面的交点xp的法向，v表示光线方向。in, represents the diffuse reflection albedo, x p represents the intersection point of the light ray and the hidden surface, /> Represents the intersection point of the ray and the initial rough mesh, /> Indicates the local location features corresponding to the j-th picture,/> Indicates the global image features corresponding to the j-th image,/> Represents diffuse shadow, /> Represents the intersection point of the ray and the initial rough mesh/> normal direction of ,/> represents the global depth feature corresponding to the j-th picture, a s represents the specular reflection albedo,/> Represents the specular reflection shadow, n p represents the normal direction of the intersection point x p between the light and the hidden surface, and v represents the direction of the light. 7.根据权利要求1所述的单张图像细节三维人脸重建方法，其特征在于，细节的三维人脸几何模型为：7. The detailed three-dimensional face reconstruction method of a single image according to claim 1, characterized in that the detailed three-dimensional face geometric model is: 其中，Gd表示细节的三维人脸几何模型，V表示人脸顶点集合，n表示人脸顶点的法向，表示人脸顶点的符号距离值，Gc表示初始粗糙网格。Among them, G d represents the detailed three-dimensional face geometric model, V represents the face vertex set, n represents the normal direction of the face vertices, represents the signed distance value of the face vertices, and G c represents the initial rough mesh. 8.一种低质量三维人脸识别系统，其特征在于，包括：8. A low-quality three-dimensional face recognition system, characterized by: 二维人脸图像获取模块，用于获取二维人脸图像；Two-dimensional face image acquisition module, used to acquire two-dimensional face images; 全局特征获取模块，用于对二维人脸图像进行三维人脸重建、渲染与提取特征，得到全局特征；The global feature acquisition module is used to perform three-dimensional face reconstruction, rendering and feature extraction on two-dimensional face images to obtain global features; 匹配与计算模块，用于对二维人脸图像进行三维人脸重建、渲染后的图片进行光线匹配，并计算人脸顶点的符号距离值以及局部位置特征；The matching and calculation module is used to perform three-dimensional face reconstruction on two-dimensional face images, perform light matching on the rendered pictures, and calculate the symbolic distance values and local position features of the face vertices; 隐式双向反射函数计算模块，用于根据全局特征以及局部位置特征，计算得到隐式双向反射函数；The implicit bidirectional reflection function calculation module is used to calculate the implicit bidirectional reflection function based on global features and local location features; 优化模块，用于优化SDF-Net以及隐式双向反射函数的各个分量网络；Optimization module, used to optimize each component network of SDF-Net and implicit bidirectional reflection function; 细节的三维人脸几何模型计算模块，用于根据优化后的SDF-Net以及人脸顶点的符号距离值，计算细节的三维人脸几何模型；The detailed three-dimensional face geometric model calculation module is used to calculate the detailed three-dimensional face geometric model based on the optimized SDF-Net and the signed distance value of the face vertices; 颜色值计算模块，用于根据优化后的双向反射函数的各个分量网络，计算得到每个顶点对应的颜色值；The color value calculation module is used to calculate the color value corresponding to each vertex based on each component network of the optimized bidirectional reflection function; 组合模块，用于将包含细节的三维人脸几何模型以及每个顶点对应的颜色值组合，构成带有高保真纹理的细节三维人脸模型。The combination module is used to combine the detailed 3D face geometric model and the color value corresponding to each vertex to form a detailed 3D face model with high-fidelity texture. 9.一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至7任一项所述单张图像细节三维人脸重建方法的步骤。9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, it implements the claims as claimed in The steps of the single image detail three-dimensional face reconstruction method described in any one of 1 to 7. 10.一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述单张图像细节三维人脸重建方法的步骤。10. A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that when the computer program is executed by a processor, the single image as claimed in any one of claims 1 to 7 is realized. Detailed steps of the 3D face reconstruction method.
CN202311463711.8A 2023-11-06 2023-11-06 A single image detail three-dimensional face reconstruction method, system, equipment and medium Pending CN117557721A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311463711.8A CN117557721A (en) 2023-11-06 2023-11-06 A single image detail three-dimensional face reconstruction method, system, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311463711.8A CN117557721A (en) 2023-11-06 2023-11-06 A single image detail three-dimensional face reconstruction method, system, equipment and medium

Publications (1)

Publication Number Publication Date
CN117557721A true CN117557721A (en) 2024-02-13

Family

ID=89813823

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311463711.8A Pending CN117557721A (en) 2023-11-06 2023-11-06 A single image detail three-dimensional face reconstruction method, system, equipment and medium

Country Status (1)

Country Link
CN (1) CN117557721A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117876609A (en) * 2024-03-11 2024-04-12 国网电商科技有限公司 A method, system, device and storage medium for multi-feature three-dimensional face reconstruction
CN118941723A (en) * 2024-10-12 2024-11-12 南昌大学 A 3D face reconstruction method based on differentiable ray tracing
CN119068098A (en) * 2024-11-05 2024-12-03 浙江大学 Method and device for generating rendering texture map based on ultra-high resolution portrait

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117876609A (en) * 2024-03-11 2024-04-12 国网电商科技有限公司 A method, system, device and storage medium for multi-feature three-dimensional face reconstruction
CN117876609B (en) * 2024-03-11 2024-05-24 国网电商科技有限公司 Multi-feature three-dimensional face reconstruction method, system, equipment and storage medium
CN118941723A (en) * 2024-10-12 2024-11-12 南昌大学 A 3D face reconstruction method based on differentiable ray tracing
CN119068098A (en) * 2024-11-05 2024-12-03 浙江大学 Method and device for generating rendering texture map based on ultra-high resolution portrait

Similar Documents

Publication Publication Date Title
CN109255831B (en) A method for single-view face 3D reconstruction and texture generation based on multi-task learning
Niklaus et al. 3d ken burns effect from a single image
JP7448566B2 (en) Scalable 3D object recognition in cross-reality systems
CN110223370B (en) Method for generating complete human texture map from single-view picture
CN117557721A (en) A single image detail three-dimensional face reconstruction method, system, equipment and medium
Rematas et al. Image-based synthesis and re-synthesis of viewpoints guided by 3d models
CN115115805B (en) Training method, device, equipment and storage medium for three-dimensional reconstruction model
Condorelli et al. A comparison between 3D reconstruction using nerf neural networks and mvs algorithms on cultural heritage images
CN117011493B (en) Three-dimensional face reconstruction method, device and equipment based on symbol distance function representation
CN116416376A (en) Three-dimensional hair reconstruction method, system, electronic equipment and storage medium
CN118138741B (en) Naked eye 3D data communication method
Maxim et al. A survey on the current state of the art on deep learning 3D reconstruction
CN116228986A (en) Indoor scene illumination estimation method based on local-global completion strategy
Purushwalkam et al. Conrad: Image constrained radiance fields for 3d generation from a single image
Dai et al. PBR-Net: Imitating physically based rendering using deep neural network
CN114255328B (en) A 3D reconstruction method of ancient artifacts based on single view and deep learning
JP6898264B2 (en) Synthesizers, methods and programs
Zhang et al. Virtual lighting environment and real human fusion based on multiview videos
CN115619974A (en) Large scene three-dimensional reconstruction method, reconstruction device, equipment and storage medium based on improved PatchMatch network
Zhu et al. Relighting scenes with object insertions in neural radiance fields
CN111243099A (en) Method and device for processing image and method and device for displaying image in AR (augmented reality) device
CN114677292B (en) High-resolution material restoration method based on two-image inverse rendering neural network
CN119048369B (en) A method for synthesizing new views using noisy images
Wang et al. 3d reconstruction of plant based on nerf
CN119512490B (en) Intelligent mainboard display driving system based on 3D display and reality enhancement

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination