WO2020207270A1 - 一种三维人脸重建方法及其系统、装置、存储介质 - Google Patents

一种三维人脸重建方法及其系统、装置、存储介质 Download PDF

Info

Publication number
WO2020207270A1
WO2020207270A1 PCT/CN2020/081684 CN2020081684W WO2020207270A1 WO 2020207270 A1 WO2020207270 A1 WO 2020207270A1 CN 2020081684 W CN2020081684 W CN 2020081684W WO 2020207270 A1 WO2020207270 A1 WO 2020207270A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
original
dimensional
face image
human body
Prior art date
Application number
PCT/CN2020/081684
Other languages
English (en)
French (fr)
Inventor
徐颖
翟懿奎
江子义
李基伟
甘俊英
周健文
杜锡坤
郑俊杰
Original Assignee
五邑大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 五邑大学 filed Critical 五邑大学
Publication of WO2020207270A1 publication Critical patent/WO2020207270A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation

Definitions

  • the invention relates to the technical field of face recognition, in particular to a three-dimensional face reconstruction method and its system, device and storage medium.
  • Human face is the most important and direct carrier of human daily emotion expression and communication.
  • Three-dimensional face modeling has always attracted attention. It has very broad application prospects in the fields of games, film and television special effects, virtual reality, and auxiliary medical care. It is a computer An important research topic in vision.
  • most methods of 3D face reconstruction are to obtain multiple images of the same face from different shooting angles, and obtain 3D face images through operations such as alignment, stitching, and fitting.
  • this kind of method has a large amount of calculation and low efficiency, and the accuracy of the generated three-dimensional face image is not high.
  • the result obtained is only a three-dimensional facial image, not a three-dimensional human image model, which lacks completeness.
  • the purpose of the present invention is to provide a three-dimensional face reconstruction method, system, device, and storage medium, which can improve the efficiency and accuracy of three-dimensional face reconstruction and can output a three-dimensional human body model.
  • an embodiment of the present invention provides a three-dimensional face reconstruction method, including:
  • said acquiring the face feature points, face region and rotation matrix of the original face image, and using a convolutional neural network model to locate the mapping of the original face image from a two-dimensional plane structure to a three-dimensional space structure to obtain Three-dimensional face images in standard poses including:
  • the three-dimensional face coordinate point set and the rotation matrix are used to perform matrix operations to obtain a three-dimensional face image in a standard posture.
  • the acquiring the gender attribute of the original face image and the original human body model corresponding to the gender includes:
  • the acquiring the face data of the original human body model includes:
  • the scaling the three-dimensional face image to a size similar to the face of the original human model, and then making the face area of the original human model fit and deform the scaled three-dimensional face image includes:
  • zoom factor to zoom the three-dimensional face image to a size similar to the face area of the original human model
  • the cost function is constructed to solve the optimal solution, so that the face area of the original human body model is fitted and deformed to the scaled three-dimensional face image.
  • the ratio of the part distance includes at least one of the ratio of the horizontal distance from the corner of the left eye to the right corner of the eye, the ratio of the horizontal distance between the corners of the mouth, the ratio of the vertical distance between the eyebrows and the lower lip, and the ratio of the vertical distance from the tip of the nose to the chin.
  • an embodiment of the present invention also proposes a three-dimensional face reconstruction system, including:
  • the face reconstruction unit is used to obtain the face feature points, face area and rotation matrix of the original face image, and use the convolutional neural network model to locate the original face image from a two-dimensional plane structure to a three-dimensional space structure Mapping to obtain a three-dimensional face image in a standard pose;
  • the fitting deformation unit is used to scale the three-dimensional face image to the size of the face of the original human model, and then make the face area of the original human model fit and deform the scaled three-dimensional face image;
  • the rendering unit is used to render the original human body model after fitting and deformation to obtain a three-dimensional human body model.
  • the personalized setting unit is used to perform expression fusion deformation and facial shape adjustment on the facial area of the three-dimensional human body model
  • the display unit is used to display a three-dimensional human body model and a personalized setting interface.
  • an embodiment of the present invention also provides a three-dimensional face reconstruction device, including:
  • At least one processor and,
  • a memory communicatively connected with the at least one processor; wherein,
  • the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method according to the first aspect of the present invention.
  • an embodiment of the present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to make a computer execute The method of the first aspect of the invention.
  • a three-dimensional face reconstruction method provided by the embodiments of the present invention directly obtains a standard pose from an original face image through a convolutional neural network model
  • the 3D face image is then scaled to fit and deform the original human body model, which can effectively reduce the limitation on the input original face image and reduce the calculation of the 3D face reconstruction process. It improves the efficiency of 3D face reconstruction and the accuracy of the reconstructed 3D face image.
  • the final output of the 3D human body model can be achieved. It is not only a three-dimensional face image, which effectively guarantees the integrity of the output after reconstruction.
  • a face reconstruction unit directly obtains a three-dimensional face image in a standard pose from an original face image through a convolutional neural network model, and the fitting deformation unit then converts the three-dimensional face
  • the face image is scaled to fit and deform the original human body model, which can effectively reduce the limitation on the input original face image, reduce the amount of calculation in the 3D face reconstruction process, and improve the efficiency of 3D face reconstruction.
  • the accuracy of the reconstructed three-dimensional face image is improved; and the fitting deformation unit can realize the final output of the three-dimensional human body model through the fitting and deformation of the three-dimensional face image and the original human body model, not just the three-dimensional face image. This effectively guarantees the integrity of the output after reconstruction.
  • Fig. 1 is a flowchart of an embodiment of a method for 3D face reconstruction of the present invention
  • Figure 2 is an embodiment of the method for 3D face reconstruction of the present invention to obtain facial feature points, face regions and rotation matrix of the original face image, and use a convolutional neural network model to locate the original face image From the mapping of the two-dimensional plane structure to the three-dimensional space structure, the flow chart of the three-dimensional face image in the standard pose is obtained;
  • FIG. 3 is a flowchart of acquiring the gender attributes of the original face image and the original human body model corresponding to the gender in an embodiment of a method for reconstructing a three-dimensional face of the present invention
  • FIG. 4 is a flowchart of acquiring the face data of the original human body model in an embodiment of a method for 3D face reconstruction of the present invention
  • Fig. 5 is an embodiment of a method for reconstructing a three-dimensional face of the present invention.
  • the three-dimensional face image is scaled to the size of the face of the original human model, and then the face area of the original human model is scaled to Flow chart of fitting and deforming three-dimensional face images;
  • FIG. 6 is a flowchart of another embodiment of a three-dimensional face reconstruction method of the present invention.
  • Figure 7 is a schematic diagram of a three-dimensional face reconstruction system of the present invention.
  • Fig. 8 is a schematic diagram of a three-dimensional face reconstruction device of the present invention.
  • Human face is the most important and direct carrier of human daily emotion expression and communication.
  • Three-dimensional face modeling has always attracted attention. It has very broad application prospects in the fields of games, film and television special effects, virtual reality, and auxiliary medical care. It is a computer An important research topic in vision.
  • most methods of 3D face reconstruction are to obtain multiple images of the same face from different shooting angles, and obtain 3D face images through operations such as alignment, stitching, and fitting.
  • this kind of method has a large amount of calculation and low efficiency, and the accuracy of the generated three-dimensional face image is not high.
  • the result obtained is only a three-dimensional facial image, not a three-dimensional human image model, which lacks completeness.
  • the present invention provides a three-dimensional face reconstruction method and its system, device, and storage medium.
  • the three-dimensional face image in the standard pose is directly obtained from the original face image through the convolutional neural network model, and then the The 3D face image is scaled to fit and deform the original human body model, which can effectively reduce the limitation on the input original face image, reduce the amount of calculation in the 3D face reconstruction process, and improve the efficiency of 3D face reconstruction.
  • the accuracy of the reconstructed 3D face image is improved; and through the fitting deformation of the 3D face image and the original human body model, the final output of the 3D human body model can be realized, not just the 3D face image, thus effectively Ensure the integrity of the output after reconstruction.
  • the first embodiment of the present invention provides a three-dimensional face reconstruction method, including but not limited to the following steps:
  • S200 Obtain face feature points, face regions, and rotation matrix of the original face image, and use a convolutional neural network model to locate the original face image from a two-dimensional planar structure to a three-dimensional spatial structure to obtain a standard pose Three-dimensional face image below;
  • S400 Scale the three-dimensional face image to a size similar to the face of the original human model, and then make the face area of the original human model fit and deform the scaled three-dimensional face image;
  • S500 Render the original human body model after fitting and deformation to obtain a three-dimensional human body model.
  • only one original face image needs to be acquired, and there are many ways to acquire it, for example, it can be downloaded from the Internet or read locally.
  • the shooting angle of the original face image is not limited, and it may be a front face image or a side face image.
  • the convolutional neural network model is used to directly obtain a three-dimensional face image in a standard pose from the original face image, and then the three-dimensional face image is scaled to fit and deform the original human body model, which can be effective It reduces the limitation on the input original face image, reduces the calculation amount of the 3D face reconstruction process, improves the efficiency of 3D face reconstruction, and improves the accuracy of the reconstructed 3D face image; and, through The fitting deformation of the three-dimensional face image and the original human body model can realize the final output of the three-dimensional human body model, not just the three-dimensional face image, thereby effectively ensuring the integrity of the reconstructed output.
  • the second embodiment of the present invention also provides a three-dimensional face reconstruction method, wherein the step S200: obtain the face feature points of the original face image , Face area and rotation matrix, using the convolutional neural network model to locate the original face image from a two-dimensional plane structure to a three-dimensional space structure mapping, to obtain a three-dimensional face image in a standard pose, which specifically includes:
  • S210 Use the OpenCV library and the Dlib library to obtain the face feature points, the face area and the rotation matrix of the corresponding face of the original face image, and crop the face area into an image of a specific resolution;
  • S220 Use the cropped image as the input of the convolutional neural network model, and output the preliminary 3D face image data in the corresponding pose, including the 3D face coordinate point set, the RGB value corresponding to the coordinate point, and the triangular mesh face set;
  • S230 Perform a matrix operation using a three-dimensional face coordinate point set and a rotation matrix to obtain a three-dimensional face image in a standard pose.
  • OpenCV is a cross-platform computer vision library released under the BSD license (open source), which can support a variety of operating systems and interfaces to implement many common algorithms in image processing and computer vision; and Dlib is a C++ open source tool containing machine learning algorithms The package can provide a large number of machine learning/image processing algorithms. Using these two libraries to process the original face image can effectively improve the processing efficiency.
  • the third embodiment of the present invention also provides a three-dimensional face reconstruction method, wherein, the step S300: obtain the gender attribute and correspondence of the original face image Original mannequins of gender, including:
  • S310 Perform gender recognition on the original face image using a neural network combining high-level features and low-level features;
  • the neural network combining high-level features and low-level features is mainly composed of a high-level feature extraction module, a low-level feature extraction module, and a joint voting output module.
  • the high-level feature module is mainly composed of a convolutional layer and a downsampling layer. In the convolutional layer, multiple receptive fields with different weights are used to identify the original face image, thereby extracting different local features of the original face image, reducing The amount of parameters to be trained.
  • the low-level feature extraction module is mainly composed of a flattening processing unit and a reconstruction network. The flattening processing unit first converts the original face image into a one-dimensional original face feature vector, and then inputs it to the reconstruction network for vector reconstruction.
  • the reconstruction network adopts an artificial neural network structure, and through the processing of multiple layers of neurons, the original feature vector of the face is re-represented and described.
  • the high-level feature extraction module convolves and down-samples the input face image, a high-level feature vector is finally obtained through the flattening processing unit.
  • the low-level feature extraction module re-represents the original face image through the reconstruction network to obtain the low-level feature vector
  • the joint voting layer merges the high-level feature vector and the low-level feature vector, and is fully connected with the two neurons of the output layer (corresponding to male and female genders), and completes gender recognition after the final classification judgment.
  • the fourth embodiment of the present invention also provides a three-dimensional face reconstruction method, wherein the step S330: acquiring the face data of the original human body model, specifically include:
  • S331 Perform key point detection on the original human model to determine the face area of the original human model
  • S332 Assign different values to the coordinate points of the face area and other parts of the original human body model to obtain the face data of the original human body model;
  • the coordinate points of the face area and other parts of the original human body model are assigned different values, which can be used to respectively indicate that the area can and cannot be deformed, so as to facilitate subsequent adjustments to the model.
  • the fifth embodiment of the present invention also provides a three-dimensional face reconstruction method, wherein the step S400: zoom the three-dimensional face image to the same
  • the face size of the original human model is similar, and then the face area of the original human model is fitted and deformed to the scaled 3D face image, which specifically includes:
  • S410 Determine the zoom factor based on the ratio of the distance between the three-dimensional face image and the face area of the original human body model
  • S420 Use the zoom factor to zoom the three-dimensional face image to a size similar to the face area of the original human model
  • S430 Construct a cost function to solve an optimal solution, so that the face area of the original human body model is fitted and deformed on the scaled three-dimensional face image.
  • the face area of the original human body model is fitted and deformed to the scaled three-dimensional face image, which can effectively improve the effect of fitting and make the subsequent three-dimensional face The quality of face reconstruction is improved.
  • the sixth embodiment of the present invention also provides a three-dimensional face reconstruction method, wherein the ratio of the part distance includes the ratio of the horizontal distance from the left eye corner to the right eye corner, and the horizontal distance between the two mouth corners. At least one of the ratio of, the ratio of the vertical distance from the eyebrows to the lower lip, and the ratio of the vertical distance from the tip of the nose to the chin.
  • the seventh embodiment of the present invention also provides a three-dimensional face reconstruction method, including but not limited to the following steps:
  • S520 Use the OpenCV library and the Dlib library to obtain the face feature points, the face area, and the rotation matrix of the corresponding face of the original face image, and crop the face area into an image of a specific resolution, and use the cropped image as a roll
  • the input and output of the product neural network model get the preliminary 3D face image data in the corresponding pose, including the 3D face coordinate point set, the RGB value corresponding to the coordinate point, and the triangular mesh face set, using the 3D face coordinate point set and rotation Perform matrix operations on the matrix to obtain a three-dimensional face image in a standard pose;
  • S530 Use a neural network combining high-level features and low-level features to perform gender recognition on the original face image, select the corresponding original human model based on the gender of the original face image, perform key point detection on the original human model, and determine the face of the original human model Part area, assign different values to the coordinate points of the face area and other parts of the original human body model to obtain the face data of the original human body model, and write the face data of the original human body model into the storage file;
  • S540 Determine a zoom factor based on the ratio of the distance between the three-dimensional face image and the face area of the original human body model, and use the zoom factor to zoom the three-dimensional face image to a size similar to the face area of the original human body model, and the construction cost Function to solve the optimal solution, so that the face area of the original human body model fits and deforms the scaled three-dimensional face image;
  • S550 Render the original human body model after fitting and deformation through feature point mapping to obtain a three-dimensional human body model.
  • the original human body model can be freely obtained by constructing an original human body model library, and the source of the model can be local acquisition or network acquisition.
  • a three-dimensional face image in a standard pose is directly obtained from the original face image through a convolutional neural network model, and then the three-dimensional face image is scaled to fit and deform the original human body model, which can effectively reduce
  • the limitation on the input original face image reduces the computational complexity of the 3D face reconstruction process, improves the efficiency of 3D face reconstruction, and improves the accuracy of the reconstructed 3D face image; and, through the 3D face
  • the fitting and deformation of the image and the original human body model can achieve the final output of the three-dimensional human body model, not just the three-dimensional face image, thereby effectively ensuring the integrity of the reconstructed output.
  • the eighth embodiment of the present invention also provides a three-dimensional face reconstruction system, including:
  • the data acquisition unit 110 is configured to acquire an original face image used for three-dimensional face reconstruction, and acquire the gender attribute of the original face image and the original human body model corresponding to the gender;
  • the face reconstruction unit 120 is used to obtain the face feature points, face area and rotation matrix of the original face image, and use a convolutional neural network model to locate the original face image from a two-dimensional plane structure to a three-dimensional space structure Mapping to obtain a three-dimensional face image in a standard pose;
  • the fitting deformation unit 130 is configured to scale the three-dimensional face image to a size similar to the face size of the original human model, and then make the face area of the original human model fit and deform the scaled three-dimensional face image;
  • the rendering unit 140 is configured to render the original human body model after fitting and deformation to obtain a three-dimensional human body model.
  • the face reconstruction unit directly obtains the 3D face image in the standard pose from the original face image through the convolutional neural network model, and the fitting deformation unit then scales the 3D face image to the original human body image.
  • Fitting and deforming the model can effectively reduce the limitation on the input original face image, reduce the amount of calculation in the 3D face reconstruction process, improve the efficiency of 3D face reconstruction, and improve the 3D face obtained by reconstruction The accuracy of the image; and the fitting deformation unit can realize the final output of the three-dimensional human body model through the fitting and deformation of the three-dimensional face image and the original human body model, not just the three-dimensional face image, thereby effectively ensuring the reconstructed output Completeness.
  • the ninth embodiment of the present invention also provides a three-dimensional face reconstruction system, which further includes:
  • the personalization setting unit 150 is used to perform expression fusion deformation and facial shape adjustment on the face area of the three-dimensional human body model
  • the display unit 160 is used to display a three-dimensional human body model and display a personalized setting interface.
  • the personalization setting unit can be used to perform expression fusion deformation and facial shape adjustment on the model face area through the libigl library, to realize expression switching and model beautification, so that the quality of the finally constructed three-dimensional human body model is higher.
  • the tenth embodiment of the present invention also provides a three-dimensional face reconstruction device, including:
  • At least one processor At least one processor
  • the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the first to seventh embodiments described above. Any of the three-dimensional face reconstruction methods in the.
  • the device 200 can be any type of smart terminal, such as a mobile phone, a tablet computer, a personal computer, and so on.
  • the processor and the memory may be connected through a bus or in other ways.
  • the connection through a bus is taken as an example.
  • the memory can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, such as program instructions corresponding to the three-dimensional face reconstruction method in the embodiment of the present invention.
  • the processor executes various functional applications and data processing of the device 200 by running non-transitory software programs, instructions, and modules stored in the memory, that is, implements the three-dimensional face reconstruction method of any of the foregoing method embodiments.
  • the memory may include a program storage area and a data storage area.
  • the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the device 200 and the like.
  • the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device.
  • the storage may optionally include storage remotely arranged with respect to the processor, and these remote storages may be connected to the apparatus 200 via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
  • the one or more modules are stored in the memory, and when executed by the one or more processors, the three-dimensional face reconstruction method in any of the above-mentioned method embodiments is executed, for example, the above-described three-dimensional face reconstruction method in FIG.
  • the eleventh embodiment of the present invention also provides a computer-readable storage medium that stores computer-executable instructions, and the computer-executable instructions are executed by one or more control processors, for example, Being executed by a processor in FIG. 8 can cause the above one or more processors to execute a three-dimensional face reconstruction method in the above method embodiment, for example, to execute the above described method steps S100 to S500 in FIG.
  • the method steps S210 to S230 in FIG. 2, the method steps S310 to S330 in FIG. 3, the method steps S331 to S333 in FIG. 4, the method steps S410 to S430 in FIG. 5, and the method steps S510 to S550 in FIG. 6, Realize the functions of each unit of the 3D face reconstruction system in Figure 7.
  • the device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • each implementation manner can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware.
  • a person of ordinary skill in the art can understand that all or part of the processes in the method of the foregoing embodiments can be implemented by instructing relevant hardware through a computer program.
  • the program can be stored in a computer readable storage medium. When executed, it may include the processes of the above-mentioned method embodiments.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种三维人脸重建方法及其系统、装置、存储介质，通过卷积神经网络模型直接从原始人脸图像中得到标准姿态下的三维人脸图像，再将所述三维人脸图像缩放后与原始人体模型进行拟合变形，可以有效地减少了对输入的原始人脸图像的限制，降低了三维人脸重建过程的运算量，提高了三维人脸重建的效率，提高了重建得出的三维人脸图像的精确度；并且，通过三维人脸图像和原始人体模型的拟合变形，可以实现最终输出三维人体模型，而不单单是三维脸部图像，从而有效地保证了重建后输出的完整性。

Description

一种三维人脸重建方法及其系统、装置、存储介质 技术领域
本发明涉及人脸识别技术领域，尤其是一种三维人脸重建方法及其系统、装置、存储介质。
背景技术
人脸是人类日常情感表达、交流最重要和最直接的载体，三维人脸建模一直备受关注，在游戏、影视特效、虚拟现实、辅助医疗等领域都有非常广阔的应用前景，是计算机视觉中的一个重要研究课题。目前三维人脸重建的方法大多是获取多张同一人脸不同拍摄角度的图像，通过对齐、拼接、拟合等运算得到三维人脸图像。但这类方法的运算量较大，效率低下，生成的三维人脸图像精度不高。并且得出的效果只是三维脸部图像，并非三维人体图像模型，缺少完整性。
发明内容
为解决上述问题，本发明的目的在于提供一种三维人脸重建方法及其系统、装置、存储介质，能够提高三维人脸重建的效率和精度，并且能够输出三维人体模型。
本发明解决其问题所采用的技术方案是：
第一方面，本发明实施例提出了一种三维人脸重建方法，包括：
获取用于进行三维人脸重建的原始人脸图像；
获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像；
获取所述原始人脸图像的性别属性和对应性别的原始人体模型；将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形；
对所述拟合变形后的原始人体模型进行渲染，得到三维人体模型。
进一步，所述获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像，包括：
采用OpenCV库和Dlib库获得原始人脸图像的人脸特征点、人脸区域及对应人脸的旋转矩阵，并将人脸区域裁剪成特定分辨率的图像；
以裁剪后的图像作为卷积神经网络模型的输入，输出得到对应姿态下的初步三维人脸图像数据，包括三维人脸坐标点集、坐标点对应的RGB值以及三角网格面集；
利用三维人脸坐标点集和旋转矩阵进行矩阵运算，得到标准姿态下的三维人脸图像。
进一步，所述获取所述原始人脸图像的性别属性和对应性别的原始人体模型，包括：
采用高层特征和低层特征结合的神经网络对原始人脸图像进行性别识别；
根据原始人脸图像的性别选择对应的原始人体模型；
获取所述原始人体模型的脸部数据。
进一步，所述获取所述原始人体模型的脸部数据，包括：
对原始人体模型进行关键点检测，确定原始人体模型的脸部区域；
对原始人体模型的脸部区域和其他部位区域的坐标点赋予不同的值，得到原始人体模型的脸部数据；
将原始人体模型的脸部数据写入储存文件。
进一步，所述将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形，包括：
通过三维人脸图像和原始人体模型的脸部区域的部位距离的比值确定缩放系数；
利用所述缩放系数将三维人脸图像缩放至与原始人体模型的脸部区域的大小相近；
构建代价函数求解最优解，使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形。
进一步，所述部位距离的比值包括左眼角到右眼角水平距离的比值、两嘴角水平距离的比值、眉毛到下嘴唇垂直距离的比值、鼻尖到下巴垂直距离的比值中的至少一种。
第二方面，本发明实施例还提出了一种三维人脸重建系统，包括：
数据获取单元，用于获取用于进行三维人脸重建的原始人脸图像，以及获取所述原始人脸图像的性别属性和对应性别的原始人体模型；
人脸重建单元，用于获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像；
拟合变形单元，用于将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形；
渲染单元，用于对所述拟合变形后的原始人体模型进行渲染，得到三维人体模型。
进一步，还包括：
个性化设置单元，用于对三维人体模型的脸部区域进行表情融合变形和脸部形状调整；
显示单元，用于显示三维人体模型，以及显示个性化设置界面。
第三方面，本发明实施例还提出了一种三维人脸重建装置，包括：
至少一个处理器；以及，
与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如本发明第一方面所述的方法。
第四方面，本发明实施例还提出了一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机可执行指令，所述计算机可执行指令用于使计算机执行如本发明第一方面所述的方法。
本发明实施例中提供的一个或多个技术方案，至少具有如下有益效果：本发明实施例提供的一种三维人脸重建方法，通过卷积神经网络模型直接从原始人脸图像中得到标准姿态下的三维人脸图像，再将所述三维人脸图像缩放后与原始人体模型进行拟合变形，可以有效地减少了对输入的原始人脸图像的限制，降低了三维人脸重建过程的运算量，提高了三维人脸重建的效率，提高了重建得出的三维人脸图像的精确度；并且，通过三维人脸图像和原始人体模型的拟合变形，可以实现最终输出三维人体模型，而不单单是三维脸部图像，从而有效地保证了重建后输出的完整性。
本发明实施例提供的一种三维人脸重建系统，人脸重建单元通过卷积神经网络模型直接从原始人脸图像中得到标准姿态下的三维人脸图像，拟合变形单元再将所述三维人脸图像缩放后与原始人体模型进行拟合变形，可以有效地减少了对输入的原始人脸图像的限制，降低了三维人脸重建过程的运算量，提高了三维人脸重建的效率，提高了重建得出的三维人脸图像的精确度；并且，拟合变形单元通过三维人脸图像和原始人体模型的拟合变形，可以实现最终输出三维人体模型，而不单单是三维脸部图像，从而有效地保证了重建后输出的完整性。
附图说明
下面结合附图和实例对本发明作进一步说明。
图1是本发明一种三维人脸重建方法的一个实施例的流程图；
图2是本发明一种三维人脸重建方法的一个实施例中获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像的流程图；
图3是本发明一种三维人脸重建方法的一个实施例中获取所述原始人脸图像的性别属性和对应性别的原始人体模型的流程图；
图4是本发明一种三维人脸重建方法的一个实施例中获取所述原始人体模型的脸部数据的流程图；
图5是本发明一种三维人脸重建方法的一个实施例中将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形的流程图；
图6是本发明一种三维人脸重建方法的另一个实施例的流程图；
图7是本发明一种三维人脸重建系统的示意图；
图8是本发明一种三维人脸重建装置的示意图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本发明，并不用于限定本发明。
需要说明的是，如果不冲突，本发明实施例中的各个特征可以相互结合，均在本发明的保护范围之内。另外，虽然在系统示意图中进行了功能模块划分，在流程图中示出了逻辑顺序，但是在某些情况下，可以 以不同于系统中的单元划分，或流程图中的顺序执行所示出或描述的步骤。
人脸是人类日常情感表达、交流最重要和最直接的载体，三维人脸建模一直备受关注，在游戏、影视特效、虚拟现实、辅助医疗等领域都有非常广阔的应用前景，是计算机视觉中的一个重要研究课题。目前三维人脸重建的方法大多是获取多张同一人脸不同拍摄角度的图像，通过对齐、拼接、拟合等运算得到三维人脸图像。但这类方法的运算量较大，效率低下，生成的三维人脸图像精度不高。并且得出的效果只是三维脸部图像，并非三维人体图像模型，缺少完整性。
基于此，本发明提供了一种三维人脸重建方法及其系统、装置、存储介质，通过卷积神经网络模型直接从原始人脸图像中得到标准姿态下的三维人脸图像，再将所述三维人脸图像缩放后与原始人体模型进行拟合变形，可以有效地减少了对输入的原始人脸图像的限制，降低了三维人脸重建过程的运算量，提高了三维人脸重建的效率，提高了重建得出的三维人脸图像的精确度；并且，通过三维人脸图像和原始人体模型的拟合变形，可以实现最终输出三维人体模型，而不单单是三维脸部图像，从而有效地保证了重建后输出的完整性。下面结合附图，对本发明实施例作进一步阐述。
参照图1，本发明的第一实施例提供了一种三维人脸重建方法，包括但不限于以下步骤：
S100:获取用于进行三维人脸重建的原始人脸图像；
S200:获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像；
S300:获取所述原始人脸图像的性别属性和对应性别的原始人体模型；
S400:将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形；
S500:对所述拟合变形后的原始人体模型进行渲染，得到三维人体模型。
在本实施例中，所述原始人脸图像只需要获取一张即可，获取方式有多种，例如可以从网上下载，或者通过本地读取等。而所述原始人脸图像的拍摄角度没有限定，可以是正面的人脸图像，也可以是侧面的人脸图像。
在本实施例中，通过卷积神经网络模型直接从原始人脸图像中得到标准姿态下的三维人脸图像，再将所述三维人脸图像缩放后与原始人体模型进行拟合变形，可以有效地减少了对输入的原始人脸图像的限制，降低了三维人脸重建过程的运算量，提高了三维人脸重建的效率，提高了重建得出的三维人脸图像的精确度；并且，通过三维人脸图像和原始人体模型的拟合变形，可以实现最终输出三维人体模型，而不单单是三维脸部图像，从而有效地保证了重建后输出的完整性。
进一步地，参照图2，基于第一实施例，本发明的第二实施例还提供了一种三维人脸重建方法，其中，所述步骤S200:获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像，具体包括：
S210:采用OpenCV库和Dlib库获得原始人脸图像的人脸特征点、人脸区域及对应人脸的旋转矩阵，并将人脸区域裁剪成特定分辨率的图像；
S220:以裁剪后的图像作为卷积神经网络模型的输入，输出得到对应姿态下的初步三维人脸图像数据，包括三维人脸坐标点集、坐标点对应的RGB值以及三角网格面集；
S230:利用三维人脸坐标点集和旋转矩阵进行矩阵运算，得到标准姿态下的三维人脸图像。
OpenCV是一个基于BSD许可(开源)发行的跨平台计算机视觉库，可以支持多种操作系统和接口，实现图像处理和计算机视觉方面的很多通用算法；而Dlib是一个包含机器学习算法的C++开源工具包，可以提供大量的机器学习/图像处理算法，采用这两个库进行原始人脸图像的处理，可以有效提高处理的效率。
进一步地，参照图3，基于第一实施例，本发明的第三实施例还提供了一种三维人脸重建方法，其中，所述步骤S300:获取所述原始人脸图像的性别属性和对应性别的原始人体模型，具体包括：
S310:采用高层特征和低层特征结合的神经网络对原始人脸图像进行性别识别；
S320:根据原始人脸图像的性别选择对应的原始人体模型；
S330:获取所述原始人体模型的脸部数据。
在本实施例中，高层特征和低层特征结合的神经网络主要由高层特征提取模块，低层特征提取模块和联合表决输出模块组成。高层特征模块主要由卷积层和下采样层组成，在卷积层中，采用多个不同权值的接收域对原始人脸图像进行识别，从而提取出原始人脸图像的不同局部特征，减少需要训练的参数量。低层特征提取模块主要由扁平化处理单元和重建网络组成，扁平化处理单元首先将原始人脸图像转化为一维的人脸原始特征向量，然后输入到重建网络进行向量的重建。重建网络采用人工神经网络结构，通过多层神经元的处理，对人脸原始特征向量进行重新表示和描述。高层特征提取模块对输入的人脸图像进行卷积和下采样后，最后通过扁平化处理单元得到一个高层特征向量，低层特征提取模块经过重建网络对原始人脸图像进行重新表示，得到低层特征向量，最后，联合表决层将高层特征向量和低层特征向量合并在一起，与输出层的两个神经元(对应男女两种性别)全连接，进行最终的分类判决后完成性别识别。
进一步地，参照图4，基于第三实施例，本发明的第四实施例还提供了一种三维人脸重建方法，其中，所述步骤S330:获取所述原始人体模型的脸部数据，具体包括：
S331:对原始人体模型进行关键点检测，确定原始人体模型的脸部区域；
S332:对原始人体模型的脸部区域和其他部位区域的坐标点赋予不同的值，得到原始人体模型的脸部数据；
S333:将原始人体模型的脸部数据写入储存文件。
在本实施例中，通过原始人体模型的脸部区域和其他部位区域的坐标点赋予不同的值，可以用于分别表示该区域可发生变形和不可发生变形，便于后续对模型的调整。
进一步地，参照图5，基于第一实施例，本发明的第五实施例还提供了一种三维人脸重建方法，其中， 所述步骤S400:将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形，具体包括：
S410:通过三维人脸图像和原始人体模型的脸部区域的部位距离的比值确定缩放系数；
S420:利用所述缩放系数将三维人脸图像缩放至与原始人体模型的脸部区域的大小相近；
S430:构建代价函数求解最优解，使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形。
在本实施例中，通过构建代价函数，得到最优解后再将原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形，可以有效提高拟合的效果，使得后续三维人脸重建的质量得到提升。
进一步地，基于第五实施例，本发明的第六实施例还提供了一种三维人脸重建方法，其中，所述部位距离的比值包括左眼角到右眼角水平距离的比值、两嘴角水平距离的比值、眉毛到下嘴唇垂直距离的比值、鼻尖到下巴垂直距离的比值中的至少一种。
另外，参照图6，本发明的第七实施例还提供了一种三维人脸重建方法，包括但不限于以下步骤：
S510:获取一张用于进行三维人脸重建的原始人脸图像；
S520:采用OpenCV库和Dlib库获得原始人脸图像的人脸特征点、人脸区域及对应人脸的旋转矩阵，并将人脸区域裁剪成特定分辨率的图像，以裁剪后的图像作为卷积神经网络模型的输入，输出得到对应姿态下的初步三维人脸图像数据，包括三维人脸坐标点集、坐标点对应的RGB值以及三角网格面集，利用三维人脸坐标点集和旋转矩阵进行矩阵运算，得到标准姿态下的三维人脸图像；
S530:采用高层特征和低层特征结合的神经网络对原始人脸图像进行性别识别，根据原始人脸图像的性别选择对应的原始人体模型，对原始人体模型进行关键点检测，确定原始人体模型的脸部区域，对原始人体模型的脸部区域和其他部位区域的坐标点赋予不同的值，得到原始人体模型的脸部数据，将原始人体模型的脸部数据写入储存文件；
S540:通过三维人脸图像和原始人体模型的脸部区域的部位距离的比值确定缩放系数，利用所述缩放系数将三维人脸图像缩放至与原始人体模型的脸部区域的大小相近，构建代价函数求解最优解，使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形；
S550:通过特征点映射对所述拟合变形后的原始人体模型进行渲染，得到三维人体模型。
优选地，可以通过构建原始人体模型库，实现自由获取原始人体模型，模型来源可以是本地获取或者网络获取。
本实施例通过卷积神经网络模型直接从原始人脸图像中得到标准姿态下的三维人脸图像，再将所述三维人脸图像缩放后与原始人体模型进行拟合变形，可以有效地减少了对输入的原始人脸图像的限制，降低了三维人脸重建过程的运算量，提高了三维人脸重建的效率，提高了重建得出的三维人脸图像的精确度；并且，通过三维人脸图像和原始人体模型的拟合变形，可以实现最终输出三维人体模型，而不单单是三维脸部图像，从而有效地保证了重建后输出的完整性。
另外，参照图7，本发明的第八实施例还提供了一种三维人脸重建系统，包括：
数据获取单元110，用于获取用于进行三维人脸重建的原始人脸图像，以及获取所述原始人脸图像的性别属性和对应性别的原始人体模型；
人脸重建单元120，用于获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像；
拟合变形单元130，用于将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形；
渲染单元140，用于对所述拟合变形后的原始人体模型进行渲染，得到三维人体模型。
在本实施例中，人脸重建单元通过卷积神经网络模型直接从原始人脸图像中得到标准姿态下的三维人脸图像，拟合变形单元再将所述三维人脸图像缩放后与原始人体模型进行拟合变形，可以有效地减少了对输入的原始人脸图像的限制，降低了三维人脸重建过程的运算量，提高了三维人脸重建的效率，提高了重建得出的三维人脸图像的精确度；并且，拟合变形单元通过三维人脸图像和原始人体模型的拟合变形，可以实现最终输出三维人体模型，而不单单是三维脸部图像，从而有效地保证了重建后输出的完整性。
进一步地，基于第八实施例，本发明的第九实施例还提供了一种三维人脸重建系统，其中，还包括：
个性化设置单元150，用于对三维人体模型的脸部区域进行表情融合变形和脸部形状调整；
显示单元160，用于显示三维人体模型，以及显示个性化设置界面。
在本实施例中，可以利用个性化设置单元通过libigl库对模型脸部区域进行表情融合变形和脸部形状调整，实现表情切换和模型美化，使得最终构建的三维人体模型的质量更高。
参照图8，本发明的第十实施例还提供了一种三维人脸重建装置，包括：
至少一个处理器；
以及与所述至少一个处理器通信连接的存储器；
其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如上述第一至第七实施例中任意一种三维人脸重建方法。
该装置200可以是任意类型的智能终端，例如手机、平板电脑、个人计算机等。
处理器和存储器可以通过总线或者其他方式连接，图8中以通过总线连接为例。
存储器作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序、非暂态性计算机可执行程序以及模块，如本发明实施例中的三维人脸重建方法对应的程序指令/模块。处理器通过运行存储在存储器中的非暂态软件程序、指令以及模块，从而执行装置200的各种功能应用以及数据处理，即实现上述任一方法实施例的三维人脸重建方法。
存储器可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据装置200的使用所创建的数据等。此外，存储器可以包括高速随机存 取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施例中，存储器可选包括相对于处理器远程设置的存储器，这些远程存储器可以通过网络连接至该装置200。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
所述一个或者多个模块存储在所述存储器中，当被所述一个或者多个处理器执行时，执行上述任意方法实施例中的三维人脸重建方法，例如，执行以上描述的图1中的方法步骤S100至S500，图6中的方法步骤S510至S550。
本发明的第十一实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机可执行指令，该计算机可执行指令被一个或多个控制处理器执行，例如，被图8中的一个处理器执行，可使得上述一个或多个处理器执行上述方法实施例中的一种三维人脸重建方法，例如，执行以上描述的图1中的方法步骤S100至S500、图2中的方法步骤S210至S230、图3中的方法步骤S310至S330、图4中的方法步骤S331至S333，图5中的方法步骤S410至S430，图6中的方法步骤S510至S550，实现图7中三维人脸重建系统各个单元的功能。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
通过以上的实施方式的描述，本领域普通技术人员可以清楚地了解到各实施方式可借助软件加通用硬件平台的方式来实现，当然也可以通过硬件。本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory,ROM)或随机存储记忆体(Random Access Memory,RAM)等。
以上是对本发明的较佳实施进行了具体说明，但本发明并不局限于上述实施方式，熟悉本领域的技术人员在不违背本发明精神的前提下还可作出种种的等同变形或替换，这些等同的变形或替换均包含在本申请权利要求所限定的范围内。

Claims (10)

  1. 一种三维人脸重建方法，其特征在于，包括：
    获取用于进行三维人脸重建的原始人脸图像；
    获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像；
    获取所述原始人脸图像的性别属性和对应性别的原始人体模型；将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形；
    对所述拟合变形后的原始人体模型进行渲染，得到三维人体模型。
  2. 根据权利要求1所述的一种三维人脸重建方法，其特征在于，所述获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像，包括：
    采用OpenCV库和Dlib库获得原始人脸图像的人脸特征点、人脸区域及对应人脸的旋转矩阵，并将人脸区域裁剪成特定分辨率的图像；
    以裁剪后的图像作为卷积神经网络模型的输入，输出得到对应姿态下的初步三维人脸图像数据，包括三维人脸坐标点集、坐标点对应的RGB值以及三角网格面集；
    利用三维人脸坐标点集和旋转矩阵进行矩阵运算，得到标准姿态下的三维人脸图像。
  3. 根据权利要求1所述的一种三维人脸重建方法，其特征在于，所述获取所述原始人脸图像的性别属性和对应性别的原始人体模型，包括：
    采用高层特征和低层特征结合的神经网络对原始人脸图像进行性别识别；
    根据原始人脸图像的性别选择对应的原始人体模型；
    获取所述原始人体模型的脸部数据。
  4. 根据权利要求3所述的一种三维人脸重建方法，其特征在于，所述获取所述原始人体模型的脸部数据，包括：
    对原始人体模型进行关键点检测，确定原始人体模型的脸部区域；
    对原始人体模型的脸部区域和其他部位区域的坐标点赋予不同的值，得到原始人体模型的脸部数据；
    将原始人体模型的脸部数据写入储存文件。
  5. 根据权利要求1所述的一种三维人脸重建方法，其特征在于，所述将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形，包括：
    通过三维人脸图像和原始人体模型的脸部区域的部位距离的比值确定缩放系数；
    利用所述缩放系数将三维人脸图像缩放至与原始人体模型的脸部区域的大小相近；
    构建代价函数求解最优解，使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形。
  6. 根据权利要求5所述的一种三维人脸重建方法，其特征在于：所述部位距离的比值包括左眼角到右眼角水平距离的比值、两嘴角水平距离的比值、眉毛到下嘴唇垂直距离的比值、鼻尖到下巴垂直距离的比值中的至少一种。
  7. 一种三维人脸重建系统，其特征在于，包括：
    数据获取单元，用于获取用于进行三维人脸重建的原始人脸图像，以及获取所述原始人脸图像的性别属性和对应性别的原始人体模型；
    人脸重建单元，用于获取所述原始人脸图像的人脸特征点、人脸区域和旋转矩阵，采用卷积神经网络模型定位所述原始人脸图像从二维平面结构到三维空间结构的映射，得到标准姿态下的三维人脸图像；
    拟合变形单元，用于将所述三维人脸图像缩放至与所述原始人体模型脸部大小相近，再使原始人体模型的脸部区域对缩放后的三维人脸图像进行拟合变形；
    渲染单元，用于对所述拟合变形后的原始人体模型进行渲染，得到三维人体模型。
  8. 根据权利要求7所述的一种三维人脸重建系统，其特征在于，还包括：
    个性化设置单元，用于对三维人体模型的脸部区域进行表情融合变形和脸部形状调整；
    显示单元，用于显示三维人体模型，以及显示个性化设置界面。
  9. 一种三维人脸重建装置，其特征在于，包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如权利要求1-6任一项所述的方法。
  10. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机可执行指令，所述计算机可执行指令用于使计算机执行如权利要求1-6任一项所述的方法。
PCT/CN2020/081684 2019-04-09 2020-03-27 一种三维人脸重建方法及其系统、装置、存储介质 WO2020207270A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910278736.8 2019-04-09
CN201910278736.8A CN110136243B (zh) 2019-04-09 2019-04-09 一种三维人脸重建方法及其系统、装置、存储介质

Publications (1)

Publication Number Publication Date
WO2020207270A1 true WO2020207270A1 (zh) 2020-10-15

Family

ID=67569250

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/081684 WO2020207270A1 (zh) 2019-04-09 2020-03-27 一种三维人脸重建方法及其系统、装置、存储介质

Country Status (2)

Country Link
CN (1) CN110136243B (zh)
WO (1) WO2020207270A1 (zh)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112508778A (zh) * 2020-12-18 2021-03-16 咪咕文化科技有限公司 一种3d人脸道具贴图方法、终端及存储介质
CN112561784A (zh) * 2020-12-17 2021-03-26 咪咕文化科技有限公司 图像合成方法、装置、电子设备及存储介质
CN112633252A (zh) * 2021-01-09 2021-04-09 浙江臻享网络科技有限公司 一种证件照标准化处理方法及装置
CN112966574A (zh) * 2021-02-22 2021-06-15 厦门艾地运动科技有限公司 人体三维关键点预测方法、装置及电子设备
CN113012282A (zh) * 2021-03-31 2021-06-22 深圳市慧鲤科技有限公司 三维人体重建方法、装置、设备及存储介质
CN113221840A (zh) * 2021-06-02 2021-08-06 广东工业大学 一种人像视频处理方法
CN113632098A (zh) * 2021-07-02 2021-11-09 华为技术有限公司 人脸图像处理方法、装置和车辆
CN114241102A (zh) * 2021-11-11 2022-03-25 清华大学 基于参数化模型的人脸细节重建和编辑方法及装置
CN114723888A (zh) * 2022-04-08 2022-07-08 北京百度网讯科技有限公司 三维发丝模型生成方法、装置、设备、存储介质及产品
CN115661370A (zh) * 2022-12-14 2023-01-31 长沙观谱红外科技有限公司 一种红外3d人体模型构建方法及存储介质
CN116051364A (zh) * 2022-12-14 2023-05-02 长沙观谱红外科技有限公司 一种红外图像处理方法及存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110136243B (zh) * 2019-04-09 2023-03-17 五邑大学 一种三维人脸重建方法及其系统、装置、存储介质
CN110827342B (zh) 2019-10-21 2023-06-02 中国科学院自动化研究所 三维人体模型重建方法及存储设备、控制设备
CN114758090A (zh) * 2020-12-29 2022-07-15 华为技术有限公司 三维模型生成方法及装置
CN113284229B (zh) * 2021-05-28 2023-04-18 上海星阑信息科技有限公司 三维人脸模型生成方法、装置、设备及存储介质
CN117372604B (zh) * 2023-12-06 2024-03-08 国网电商科技有限公司 一种3d人脸模型生成方法、装置、设备及可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106652025A (zh) * 2016-12-20 2017-05-10 五邑大学 一种基于视频流与人脸多属性匹配的三维人脸建模方法和打印装置
US9734628B2 (en) * 2012-05-17 2017-08-15 Disney Enterprises, Inc. Techniques for processing reconstructed three-dimensional image data
CN107680167A (zh) * 2017-09-08 2018-02-09 郭睿 一种基于用户形象的三维人体模型创建方法及系统
CN108805977A (zh) * 2018-06-06 2018-11-13 浙江大学 一种基于端到端卷积神经网络的人脸三维重建方法
CN110136243A (zh) * 2019-04-09 2019-08-16 五邑大学 一种三维人脸重建方法及其系统、装置、存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106372333A (zh) * 2016-08-31 2017-02-01 北京维盛视通科技有限公司 基于人脸模型的服装展示方法及装置
CN107122705B (zh) * 2017-03-17 2020-05-19 中国科学院自动化研究所 基于三维人脸模型的人脸关键点检测方法
CN107194964B (zh) * 2017-05-24 2020-10-09 电子科技大学 一种基于实时人体三维重建的vr社交系统及其方法
CN107423678A (zh) * 2017-05-27 2017-12-01 电子科技大学 一种提取特征的卷积神经网络的训练方法及人脸识别方法
CN107578469A (zh) * 2017-09-08 2018-01-12 明利 一种基于单张照片的3d人体建模方法及装置
CN109255830B (zh) * 2018-08-31 2020-06-05 百度在线网络技术（北京）有限公司 三维人脸重建方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9734628B2 (en) * 2012-05-17 2017-08-15 Disney Enterprises, Inc. Techniques for processing reconstructed three-dimensional image data
CN106652025A (zh) * 2016-12-20 2017-05-10 五邑大学 一种基于视频流与人脸多属性匹配的三维人脸建模方法和打印装置
CN107680167A (zh) * 2017-09-08 2018-02-09 郭睿 一种基于用户形象的三维人体模型创建方法及系统
CN108805977A (zh) * 2018-06-06 2018-11-13 浙江大学 一种基于端到端卷积神经网络的人脸三维重建方法
CN110136243A (zh) * 2019-04-09 2019-08-16 五邑大学 一种三维人脸重建方法及其系统、装置、存储介质

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112561784B (zh) * 2020-12-17 2024-04-09 咪咕文化科技有限公司 图像合成方法、装置、电子设备及存储介质
CN112561784A (zh) * 2020-12-17 2021-03-26 咪咕文化科技有限公司 图像合成方法、装置、电子设备及存储介质
CN112508778A (zh) * 2020-12-18 2021-03-16 咪咕文化科技有限公司 一种3d人脸道具贴图方法、终端及存储介质
CN112508778B (zh) * 2020-12-18 2024-04-12 咪咕文化科技有限公司 一种3d人脸道具贴图方法、终端及存储介质
CN112633252A (zh) * 2021-01-09 2021-04-09 浙江臻享网络科技有限公司 一种证件照标准化处理方法及装置
CN112966574A (zh) * 2021-02-22 2021-06-15 厦门艾地运动科技有限公司 人体三维关键点预测方法、装置及电子设备
CN113012282A (zh) * 2021-03-31 2021-06-22 深圳市慧鲤科技有限公司 三维人体重建方法、装置、设备及存储介质
CN113221840A (zh) * 2021-06-02 2021-08-06 广东工业大学 一种人像视频处理方法
CN113221840B (zh) * 2021-06-02 2022-07-26 广东工业大学 一种人像视频处理方法
CN113632098A (zh) * 2021-07-02 2021-11-09 华为技术有限公司 人脸图像处理方法、装置和车辆
CN114241102A (zh) * 2021-11-11 2022-03-25 清华大学 基于参数化模型的人脸细节重建和编辑方法及装置
CN114241102B (zh) * 2021-11-11 2024-04-19 清华大学 基于参数化模型的人脸细节重建和编辑方法
CN114723888A (zh) * 2022-04-08 2022-07-08 北京百度网讯科技有限公司 三维发丝模型生成方法、装置、设备、存储介质及产品
CN116051364B (zh) * 2022-12-14 2024-03-29 长沙观谱红外科技有限公司 一种红外图像处理方法及存储介质
CN116051364A (zh) * 2022-12-14 2023-05-02 长沙观谱红外科技有限公司 一种红外图像处理方法及存储介质
CN115661370B (zh) * 2022-12-14 2023-04-28 长沙观谱红外科技有限公司 一种红外3d人体模型构建方法及存储介质
CN115661370A (zh) * 2022-12-14 2023-01-31 长沙观谱红外科技有限公司 一种红外3d人体模型构建方法及存储介质

Also Published As

Publication number Publication date
CN110136243B (zh) 2023-03-17
CN110136243A (zh) 2019-08-16

Similar Documents

Publication Publication Date Title
WO2020207270A1 (zh) 一种三维人脸重建方法及其系统、装置、存储介质
US11302064B2 (en) Method and apparatus for reconstructing three-dimensional model of human body, and storage medium
WO2021093453A1 (zh) 三维表情基的生成方法、语音互动方法、装置及介质
CN110807836B (zh) 三维人脸模型的生成方法、装置、设备及介质
EP3992919B1 (en) Three-dimensional facial model generation method and apparatus, device, and medium
US10891789B2 (en) Method to produce 3D model from one or several images
US20180253895A1 (en) System and method for creating a full head 3d morphable model
CN111710036B (zh) 三维人脸模型的构建方法、装置、设备及存储介质
WO2022057526A1 (zh) 三维模型重建方法、三维重建模型的训练方法和装置
CN112819947A (zh) 三维人脸的重建方法、装置、电子设备以及存储介质
CN109801380A (zh) 一种虚拟试衣的方法、装置、存储介质及计算机设备
CN111652123B (zh) 图像处理和图像合成方法、装置和存储介质
WO2024007478A1 (zh) 基于单手机的人体三维建模数据采集与重建方法及系统
JP2023545190A (ja) 画像の視線補正方法、装置、電子機器、及びコンピュータプログラム
WO2022135518A1 (zh) 基于三维卡通模型的眼球配准方法、装置、服务器和介质
WO2023066120A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN109961496A (zh) 表情驱动方法及表情驱动装置
US11631154B2 (en) Method, apparatus, device and storage medium for transforming hairstyle
CN116881886A (zh) 身份识别方法、装置、计算机设备和存储介质
CN116977539A (zh) 图像处理方法、装置、计算机设备、存储介质和程序产品
CN113610864B (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN112150387B (zh) 对照片中的人像增强五官立体感的方法及装置
CN113223103A (zh) 素描画生成的方法、装置、电子设备及介质
CN117557699B (zh) 动画数据生成方法、装置、计算机设备和存储介质
US20240013500A1 (en) Method and apparatus for generating expression model, device, and medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20787921

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20787921

Country of ref document: EP

Kind code of ref document: A1