WO2022012192A1 - 三维人脸模型的构建方法、装置、设备及存储介质 - Google Patents

三维人脸模型的构建方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2022012192A1
WO2022012192A1 PCT/CN2021/097567 CN2021097567W WO2022012192A1 WO 2022012192 A1 WO2022012192 A1 WO 2022012192A1 CN 2021097567 W CN2021097567 W CN 2021097567W WO 2022012192 A1 WO2022012192 A1 WO 2022012192A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
dimensional
point cloud
area
target object
Prior art date
Application number
PCT/CN2021/097567
Other languages
English (en)
French (fr)
Inventor
林祥凯
王盛
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to JP2022549363A priority Critical patent/JP7456670B2/ja
Priority to EP21842458.8A priority patent/EP4057234A4/en
Publication of WO2022012192A1 publication Critical patent/WO2022012192A1/zh
Priority to US17/751,496 priority patent/US20220284679A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G06T17/205—Re-meshing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • G06T5/70—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/64—Analysis of geometric attributes of convexity or concavity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20216—Image averaging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/56—Particle system, point based geometry or rendering

Definitions

  • the present application relates to the field of computer vision technology of artificial intelligence, and in particular, to the construction technology of a three-dimensional face model.
  • 3D (3 Dimensions, three-dimensional) face reconstruction refers to reconstructing a 3D model (mesh) of a human face from one or more 2D (2 Dimensions, two-dimensional) face images.
  • three-dimensional face reconstruction is usually carried out in the following manner: obtaining one or more face images of the target object, obtaining points on various parts of the face from the face images, The points form three-dimensional point cloud data, and further perform Poisson reconstruction on the three-dimensional point cloud data to obtain a three-dimensional face model of the target object.
  • the embodiments of the present application provide a method, apparatus, device, and storage medium for constructing a three-dimensional face model, which improve the efficiency of three-dimensional face reconstruction.
  • the technical solution is as follows:
  • an embodiment of the present application provides a method for constructing a three-dimensional face model, the method comprising:
  • triangulating the pixels on the face texture image to obtain triangulation information, where the triangulation information is used to indicate the triangle topology corresponding to the pixels;
  • non-critical area refers to a non-face area in the initial three-dimensional face model
  • the non-critical areas in the initial three-dimensional face model are replaced with the smoothed non-critical areas to obtain the three-dimensional face model of the target object.
  • an embodiment of the present application provides an apparatus for constructing a three-dimensional face model, the apparatus comprising:
  • the point cloud acquisition module is used to acquire the face point cloud of the target object
  • an index obtaining module configured to determine, through expansion calculation, the corresponding pixel coordinates of the three-dimensional data points in the face point cloud on the face texture image of the target object, and obtain index information of the three-dimensional data points;
  • a triangulation module configured to triangulate the pixels on the face texture image to obtain triangulation information, where the triangulation information is used to indicate the triangle topology corresponding to the pixels;
  • Model initial building module for obtaining initial three-dimensional face model according to described triangulation information, described index information and described face point cloud;
  • an area determination module configured to determine a non-critical area in the initial three-dimensional face model, where the non-critical area refers to a non-face area in the initial three-dimensional face model;
  • an area smoothing module used for smoothing the non-critical areas in the initial three-dimensional face model to obtain smoothed non-critical areas
  • a region replacement module configured to replace the non-critical regions in the initial three-dimensional face model with the smoothed non-critical regions to obtain the three-dimensional face model of the target object.
  • an embodiment of the present application provides a computer device, the computer device includes a processor and a memory, and the memory stores at least one instruction, at least a piece of program, code set or instruction set, the at least one instruction, The at least one piece of program, the code set or the instruction set is loaded and executed by the processor to implement the method as described in the above aspects.
  • an embodiment of the present application provides a computer-readable storage medium, where at least one instruction, at least one program, code set or instruction set is stored in the storage medium, the at least one instruction, the at least one program, The set of codes or instructions is loaded and executed by a processor to implement the method as described in the above aspects.
  • an embodiment of the present application provides a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • a processor of the computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform the methods provided in the above aspects.
  • the triangulation information is obtained by triangulating the pixels on the face texture image determined by the expansion calculation, and then a three-dimensional face model is obtained based on the triangulation information.
  • the embodiment of the present application constructs a three-dimensional human face based on the triangulation. Face model, fast and efficient.
  • the non-critical regions in the initial three-dimensional face model are smoothed, so that the smoothed non-critical regions are used to replace the original non-critical regions to obtain a three-dimensional face model.
  • the effect of 3D face model is better.
  • Fig. 1 is the frame diagram of the reconstruction process of the three-dimensional face model provided by the application
  • FIG. 2 is a flowchart of a method for constructing a three-dimensional face model provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a face point cloud provided by an embodiment of the present application under different viewing angles;
  • FIG. 4 is a schematic diagram of a triangle topology provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an initial three-dimensional face model provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a three-dimensional face model provided by an embodiment of the present application.
  • FIG. 7 is a flowchart of a method for constructing a three-dimensional face model provided by another embodiment of the present application.
  • FIG. 8 is a schematic diagram of a face point cloud under three-dimensional coordinates provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a pixel on a face texture image provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a target area provided by an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a triangle topology provided by another embodiment of the present application.
  • FIG. 12 is a schematic diagram of a non-critical area provided by an embodiment of the present application.
  • FIG. 13 is a schematic diagram of a three-dimensional face model provided by another embodiment of the present application.
  • FIG. 14 is a block diagram of a construction process of a three-dimensional face model provided by an embodiment of the present application.
  • FIG. 15 is a schematic diagram of a face image provided by an embodiment of the present application.
  • 16 is a schematic diagram of a region texture image provided by an embodiment of the present application.
  • FIG. 17 is a schematic diagram of a face texture image provided by an embodiment of the present application.
  • FIG. 18 is a schematic diagram of a textured three-dimensional face model provided by an embodiment of the present application.
  • FIG. 19 is a block diagram of an apparatus for constructing a three-dimensional face model provided by an embodiment of the present application.
  • FIG. 20 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • the solutions provided by the embodiments of the present application relate to the technical field of 3D face reconstruction.
  • Computer vision technology is used to analyze and process a face image of a target object to construct a 3D face model of the target object, and the face texture of the target object can be generated.
  • image, and the above-mentioned three-dimensional face model is rendered through the face texture image to generate a textured face model.
  • FIG. 1 it shows a frame diagram of the reconstruction process of the three-dimensional face model provided by the present application.
  • the input data includes the face point cloud 11 of the target object
  • the output data includes the three-dimensional face model 12 of the target object.
  • the corresponding pixel coordinates of the three-dimensional data points in the face point cloud 11 on the face texture image 13 of the target object are determined through expansion calculation, and the three-dimensional data point is obtained.
  • triangulate the pixels on the face texture image 13 to obtain triangulation information, and the triangulation information is used to indicate the triangle topology 14 corresponding to the pixel.
  • an initial three-dimensional face model 15 is generated. After the initial three-dimensional face model 15 is obtained, it is necessary to smooth the non-critical areas in the initial three-dimensional face model 15 to obtain the smoothed non-critical areas; finally, the non-critical areas in the initial three-dimensional face model 15 are Substitute the above-mentioned smoothed non-critical area to obtain a three-dimensional face model 12 of the target object.
  • the execution subject of each step may be a terminal such as a mobile phone, a tablet computer, a multimedia playback device, a wearable device, a PC (Personal Computer, personal computer), or a server.
  • a terminal such as a mobile phone, a tablet computer, a multimedia playback device, a wearable device, a PC (Personal Computer, personal computer), or a server.
  • a computer device is used as an example for the execution of each step.
  • the computer device may be any electronic device with computing and storage capabilities, such as the terminal or server described above.
  • FIG. 2 shows a flowchart of a method for constructing a three-dimensional face model provided by an embodiment of the present application.
  • the method may include the following steps.
  • Step 201 acquiring a face point cloud of a target object.
  • the target object refers to any user who has the requirement of constructing a three-dimensional face model.
  • the target object can be any person, and the target object can be a real person, a robot or other three-dimensional characters occupying a certain spatial volume. Not limited.
  • the face point cloud refers to the point cloud corresponding to the face, which is the three-dimensional data that uses multiple three-dimensional data points to describe the face of the target object.
  • a face point cloud includes multiple 3D data points.
  • a 3D data point refers to a point in a 3D space. Each 3D data point can be represented by a (x, y, z) 3D coordinate in the 3D space.
  • the computer device collects at least one image pair of the target object, each image pair includes a color image and a depth image; and then extracts a face point cloud of the target object according to the color image and the depth image.
  • the x-coordinate and y-coordinate of each three-dimensional data point in the face point cloud can be determined by the color image, and the z-coordinate of the three-dimensional data point can be determined by the depth image.
  • Figure 3 shows the schematic diagram of the face point cloud under different viewing angles.
  • Part (a) of Figure 3 shows the schematic diagram of the face point cloud in the frontal pose
  • Part and part (c) of Figure 3 show a schematic diagram of the face point cloud in the right side face pose
  • part (d) in Figure 3 shows a schematic diagram of the face point cloud in the right side face looking up attitude
  • Part (e) in 3 shows a schematic diagram of the face point cloud in the head-up pose.
  • Step 202 Determine the pixel coordinates corresponding to the three-dimensional data points in the face point cloud on the face texture image of the target object through expansion calculation, and obtain index information of the three-dimensional data points.
  • the index information may also be called a uv index, and the uv index of each 3D data point in the face point cloud refers to the pixel coordinates (u, v) corresponding to the 3D data point on the face texture image. ).
  • Step 203 Triangulate the pixels on the face texture image to obtain triangulation information.
  • the triangulation information is used to indicate the triangle topology corresponding to the pixel point.
  • three adjacent pixels can be connected to form a triangular topology (or called a triangular patch).
  • triangulation is to give a set of pixel points, and connect each pixel point with triangles to form a triangular mesh, which is essentially a topology between a pixel point and its adjacent pixels. Relationships are reflected in the form of a triangular mesh.
  • Delaunay triangulation is performed on the pixels on the face texture image to obtain triangulation information.
  • the computer device triangulates the face texture image 41 on the left to obtain a triangle topology 42 on the right, and triangulation information can be determined according to the triangle topology 42 .
  • Step 204 Obtain an initial three-dimensional face model according to the triangulation information, the index information and the face point cloud.
  • the triangle topology corresponding to each pixel point can be determined, and according to the index information, the three-dimensional data points in the face point cloud corresponding to the pixel points on the face texture image can be determined, so the face point can be obtained.
  • the triangle topology corresponding to each 3D data point in the cloud based on the triangle topology corresponding to each 3D data point, the initial 3D face model shown in Figure 5 can be obtained, and part (a) in Figure 5 shows the frontal face The initial 3D face model in the pose, part (b) in Figure 5 shows the initial 3D face model in the right side face pose, and part (c) in Figure 5 shows the initial 3D face model in the head-up pose .
  • the effect of the chin area in the initial 3D face model is poor due to the overlapping of the chin area. Therefore, it is necessary to adjust and optimize the chin area in the initial 3D face model. It should be noted that the chin area belongs to non- critical area.
  • Step 205 Determine non-critical regions in the initial three-dimensional face model.
  • the non-critical area refers to the non-face area in the initial 3D face model, that is, the area covering the non-face key 3D data points.
  • the non-face key three-dimensional data points refer to the three-dimensional data points corresponding to the non-face key points.
  • the face key points may include key points on key parts of the face such as eyebrows, eyes, lips, etc.
  • the non-critical area can be adjusted and optimized.
  • Step 206 smoothing the non-critical regions in the initial three-dimensional face model to obtain the smoothed non-critical regions.
  • the non-critical regions in the initial three-dimensional face model are sent to the Laplace grid smoothing algorithm for smoothing, and the smoothed non-critical regions are obtained.
  • Step 207 Replace the non-critical regions in the initial three-dimensional face model with the smoothed non-critical regions to obtain a three-dimensional face model of the target object.
  • the three-dimensional face model of the target object refers to the three-dimensional model of the face of the target object reconstructed based on the face point cloud.
  • a 3D face model can include point clouds and triangle topology.
  • FIG 6 it shows a schematic diagram of the three-dimensional face model of the target object generated after the face point cloud is reconstructed under different viewing angles.
  • part (a) is the three-dimensional face model in The schematic diagram under the frontal face angle
  • the part (b) in Figure 6 is the schematic diagrams of the three-dimensional face model in the left side face pose
  • the part (e) in Figure 6 is a schematic diagram of the 3D face model under the head-up angle.
  • triangulation is performed on the pixels on the face texture image determined by the expansion calculation to obtain triangulation information, and then three-dimensional triangulation information is obtained based on the triangulation information.
  • the embodiment of the present application constructs a three-dimensional face model based on triangulation, which is fast and efficient.
  • the non-critical regions in the initial 3D face model are smoothed, so that the non-critical regions in the initial 3D face model are replaced by the smoothed non-critical regions to obtain the target object
  • the 3D face model is better.
  • FIG. 7 shows a flowchart of a method for constructing a three-dimensional face model provided by another embodiment of the present application.
  • the method may include the following steps.
  • Step 701 Obtain a face point cloud of the target object.
  • step 701 For the introduction and description of step 701, reference may be made to the above embodiments, and details are not repeated here.
  • Step 702 Perform grid downsampling on the face point cloud to obtain a downsampled face point cloud.
  • the grid densities corresponding to the neck region and the non-neck region in the down-sampled face point cloud are different.
  • the neck area is a non-critical area, so a substantial downsampling can be done on the neck area, for example, a substantial downsampling can be achieved by reducing the grid density corresponding to the neck area.
  • step 702 includes the following sub-steps:
  • Step 702a in the three-dimensional space, divide multiple grids corresponding to the neck area according to the first grid density, and divide multiple grids corresponding to the non-neck area according to the second grid density.
  • the density of the first grid is lower than the density of the second grid, so as to achieve a relatively large downsampling of the neck region.
  • Step 702b fuse the face point cloud into multiple grids in the three-dimensional space.
  • Step 702c averaging the three-dimensional coordinates of the three-dimensional data points in the face point cloud included in each grid under the same coordinate dimension to obtain a down-sampled face point cloud.
  • one grid includes three 3D data points: 3D data point 1, 3D data point 2, and 3D data point 3.
  • the 3D coordinates of 3D data point 1 are (x1, y1, z1)
  • the 3D coordinates of 3D data point 2 are (x2, y2, z2)
  • the 3D coordinates of 3D data point 3 are (x3, y3, z3)
  • the grid includes a 3D data point whose 3D coordinates are ((x1+x2+x3)/3, (y1+y2+y3)/3, (z1+z2+z3)/3 ).
  • the down-sampling processing method of each grid in the three-dimensional space is similar to this, and details are not described here.
  • Downsampling the face point cloud can make the face point cloud more evenly distributed.
  • the grid down-sampling is performed on the face point cloud first, and then the subsequent processing flow is performed based on the down-sampled face point cloud, which can reduce the processing overhead of computer equipment.
  • Step 703 Determine the pixel coordinates corresponding to the three-dimensional data points in the down-sampled face point cloud on the face texture image of the target object through expansion calculation, and obtain index information of the three-dimensional data points.
  • step 703 includes the following sub-steps.
  • the face point cloud in the following steps may be the face point cloud after downsampling, or may be the face point cloud without downsampling;
  • the face point cloud in the subsequent steps should be the down-sampled face point cloud.
  • step 702 is not performed and the down-sampling processing is not performed on the face point cloud, then in the subsequent steps The face point cloud should be the face point cloud without downsampling.
  • Step 703a obtaining the three-dimensional coordinates of the three-dimensional data points in the face point cloud in the three-dimensional coordinate system.
  • the three-dimensional coordinates of the three-dimensional data points in the face point cloud in the three-dimensional coordinate system can be obtained, and the three-dimensional coordinates can be expressed as (x, y, z).
  • a three-dimensional coordinate system is constructed first.
  • the three-dimensional coordinate system may include x, y, and z axes, which are perpendicular to each other.
  • the x-axis is the coordinate axis in the horizontal direction, such as it can be the direction from the left eye to the right eye;
  • the y-axis is the coordinate axis in the vertical direction, such as it can be the direction from the center of the eyebrow to the mouth;
  • the z-axis is The coordinate axis in the depth direction, such as it can be the direction from the back of the head to the tip of the nose.
  • part (a) of Figure 8 shows a schematic diagram of the face point cloud observed when viewed from left to right. From part (a) of Figure 8, it can be clearly seen that the z-axis direction and The y-axis direction; part (b) of FIG. 8 shows a schematic diagram of the face point cloud viewed from the top of the head downwards, from which the x-axis and z-axis directions can be clearly seen.
  • Step 703b according to the three-dimensional coordinates of the three-dimensional data points, determine the pixel point coordinates corresponding to the three-dimensional data points on the face texture image through cylindrical expansion calculation, and obtain index information.
  • the face point cloud can be regarded as a cylinder, and the direction from the top surface to the bottom surface of the cylinder is the direction looking down from the top of the head.
  • the pixel coordinates (u, v) corresponding to each three-dimensional data point (x, y, z) on the face texture image can be calculated through cylindrical expansion.
  • the calculation formula can be as follows:
  • v represents the unfolded ordinate
  • u represents the unfolded abscissa
  • ymax and ymin represent the maximum and minimum values of the three-dimensional data points included in the face point cloud on the y-axis, respectively
  • the atan() function represents the arctangent function.
  • u-sub-quadrant processing is performed to correspond to the position of the face point cloud.
  • the face point cloud can be expanded into a plane to obtain a two-dimensional uv index, as shown in FIG. 9 , which shows the pixel points on the face texture image 91 provided by an embodiment of the present application schematic diagram.
  • the pixel coordinates can also be determined by the spherical expansion calculation.
  • v atan(z/y)
  • u atan(z/x)
  • the pixel coordinates can also be determined by the stereo expansion calculation.
  • u (x-xmin)/(xmax-xmin)
  • v (y-ymin)/(ymax-ymin); this application does not calculate the expansion here
  • the calculation method used is not limited in any way.
  • Step 704 Triangulate the pixels on the face texture image to obtain triangulation information.
  • the triangulation information is used to indicate the triangle topology corresponding to the pixel point.
  • Step 705 Obtain an initial three-dimensional face model according to the triangulation information, the index information and the face point cloud.
  • Step 706 Determine non-face key pixels in the face texture image.
  • step 706 includes the following sub-steps:
  • Step 706a Determine the convex hull corresponding to the key region in the face texture image.
  • the key area refers to the area covered by the key pixels of the face in the face texture image.
  • the face key pixels refer to the pixels corresponding to the face key points, and the convex hull refers to the area that just encloses the above-mentioned face key pixels.
  • Step 706b Enlarge the convex hull based on the dilation threshold to obtain the target area.
  • the dilation threshold is set based on the resolution.
  • FIG. 10 which shows a schematic diagram of the target area provided by an embodiment of the present application
  • the convex hull 101 is enlarged based on the expansion threshold to obtain the target area 102 .
  • Step 706c Determine pixels in areas other than the target area in the face texture image as non-face key pixels.
  • Pixels included in areas other than the target area can be considered as neck points farther from the face, and neck points farther from the face are non-face key pixels.
  • the pixels included in the area 103 in FIG. 10 can be regarded as non-face key pixels.
  • the target area is obtained, and the non-face key pixels are determined based on the target area, which can make the subsequent smoothing process more accurate, and the smoothed non-key area and the initial 3D face model are more fused. nature.
  • Step 707 Determine the triangle topology corresponding to the non-face key 3D data points and the non-face key 3D data points based on the non-face key pixel points, triangulation information and index information.
  • the triangle topology corresponding to the non-face key pixels can be determined. As shown in FIG. 11 , the area 110 with a diagonal line is the triangle topology corresponding to the non-face key pixels. .
  • the index information reflects the correspondence between the pixels on the face texture image and the 3D data points in the face point cloud, so the non-face key 3D data points can be determined based on the non-face key pixels and index information. Based on the non-face key 3D data points and triangulation information, the triangle topology corresponding to the non-face key 3D data points can be determined.
  • Step 708 Determine the area formed by the triangle topology corresponding to the non-face key three-dimensional data points as the non-key area.
  • the dark area 120 in FIG. 12 represents a non-critical area, part (a) in Schematic representation of non-critical regions in profile pose.
  • Step 709 smoothing the non-critical regions in the initial three-dimensional face model to obtain smoothed non-critical regions.
  • Step 710 Replace the non-critical areas in the initial three-dimensional face model with the smoothed non-critical areas to obtain a three-dimensional face model of the target object.
  • FIG. 14 it shows a flowchart of the construction of a three-dimensional face model provided by an embodiment of the present application.
  • the flow chart of the construction of the 3D face model includes a grid downsampling module 1401 , a column expansion module 1402 , a triangulation module 1403 , and a subregion smoothing module 1404 .
  • the grid downsampling module 1401 is used to downsample the face point cloud to obtain the downsampled face point cloud.
  • the columnar expansion module 1402 is configured to perform columnar expansion calculation on the down-sampled face point cloud to determine index information.
  • the triangulation module 1403 is used to triangulate the pixels on the face texture image to obtain triangulation information, so as to obtain an initial three-dimensional face model based on the triangulation information, index information and face point cloud.
  • the sub-region smoothing module 1404 is used for smoothing the non-critical regions in the initial three-dimensional face model to obtain a three-dimensional face model of the target object.
  • the above-mentioned construction flowchart of the three-dimensional face model further includes a texture extraction module 1405 .
  • the texture extraction block 1405 is used to generate the face texture image of the target object.
  • For the specific process of generating the face texture image of the target object reference may be made to the following embodiments, which will not be repeated here.
  • the distribution of the face point cloud can be made more uniform, and a certain smoothing effect can be achieved.
  • the down-sampling of the face point cloud is performed first, and the processing overhead of the computer equipment can be reduced when the computer equipment subsequently processes the down-sampled face point cloud.
  • the embodiment of the present application generates a face texture image of the target object in the following manner
  • n face images corresponding to n head poses of the target object where n is a positive integer.
  • the face image of the target object refers to an RGB image (or called a color image) containing the face of the target object, and the RGB image may be obtained by photographing the target object with a camera.
  • Each pixel value of an RGB image is represented by color components of three channels, R (Red, red), G (Green, green), and B (Blue, blue).
  • R Red, red
  • G Green, green
  • B Blue, blue
  • a depth image corresponding to each RGB image can also be obtained, and the depth image refers to the distance value from the camera to each point in the scene where the target object is located as the pixel value.
  • Depth images are also known as range images. The depth image reflects the geometry of the visible surface of the object.
  • a depth image is similar to a grayscale image, except that each pixel value of the depth image is the actual distance from the camera to the object.
  • the RGB image and the depth image of the target object can be registered, and there is a one-to-one correspondence between the pixels of the two.
  • the RGB image and depth image of the target object can be two images captured by a 3D camera.
  • 3D cameras can capture depth images in addition to RGB images.
  • the 3D camera may include a color camera and a depth sensor, the color camera is used for capturing RGB images, and the depth sensor is used for collecting depth information to generate a depth image.
  • the above n face images may be face images of the target object under n head poses.
  • n 1, for example, a face image of the target object in a frontal pose is obtained.
  • n head poses include a front face pose, a right side face pose, a left side face pose, and a head-up pose.
  • the computer equipment obtains the face images of the target object under which head poses, which can be set in advance.
  • the face images of the target object under four head poses are acquired: the right side face pose, the front face pose, the left side face pose, and the head-up pose.
  • the target object turns its head, and the camera collects multiple face images of the target object.
  • the image of the whole process of the target object turning its head will be captured and saved by the camera.
  • it is not necessary to calculate all the collected images because many data are repeated, and hundreds of face images will be collected during the whole process of the target object turning its head. It would be a waste of computing resources to perform computations on all of them. Therefore, it is sufficient to select several representative face images from these hundreds of face images for subsequent calculation.
  • a plurality of candidate face images of the target object are obtained; a target image with qualified quality is selected from the plurality of candidate face images; a person in n head poses is selected from the target image face image.
  • the face images with unqualified quality such as blinking, motion blur, etc.
  • the target images with qualified quality will be retained, and then n face images will be selected according to the head pose in these qualified target images.
  • mapping relationship between the three-dimensional data points and the pixels on the face image is acquired.
  • mapping relationship between the three-dimensional data points in the face point cloud and the pixel points on each face image that is, to obtain n sets of mapping relationships.
  • the mapping relationship is obtained in the following manner: for the ith face image in the n face images, the three-dimensional data point is projected on the ith face image by perspective projection, and the three-dimensional data point is obtained in the ith face image.
  • the pixel coordinates corresponding to the above are used as the ith group of mapping relationships; wherein, the above-mentioned mapping relationships include n groups, and the ith group of mapping relationships in the n group of mapping relationships corresponds to the ith face image, and i is a positive value less than or equal to n. Integer.
  • the face point cloud can be rotated and translated according to the head pose, and then projected onto the face image corresponding to the head pose according to the camera parameters. .
  • each set of mapping relationships combined with index information can generate a face texture image of the target object. Perspective fusion, and finally generate a fused face texture image.
  • this step includes the following sub-steps:
  • Face key points include key points on key parts of the face such as eyebrows, eyes, lips, and face contours.
  • the effective area in the ith face image refers to the area determined according to the head pose corresponding to the ith face image and used for fusion to generate the final face texture image.
  • the size of the circle is determined by all the faces Among the key points, it is determined by the key point of the face with the largest distance from the tip of the nose, that is, the circle is the smallest circle with the tip of the nose as the center including all the detected key points of the face. Then, from the circular area, an area containing the completely visible face parts under the current head pose is selected as an effective area.
  • the area contained in the circle can be directly determined as an effective area; if the ith face image is a face image of the left side face posture, Then generate a line that passes through the three key points of the left corner of the left eye, the left corner of the mouth, and the center of the chin, and the area containing the left side face formed by the intersection of the line and the above circle is used as the effective area; if the ith face The image is a face image of the right side face pose, then generate a line that passes through the three key points of the right corner of the right eye, the corner of the right mouth, and the center of the chin.
  • the i-th face image is a face image in a head-up posture, then the left corner of the mouth, the right corner of the mouth, the contour point of the left cheek, the contour point of the right cheek and the above circle are enclosed, including the chin Part of the area as the effective area.
  • the target pixel is rendered to generate an area texture image corresponding to the ith effective area; wherein, the pixel value of the target pixel is determined according to the pixel value of the pixel in the ith effective area.
  • part (a) of Fig. 16 shows the region texture image corresponding to the valid region in the face image of the frontal face pose
  • part (b) of Fig. 16 shows the face of the left side face pose
  • part (c) in Figure 16 shows the area texture image corresponding to the effective area in the face image of the right side face pose
  • part (d) in Figure 16 shows the head-up image
  • image fusion is performed on the texture images of the n regions to generate a face texture image of the target object.
  • the above n region texture images can be combined to generate a face texture image of the target object.
  • this step includes the following sub-steps:
  • i is a positive integer less than or equal to n.
  • FIG. 17 it shows a schematic diagram of a face texture image of a target object generated by fusing the texture images of various regions in FIG. 16 .
  • the face texture image can be used to render the 3D face model of the target object to generate the textured 3D face model of the target object, and the process can include the following steps :
  • the three-dimensional face model is rendered, and the textured three-dimensional face model is generated.
  • the pixel value of the corresponding pixel of the three-dimensional data point on the face texture image of the target object can be obtained according to the uv index corresponding to the three-dimensional data point, as the pixel value of the three-dimensional data point value.
  • the pixel value of each triangular patch in the three-dimensional face model can be represented by the average value of the pixel values of the three vertices of the triangular patch. In this way, the textured 3D face model of the target object as shown in Figure 18 can be generated by rendering.
  • Part (a) in Figure 18 shows a schematic diagram of the 3D face model in the frontal pose
  • Part ( Part b) and part (c) in Figure 18 show a schematic diagram of the 3D face model in the right side face pose
  • part (d) in Figure 18 shows a schematic diagram of the 3D face model in the left side face pose
  • Part (e) of FIG. 18 shows a schematic diagram of a three-dimensional face model in a head-up posture.
  • the 3D face model can be driven to make different expressions, such as different facial action effects such as opening, closing, blinking, etc. .
  • an emoticon sequence corresponding to the voice information can be obtained, and the emoticon sequence includes at least one emoticon;
  • the sequence of expressions which sequentially displays the textured 3D face model of the target object under each expression.
  • the mapping relationship between the voice information and the expression sequence may be stored in advance, and after the voice information to be played is determined, the expression sequence corresponding to the voice information may be found according to the above mapping relationship.
  • the mapping relationship between each pronunciation and expressions can be stored, and after the voice information to be played is determined, the pronunciation sequence corresponding to the voice information can be determined, and the pronunciation sequence includes at least one pronunciation arranged in order, and obtain For the expression corresponding to each pronunciation, the expression sequence corresponding to the speech information can be obtained.
  • FIG. 19 shows a block diagram of an apparatus for constructing a three-dimensional face model provided by an embodiment of the present application.
  • the apparatus has the function of implementing the above method example, and the function may be implemented by hardware or by executing corresponding software by hardware.
  • the apparatus 1900 may include: a point cloud acquisition module 1910, an index acquisition module 1920, a triangulation module 1930, a model initial construction module 1940, a region determination module 1950, a region smoothing module 1960, and a region replacement module 1970.
  • the point cloud acquisition module 1910 is used to acquire the face point cloud of the target object
  • the index obtaining module 1920 is configured to determine, through expansion calculation, the corresponding pixel coordinates of the three-dimensional data points in the face point cloud on the face texture image of the target object, and obtain the index information of the three-dimensional data points;
  • Triangulation module 1930 configured to triangulate the pixels on the face texture image to obtain triangulation information, where the triangulation information is used to indicate the triangle topology corresponding to the pixels;
  • Model initial building module 1940 for obtaining an initial three-dimensional face model according to the triangulation information, the index information and the face point cloud;
  • an area determination module 1950 configured to determine a non-critical area in the initial three-dimensional face model, where the non-critical area refers to a non-face area in the initial three-dimensional face model;
  • the region smoothing module 1960 is used for smoothing the non-critical regions in the initial three-dimensional face model to obtain the smoothed non-critical regions;
  • the region replacement module 1970 is configured to replace the non-critical regions in the initial three-dimensional face model with the smoothed non-critical regions to obtain the three-dimensional face model of the target object.
  • the area determination module 1950 includes: a pixel point determination unit, a topology determination unit and an area determination unit (not shown in the figure).
  • a pixel point determination unit for determining non-face key pixels in the face texture image
  • a topology determination unit configured to determine the triangle topology corresponding to the non-face key 3D data points and the non-face key 3D data points based on the non-face key pixel points, the triangulation information and the index information ;
  • a region determination unit configured to determine the region formed by the triangle topology corresponding to the non-face key three-dimensional data point as the non-critical region.
  • the pixel point determination unit is used for:
  • Pixels in areas other than the target area in the face texture image are determined as the non-face key pixels.
  • the apparatus further includes: a downsampling module (not shown in the figure).
  • a downsampling module configured to perform grid downsampling on the face point cloud to obtain a downsampled face point cloud; wherein, the neck area in the downsampled face point cloud corresponds to a non-neck area
  • the grid density is different.
  • the index obtaining module 1920 is specifically configured to: determine the pixel coordinates corresponding to the three-dimensional data points in the down-sampled face point cloud on the face texture image of the target object through expansion calculation, and obtain the The index information of the three-dimensional data point is described.
  • the downsampling module is used to:
  • the three-dimensional coordinates of the three-dimensional data points in the face point cloud included in each grid are averaged under the same coordinate dimension to obtain the down-sampled face point cloud.
  • the index obtaining module 1920 is used to:
  • the coordinates of the pixels corresponding to the three-dimensional data points on the face texture image are determined through cylindrical expansion calculation, and the index information is obtained.
  • the device further includes: an image acquisition module, a map acquisition module, a region acquisition module, a data point acquisition module, a pixel point acquisition module, a pixel point rendering module, and an image fusion module (not shown in the figure). ).
  • an image acquisition module configured to acquire n face images corresponding to n head poses of the target object, where n is a positive integer
  • mapping acquisition module for acquiring the mapping relationship between the three-dimensional data points and the pixels on the face image
  • an area acquisition module configured to acquire image areas corresponding to the head posture from the n face images, respectively, to obtain n effective areas
  • the data point acquisition module is used for obtaining the target three-dimensional data point corresponding to the pixel point in the i-th effective area according to the mapping relationship for the ith effective area in the n effective areas, and the i is a positive integer less than or equal to the n;
  • a pixel point acquisition module configured to acquire, according to the index information, the target pixel point of the target three-dimensional data point on the area texture image corresponding to the i-th effective area
  • a pixel point rendering module configured to render the target pixel point, and generate an area texture image corresponding to the i-th effective area; wherein, the pixel value of the target pixel point is based on the i-th effective area The pixel value of the pixel point is determined;
  • An image fusion module configured to perform image fusion on the n regional texture images to generate the face texture image of the target object.
  • the area acquisition module is used to:
  • the effective area in the i-th face image is acquired.
  • the apparatus further includes: a pixel value acquisition module and a model rendering module (not shown in the figure).
  • a pixel value obtaining module configured to obtain, according to the index information, the pixel value of the corresponding pixel of the three-dimensional data point on the face texture image
  • a model rendering module configured to render the three-dimensional face model according to the pixel values of the three-dimensional data points, and generate a textured three-dimensional face model.
  • FIG. 20 shows a schematic structural diagram of a computer device 2000 provided by an embodiment of the present application.
  • the computer device 2000 can be used to implement the method for constructing a three-dimensional face model provided in the above embodiments. Specifically:
  • the computer device 2000 includes a central processing unit (Central Processing Unit, CPU) 2001, a system memory 2004 including a RAM (Random Access Memory, random access memory) 2002 and a ROM (Read-Only Memory, read only memory) 2003, and A system bus 2005 that connects the system memory 2004 and the central processing unit 2001.
  • the computer device 2000 also includes a basic input/output system (I/O system, Input/Output system) 2006 that helps to transfer information between various devices in the computer, and is used to store the operating system 2013, application programs 2014 and other programs Mass storage device 2007 for module 2015.
  • I/O system, Input/Output system basic input/output system
  • the basic input/output system 2006 includes a display 2008 for displaying information and input devices 2009 such as a mouse, keyboard, etc., for user input of information.
  • the display 2008 and the input device 2009 are both connected to the central processing unit 2001 through the input and output controller 2010 connected to the system bus 2005.
  • the basic input/output system 2006 may also include an input output controller 2010 for receiving and processing input from a number of other devices such as a keyboard, mouse, or electronic stylus.
  • input output controller 2010 also provides output to a display screen, printer, or other type of output device.
  • the mass storage device 2007 is connected to the central processing unit 2001 through a mass storage controller (not shown) connected to the system bus 2005 .
  • the mass storage device 2007 and its associated computer-readable media provide non-volatile storage for the computer device 2000. That is, the mass storage device 2007 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.
  • Computer-readable media can include computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Computer storage media include RAM, ROM, EPROM (Erasable Programmable Read-Only Memory, Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory, Electrically Erasable Programmable Read-Only Memory), flash memory (Flash Memory) or other solid-state storage device, CD-ROM, DVD (Digital Versatile Disc, Digital Versatile Disc) or other optical storage, cassette, magnetic tape, magnetic disk storage or other magnetic storage device.
  • the system memory 2004 and the mass storage device 2007 described above may be collectively referred to as memory.
  • the computer device 2000 may also be connected to a remote computer on the network through a network such as the Internet to operate. That is, the computer device 2000 can be connected to the network 2012 through the network interface unit 2011 connected to the system bus 2005, or in other words, the network interface unit 2011 can also be used to connect to other types of networks or remote computer systems (not shown). ).
  • the memory also includes one or more programs stored in the memory and configured to be executed by one or more processors.
  • the above-mentioned one or more programs include instructions for implementing the above-mentioned method for constructing a three-dimensional face model on the computer device side.
  • a terminal in an exemplary embodiment, includes a processor and a memory, and the memory stores at least one instruction, at least one piece of program, code set or instruction set.
  • the at least one instruction, at least one section of program, code set or instruction set is configured to be executed by one or more processors, so as to implement the above-mentioned method for constructing a three-dimensional face model.
  • a server including a processor and a memory, the memory storing at least one instruction, at least one piece of program, code set or instruction set.
  • the at least one instruction, at least one section of program, code set or instruction set is configured to be executed by one or more processors, so as to implement the above-mentioned method for constructing a three-dimensional face model.
  • a computer-readable storage medium stores at least one instruction, at least one piece of program, code set or instruction set, the at least one instruction, the at least one piece of program .
  • the code set or the instruction set implements the above-mentioned method for constructing a three-dimensional face model when executed by a processor of a computer device.
  • a computer program product or computer program comprising computer instructions stored in a computer readable storage medium.
  • the processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the three-dimensional face model provided in the foregoing embodiment or various optional implementations of the foregoing embodiment construction method.

Abstract

本申请实施例公开了一种三维人脸模型的构建方法、装置、设备及存储介质，属于人工智能的计算机视觉技术领域。所述方法包括：获取目标对象的人脸点云；通过展开计算确定人脸点云中的三维数据点在目标对象的人脸纹理图像上对应的像素点坐标，得到三维数据点的索引信息；对人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息；根据三角剖分信息、索引信息以及人脸点云，得到初始三维人脸模型；确定初始三维人脸模型中的非关键区域；对初始三维人脸模型中的非关键区域做平滑处理，得到平滑后的非关键区域；将初始三维人脸模型中的非关键区域替换为平滑后的非关键区域，得到目标对象的三维人脸模型。本申请实施例提高了三维人脸重建的效率。

Description

三维人脸模型的构建方法、装置、设备及存储介质
本申请要求于2020年07月16日提交中国专利局、申请号为2020106851830、申请名称为“三维人脸模型的构建方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能的计算机视觉技术领域，特别涉及三维人脸模型的构建技术。
背景技术
3D(3 Dimensions，三维)人脸重建是指从一张或多张2D(2 Dimensions，二维)人脸图像中重建出人脸的3D模型(mesh)。
在相关技术中，通常通过如下方式进行三维人脸重建：获取目标对象的一张或多张人脸图像，从人脸图像中获取人脸上各个部位的点，根据该人脸上各个部位的点形成三维点云数据，进一步地对该三维点云数据进行泊松重建，得到该目标对象的三维人脸模型。
然而，上述基于泊松重建的三维人脸重建方法耗时较长，效率低。
发明内容
本申请实施例提供了一种三维人脸模型的构建方法、装置、设备及存储介质，提高了三维人脸重建的效率。技术方案如下：
一方面，本申请实施例提供一种三维人脸模型的构建方法，所述方法包括：
获取目标对象的人脸点云；
通过展开计算，确定所述人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息；
对所述人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息，所述三角剖分信息用于指示所述像素点对应的三角形拓扑；
根据所述三角剖分信息、所述索引信息以及所述人脸点云，得到初始三维人脸模型；
确定所述初始三维人脸模型中的非关键区域，所述非关键区域是指所述初始三维人脸模型中的非人脸区域；
对所述初始三维人脸模型中的非关键区域做平滑处理，得到平滑后的非关键区域；
将所述初始三维人脸模型中的非关键区域替换为所述平滑后的非关键区域，得到所述目标对象的三维人脸模型。
另一方面，本申请实施例提供一种三维人脸模型的构建装置，所述装置包括：
点云获取模块，用于获取目标对象的人脸点云；
索引获取模块，用于通过展开计算，确定所述人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息；
三角剖分模块，用于对所述人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息，所述三角剖分信息用于指示所述像素点对应的三角形拓扑；
模型初建模块，用于根据所述三角剖分信息、所述索引信息以及所述人脸点云，得到初始三维人 脸模型；
区域确定模块，用于确定所述初始三维人脸模型中的非关键区域，所述非关键区域是指所述初始三维人脸模型中的非人脸区域；
区域平滑模块，用于对所述初始三维人脸模型中的非关键区域做平滑处理，得到平滑后的非关键区域；
区域替换模块，用于将所述初始三维人脸模型中的非关键区域替换为所述平滑后的非关键区域，得到所述目标对象的三维人脸模型。
另一方面，本申请实施例提供一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如上述方面所述的方法。
再一方面，本申请实施例提供一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如上述方面所述的方法。
再一方面，本申请实施例提供一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述方面中提供的方法。
本申请实施例提供的技术方案可以带来如下有益效果：
通过对展开计算确定得到的人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息，然后基于该三角剖分信息得到三维人脸模型，本申请实施例基于三角剖分构建三维人脸模型，速度快，效率高。
另外，在得到初始三维人脸模型后，对该初始三维人脸模型中的非关键区域进行平滑处理，从而利用平滑处理后的非关键区域替换原有的非关键区域得到三维人脸模型，该三维人脸模型的效果较好。
附图说明
图1是本申请提供的三维人脸模型的重建流程的框架图；
图2是本申请一个实施例提供的三维人脸模型的构建方法的流程图；
图3是本申请一个实施例提供的人脸点云在不同观察视角下的示意图；
图4是本申请一个实施例提供的三角形拓扑的示意图；
图5是本申请一个实施例提供的初始三维人脸模型的示意图；
图6是本申请一个实施例提供的三维人脸模型的示意图；
图7是本申请另一个实施例提供的三维人脸模型的构建方法的流程图；
图8是本申请一个实施例提供的人脸点云在三维坐标下的示意图；
图9是本申请一个实施例提供的人脸纹理图像上的像素点的示意图；
图10是本申请一个实施例提供的目标区域的示意图；
图11是本申请另一个实施例提供的三角形拓扑的示意图；
图12是本申请一个实施例提供的非关键区域的示意图；
图13是本申请另一个实施例提供的三维人脸模型的示意图；
图14是本申请一个实施例提供的三维人脸模型的构建流程框图；
图15是本申请一个实施例提供的人脸图像的示意图；
图16是本申请一个实施例提供的区域纹理图像的示意图；
图17是本申请一个实施例提供的人脸纹理图像的示意图；
图18是本申请一个实施例提供的带纹理的三维人脸模型的示意图；
图19是本申请一个实施例提供的三维人脸模型的构建装置的框图；
图20是本申请一个实施例提供的计算机设备的结构示意图。
具体实施方式
本申请实施例提供的方案涉及三维人脸重建技术领域，利用计算机视觉技术对目标对象的人脸图像进行分析处理，构建出目标对象的三维人脸模型，并且可以生成该目标对象的人脸纹理图像，通过该人脸纹理图像对上述三维人脸模型进行渲染，生成带纹理的人脸模型。
参考图1，其示出了本申请提供的三维人脸模型的重建流程的框架图。从该流程中可以看出，输入数据包括目标对象的人脸点云11，输出数据包括该目标对象的三维人脸模型12。在获取到目标对象的人脸点云11之后，一方面通过展开计算确定人脸点云11中的三维数据点在目标对象的人脸纹理图像13上对应的像素点坐标，得到该三维数据点的索引信息；另一方面对人脸纹理图像13上的像素点进行三角剖分，得到三角剖分信息，该三角剖分信息用于指示该像素点对应的三角形拓扑14。然后根据上述索引信息和三角剖分信息，生成初始三维人脸模型15。在得到初始三维人脸模型15之后，需要对该初始三维人脸模型15中的非关键区域进行平滑处理，得到平滑处理后的非关键区域；最后将初始三维人脸模型15中的非关键区域替换为上述平滑处理后的非关键区域，得到目标对象的三维人脸模型12。
本申请实施例提供的方法流程，各步骤的执行主体可以是诸如手机、平板电脑、多媒体播放设备、可穿戴设备、PC(Personal Computer，个人计算机)等终端，也可以是服务器。为了便于说明，在下述方法实施例中，仅以各步骤的执行主体为计算机设备为例进行介绍说明，上述计算机设备可以是任何具备计算和存储能力的电子设备，如上文介绍的终端或服务器。
下面，将结合几个实施例对本申请技术方案进行详细的介绍说明。
请参考图2，其示出了本申请一个实施例提供的三维人脸模型的构建方法的流程图。该方法可以包括如下几个步骤。
步骤201，获取目标对象的人脸点云。
目标对象是指具有构建三维人脸模型需求的任意用户，目标对象可以是任意一个人，该目标对象可以是真人，也可以是机器人或者其它占有一定空间体积的立体人物，本申请实施例对此不作限定。
人脸点云是指人脸对应的点云，是采用多个三维数据点来描述目标对象的人脸的三维数据。人脸点云包括多个三维数据点，三维数据点是指三维立体空间中的一个点，每个三维数据点在三维立体空间中可以采用一个(x，y，z)的三维坐标来表示。
在可能的实现方式中，计算机设备采集目标对象的至少一个图像对，每个图像对包括彩色图像和深度图像；然后根据彩色图像和深度图像提取目标对象的人脸点云。其中，人脸点云中的每个三维数据点的x坐标和y坐标可以由彩色图像来确定，三维数据点的z坐标可以由深度图像来确定。
如图3所示，其示出了人脸点云在不同观察视角下的示意图，图3中(a)部分示出了人脸点云在正脸姿态下的示意图，图3中(b)部分和图3中(c)部分示出了人脸点云在右边侧脸姿态下的示意图，图3中(d)部分示出了人脸点云在右边侧脸抬头姿态下的示意图，图3中(e)部分示出了人脸点云在抬头姿态下的示意图。
步骤202，通过展开计算，确定人脸点云中的三维数据点在目标对象的人脸纹理图像上对应的像素点坐标，得到三维数据点的索引信息。
可选地，该索引信息也可以称为uv索引，人脸点云中的每个三维数据点的uv索引，是指该三维数据点在人脸纹理图像上对应的像素点坐标(u，v)。
步骤203，对人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息。
在本申请实施例中，三角剖分信息用于指示像素点对应的三角形拓扑。在目标对象的人脸纹理图像中，邻近的三个像素点可以连接形成一个三角形拓扑(或称为三角形面片)。
在本申请实施例中，三角剖分就是给定一组像素点，将各像素点之间以三角形相互连接，形成一张三角网格，其实质是将像素点与其临近的像素点间的拓扑关系以三角网格的形式反映出来。
在可能的实现方式中，对人脸纹理图像上的像素点进行Delaunay三角剖分，得到三角剖分信息。
如图4所示，计算机设备对左侧的人脸纹理图像41进行三角剖分，得到右侧的三角形拓扑42，根据该三角形拓扑42可确定三角剖分信息。
步骤204，根据三角剖分信息、索引信息以及人脸点云，得到初始三维人脸模型。
根据三角剖分信息，可以确定每个像素点对应的三角形拓扑，而根据索引信息可以确定与人脸纹理图像上的像素点对应的人脸点云中的三维数据点，因而可以得到人脸点云中的每个三维数据点对应的三角形拓扑，基于每个三维数据点对应的三角形拓扑，可以得到如图5所示的初始三维人脸模型，图5中(a)部分示出了正脸姿态下的初始三维人脸模型，图5中(b)部分示出了右边侧脸姿态下的初始三维人脸模型，图5中(c)部分示出了抬头姿态下的初始三维人脸模型。
如图5所示，由于下巴区域的重叠造成初始三维人脸模型中下巴区域效果较差，因此，需要对初始三维人脸模型中的下巴区域进行调整优化，需要说明的是，下巴区域属于非关键区域。
步骤205，确定初始三维人脸模型中的非关键区域。
在本申请实施例中，非关键区域是指初始三维人脸模型中的非人脸区域，即覆盖非人脸关键三维数据点的区域。非人脸关键三维数据点是指非人脸关键点对应的三维数据点。人脸关键点可以包括眉毛、眼睛、嘴唇等人脸关键部位上的关键点。
计算机设备确定出初始三维人脸模型中的非关键区域之后，可以对该非关键区域进行调整优化。
步骤206，对初始三维人脸模型中的非关键区域做平滑处理，得到平滑后的非关键区域。
在可能的实现方式中，将初始三维人脸模型中的非关键区域送入拉普拉斯网格光顺算法中做平滑处理，得到平滑后的非关键区域。
步骤207，将初始三维人脸模型中的非关键区域替换为平滑后的非关键区域，得到目标对象的三维人脸模型。
目标对象的三维人脸模型是指基于人脸点云重构出的目标对象的人脸的三维模型。一个三维人脸模型可以包括点云和三角形拓扑。
如图6所示，其示出了对人脸点云进行重建后，生成的该目标对象的三维人脸模型在不同观察视 角下的示意图，图6中(a)部分是三维人脸模型在正脸角度下的示意图，图6中(b)部分、图6中(c)部分和图6中(d)部分是三维人脸模型在左边侧脸姿态下的示意图，图6中(e)部分是三维人脸模型在抬头角度下的示意图。
综上所述，本申请实施例提供的技术方案中，通过对展开计算确定得到的人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息，然后基于该三角剖分信息得到三维人脸模型，本申请实施例基于三角剖分构建三维人脸模型，速度快，效率高。另外，在得到初始三维人脸模型后，对该初始三维人脸模型中的非关键区域进行平滑处理，从而利用平滑处理后的非关键区域替换初始三维人脸模型中的非关键区域得到目标对象的三维人脸模型，该三维人脸模型的效果较好。
请参考图7，其示出了本申请另一个实施例提供的三维人脸模型的构建方法的流程图。该方法可以包括如下几个步骤。
步骤701，获取目标对象的人脸点云。
有关步骤701的介绍说明可参见上文实施例，此处不再赘述。
步骤702，对人脸点云进行栅格降采样，得到降采样后的人脸点云。
在本申请实施例中，降采样后的人脸点云中的脖子区域和非脖子区域对应的栅格密度不同。脖子区域为非关键区域，因此可以对脖子区域做大幅度的降采样，如可以降低脖子区域对应的栅格密度来实现大幅度的降采样。
在可能的实现方式中，步骤702包括如下几个子步骤：
步骤702a，在三维空间中按照第一栅格密度划分脖子区域对应的多个栅格，以及按照第二栅格密度划分非脖子区域对应的多个栅格。
在可能的实现方式中，第一栅格密度低于第二栅格密度，从而实现对脖子区域做较大幅度的降采样。
步骤702b，将人脸点云融合至三维空间中的多个栅格内。
步骤702c，对每个栅格内包括的人脸点云中的三维数据点的三维坐标在相同坐标维度下取均值，得到降采样后的人脸点云。
例如，一个栅格内包括三个三维数据点：三维数据点1、三维数据点2和三维数据点3。三维数据点1的三维坐标为(x1，y1，z1)，三维数据点2的三维坐标为(x2，y2，z2)，三维数据点3的三维坐标为(x3，y3，z3)，则该栅格经过降采样后包括一个三维数据点，该三维数据点的三维坐标为((x1+x2+x3)/3，(y1+y2+y3)/3，(z1+z2+z3)/3)。需要说明的是，三维空间内的每个栅格的降采样处理方式与此类似，此处不再赘述。
对人脸点云进行降采样，可以使得人脸点云分布得较为均匀。另外，先对人脸点云进行栅格降采样，然后再基于降采样后的人脸点云进行后续的处理流程，可以降低计算机设备的处理开销。
步骤703，通过展开计算，确定降采样后的人脸点云中的三维数据点在目标对象的人脸纹理图像上对应的像素点坐标，得到三维数据点的索引信息。
在可能的实现方式中，步骤703包括如下几个子步骤。需要说明的是，下述步骤中的人脸点云可以是降采样后的人脸点云，也可以为未做降采样处理的人脸点云；即如若通过步骤702对人脸点云进行了降采样处理，则后续步骤中的人脸点云即应当是降采样后的人脸点云，反之，如若未执行步骤702，没有对人脸点云进行降采样处理，则后续步骤中的人脸点云即应当是未做降采样的人脸点云。
步骤703a，获取人脸点云中的三维数据点在三维坐标系中的三维坐标。
在构建好的三维坐标系下，可以获取人脸点云中的三维数据点在三维坐标系中的三维坐标，该三维坐标可以表示为(x，y，z)。
可选地，在获取上述三维坐标之前，先构建三维坐标系。三维坐标系可以包括x、y、z轴，且两两互相垂直。可选地，x轴为水平方向上的坐标轴，如其可以是从左眼到右眼的方向；y轴为竖直方向上的坐标轴，如其可以是从眉心到嘴巴的方向；z轴为深度方向上的坐标轴，如其可以是从后脑勺到鼻尖的方向。如图8所示，图8中(a)部分示出了从左向右看时观察得到的人脸点云的示意图，从该图8中(a)部分可以明显的看到z轴方向和y轴方向；图8中(b)部分示出了从头顶向下俯视观察得到的人脸点云的示意图，从该俯视图视角可以明显的看到x轴方向和z轴方向。
步骤703b，根据三维数据点的三维坐标，通过柱形展开计算确定三维数据点在人脸纹理图像上对应的像素点坐标，得到索引信息。
在本申请实施例中，可以将人脸点云当成一个圆柱体，从圆柱顶面到底面的方向即为从头顶向下俯视的方向。通过柱形展开可以计算出各个三维数据点(x，y，z)在人脸纹理图像上对应的像素点坐标(u，v)。可选地，该计算公式可以如下所示：
v＝(y-y min)/(y max-y min)；
u＝atan(z/x)；
其中，v表示展开纵坐标，u表示展开横坐标，ymax和ymin分别表示人脸点云中包括的三维数据点在y轴上的最大值和最小值，atan()函数表示反正切函数。
需要说明的是，u分象限处理，从而实现与人脸点云的位置对应起来。
通过上述公式，即可以将人脸点云通过柱形展开得到一个平面二维的uv索引，如图9所示，其示出了本申请一个实施例提供的人脸纹理图像91上的像素点的示意图。
在可能的实现方式中，除了用柱形展开计算确定像素点坐标，还可以通过球状展开计算确定像素点坐标，此时，v＝atan(z/y)，u＝atan(z/x)；或者还可以通过立体展开计算确定像素点坐标，此时，u＝(x-xmin)/(xmax-xmin)，v＝(y-ymin)/(ymax-ymin)；本申请在此不对展开计算所采用的计算方法做任何限定。
步骤704，对人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息。
在本申请实施例中，三角剖分信息用于指示像素点对应的三角形拓扑。
步骤705，根据三角剖分信息、索引信息以及人脸点云，得到初始三维人脸模型。
步骤706，确定人脸纹理图像中的非人脸关键像素点。
在可能的实现方式中，步骤706包括如下几个子步骤：
步骤706a，确定人脸纹理图像中的关键区域对应的凸包。
在本申请实施例中，关键区域是指人脸纹理图像中人脸关键像素点覆盖的区域。人脸关键像素点是指人脸关键点对应的像素点，凸包是指刚好包着上述人脸关键像素点的区域。
步骤706b，基于膨胀阈值放大凸包，得到目标区域。
可选地，膨胀阈值基于分辨率进行设置。如图10所示，其示出了本申请一个实施例提供的目标区域的示意图，基于膨胀阈值对凸包101进行放大，得到目标区域102。
步骤706c，将人脸纹理图像中除目标区域以外的区域中的像素点确定为非人脸关键像素点。
除目标区域以外的区域中包括的像素点可以认为是距离脸部较远的脖子点，距离脸部较远的脖子点即是非人脸关键像素点。如图10中的区域103中包括的像素点可以被认为是非人脸关键像素点。
对凸包进行放大后，得到目标区域，基于该目标区域确定非人脸关键像素点，可以使得后续的平滑处理更为精确，且平滑后的非关键区域与初始三维人脸模型的融合更为自然。
步骤707，基于非人脸关键像素点、三角剖分信息和索引信息，确定非人脸关键三维数据点和非人脸关键三维数据点对应的三角形拓扑。
基于非人脸关键像素点和三角剖分信息可以确定出非人脸关键像素点对应的三角形拓扑，如图11所示，带斜线的区域110即为非人脸关键像素点对应的三角形拓扑。
索引信息反映了人脸纹理图像上的像素点与人脸点云中的三维数据点的对应关系，因此基于非人脸关键像素点和索引信息可以确定出非人脸关键三维数据点。基于非人脸关键三维数据点和三角剖分信息，可以确定出非人脸关键三维数据点对应的三角形拓扑。
步骤708，将非人脸关键三维数据点对应的三角形拓扑形成的区域确定为非关键区域。
如图12所示，图12中的深色区域120表示非关键区域，图12中(a)部分示出了正脸姿态下的非关键区域的示意图，图12中(b)部分示出了侧脸姿态下的非关键区域的示意图。
步骤709，对初始三维人脸模型中的非关键区域做平滑处理，得到平滑后的非关键区域。
步骤710，将初始三维人脸模型中的非关键区域替换为平滑后的非关键区域，得到目标对象的三维人脸模型。
将初始三维人脸模型中的非关键区域替换为平滑后的非关键区域，消除了下巴区域的重叠，最终得到如图13所示的三维人脸模型的示意图，图13中(a)部分示出了正脸姿态下的三维人脸模型的示意图，图13中(b)部分示出了右边侧脸姿态下的三维人脸模型的示意图。
参考图14，其示出了本申请一个实施例提供的三维人脸模型的构建流程框图。该三维人脸模型的构建流程框图包括栅格降采样模块1401、柱状展开模块1402、三角剖分模块1403、分区域平滑模块1404。栅格降采样模块1401用于对人脸点云进行栅格降采样，得到降采样后的人脸点云。柱状展开模块1402用于对降采样后的人脸点云进行柱状展开计算确定索引信息。三角剖分模块1403用于对人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息，从而基于该三角剖分信息、索引信息和人脸点云，得到初始三维人脸模型。分区域平滑模块1404用于对初始三维人脸模型中的非关键区域做平滑处理，得到目标对象的三维人脸模型。
在可能的实现方式中，上述三维人脸模型的构建流程框图还包括提取纹理模块1405。提取纹理框图1405用于生成目标对象的人脸纹理图像，有关生成目标对象的人脸纹理图像的具体流程可参见下文实施例，此处不再赘述。
综上所述，本申请实施例提供的技术方案中，通过对人脸点云进行栅格降采样，可以使得人脸点云的分布较为均匀，起到一定的平滑的效果。另外，先对人脸点云进行降采样，后续计算机设备对降采样后的人脸点云进行处理时，可以降低计算机设备的处理开销。
在示意性实施例中，本申请实施例通过如下方式生成目标对象的人脸纹理图像
第一、获取目标对象的n种头部姿态对应的n个人脸图像，n为正整数。
目标对象的人脸图像是指包含该目标对象的人脸的RGB图像(或称为彩色图像)，该RGB图像可以是通过摄像头对目标对象进行拍摄得到的。RGB图像的每个像素值采用R(Red，红)、G(Green， 绿)、B(Blue，蓝)三个通道的颜色分量来表示。可选地，在拍摄获取目标对象的RGB图像时，还可以获取与每个RGB图像对应的深度图像，深度图像是指将从摄像头到目标对象所处场景中各点的距离值作为像素值的图像。深度图像也称为距离影像。深度图像反映了物体可见表面的几何形状。深度图像类似于灰度图像，只是深度图像的每个像素值是摄像头距离物体的实际距离。目标对象的RGB图像和深度图像可以是配准的，两者的像素点之间具有一对一的对应关系。
目标对象的RGB图像和深度图像可以是采用3D摄像头拍摄得到的两张图像，相较于普通2D摄像头仅能够拍摄RGB图像，3D摄像头除了能够拍摄RGB图像之外，还能够拍摄深度图像。其中，3D摄像头可以包括彩色摄像头和深度传感器，彩色摄像头用于拍摄RGB图像，深度传感器用于采集深度信息以生成深度图像。
上述n个人脸图像可以是目标对象在n种头部姿态下的人脸图像。
在一个示例中，n的取值为1，例如获取目标对象在正脸姿态下的人脸图像。
在另一个示例中，为了提升3D人脸重建的准确性，n的取值大于1，例如上述n种头部姿态包括正脸姿态、右边侧脸姿态、左边侧脸姿态和抬头姿态。
计算机设备获取目标对象在哪些头部姿态下的人脸图像，可以预先进行设定。示例性地，如图15所示，获取目标对象在4种头部姿态下的人脸图像：右边侧脸姿态、正脸姿态、左边侧脸姿态和抬头姿态。
可选地，通过固定用于采集目标对象的人脸图像的相机位置，目标对象转动头部，由相机采集目标对象的多个人脸图像。例如，目标对象转动头部的全过程的图像，都会被相机采集并保存下来。出于效率的考虑，不需要对采集的全部图像均进行计算，因为很多数据是重复的，目标对象转动头部的全过程中会采集几百张人脸图像，如果对这几百个人脸图像均进行计算会很浪费计算资源。因此，从这几百个人脸图像中筛选出几个具有代表性的人脸图像用于后续的计算即可。
在可能的实现方式中，获取目标对象的多个候选的人脸图像；从该多个候选的人脸图像中筛选出质量合格的目标图像；从目标图像中选取n种头部姿态下的人脸图像。其中，质量不合格(例如眨眼、运动模糊等)的人脸图像会被剔除，保留质量合格的目标图像，然后在这些质量合格的目标图像中按照头部姿态选出n个人脸图像。
第二、获取三维数据点与人脸图像上的像素点之间的映射关系。
当人脸图像包括n个时，需要分别获取人脸点云中的三维数据点与每个人脸图像上的像素点之间的映射关系，也即获取n组映射关系。
可选地，通过如下方式获取映射关系：对于n个人脸图像中的第i个人脸图像，采用透视投影将三维数据点投影到第i个人脸图像上，得到三维数据点在第i个人脸图像上对应的像素点坐标作为第i组映射关系；其中，上述映射关系包括n组，且n组映射关系中的第i组映射关系与第i个人脸图像相对应，i为小于等于n的正整数。
由于每个人脸图像对应的头部姿态已知，因此可以根据该头部姿态对人脸点云进行旋转、平移等处理，再根据相机参数透视投影到与该头部姿态对应的人脸图像上。
第三、分别从n个人脸图像中获取与头部姿态相对应的图像区域，得到n个有效区域。
当人脸图像的数量n大于1时，由于会生成n组映射关系，每组映射关系结合索引信息都可以生成一张目标对象的人脸纹理图像，需要对该多张人脸纹理图像进行多视角融合，最终生成一张融合后 的人脸纹理图像。
可选地，此步骤包括如下几个子步骤：
1、对于n个人脸图像中的第i个人脸图像，从第i个人脸图像中检测得到人脸关键点。
人脸关键点包括眉毛、眼睛、嘴唇、人脸轮廓等人脸关键部位上的关键点。
2、根据人脸关键点的位置和预定义的规则，获取第i个人脸图像中的有效区域。
第i个人脸图像中的有效区域，是指根据该第i个人脸图像对应的头部姿态确定的用于融合生成最终的人脸纹理图像的区域。
以确定第i个人脸图像中的有效区域为例，在从该第i个人脸图像中检测得到人脸关键点之后，以鼻尖点为中心画一个圆，该圆的大小是由所有的人脸关键点中距离鼻尖点的距离最大的人脸关键点决定的，也即该圆是以鼻尖点为圆心包含检测出的所有人脸关键点的最小圆。然后，从该圆形区域中选取一个包含在当前头部姿态下完整可见的人脸部位的区域，作为有效区域。示例性地，如果该第i个人脸图像是正脸姿态的人脸图像，那么可以直接将该圆包含的区域确定为有效区域；如果该第i个人脸图像是左边侧脸姿态的人脸图像，那么生成一条经过左眼左侧眼角、左侧嘴角以及下巴中心位置这3个关键点的线条，该线条与上述圆相交所形成的包含左边侧脸的区域作为有效区域；如果该第i个人脸图像是右边侧脸姿态的人脸图像，那么生成一条经过右眼右侧眼角、右侧嘴角以及下巴中心位置这3个关键点的线条，该线条与上述圆相交所形成的包含右边侧脸的区域作为有效区域；如果该第i个人脸图像是抬头姿态的人脸图像，那么将左侧嘴角、右侧嘴角、左脸颊轮廓点、右脸颊轮廓点以及上述圆围合而成的，包含下巴部分的区域作为有效区域。
第四、对于n个有效区域中的第i个有效区域，根据映射关系获取第i个有效区域中的像素点对应的目标三维数据点，i为小于等于n的正整数。
第五、根据索引信息获取目标三维数据点在第i个有效区域对应的区域纹理图像上的目标像素点。
第六、对目标像素点进行渲染，生成第i个有效区域对应的区域纹理图像；其中，目标像素点的像素值是根据第i个有效区域中的像素点的像素值确定的。
如图16所示，图16中(a)部分示出了正脸姿态的人脸图像中的有效区域对应的区域纹理图像，图16中(b)部分示出了左边侧脸姿态的人脸图像中的有效区域对应的区域纹理图像，图16中(c)部分示出了右边侧脸姿态的人脸图像中的有效区域对应的区域纹理图像，图16中(d)部分示出了抬头姿态的人脸图像中的有效区域对应的区域纹理图像。
第七、对n个区域纹理图像进行图像融合，生成目标对象的人脸纹理图像。
在生成n个有效区域分别对应的区域纹理图像之后，由于n个有效区域拼合形成完整人脸，因此，可以对上述n个区域纹理图像进行拼合，生成目标对象的人脸纹理图像。
由于各个头部姿态下的光照条件可能有所不同，导致各个人脸图像之间的颜色存在一定的差异，如正脸明显偏白。此时，如果直接将n个区域纹理图像进行拼合生成目标对象的人脸纹理图像，拼接位置处会出现明显的分界线。为了得到更加平滑效果，可以采用如下步骤对n个区域纹理图像进行图像融合：
1、计算n个区域纹理图像的平均颜色。
可选地，此步骤包括如下几个子步骤：
1.1、对于n个区域纹理图像中的第i个区域纹理图像，从第i个区域纹理图像的有效区域中采样获取至少一个像素点的像素值，计算得到第i个区域纹理图像的像素值均值，i为小于等于n的正整数。
1.2、根据n个区域纹理图像的像素值均值，计算得到n个区域纹理图像的平均颜色。
2、将n个区域纹理图像的颜色变换为平均颜色，得到n个变换后的区域纹理图像。
3、对n个变换后的区域纹理图像进行拼合，生成人脸纹理图像。
在得到平均颜色之后，分别将每个区域纹理图像的颜色变换为平均颜色，得到n个变换后的区域纹理图像，然后再对n个变换后的区域纹理图像进行拼合，生成人脸纹理图像，这样就可以使得最终生成的人脸纹理图像中的肤色更加平滑自然，不会出现明显的分界线。示例性地，如图17所示，其示出了对图16中的各个区域纹理图像进行融合，生成的目标对象的人脸纹理图像的示意图。
当计算机设备生成目标对象的人脸纹理图像之后，可以采用该人脸纹理图像对该目标对象的三维人脸模型进行渲染，生成目标对象的带纹理的三维人脸模型，该过程可以包括如下步骤：
1、根据索引信息，获取三维数据点在人脸纹理图像上对应像素点的像素值。
2、根据三维数据点的像素值，渲染三维人脸模型，生成带纹理的三维人脸模型。
对于点云中的每个三维数据点，可以根据该三维数据点对应的uv索引，获取该三维数据点在目标对象的人脸纹理图像上对应像素点的像素值，作为该三维数据点的像素值。另外，三维人脸模型中的每一个三角形面片的像素值，可以采用该三角形面片的三个顶点的像素值的平均值来表示。这样，就可以渲染生成如图18所示的该目标对象的带纹理的三维人脸模型，图18中(a)部分示出了正脸姿态下的三维人脸模型的示意图，图18中(b)部分和图18中(c)部分示出了右边侧脸姿态下的三维人脸模型的示意图，图18中(d)部分示出了左边侧脸姿态下的三维人脸模型的示意图，图18中(e)部分示出了抬头姿态下的三维人脸模型的示意图。
在一种可能的应用场景中，在生成目标对象的带纹理的三维人脸模型之后，可以驱动该三维人脸模型做出不同的表情，如张嘴、闭嘴、眨眼等不同的脸部动作效果。
以语音互动场景为例，在确定待播放的语音消息之后，可以获取与该语音信息对应的表情序列，该表情序列包括至少一个表情；在播放语音消息的过程中，按照表情序列中包含的各个表情的顺序，依次显示目标对象在各个表情下的带纹理的三维人脸模型。其中，语音信息与表情序列之间的映射关系可以预先存储，在确定出待播放的语音信息之后，可以根据上述映射关系，找到与该语音信息对应的表情序列。例如，可以存储每个发音与表情之间的映射关系，在确定出待播放的语音信息之后，可以确定出该语音信息对应的发音序列，该发音序列中包括至少一个按序排列的发音，获取每个发音对应的表情，即可得到与该语音信息对应的表情序列。
通过上述方式，实现了一种基于三维人脸重建得到的模型进行语音交互的方案，且能够根据实时播放的语音，显示相匹配的表情，更加逼真。
下述为本申请装置实施例，可以用于执行本申请方法实施例。对于本申请装置实施例中未披露的细节，请参照本申请方法实施例。
请参考图19，其示出了本申请一个实施例提供的三维人脸模型的构建装置的框图。该装置具有实现上述方法示例的功能，所述功能可以由硬件实现，也可以由硬件执行相应的软件实现。该装置1900可以包括：点云获取模块1910、索引获取模块1920、三角剖分模块1930、模型初建模块1940、区域 确定模块1950、区域平滑模块1960和区域替换模块1970。
点云获取模块1910，用于获取目标对象的人脸点云；
索引获取模块1920，用于通过展开计算，确定所述人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息；
三角剖分模块1930，用于对所述人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息，所述三角剖分信息用于指示所述像素点对应的三角形拓扑；
模型初建模块1940，用于根据所述三角剖分信息、所述索引信息以及所述人脸点云，得到初始三维人脸模型；
区域确定模块1950，用于确定所述初始三维人脸模型中的非关键区域，所述非关键区域是指所述初始三维人脸模型中的非人脸区域；
区域平滑模块1960，用于对所述初始三维人脸模型中的非关键区域做平滑处理，得到平滑后的非关键区域；
区域替换模块1970，用于将所述初始三维人脸模型中的非关键区域替换为所述平滑后的非关键区域，得到所述目标对象的三维人脸模型。
在示意性实施例中，所述区域确定模块1950，包括：像素点确定单元、拓扑确定单元和区域确定单元(图中未示出)。
像素点确定单元，用于确定所述人脸纹理图像中的非人脸关键像素点；
拓扑确定单元，用于基于所述非人脸关键像素点、所述三角剖分信息和所述索引信息，确定非人脸关键三维数据点和所述非人脸关键三维数据点对应的三角形拓扑；
区域确定单元，用于将所述非人脸关键三维数据点对应的三角形拓扑形成的区域确定为所述非关键区域。
在示意性实施例中，所述像素点确定单元，用于：
确定所述人脸纹理图像中的关键区域对应的凸包，所述关键区域是指所述人脸纹理图像中人脸关键像素点覆盖的区域；
基于膨胀阈值放大所述凸包，得到目标区域；
将所述人脸纹理图像中除所述目标区域以外的区域中的像素点确定为所述非人脸关键像素点。
在示意性实施例中，所述装置，还包括：降采样模块(图中未示出)。
降采样模块，用于对所述人脸点云进行栅格降采样，得到降采样后的人脸点云；其中，所述降采样后的人脸点云中的脖子区域和非脖子区域对应的栅格密度不同。
则所述索引获取模块1920具体用于：通过展开计算，确定所述降采样后的人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息。
在示意性实施例中，所述降采样模块，用于：
在三维空间中按照第一栅格密度划分所述脖子区域对应的多个栅格，以及按照第二栅格密度划分所述非脖子区域对应的多个栅格；
将所述人脸点云融合至所述三维空间中的多个栅格内；
对每个栅格内包括的所述人脸点云中的三维数据点的三维坐标在相同坐标维度下取均值，得到所述降采样后的人脸点云。
在示意性实施例中，所述索引获取模块1920，用于：
获取所述人脸点云中的三维数据点在三维坐标系中的三维坐标；
根据所述三维数据点的三维坐标，通过柱形展开计算确定所述三维数据点在所述人脸纹理图像上对应的像素点坐标，得到所述索引信息。
在示意性实施例中，所述装置，还包括：图像获取模块、映射获取模块、区域获取模块、数据点获取模块、像素点获取模块、像素点渲染模块和图像融合模块(图中未示出)。
图像获取模块，用于获取所述目标对象的n种头部姿态对应的n个人脸图像，所述n为正整数；
映射获取模块，用于获取所述三维数据点与所述人脸图像上的像素点之间的映射关系；
区域获取模块，用于分别从所述n个人脸图像中获取与头部姿态相对应的图像区域，得到n个有效区域；
数据点获取模块，用于对于所述n个有效区域中的第i个有效区域，根据所述映射关系获取所述第i个有效区域中的像素点对应的目标三维数据点，所述i为小于等于所述n的正整数；
像素点获取模块，用于根据所述索引信息获取所述目标三维数据点在所述第i个有效区域对应的区域纹理图像上的目标像素点；
像素点渲染模块，用于对所述目标像素点进行渲染，生成所述第i个有效区域对应的区域纹理图像；其中，所述目标像素点的像素值是根据所述第i个有效区域中的像素点的像素值确定的；
图像融合模块，用于对所述n个区域纹理图像进行图像融合，生成所述目标对象的所述人脸纹理图像。
在示意性实施例中，所述区域获取模块，用于：
对于所述n个人脸图像中的第i个人脸图像，从所述第i个人脸图像中检测得到人脸关键点；
根据所述人脸关键点的位置和预定义的规则，获取所述第i个人脸图像中的有效区域。
在示意性实施例中，所述装置，还包括：像素值获取模块和模型渲染模块(图中未示出)。
像素值获取模块，用于根据所述索引信息，获取所述三维数据点在所述人脸纹理图像上对应像素点的像素值；
模型渲染模块，用于根据所述三维数据点的像素值，渲染所述三维人脸模型，生成带纹理的三维人脸模型。
请参考图20，其示出了本申请一个实施例提供的计算机设备2000的结构示意图。该计算机设备2000可用于实施上述实施例中提供的三维人脸模型的构建方法。具体来讲：
所述计算机设备2000包括中央处理单元(Central Processing Unit，CPU)2001、包括RAM(Random Access Memory，随机存取存储器)2002和ROM(Read-Only Memory，只读存储器)2003的系统存储器2004，以及连接系统存储器2004和中央处理单元2001的系统总线2005。所述计算机设备2000还包括帮助计算机内的各个器件之间传输信息的基本输入/输出系统(I/O系统，Input/Output系统)2006，和用于存储操作系统2013、应用程序2014和其他程序模块2015的大容量存储设备2007。
所述基本输入/输出系统2006包括有用于显示信息的显示器2008和用于用户输入信息的诸如鼠标、键盘之类的输入设备2009。其中所述显示器2008和输入设备2009都通过连接到系统总线2005的输入输出控制器2010连接到中央处理单元2001。所述基本输入/输出系统2006还可以包括输入输出控制器2010以用于接收和处理来自键盘、鼠标、或电子触控笔等多个其他设备的输入。类似地，输入 输出控制器2010还提供输出到显示屏、打印机或其他类型的输出设备。
所述大容量存储设备2007通过连接到系统总线2005的大容量存储控制器(未示出)连接到中央处理单元2001。所述大容量存储设备2007及其相关联的计算机可读介质为计算机设备2000提供非易失性存储。也就是说，所述大容量存储设备2007可以包括诸如硬盘或者CD-ROM(Compact Disc Read-Only Memory，只读光盘)驱动器之类的计算机可读介质(未示出)。
不失一般性，所述计算机可读介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、EPROM(Erasable Programmable Read-Only Memory，可擦除可编程只读存储器)、EEPROM(Electrically Erasable Programmable Read-Only Memory，电可擦可编程只读存储器)、闪存(Flash Memory)或其他固态存储设备，CD-ROM、DVD(Digital Versatile Disc，数字通用光盘)或其他光学存储、磁带盒、磁带、磁盘存储或其他磁性存储设备。当然，本领域技术人员可知所述计算机存储介质不局限于上述几种。上述的系统存储器2004和大容量存储设备2007可以统称为存储器。
根据本申请的各种实施例，所述计算机设备2000还可以通过诸如因特网等网络连接到网络上的远程计算机运行。也即计算机设备2000可以通过连接在所述系统总线2005上的网络接口单元2011连接到网络2012，或者说，也可以使用网络接口单元2011来连接到其他类型的网络或远程计算机系统(未示出)。
所述存储器还包括一个或者一个以上的程序，所述一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行。上述一个或者一个以上程序包含用于实现上述计算机设备侧的三维人脸模型的构建方法的指令。
在示例性实施例中，还提供了一种终端，所述终端包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集。所述至少一条指令、至少一段程序、代码集或指令集经配置以由一个或者一个以上处理器执行，以实现上述三维人脸模型的构建方法。
在示例性实施例中，还提供了一种服务器，所述服务器包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集。所述至少一条指令、至少一段程序、代码集或指令集经配置以由一个或者一个以上处理器执行，以实现上述三维人脸模型的构建方法。
在示例性实施例中，还提供了一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或所述指令集在被计算机设备的处理器执行时实现上述三维人脸模型的构建方法。
在示例性实施例中，还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述实施例或者上述实施例的各种可选实现方式中提供的三维人脸模型的构建方法。

Claims (13)

  1. 一种三维人脸模型的构建方法，由计算机设备执行，所述方法包括：
    获取目标对象的人脸点云；
    通过展开计算，确定所述人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息；
    对所述人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息，所述三角剖分信息用于指示所述像素点对应的三角形拓扑；
    根据所述三角剖分信息、所述索引信息以及所述人脸点云，得到初始三维人脸模型；
    确定所述初始三维人脸模型中的非关键区域，所述非关键区域是指所述初始三维人脸模型中的非人脸区域；
    对所述初始三维人脸模型中的非关键区域做平滑处理，得到平滑后的非关键区域；
    将所述初始三维人脸模型中的非关键区域替换为所述平滑后的非关键区域，得到所述目标对象的三维人脸模型。
  2. 根据权利要求1所述的方法，所述确定所述初始三维人脸模型中的非关键区域，包括：
    确定所述人脸纹理图像中的非人脸关键像素点；
    基于所述非人脸关键像素点、所述三角剖分信息和所述索引信息，确定非人脸关键三维数据点和所述非人脸关键三维数据点对应的三角形拓扑；
    将所述非人脸关键三维数据点对应的三角形拓扑形成的区域确定为所述非关键区域。
  3. 根据权利要求2所述的方法，所述确定所述人脸纹理图像中的非人脸关键像素点，包括：
    确定所述人脸纹理图像中的关键区域对应的凸包，所述关键区域是指所述人脸纹理图像中人脸关键像素点覆盖的区域；
    基于膨胀阈值放大所述凸包，得到目标区域；
    将所述人脸纹理图像中除所述目标区域以外的区域中的像素点确定为所述非人脸关键像素点。
  4. 根据权利要求1所述的方法，所述通过展开计算，确定所述人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息之前，还包括：
    对所述人脸点云进行栅格降采样，得到降采样后的人脸点云；其中，所述降采样后的人脸点云中的脖子区域和非脖子区域对应的栅格密度不同；
    则所述通过展开计算，确定所述人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息，包括：
    通过展开计算，确定所述降采样后的人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息。
  5. 根据权利要求4所述的方法，所述对所述人脸点云进行栅格降采样，得到降采样后的人脸点云，包括：
    在三维空间中按照第一栅格密度划分所述脖子区域对应的多个栅格，以及按照第二栅格密度划分所述非脖子区域对应的多个栅格；
    将所述人脸点云融合至所述三维空间中的多个栅格内；
    对每个栅格内包括的所述人脸点云中的三维数据点的三维坐标在相同坐标维度下取均值，得到所 述降采样后的人脸点云。
  6. 根据权利要求1所述的方法，所述通过展开计算，确定所述人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息，包括：
    获取所述人脸点云中的三维数据点在三维坐标系中的三维坐标；
    根据所述三维数据点的三维坐标，通过柱形展开计算确定所述三维数据点在所述人脸纹理图像上对应的像素点坐标，得到所述索引信息。
  7. 根据权利要求1至6任一项所述的方法，还包括：
    获取所述目标对象的n种头部姿态对应的n个人脸图像，所述n为正整数；
    获取所述三维数据点与所述人脸图像上的像素点之间的映射关系；
    分别从所述n个人脸图像中获取与头部姿态相对应的图像区域，得到n个有效区域；
    对于所述n个有效区域中的第i个有效区域，根据所述映射关系获取所述第i个有效区域中的像素点对应的目标三维数据点，所述i为小于等于所述n的正整数；
    根据所述索引信息获取所述目标三维数据点在所述第i个有效区域对应的区域纹理图像上的目标像素点；
    对所述目标像素点进行渲染，生成所述第i个有效区域对应的区域纹理图像；其中，所述目标像素点的像素值是根据所述第i个有效区域中的像素点的像素值确定的；
    对所述n个区域纹理图像进行图像融合，生成所述目标对象的所述人脸纹理图像。
  8. 根据权利要求7所述的方法，所述分别从所述n个人脸图像中获取与头部姿态相对应的图像区域，得到n个有效区域，包括：
    对于所述n个人脸图像中的第i个人脸图像，从所述第i个人脸图像中检测得到人脸关键点；
    根据所述人脸关键点的位置和预定义的规则，获取所述第i个人脸图像中的有效区域。
  9. 根据权利要求7所述的方法，所述对所述n个区域纹理图像进行图像融合，生成所述目标对象的所述人脸纹理图像之后，还包括：
    根据所述索引信息，获取所述三维数据点在所述人脸纹理图像上对应像素点的像素值；
    根据所述三维数据点的像素值，渲染所述三维人脸模型，生成带纹理的三维人脸模型。
  10. 一种三维人脸模型的构建装置，所述装置包括：
    点云获取模块，用于获取目标对象的人脸点云；
    索引获取模块，用于通过展开计算，确定所述人脸点云中的三维数据点在所述目标对象的人脸纹理图像上对应的像素点坐标，得到所述三维数据点的索引信息；
    三角剖分模块，用于对所述人脸纹理图像上的像素点进行三角剖分，得到三角剖分信息，所述三角剖分信息用于指示所述像素点对应的三角形拓扑；
    模型初建模块，用于根据所述三角剖分信息、所述索引信息以及所述人脸点云，得到初始三维人脸模型；
    区域确定模块，用于确定所述初始三维人脸模型中的非关键区域，所述非关键区域是指所述初始三维人脸模型中的非人脸区域；
    区域平滑模块，用于对所述初始三维人脸模型中的非关键区域做平滑处理，得到平滑后的非关键区域；
    区域替换模块，用于将所述初始三维人脸模型中的非关键区域替换为所述平滑后的非关键区域，得到所述目标对象的三维人脸模型。
  11. 一种计算机设备，所述计算机设备包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如权利要求1至9任一项所述的方法。
  12. 一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由处理器加载并执行以实现如权利要求1至9任一项所述的方法。
  13. 一种计算机程序产品，包括指令，当其在计算机上运行时，使得计算机执行权利要求1至9任一项所述的方法。
PCT/CN2021/097567 2020-07-16 2021-06-01 三维人脸模型的构建方法、装置、设备及存储介质 WO2022012192A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022549363A JP7456670B2 (ja) 2020-07-16 2021-06-01 ３次元顔モデルの構築方法、３次元顔モデルの構築装置、コンピュータ機器、及びコンピュータプログラム
EP21842458.8A EP4057234A4 (en) 2020-07-16 2021-06-01 METHOD AND DEVICE FOR GENERATION OF A THREE-DIMENSIONAL FACIAL MODEL, DEVICE AND STORAGE MEDIUM
US17/751,496 US20220284679A1 (en) 2020-07-16 2022-05-23 Method and apparatus for constructing three-dimensional face mesh, device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010685183.0A CN111710036B (zh) 2020-07-16 2020-07-16 三维人脸模型的构建方法、装置、设备及存储介质
CN202010685183.0 2020-07-16

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/751,496 Continuation US20220284679A1 (en) 2020-07-16 2022-05-23 Method and apparatus for constructing three-dimensional face mesh, device, and storage medium

Publications (1)

Publication Number Publication Date
WO2022012192A1 true WO2022012192A1 (zh) 2022-01-20

Family

ID=72546119

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/097567 WO2022012192A1 (zh) 2020-07-16 2021-06-01 三维人脸模型的构建方法、装置、设备及存储介质

Country Status (5)

Country Link
US (1) US20220284679A1 (zh)
EP (1) EP4057234A4 (zh)
JP (1) JP7456670B2 (zh)
CN (1) CN111710036B (zh)
WO (1) WO2022012192A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115830262A (zh) * 2023-02-14 2023-03-21 济南市勘察测绘研究院 一种基于对象分割的实景三维模型建立方法及装置
CN116452403A (zh) * 2023-06-16 2023-07-18 瀚博半导体(上海)有限公司 点云数据处理方法、装置、计算机设备及存储介质

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111710036B (zh) * 2020-07-16 2023-10-17 腾讯科技（深圳）有限公司 三维人脸模型的构建方法、装置、设备及存储介质
CN112418030B (zh) * 2020-11-11 2022-05-13 中国标准化研究院 一种基于三维点云坐标的头面部号型分类方法
CN112613460A (zh) * 2020-12-30 2021-04-06 深圳威富优房客科技有限公司 人脸生成模型的建立方法和人脸生成方法
CN112801082B (zh) * 2021-01-22 2024-01-12 北京百度网讯科技有限公司 图像的采样方法、装置及电子设备
CN114363405B (zh) * 2021-05-26 2023-09-22 科大国创云网科技有限公司 基于上网话单与mr用户轨迹的司机群体识别方法及系统
CN113343925B (zh) * 2021-07-02 2023-08-29 厦门美图宜肤科技有限公司 人脸三维重建方法、装置、电子设备及存储介质
CN116385705B (zh) * 2023-06-06 2023-08-29 北京智拓视界科技有限责任公司 用于对三维数据进行纹理融合的方法、设备和存储介质
CN116912402A (zh) * 2023-06-30 2023-10-20 北京百度网讯科技有限公司 人脸重建的方法、装置、电子设备、存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150325044A1 (en) * 2014-05-09 2015-11-12 Adornably, Inc. Systems and methods for three-dimensional model texturing
CN107945278A (zh) * 2017-12-14 2018-04-20 桂林电子科技大学 一种生物体结构的三维重构与有限元分析方法
CN109191552A (zh) * 2018-08-16 2019-01-11 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备和存储介质
CN111325823A (zh) * 2020-02-05 2020-06-23 腾讯科技（深圳）有限公司 人脸纹理图像的获取方法、装置、设备及存储介质
CN111710036A (zh) * 2020-07-16 2020-09-25 腾讯科技（深圳）有限公司 三维人脸模型的构建方法、装置、设备及存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663810B (zh) * 2012-03-09 2014-07-16 北京航空航天大学 一种基于相位偏移扫描的三维人脸全自动建模方法
US9940504B2 (en) * 2015-06-17 2018-04-10 Itseez3D, Inc. Method to produce consistent face texture
CN107958489B (zh) * 2016-10-17 2021-04-02 杭州海康威视数字技术股份有限公司 一种曲面重建方法及装置
CN107945267B (zh) * 2017-12-13 2021-02-26 四川川大智胜软件股份有限公司 一种用于人脸三维模型纹理融合的方法和设备
CN110210374B (zh) * 2018-05-30 2022-02-25 沈阳工业大学 基于光栅条纹投影的三维人脸定位方法
DE102018129600A1 (de) * 2018-11-23 2020-05-28 Fielmann Ventures GmbH Verfahren und System zur virtuellen Anprobe einer Brille
CN110807836B (zh) * 2020-01-08 2020-05-12 腾讯科技（深圳）有限公司 三维人脸模型的生成方法、装置、设备及介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150325044A1 (en) * 2014-05-09 2015-11-12 Adornably, Inc. Systems and methods for three-dimensional model texturing
CN107945278A (zh) * 2017-12-14 2018-04-20 桂林电子科技大学 一种生物体结构的三维重构与有限元分析方法
CN109191552A (zh) * 2018-08-16 2019-01-11 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备和存储介质
CN111325823A (zh) * 2020-02-05 2020-06-23 腾讯科技（深圳）有限公司 人脸纹理图像的获取方法、装置、设备及存储介质
CN111710036A (zh) * 2020-07-16 2020-09-25 腾讯科技（深圳）有限公司 三维人脸模型的构建方法、装置、设备及存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115830262A (zh) * 2023-02-14 2023-03-21 济南市勘察测绘研究院 一种基于对象分割的实景三维模型建立方法及装置
CN116452403A (zh) * 2023-06-16 2023-07-18 瀚博半导体(上海)有限公司 点云数据处理方法、装置、计算机设备及存储介质
CN116452403B (zh) * 2023-06-16 2023-09-01 瀚博半导体(上海)有限公司 点云数据处理方法、装置、计算机设备及存储介质

Also Published As

Publication number Publication date
US20220284679A1 (en) 2022-09-08
EP4057234A4 (en) 2023-07-05
JP2023514289A (ja) 2023-04-05
CN111710036A (zh) 2020-09-25
CN111710036B (zh) 2023-10-17
JP7456670B2 (ja) 2024-03-27
EP4057234A1 (en) 2022-09-14

Similar Documents

Publication Publication Date Title
WO2022012192A1 (zh) 三维人脸模型的构建方法、装置、设备及存储介质
CN111325823B (zh) 人脸纹理图像的获取方法、装置、设备及存储介质
US11410320B2 (en) Image processing method, apparatus, and storage medium
WO2021093453A1 (zh) 三维表情基的生成方法、语音互动方法、装置及介质
Wei et al. Fisheye video correction
CN111243093B (zh) 三维人脸网格的生成方法、装置、设备及存储介质
US9639914B2 (en) Portrait deformation method and apparatus
EP3057066B1 (en) Generation of three-dimensional imagery from a two-dimensional image using a depth map
KR102120046B1 (ko) 오브젝트를 표시하는 방법
EP3992919B1 (en) Three-dimensional facial model generation method and apparatus, device, and medium
EP4165547A1 (en) Image augmentation for analytics
JPWO2006049147A1 (ja) 三次元形状推定システム及び画像生成システム
CN109906600B (zh) 模拟景深
JP2023545200A (ja) パラメータ推定モデルの訓練方法、パラメータ推定モデルの訓練装置、デバイスおよび記憶媒体
US11928778B2 (en) Method for human body model reconstruction and reconstruction system
WO2024007478A1 (zh) 基于单手机的人体三维建模数据采集与重建方法及系统
CN112802208B (zh) 一种航站楼内三维可视化方法及装置
CN113989434A (zh) 一种人体三维重建方法及设备
Anasosalu et al. Compact and accurate 3-D face modeling using an RGB-D camera: let's open the door to 3-D video conference
US11080920B2 (en) Method of displaying an object
JP2022133133A (ja) 生成装置、生成方法、システム、およびプログラム
CN112562060A (zh) 三维人脸建模方法、装置、电子设备及存储介质
CN112561784A (zh) 图像合成方法、装置、电子设备及存储介质
CN111652807A (zh) 眼部的调整、直播方法、装置、电子设备和存储介质
EP4227903A1 (en) Method for head image registration and head model generation and corresponding devices

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21842458

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021842458

Country of ref document: EP

Effective date: 20220610

ENP Entry into the national phase

Ref document number: 2022549363

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE