CN114049403A - Multi-angle three-dimensional face reconstruction method and device and storage medium - Google Patents
Multi-angle three-dimensional face reconstruction method and device and storage medium Download PDFInfo
- Publication number
- CN114049403A CN114049403A CN202111409835.9A CN202111409835A CN114049403A CN 114049403 A CN114049403 A CN 114049403A CN 202111409835 A CN202111409835 A CN 202111409835A CN 114049403 A CN114049403 A CN 114049403A
- Authority
- CN
- China
- Prior art keywords
- face
- dimensional
- point cloud
- angle
- dense
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Processing Or Creating Images (AREA)
Abstract
A multi-angle three-dimensional face reconstruction method, a device and a storage medium are provided, the method comprises the following steps: acquiring a multi-angle face picture; generating a three-dimensional face point cloud model according to the multi-angle face picture; and rendering the three-dimensional face point cloud model and obtaining the three-dimensional face model. The multi-angle three-dimensional face reconstruction method, the multi-angle three-dimensional face reconstruction device and the storage medium can realize parallel computation to improve the modeling speed, can remove outlier point clouds and better supplement missing details which may exist, and further render a generated face model to obtain a more vivid face model.
Description
Technical Field
The invention belongs to the technical field of three-dimensional face reconstruction, and particularly relates to a multi-angle three-dimensional face reconstruction method, a multi-angle three-dimensional face reconstruction device and a storage medium.
Background
In recent years, computer vision has a great leap under the promotion of various technologies, and people's faces contain most important biological information of one person and are always concerned. When the mobile phone bank app is opened to verify identity, the mobile phone automatically jumps to a face recognition interface, which generally relies on two-dimensional face image recognition. For three-dimensional animals, information is inevitably lost when a single picture is input, single individual identity information may not be easily recognized when the single picture is input at a certain angle, and in order to achieve higher recognition accuracy, face three-dimensional modeling is considered to be further used for assisting face recognition in many high-end face recognition technologies.
In addition, not only the face recognition accuracy is improved, but also the three-dimensional modeling of the face can be used in scenes such as VR and cloud games, which is already trending under the 5G background. By taking VR as an example, the user can generate characters with the same proportion size as the user in the virtual game space according to the image of the user, and better immersive experience is brought to the user. In addition, the reconstruction of the three-dimensional face plays an indispensable role in the aspects of movie special effects, beauty treatment and the like. Therefore, how to construct a vivid three-dimensional face in proportion to the real world has practical significance.
However, the reconstruction of three-dimensional faces can often be done in the prevalent two-dimensional storage format of photographs only. In order to obtain face depth information, a three-dimensional face model can be effectively constructed by using an RGB-D camera capable of detecting depth, but a face texture pattern cannot be obtained, and the method has high power consumption and high cost. Meanwhile, with the great splendid heterology of deep learning in the aspect of computer vision, more and more researchers consider realizing the three-dimensional reconstruction of the human face by taking the deep learning as a technical means.
At present, the technology for realizing better fitting of the three-dimensional face is still a supervised deep learning method, which means that a large number of high-precision three-dimensional face labels are required to be used as samples for training. Therefore, how to rapidly obtain more high-precision three-dimensional face models by various physical means is a problem which needs to be solved urgently.
Disclosure of Invention
In view of the above, the present invention provides a multi-angle three-dimensional face reconstruction method, apparatus and storage medium that overcome or at least partially solve the above problems.
In order to solve the technical problem, the invention provides a multi-angle three-dimensional face reconstruction method, which comprises the following steps:
acquiring a multi-angle face picture;
generating a three-dimensional face point cloud model according to the multi-angle face picture;
and rendering the three-dimensional face point cloud model and obtaining the three-dimensional face model.
Preferably, the generating of the three-dimensional face point cloud model according to the multi-angle face picture comprises the following steps:
reconstructing sparse feature points of the multi-angle face picture;
acquiring a camera pose;
optimizing the sparse feature points according to the camera pose;
generating dense three-dimensional point cloud on the face surface according to the camera pose;
and correcting the dense three-dimensional point cloud on the face surface.
Preferably, the acquiring of the camera pose comprises the steps of:
arranging a calibration pattern in a photographic field;
measuring the space coordinate of the calibration graph under the absolute space coordinate system of the photographic field;
finding the corresponding position of the calibration graph in the photographic image;
determining the corresponding relation between the multi-angle face picture and the photography field;
solving the matrix parameters of the camera;
and solving the actual three-dimensional position coordinates of the position points in the multi-angle face picture in the space points by using the matrix parameters.
Preferably, the generating of the dense three-dimensional point cloud of the face surface according to the camera pose comprises the following steps:
searching a target point with image consistency in the space;
calculating a depth value of each pixel according to the target point;
carrying out consistency aggregation by utilizing the association between adjacent pixel blocks and obtaining a face depth image;
and aggregating the face depth map to obtain a face surface dense three-dimensional point cloud.
Preferably, the step of correcting the dense three-dimensional point cloud on the face surface comprises the following steps:
meshing the dense three-dimensional point cloud on the surface of the human face;
filling up the missing part of the dense three-dimensional point cloud on the face surface;
reducing the deletion portion;
and carrying out three times of characteristic sampling reduction on the dense three-dimensional point cloud on the surface of the face to obtain a fine point cloud model.
Preferably, the generating of the three-dimensional face point cloud model according to the multi-angle face picture comprises the following steps:
acquiring dense three-dimensional point cloud on the surface of the face;
estimating a normal vector of the dense three-dimensional point cloud on the surface of the face;
projecting the dense three-dimensional point cloud on the face surface onto a two-dimensional plane according to the normal vector;
and carrying out triangularization topological connection on the dense three-dimensional point cloud on the face surface to obtain a three-dimensional face point cloud model.
Preferably, the rendering the three-dimensional face point cloud model and obtaining the three-dimensional face model comprises the following steps:
acquiring a camera pose;
acquiring two-dimensional projection of the three-dimensional face point cloud model under the camera pose;
acquiring an original picture of the three-dimensional face point cloud model;
acquiring a color difference part between the two-dimensional projection and the original photo;
optimizing the color difference portion;
and carrying out color adjustment on the three-dimensional human face point cloud model according to the color difference part.
The application also provides a device is rebuild to three-dimensional face of multi-angle, the device includes:
the multi-angle face picture acquisition module is used for acquiring a multi-angle face picture;
the three-dimensional face point cloud model generating module is used for generating a three-dimensional face point cloud model according to the multi-angle face picture;
and the three-dimensional face model generating module is used for rendering the three-dimensional face point cloud model and obtaining a three-dimensional face model.
The present application further provides an electronic device, which includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform any one of the foregoing multi-angle three-dimensional face reconstruction methods.
The present application further provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute any one of the above multi-angle three-dimensional face reconstruction methods.
One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages: the multi-angle three-dimensional face reconstruction method, the multi-angle three-dimensional face reconstruction device and the storage medium can realize parallel computation to improve the modeling speed, can remove outlier point clouds and better supplement missing details which may exist, and further render a generated face model to obtain a more vivid face model.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a multi-angle three-dimensional face reconstruction method according to an embodiment of the present invention;
FIG. 2 is a block diagram of an embodiment of the present invention
The structure schematic diagram of the multi-angle three-dimensional face reconstruction device;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a non-transitory computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail below with reference to specific embodiments and examples, and the advantages and various effects of the present invention will be more clearly apparent therefrom. It will be understood by those skilled in the art that these specific embodiments and examples are for the purpose of illustrating the invention and are not to be construed as limiting the invention.
Throughout the specification, unless otherwise specifically noted, terms used herein should be understood as having meanings as commonly used in the art. Accordingly, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. If there is a conflict, the present specification will control.
Unless otherwise specifically stated, various raw materials, reagents, instruments, equipment and the like used in the present invention are commercially available or can be prepared by existing methods.
As shown in fig. 1, in the embodiment of the present application, the present invention provides a multi-angle three-dimensional face reconstruction method, where the method includes the steps of:
s1: acquiring a multi-angle face picture;
in the embodiment of the application, the multi-angle face picture can be acquired by a camera. The multi-angle face picture acquisition steps are as follows: firstly, an annular photography field is manufactured, cameras which are arranged annularly are arranged in the annular photography field, and then shooting is carried out from the left side and the right side of a human face, the three main directions of a front face and different pitching angles, so that high-definition human face pictures at multiple angles are obtained.
S2: generating a three-dimensional face point cloud model according to the multi-angle face picture;
in this embodiment of the present application, the generating a three-dimensional face point cloud model according to the multi-angle face picture includes:
reconstructing sparse feature points of the multi-angle face picture;
acquiring a camera pose;
optimizing the sparse feature points according to the camera pose;
generating dense three-dimensional point cloud on the face surface according to the camera pose;
and correcting the dense three-dimensional point cloud on the face surface.
In the embodiment of the application, when a three-dimensional face point cloud model is generated according to the multi-angle face picture, sparse feature points of the multi-angle face picture are reconstructed in a practical training mode, then the camera pose obtained in the step S1 is obtained, the sparse feature points are optimized according to the camera pose, then a face surface dense three-dimensional point cloud is generated according to the camera pose, and the face surface dense three-dimensional point cloud is corrected.
In an embodiment of the present application, the acquiring a pose of a camera includes:
arranging a calibration pattern in a photographic field;
measuring the space coordinate of the calibration graph under the absolute space coordinate system of the photographic field;
finding the corresponding position of the calibration graph in the photographic image;
determining the corresponding relation between the multi-angle face picture and the photography field;
solving the matrix parameters of the camera;
and solving the actual three-dimensional position coordinates of the position points in the multi-angle face picture in the space points by using the matrix parameters.
In the embodiment of the application, when the pose of a camera is acquired, firstly, a calibration graph is arranged in a photographic field, the calibration graph can be selected according to needs, then, the space coordinate of the calibration graph is measured in an absolute space coordinate system of the photographic field, meanwhile, the corresponding position of the calibration graph is found in a photographic image, then, the corresponding relation between the multi-angle face picture and the photographic field is determined, the matrix parameter of the camera is solved, and then, the actual three-dimensional position coordinate of the position point in the multi-angle face picture in the space point is obtained by utilizing the matrix parameter.
In an embodiment of the present application, the generating a dense three-dimensional point cloud of a face surface according to the camera pose includes:
searching a target point with image consistency in the space;
calculating a depth value of each pixel according to the target point;
carrying out consistency aggregation by utilizing the association between adjacent pixel blocks and obtaining a face depth image;
and aggregating the face depth map to obtain a face surface dense three-dimensional point cloud.
In the embodiment of the application, when the dense three-dimensional point cloud on the face surface is generated according to the camera pose, a target point with image consistency is firstly searched in space, the depth value of each pixel is calculated according to the target point, then consistency aggregation is carried out by utilizing the association between adjacent pixel blocks to obtain a face depth map, and then the face depth map is aggregated to obtain the dense three-dimensional point cloud on the face surface.
In the embodiment of the present application, the correcting the dense three-dimensional point cloud on the face surface includes:
meshing the dense three-dimensional point cloud on the surface of the human face;
filling up the missing part of the dense three-dimensional point cloud on the face surface;
reducing the deletion portion;
and carrying out three times of characteristic sampling reduction on the dense three-dimensional point cloud on the surface of the face to obtain a fine point cloud model.
In the embodiment of the application, when the dense three-dimensional point cloud on the face surface is corrected, firstly, gridding is carried out on the dense three-dimensional point cloud on the face surface, then, the missing part of the dense three-dimensional point cloud on the face surface is completed, then, the missing part is restored, and then, the dense three-dimensional point cloud on the face surface is subjected to three times of characteristic sampling restoration to obtain a fine point cloud model.
In this embodiment of the present application, the generating a three-dimensional face point cloud model according to the multi-angle face picture includes:
acquiring dense three-dimensional point cloud on the surface of the face;
estimating a normal vector of the dense three-dimensional point cloud on the surface of the face;
projecting the dense three-dimensional point cloud on the face surface onto a two-dimensional plane according to the normal vector;
and carrying out triangularization topological connection on the dense three-dimensional point cloud on the face surface to obtain a three-dimensional face point cloud model.
In the embodiment of the application, when the three-dimensional face point cloud model is generated according to the multi-angle face picture, firstly, dense three-dimensional point cloud on the face surface is obtained, then, a normal vector of the dense three-dimensional point cloud on the face surface is estimated, then, the dense three-dimensional point cloud on the face surface is projected onto a two-dimensional plane according to the normal vector, and then, triangularization topological connection is carried out on the dense three-dimensional point cloud on the face surface to obtain the three-dimensional face point cloud model.
S3: and rendering the three-dimensional face point cloud model and obtaining the three-dimensional face model.
In the embodiment of the present application, the rendering the three-dimensional face point cloud model and obtaining the three-dimensional face model includes:
acquiring a camera pose;
acquiring two-dimensional projection of the three-dimensional face point cloud model under the camera pose;
acquiring an original picture of the three-dimensional face point cloud model;
acquiring a color difference part between the two-dimensional projection and the original photo;
optimizing the color difference portion;
and carrying out color adjustment on the three-dimensional human face point cloud model according to the color difference part.
In the embodiment of the application, when the three-dimensional face point cloud model is rendered and the three-dimensional face model is obtained, a camera pose is firstly obtained, then a two-dimensional projection of the three-dimensional face point cloud model under the camera pose is obtained, then an original photo of the three-dimensional face point cloud model is obtained, then a color difference part between the two-dimensional projection and the original photo is obtained, then the color difference part is optimized, and then the color of the three-dimensional face point cloud model is adjusted according to the color difference part.
As shown in fig. 2, in the embodiment of the present application, the present application further provides a multi-angle three-dimensional face reconstruction device, where the device includes:
a multi-angle face picture obtaining module 10, configured to obtain a multi-angle face picture;
the three-dimensional face point cloud model generating module 20 is used for generating a three-dimensional face point cloud model according to the multi-angle face picture;
and the three-dimensional face model generating module 30 is used for rendering the three-dimensional face point cloud model and obtaining a three-dimensional face model.
The multi-angle three-dimensional face reconstruction device can execute the multi-angle three-dimensional face reconstruction method provided by the steps.
Referring now to FIG. 3, a block diagram of an electronic device 100 suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 3 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 3, the electronic device 100 may include a processing means (e.g., a central processing unit, a graphic processor, etc.) 101 that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)102 or a program loaded from a storage means 108 into a Random Access Memory (RAM) 103. In the RAM 103, various programs and data necessary for the operation of the electronic apparatus 100 are also stored. The processing device 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input/output (I/0) interface 105 is also connected to bus 104.
Generally, the following devices may be connected to the I/0 interface 105: input devices 106 including, for example, a touch screen, touch pad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 107 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 108 including, for example, magnetic tape, hard disk, etc.; and a communication device 109. The communication means 109 may allow the electronic device 100 to communicate wirelessly or by wire with other devices to exchange data. While the figures illustrate an electronic device 100 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication means 109, or installed from the storage means 108, or installed from the ROM 102. The computer program, when executed by the processing device 101, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
Reference is now made to fig. 4, which shows a schematic structural diagram of a computer-readable storage medium suitable for implementing an embodiment of the present disclosure, the computer-readable storage medium storing a computer program, which when executed by a processor is capable of implementing the multi-angle three-dimensional face reconstruction method as described in any one of the above.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring at least two internet protocol addresses; sending a node evaluation request comprising the at least two internet protocol addresses to node evaluation equipment, wherein the node evaluation equipment selects the internet protocol addresses from the at least two internet protocol addresses and returns the internet protocol addresses; receiving an internet protocol address returned by the node evaluation equipment; wherein the obtained internet protocol address indicates an edge node in the content distribution network.
Alternatively, the computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a node evaluation request comprising at least two internet protocol addresses; selecting an internet protocol address from the at least two internet protocol addresses; returning the selected internet protocol address; wherein the received internet protocol address indicates an edge node in the content distribution network.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".
The multi-angle three-dimensional face reconstruction method, the multi-angle three-dimensional face reconstruction device and the storage medium can realize parallel computation to improve the modeling speed, can remove outlier point clouds and better supplement missing details which may exist, and further render a generated face model to obtain a more vivid face model.
It is to be understood that the above-described embodiments of the present invention are merely illustrative of or explaining the principles of the invention and are not to be construed as limiting the invention. Therefore, any modification, equivalent replacement, improvement and the like made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. Further, it is intended that the appended claims cover all such variations and modifications as fall within the scope and boundaries of the appended claims or the equivalents of such scope and boundaries.
Claims (10)
1. A multi-angle three-dimensional face reconstruction method is characterized by comprising the following steps:
acquiring a multi-angle face picture;
generating a three-dimensional face point cloud model according to the multi-angle face picture;
and rendering the three-dimensional face point cloud model and obtaining the three-dimensional face model.
2. The multi-angle three-dimensional human face reconstruction method according to claim 1, wherein the generating of the three-dimensional human face point cloud model according to the multi-angle human face picture comprises the steps of:
reconstructing sparse feature points of the multi-angle face picture;
acquiring a camera pose;
optimizing the sparse feature points according to the camera pose;
generating dense three-dimensional point cloud on the face surface according to the camera pose;
and correcting the dense three-dimensional point cloud on the face surface.
3. The multi-angle three-dimensional face reconstruction method according to claim 2, wherein the acquiring of the pose of the camera comprises the steps of:
arranging a calibration pattern in a photographic field;
measuring the space coordinate of the calibration graph under the absolute space coordinate system of the photographic field;
finding the corresponding position of the calibration graph in the photographic image;
determining the corresponding relation between the multi-angle face picture and the photography field;
solving the matrix parameters of the camera;
and solving the actual three-dimensional position coordinates of the position points in the multi-angle face picture in the space points by using the matrix parameters.
4. The multi-angle three-dimensional face reconstruction method according to claim 2, wherein the generating a dense three-dimensional point cloud of the face surface according to the camera pose comprises the steps of:
searching a target point with image consistency in the space;
calculating a depth value of each pixel according to the target point;
carrying out consistency aggregation by utilizing the association between adjacent pixel blocks and obtaining a face depth image;
and aggregating the face depth map to obtain a face surface dense three-dimensional point cloud.
5. The multi-angle three-dimensional face reconstruction method according to claim 2, wherein the correcting the dense three-dimensional point cloud on the face surface comprises the steps of:
meshing the dense three-dimensional point cloud on the surface of the human face;
filling up the missing part of the dense three-dimensional point cloud on the face surface;
reducing the deletion portion;
and carrying out three times of characteristic sampling reduction on the dense three-dimensional point cloud on the surface of the face to obtain a fine point cloud model.
6. The multi-angle three-dimensional human face reconstruction method according to claim 2, wherein the generating of the three-dimensional human face point cloud model according to the multi-angle human face picture comprises the steps of:
acquiring dense three-dimensional point cloud on the surface of the face;
estimating a normal vector of the dense three-dimensional point cloud on the surface of the face;
projecting the dense three-dimensional point cloud on the face surface onto a two-dimensional plane according to the normal vector;
and carrying out triangularization topological connection on the dense three-dimensional point cloud on the face surface to obtain a three-dimensional face point cloud model.
7. The multi-angle three-dimensional face reconstruction method according to claim 2, wherein the rendering the three-dimensional face point cloud model and obtaining the three-dimensional face model comprises the steps of:
acquiring a camera pose;
acquiring two-dimensional projection of the three-dimensional face point cloud model under the camera pose;
acquiring an original picture of the three-dimensional face point cloud model;
acquiring a color difference part between the two-dimensional projection and the original photo;
optimizing the color difference portion;
and carrying out color adjustment on the three-dimensional human face point cloud model according to the color difference part.
8. A multi-angle three-dimensional face reconstruction apparatus, the apparatus comprising:
the multi-angle face picture acquisition module is used for acquiring a multi-angle face picture;
the three-dimensional face point cloud model generating module is used for generating a three-dimensional face point cloud model according to the multi-angle face picture;
and the three-dimensional face model generating module is used for rendering the three-dimensional face point cloud model and obtaining a three-dimensional face model.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the multi-angle three-dimensional face reconstruction method of any one of claims 1-7.
10. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the multi-angle three-dimensional face reconstruction method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111409835.9A CN114049403A (en) | 2021-11-23 | 2021-11-23 | Multi-angle three-dimensional face reconstruction method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111409835.9A CN114049403A (en) | 2021-11-23 | 2021-11-23 | Multi-angle three-dimensional face reconstruction method and device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114049403A true CN114049403A (en) | 2022-02-15 |
Family
ID=80210946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111409835.9A Pending CN114049403A (en) | 2021-11-23 | 2021-11-23 | Multi-angle three-dimensional face reconstruction method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114049403A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115409953A (en) * | 2022-11-02 | 2022-11-29 | 汉斯夫(杭州)医学科技有限公司 | Multi-camera color consistency-based maxillofacial reconstruction method, equipment and medium |
-
2021
- 2021-11-23 CN CN202111409835.9A patent/CN114049403A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115409953A (en) * | 2022-11-02 | 2022-11-29 | 汉斯夫(杭州)医学科技有限公司 | Multi-camera color consistency-based maxillofacial reconstruction method, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110058685B (en) | Virtual object display method and device, electronic equipment and computer-readable storage medium | |
| CN106846497B (en) | Method and device for presenting three-dimensional map applied to terminal | |
| CN113396443A (en) | Augmented reality system | |
| CN110189246B (en) | Image stylization generation method and device and electronic equipment | |
| CN109754464B (en) | Method and apparatus for generating information | |
| CN111882634B (en) | Image rendering method, device, equipment and storage medium | |
| CN110378947B (en) | 3D model reconstruction method and device and electronic equipment | |
| CN107329671B (en) | Model display method and device | |
| CN112766215A (en) | Face fusion method and device, electronic equipment and storage medium | |
| WO2024104248A1 (en) | Rendering method and apparatus for virtual panorama, and device and storage medium | |
| WO2022247630A1 (en) | Image processing method and apparatus, electronic device and storage medium | |
| US11494961B2 (en) | Sticker generating method and apparatus, and medium and electronic device | |
| CN114842120A (en) | Image rendering processing method, device, equipment and medium | |
| CN114445269A (en) | Image special effect processing method, device, equipment and medium | |
| CN111862349A (en) | Virtual brush implementation method and device and computer readable storage medium | |
| CN114049674A (en) | Three-dimensional face reconstruction method, device and storage medium | |
| CN111833459B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN114049403A (en) | Multi-angle three-dimensional face reconstruction method and device and storage medium | |
| CN111292406B (en) | Model rendering method, device, electronic equipment and medium | |
| CN109816791B (en) | Method and apparatus for generating information | |
| CN110197459B (en) | Image stylization generation method and device and electronic equipment | |
| CN110378948B (en) | 3D model reconstruction method and device and electronic equipment | |
| CN110390717B (en) | 3D model reconstruction method and device and electronic equipment | |
| CN114900621A (en) | Special effect video determination method and device, electronic equipment and storage medium | |
| CN111862342A (en) | Texture processing method and device for augmented reality, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |