CN113379893B - Method for synthesizing 3D face model by utilizing optical reflection - Google Patents
Method for synthesizing 3D face model by utilizing optical reflection Download PDFInfo
- Publication number
- CN113379893B CN113379893B CN202110583944.6A CN202110583944A CN113379893B CN 113379893 B CN113379893 B CN 113379893B CN 202110583944 A CN202110583944 A CN 202110583944A CN 113379893 B CN113379893 B CN 113379893B
- Authority
- CN
- China
- Prior art keywords
- face
- reflector
- image
- degrees
- synthesizing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003287 optical effect Effects 0.000 title claims abstract description 51
- 230000002194 synthesizing effect Effects 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000009877 rendering Methods 0.000 claims abstract description 8
- 210000003128 head Anatomy 0.000 claims description 33
- 230000000284 resting effect Effects 0.000 claims description 12
- 210000001061 forehead Anatomy 0.000 claims description 10
- 238000003384 imaging method Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000002310 reflectometry Methods 0.000 claims description 5
- 230000015572 biosynthetic process Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 238000003786 synthesis reaction Methods 0.000 claims description 4
- 238000007493 shaping process Methods 0.000 abstract 1
- 230000008901 benefit Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 210000001331 nose Anatomy 0.000 description 4
- 230000001815 facial effect Effects 0.000 description 3
- 230000037303 wrinkles Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000001559 infrared map Methods 0.000 description 1
- 239000004816 latex Substances 0.000 description 1
- 229920000126 latex Polymers 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011148 porous material Substances 0.000 description 1
- 238000004659 sterilization and disinfection Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
- G06T15/20—Perspective computation
- G06T15/205—Image-based rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
- G06T7/596—Depth or shape recovery from multiple images from stereo images from three or more stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a method for synthesizing a 3D face model by optical reflection, which comprises the following steps of acquiring a plurality of face three-dimensional images and RGB color images at a plurality of angles relative to a front face from an optical reflector by rotating a 3D structured light rotating module, wherein the optical reflector corresponds to the acquisition positions of the face three-dimensional images at all angles respectively; generating point cloud images and color maps corresponding to all angles according to the human face three-dimensional image and the RGB color image; synthesizing corresponding point cloud images in two human face three-dimensional images with the smallest relative frontal face angle absolute value into a first synthesized image; sequentially synthesizing the corresponding point cloud images in the human face three-dimensional images with the residual angles with the first synthetic image; and rendering the synthesized image according to the color map to obtain a 3D face model. The method has high image precision, can obtain more accurate face depth information, and provides good reference for medical and aesthetic shaping.
Description
Technical Field
The invention relates to a method for synthesizing a 3D face model, in particular to a method for synthesizing a 3D face model by utilizing optical reflection.
Background
With the continuous progress of the society of times, people pay more and more attention to the appearance characteristics of the skin and the like, so the need for understanding the skin state of the people is stronger and stronger. In the fields of medical cosmetology and the like, such as skin care product recommendation, skin analysis and the like, people generally need to perform certain detection and analysis on the skin condition of the whole face, and then make a corresponding strategy according to the analysis result. The skin is detected by using an analytical instrument, and a proper skin care product is recommended according to the detection result or a corresponding skin care strategy is made. The skin detector is an instrument for detecting skin, can help consumers to intuitively and quickly know the health condition of the skin by matching with professional detection software, has multiple functions of attractive interface and comprehensive practical functions, also has a plurality of analysis modules and can accurately analyze skin oil content, moisture, color spots, pores and skin age, directly acquires moisture data by acquiring the data of the moisture under the skin cuticle, and has high data accuracy.
However, the current skin detector usually only has a single camera to shoot, or shoots the human face by rotating the device, the former can only shoot a facial image at an angle, if facial images at different angles need to be shot, the head needs to be artificially rotated, and the shot picture is basically a two-dimensional graph, the image cannot be rotated in a three-dimensional way, the contour analysis is not accurate, and the latter has extremely high cost and high price.
In summary, a 3D structured light full-face imaging device combined with optical reflection imaging, which can satisfy the increasing demand of consumers, is urgently needed.
Disclosure of Invention
The present invention is directed to the above problems in the prior art, and provides a method for synthesizing a 3D human face model by using optical reflection.
In order to realize the purpose of the invention, the invention adopts the following technical scheme: a method for synthesizing a 3D face model by utilizing optical reflection comprises the following steps:
s100, acquiring a plurality of human face three-dimensional images and RGB (red, green and blue) color images at a plurality of angles relative to a front face from an optical reflector by rotating a 3D structured light rotating module, wherein the optical reflector corresponds to the human face three-dimensional image acquisition positions at all angles respectively;
s200, generating point cloud images and color maps corresponding to all angles according to the human face three-dimensional images and the RGB color images.
S300, synthesizing corresponding point cloud images in two human face three-dimensional images with the minimum positive face angle absolute value into a first synthesized image;
s400, sequentially synthesizing point cloud images corresponding to the human face three-dimensional images with the residual angles with a first synthetic image;
and S500, rendering the synthesized image according to the color map to obtain a 3D face model.
The working principle and the beneficial effects are as follows: 1. compared with the prior art, the whole equipment does not need to be rotated to shoot the face of the user, so that a large-structure driving mechanism is omitted, only a simple and small rotating mechanism is needed to rotate the 3D structured light rotating module, and the shooting of all angles of the face can be realized under the conditions that the equipment does not rotate and the face does not rotate by matching with the multi-surface light reflector, so that the defects of slow rotation of the whole machine and long imaging time are overcome, the equipment cost is obviously reduced, and the equipment has the advantages of simple structure, convenience in operation, short shooting time and good user experience;
2. compared with the fixed single-lens scheme in the prior art, the method can generate the human face three-dimensional image in the same-size equipment, and the structural light camera is adopted, so that more accurate human face depth information can be obtained, and the single-lens scheme can only generate a two-dimensional human face image, so that the two human face three-dimensional image and the single-lens scheme have obvious difference;
3. compared with the scheme of multiple lenses, the method has the advantages that only one lens is provided, the cost is obviously lower, and the structure is simpler.
4. The two human face three-dimensional images with the minimum relative frontal face angle absolute value are the two human face three-dimensional images closest to the front face of the human face, and comprise lips, noses and eyes of the human face, the depth information has large fluctuation and is easy to register, so that the two human face three-dimensional images are used as registration starting points and can be conveniently positioned.
Further, the number of the optical reflectors is four, and the four optical reflectors correspond to-15 degrees, -45 degrees and 45 degrees of the opposite frontal face respectively, wherein the synthesizing step comprises:
s300, synthesizing point cloud images corresponding to the two human face three-dimensional images at the angles of-15 degrees and 15 degrees to obtain a first synthesized image;
s400, synthesizing a point cloud image corresponding to a human face three-dimensional image at-45 degrees or 45 degrees with the first synthetic image to obtain a second synthetic image;
s500, synthesizing the corresponding point cloud image in the remaining three-dimensional face image with a second synthetic image to obtain a complete 3D face image;
s600, rendering the complete face 3D image according to the color map to obtain a 3D face model.
Further, the step of synthesizing the image comprises:
s410, point cloud data output by the 3D structured light rotating module are obtained;
s420, performing coarse registration of the position and the rotation angle of a preset 3D structured light rotation module according to the point cloud data;
s430, performing fine registration according to the coincident points in the point cloud data;
s440, filtering coincident points to complete synthesis and splicing.
Through the steps, the two synthetic images can be spliced quickly to form a complete human face 3D model for subsequent rendering.
Further, the rendering step includes:
s450, performing three-dimensional reconstruction on the spliced point cloud data to produce a triangular mesh;
and S460, obtaining a mapping picture obtained by the RGB camera in the 3D structured light rotation module, and pasting the mapping picture on the grid of the reconstructed three-dimensional model to generate the 3D face model.
Through the steps, the face 3D model can be covered with the color two-dimensional photos to obtain the vivid 3D face model, the face model can be rotatably observed on computer equipment, doctors can observe the face model conveniently, and the height of the nose, the depression of eyes, the depth of wrinkles, the size of the mouth and the protruding condition can be clearly known.
The 3D structured light full-face imaging device comprises a head holding area, a 3D structured light rotating module and a plurality of optical reflectors; the 3D structured light rotating module is positioned in the center of the front of the head holding area and can rotate by taking the 3D structured light rotating module as an axis by being driven by the rotating shaft; the optical reflector is arranged around the 3D structured light rotating module; the number of the optical reflectors is four, and the optical reflectors are respectively correspondingly positioned at-45 degrees, -15 degrees, 15 degrees and 45 degrees of the human face in the head resting area. The device can quickly scan the face of the user, thereby conveniently forming the 3D face model of the user.
Furthermore, the optical reflector is divided into a first reflector, a second reflector, a third reflector and a fourth reflector, the connecting line of the 3D structured light rotating module and the head holding area is a 0-degree line, the included angle between the connecting line of the first reflector and the head holding area and the 0-degree line is 90 degrees, the included angle between the connecting line of the second reflector and the head holding area and the 0-degree line is 45 degrees, the third reflector and the second reflector are arranged in a mirror image mode with the 0-degree line, and the fourth reflector and the first reflector are arranged in a mirror image mode with the 0-degree line. According to the scheme, the position of each reflector can be quickly positioned, and the installation of each reflector is convenient. This scheme only need rotate the pivot in proper order can, pivot clockwise or anticlockwise rotation promptly, get back to initial position at last, rotate at every turn all accessible motor and fix a position, also can stop through opening of control motor and stop and shoot the operation, and control is convenient, realizes that the degree of difficulty is low.
Furthermore, the rotating shaft drives the 3D structured light rotating module to sequentially turn to the first reflecting mirror, the second reflecting mirror, the third reflecting mirror and the fourth reflecting mirror. This scheme only need rotate the pivot in proper order can, pivot clockwise or anticlockwise rotation promptly, get back to initial position at last, rotate at every turn all accessible motor and fix a position, also can stop through opening of control motor and stop and shoot the operation, and control is convenient, realizes that the degree of difficulty is low.
Furthermore, the three-dimensional light source further comprises a light shield and an illuminating lamp arranged in the light shield, and the light shield shields the head holding area, the 3D structured light rotating module and the plurality of optical reflectors. This scheme can remove external light source's influence from effectively, and the light that inside was opened can be opened according to the demand.
Furthermore, the head resting area is provided with a chin rest and a forehead rest, and the face of a person is fixed through the chin rest and the forehead rest. According to the scheme, the head of the user can be conveniently fixed, the use experience is good, the head of the user can be always located at the optimal height, and a complete human face model is generated.
Further, the reflectivity of each optical reflector is greater than or equal to 90%. The optical reflector with the reflectivity of more than 90% can provide better collected images for the 3D structured light rotating module, and the quality of the final human face model is remarkably improved.
Drawings
FIG. 1 is a schematic structural view of one embodiment of the apparatus of the present invention;
FIG. 2 is a first schematic view of the apparatus according to the present invention;
FIG. 3 is a second schematic view of the apparatus of the present invention in use;
FIG. 4 is a flow chart of one embodiment of the method of the present invention;
FIG. 5 is a point cloud image obtained at-15 degrees by the method of the present invention;
FIG. 6 is a point cloud image of 15 degrees obtained by the method of the present invention;
FIG. 7 is a point cloud image of 45 degrees obtained by the method of the present invention;
FIG. 8 is a point cloud image obtained at-45 degrees by the method of the present invention;
FIG. 9 is a schematic diagram of the present invention generating an image.
In the figure, 1, a head rest area; 2. a 3D structured light rotation module; 3. a chin rest; 4. a user; 5. a rotating shaft; 6. a motor; 7. a light shield; 8. an illuminating lamp; 9. a first reflector; 10. a second reflector; 11. a third reflector; 12. a fourth mirror; 13. forehead support.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
It will be understood by those skilled in the art that in the present disclosure, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in an orientation or positional relationship indicated in the drawings for ease of description and simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and thus, the above terms should not be construed as limiting the present invention.
For ease of understanding, the principle of 3D structured light is explained first: the basic principle of the 3D structured light technology is that light rays with certain structural characteristics are projected to a shot object through a near-infrared laser, and then collected through a special infrared camera. The light with a certain structure can acquire different image phase information according to different depth areas of a shot object, and then the change of the structure is converted into depth information through an arithmetic unit, so that a three-dimensional structure is obtained. In short, the three-dimensional structure of the object to be photographed is acquired by an optical means, and the acquired information is applied more deeply.
A depth image is also called a distance image, and is an image in which a distance (depth) value from an image pickup to each point in a scene is set as a pixel value.
Point cloud: when a laser beam irradiates the surface of an object, the reflected laser beam carries information such as direction, distance and the like. When the laser beam is scanned along a certain trajectory, the reflected laser spot information is recorded while scanning, and since the scanning is extremely fine, a large number of laser spots can be obtained, and a laser point cloud can be formed. The point cloud data is discrete data obtained by scanning and sampling the surface of a real object.
The depth image can be calculated into point cloud data through coordinate conversion; the point cloud data with regular and necessary information can be inversely calculated as a depth image. The two can be mutually converted under certain conditions.
As shown in fig. 1-3, the device used in the method includes a head resting area 1, a 3D structured light rotation module 2 and a plurality of optical reflectors, wherein a chin rest 3 matched with the chin of the human face and a forehead rest 13 used for abutting against the forehead of the human face are arranged at the head resting area 1, the head can be fixed by a support similar to the chin rest 3 abutting against the top of the head, soft pads such as latex pads can be installed on the chin rest 3 and can be replaced by itself, and the disinfection is facilitated, when the device is used, the chin rest 3 of a user 4 is on the chin rest 3, the 3D structured light rotation module 2 is installed on a rotating shaft 5, and the 3D structured light rotation module 2 at least includes an RGB camera and a structured light camera. The establishment of the human face model is realized through the RGB camera and the structured light camera.
Specifically, 3D structure light rotates module 2 and is located 1 the place ahead in head and shelve the district and just put the position, can rotate as the axle center by oneself through 5 drives of pivot, and pivot 5 is driven through common step motor 6 or servo motor 6, and motor 6 is through common computer equipment control, and 3D structure light rotates module 2 also through computer equipment control, generates the people's face model on computer equipment.
Specifically, the optical reflector is disposed around the 3D structured light rotation module 2 and used for reflecting light emitted from the 3D structured light rotation module 2, and the optical reflector is a commercially available product and is mounted on an external fixing bracket by means of screws, glue, and the like, and the fixing bracket is not shown in the drawing.
Preferably, the reflectivity of each of the optical mirrors is equal to or greater than 90%. The optical reflector with the reflectivity of more than 90% can provide better collected images for the 3D structured light rotating module 2, and the quality of the final human face model is remarkably improved.
In this embodiment, the head support device further comprises a light shield 7 and an illuminating lamp 8 arranged in the light shield 7, the head resting area 1, the 3D structured light rotating module 2 and the plurality of optical reflectors are shielded by the light shield 7, wherein the light shield 7 can be common black cloth and can be directly covered on the device or a cover made of other opaque materials, as shown in fig. 1, the device is made into a semi-closed structure, only an opening for the head of a user to enter is exposed, and a forehead support 13 for supporting the forehead of the user is arranged, and when the 3D structured light rotating module 2 shoots, the illuminating lamp 8 is closed. Mainly in order to avoid the lighting lamp 8 from influencing the shooting. This scheme can remove the influence of external light source from effectively, and the light 8 of inside opening can be opened according to the demand.
Preferably, the number of the optical reflectors is four, the optical reflectors are respectively corresponding to-45 degrees, -15 degrees, 15 degrees and 45 degrees of the face of the head resting area 1, the optical reflectors are divided into a first reflector 9, a second reflector 10, a third reflector 11 and a fourth reflector 12, a connecting line between the 3D structured light rotating module 2 and the head resting area 1 is a 0-degree line, an included angle between the connecting line between the first reflector 9 and the head resting area 1 and the 0-degree line is 90 degrees, an included angle between the connecting line between the second reflector 10 and the head resting area 1 and the 0-degree line is 45 degrees, the third reflector 11 and the second reflector 10 are arranged in a 0-degree line mirror image mode, and the fourth reflector 12 and the first reflector 9 are arranged in a 0-degree line mirror image mode. According to the scheme, the position of each reflector can be quickly positioned, and the installation of each reflector is convenient. According to the figure 3, the mounting angles corresponding to the reflectors are-90 degrees, -45 degrees, 45 degrees and 90 degrees, the angles of the motor 6 required to rotate are-45 degrees, -105 degrees, 105 degrees and 45 degrees in sequence, and when the motor 6 rotates to-45 degrees, the motor corresponds to the first reflector 9 of-90 degrees, and human face data of-45 degrees is obtained;
when the motor 6 rotates to-105 degrees, the second reflector 10 at-45 degrees is corresponded to obtain-15 degrees face data;
when the motor 6 rotates to 105 degrees, the third reflector 11 with 45 degrees is corresponded to, and 15-degree face data is obtained;
when the motor 6 rotates to 45 degrees, the fourth reflector 12 corresponds to 90 degrees, and 45-degree face data is obtained.
Therefore, when the three-dimensional display device is used, the rotating shaft 5 drives the 3D structured light rotating module 2 to sequentially turn to the first reflecting mirror 9, the second reflecting mirror 10, the third reflecting mirror 11 and the fourth reflecting mirror 12, the rotating shaft 5 is started to rotate to a-45-degree position, RGB light, PL and UV light of the 3D structured light rotating module 2 are sequentially turned on to shoot to obtain a group of pictures and corresponding depth data information, then the rotating shaft 5 is turned to-105 degrees, 105 degrees and 45-degree positions to respectively sequentially turn on the RGB light, PL and UV light to shoot to respectively obtain 3 different groups of pictures and corresponding depth data information, and the illuminating lamp 8 is turned off when shooting. This scheme only need rotate pivot 5 in proper order can, 5 clockwise or anticlockwise rotations of pivot promptly, get back to initial position at last, rotate at every turn and all can fix a position through motor 6, also can stop through opening of control motor 6 and stop and shoot the operation, and control is convenient, realizes that the degree of difficulty is low.
As shown in fig. 4, the method for synthesizing a 3D face model by using optical reflection includes the following steps:
s000, placing the face of the user on a head resting area 1 on the device, and starting the device;
s100, driving a 3D structured light rotating module 2 to rotate through a motor 6 to obtain face three-dimensional images and RGB color images which are-15 degrees, -45 degrees and 45 degrees relative to the front face from optical reflectors, wherein the number of the optical reflectors is four, and the optical reflectors correspond to the face three-dimensional image obtaining positions of all angles respectively;
compared with the prior art, need not to rotate whole equipment and come to shoot user's face, consequently, the drive mechanism that has huge structure has been saved, only need simple small and exquisite slewing mechanism rotate 3D structure light rotate module 2 can, cooperation multiaspect light speculum can realize shooing each angle of face under the equipment does not rotate and the non-pivoted condition of face, is showing and is reducing equipment cost, and has simple structure, convenient operation, the shooting time is short, user experience is good advantage.
Compared with the fixed single-lens scheme in the prior art, the method can generate the human face three-dimensional image in the equipment with the same size, and the single-lens scheme can only generate the two-dimensional human face image, so that the human face three-dimensional image and the single-lens scheme have obvious difference.
Compared with the scheme of multiple lenses, the method has the advantages that only one lens is provided, the cost is obviously lower, and the structure is simpler.
S200, generating a 3D point cloud image and a color map corresponding to each angle through the 3D structured light rotation module 2 and internal and external parameter parameters of the RGB color camera, wherein the generated algorithm is the prior art;
s300, synthesizing point cloud images corresponding to the two human face three-dimensional images at the angles of-15 degrees and 15 degrees to obtain a first synthesized image;
15-degree and 15-degree images contain lips, noses and eyes of the human face, the depth information is large in fluctuation and easy to register, and therefore the two images are used as registration starting points
S400, synthesizing point cloud images corresponding to the two human face three-dimensional images at the angle of minus 45 degrees to obtain a first synthesized image and a second synthesized image;
s500, synthesizing the point cloud images corresponding to the two human face three-dimensional images with the angle of 45 degrees with a second synthetic image to complete a human face 3D image;
s600, point cloud data output by the 3D structured light rotating module 2 are obtained, n down-sampling is carried out on the corresponding point cloud data, the number of points in the point cloud data is reduced, filtering operation is equivalent, and the synthesis speed is improved;
s610, performing preset coarse registration of the position and the rotation angle of the 3D structured light rotation module 2 according to the point cloud data, specifically, performing translation and rotation on the selected point cloud data according to the shooting distance and the shooting angle of the 3D structured light rotation module 2 to enable the selected point cloud data to be located at the spatial coordinate (0, 0) position, and completing the coarse registration;
s620, performing fine registration according to the coincident points in the point cloud data, specifically by setting parameters of the existing registration algorithm;
s630, filtering coincident points to complete synthesis and splicing;
s640, performing three-dimensional reconstruction on the spliced point cloud data to produce a triangular mesh;
s650, obtaining a mapping picture obtained by the RGB camera in the 3D structured light rotation module 2, and pasting the mapping picture on a grid of the reconstructed three-dimensional model to generate a 3D face model. The data that the structure light camera of rotating in the module 2 was obtained through 3D structure light, we can accurately know every point in the image and apart from the camera distance, add the (x, y) coordinate of this point in the 2D image of RGB camera like this, just can acquire the three-dimensional space coordinate of every point in the image, it can cover colored two-dimensional photo with people's face 3D model to correspond the net, obtain lifelike 3D face model, can rotate on the computer equipment and observe, very big convenient doctor observes, can clearly know the height of nose, the subsides of eyes, the depth of wrinkle, the size and the outstanding condition of mouth shape.
Referring to fig. 9, 1-4 in the drawings represent the images collected by the first mirror 9, the second mirror 10, the third mirror 11, and the fourth mirror 12, respectively, see fig. 5-8, and the middle of fig. 9 is a merged rendered image, the original image is a color image and has more facial details, and the generated face model can be used for analyzing the face skin, such as wrinkle depth, pox size, and the like.
The advantage that this scheme adopted structured light (speckle) mainly has:
1) the scheme is mature, the camera base line can be made smaller, and the miniaturization is facilitated.
2) The resource consumption is low, the depth map can be calculated by a single frame IR map, and the power consumption is low.
3) Active light source, can also be used at night.
4) The precision is high in a certain range, the resolution is high, the resolution can reach 1280x1024, and the frame rate can reach 60 FPS.
The present invention is not described in detail in the prior art, and therefore, the present invention is not described in detail.
It is understood that the terms "a" and "an" should be interpreted as meaning that a number of one element or element is one in one embodiment, while a number of other elements is one in another embodiment, and the terms "a" and "an" should not be interpreted as limiting the number.
Although the terms head rest 1, 3D structured light turning module 2, chin rest 3, user 4, spindle 5, motor 6, light shield 7, illumination lamp 8, first mirror 9, second mirror 10, third mirror 11, fourth mirror 12, forehead rest 13, etc. are used more herein, the possibility of using other terms is not excluded. These terms are used merely to more conveniently describe and explain the nature of the present invention; they are to be construed as being without limitation to any additional limitations that may be imposed by the spirit of the present invention.
The present invention is not limited to the above-mentioned preferred embodiments, and any other products in various forms can be obtained by anyone in the light of the present invention, but any changes in the shape or structure thereof, which have the same or similar technical solutions as those of the present application, fall within the protection scope of the present invention.
Claims (9)
1. A method for synthesizing a 3D face model by using optical reflection is characterized by comprising the following steps:
acquiring a plurality of human face three-dimensional images and RGB color images at a plurality of angles relative to a front face from an optical reflector by rotating a 3D structured light rotating module, wherein the optical reflector corresponds to the human face three-dimensional image acquisition positions at the angles respectively;
generating point cloud images and color maps corresponding to all angles according to the human face three-dimensional image and the RGB color image;
synthesizing corresponding point cloud images in two human face three-dimensional images with the smallest relative frontal face angle absolute value into a first synthesized image;
sequentially synthesizing the corresponding point cloud images in the human face three-dimensional images with the residual angles with the first synthetic image;
rendering the synthesized image according to the color map to obtain a 3D face model;
the number of the optical reflectors is four, and the optical reflectors correspond to-15 degrees, -45 degrees and 45 degrees of the opposite frontal face respectively, wherein the synthesizing step comprises the following steps:
synthesizing corresponding point cloud images in two human face three-dimensional images at the angles of-15 degrees and 15 degrees to obtain a first synthesized image;
synthesizing a point cloud image corresponding to a face three-dimensional image at-45 degrees or 45 degrees with the first synthetic image to obtain a second synthetic image;
synthesizing the corresponding point cloud image in the remaining three-dimensional face image with the second synthesized image to obtain a complete 3D face image;
and rendering the complete face 3D image according to the color map to obtain a 3D face model.
2. A method for synthesizing a 3D face model using optical reflection as claimed in claim 1, wherein the image synthesizing step comprises:
acquiring point cloud data output by the 3D structured light rotating module;
performing coarse registration of the position and the rotation angle of a preset 3D structured light rotation module according to the point cloud data;
performing fine registration according to the coincident points in the point cloud data;
and filtering the coincident points to complete synthesis and splicing.
3. The method of claim 2, wherein the rendering step comprises:
performing three-dimensional reconstruction on the spliced point cloud data to produce a triangular mesh;
and acquiring a mapping picture obtained by an RGB camera in the 3D structured light rotating module, and pasting the mapping picture on the grid of the reconstructed three-dimensional model to generate a 3D face model.
4. The method for synthesizing a 3D human face model by using optical reflection according to claim 1, further comprising a 3D structured light full-face imaging device, wherein the 3D structured light full-face imaging device comprises a head resting area, a 3D structured light rotating module and a plurality of optical reflectors; the 3D structured light rotating module is positioned in the center of the front of the head holding area and can rotate by taking the 3D structured light rotating module as an axis by being driven by the rotating shaft; the optical reflector is arranged around the 3D structured light rotating module.
5. The method for synthesizing a 3D human face model by using optical reflection as claimed in claim 4, wherein each optical reflector is vertically arranged and is divided into a first reflector, a second reflector, a third reflector and a fourth reflector, which are respectively correspondingly positioned at-45 degrees, -15 degrees, 15 degrees and 45 degrees of the human face in the head rest area; a connecting line between the center of the 3D structured light rotating module and the center of the head shelving area is taken as a 0-degree line, the first reflector and the fourth reflector are respectively positioned at the left front position and the right front position of the head shelving area, and the second reflector and the third reflector are respectively positioned at the left front position and the right front position of the 3D structured light rotating module; the third reflector and the second reflector are arranged in a mirror image mode by taking a 0-degree line as a central line and are mutually vertical, and the fourth reflector and the first reflector are arranged in a mirror image mode by taking the 0-degree line as the central line and are mutually parallel;
when the rotating shaft rotates to-45 degrees, the 3D structured light rotating module can face and correspond to the first reflector, and-45-degree face data are obtained from the first reflector;
when the rotating shaft rotates to-105 degrees, the 3D structured light rotating module can face and correspond to the second reflector, and-15-degree face data are obtained from the second reflector;
when the rotating shaft rotates to 105 degrees, the 3D structured light rotating module can face and correspond to the third reflector, and 15-degree face data are obtained from the third reflector;
when the rotating shaft rotates to 45 degrees, the 3D structured light rotating module can face and correspond to the fourth reflector, and 45-degree face data are obtained from the fourth reflector.
6. The method of claim 5, wherein the rotating shaft drives the 3D structured light rotating module to sequentially turn to the first mirror, the second mirror, the third mirror and the fourth mirror.
7. The method for synthesizing the 3D human face model by using the optical reflection as claimed in claim 6, further comprising a light shield and an illuminating lamp arranged in the light shield, wherein the light shield shields the head resting area, the 3D structure light rotation module and the plurality of optical reflectors.
8. A method for synthesizing a 3D face model by using optical reflection as claimed in any one of claims 5-7, wherein the head rest area is provided with a chin rest and a forehead rest, and the face is fixed by the chin rest and the forehead rest.
9. The method for synthesizing a 3D human face model by using optical reflection according to any one of claims 1-7, wherein the reflectivity of each optical mirror is greater than or equal to 90%.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110583944.6A CN113379893B (en) | 2021-05-27 | 2021-05-27 | Method for synthesizing 3D face model by utilizing optical reflection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110583944.6A CN113379893B (en) | 2021-05-27 | 2021-05-27 | Method for synthesizing 3D face model by utilizing optical reflection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113379893A CN113379893A (en) | 2021-09-10 |
| CN113379893B true CN113379893B (en) | 2022-02-11 |
Family
ID=77572463
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110583944.6A Active CN113379893B (en) | 2021-05-27 | 2021-05-27 | Method for synthesizing 3D face model by utilizing optical reflection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113379893B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024098321A1 (en) * | 2022-11-10 | 2024-05-16 | 杭州小肤科技有限公司 | 3d structured light full-face imaging apparatus combined with optical reflection imaging |
| WO2024098322A1 (en) * | 2022-11-10 | 2024-05-16 | 杭州小肤科技有限公司 | Method for synthesizing 3d human face model by using optical reflection |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108537191A (en) * | 2018-04-17 | 2018-09-14 | 广州云从信息科技有限公司 | A kind of three-dimensional face identification method based on structure light video camera head |
| CN109583304A (en) * | 2018-10-23 | 2019-04-05 | 宁波盈芯信息科技有限公司 | A kind of quick 3D face point cloud generation method and device based on structure optical mode group |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109345621A (en) * | 2018-08-28 | 2019-02-15 | 广州智美科技有限公司 | Interactive face three-dimensional modeling method and device |
| CN109242961B (en) * | 2018-09-26 | 2021-08-10 | 北京旷视科技有限公司 | Face modeling method and device, electronic equipment and computer readable medium |
| CN110986757B (en) * | 2019-10-08 | 2021-05-04 | 新拓三维技术(深圳)有限公司 | Three-dimensional human body scanning method, device and system |
-
2021
- 2021-05-27 CN CN202110583944.6A patent/CN113379893B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108537191A (en) * | 2018-04-17 | 2018-09-14 | 广州云从信息科技有限公司 | A kind of three-dimensional face identification method based on structure light video camera head |
| CN109583304A (en) * | 2018-10-23 | 2019-04-05 | 宁波盈芯信息科技有限公司 | A kind of quick 3D face point cloud generation method and device based on structure optical mode group |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113379893A (en) | 2021-09-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11852461B2 (en) | Generation of one or more edges of luminosity to form three-dimensional models of objects | |
| JP6544696B2 (en) | Dental computed tomography | |
| CN113379893B (en) | Method for synthesizing 3D face model by utilizing optical reflection | |
| US4922336A (en) | Three dimensional display system | |
| US6594539B1 (en) | Three-dimensional dental imaging method and apparatus having a reflective member | |
| KR101608848B1 (en) | System and method for generating a multi-dimensional image | |
| US7193773B2 (en) | Stereomicroscopy method and stereomicroscopy system | |
| CN109443199B (en) | 3D information measuring system based on intelligent light source | |
| KR20080055863A (en) | Capturing and processing facial motion data | |
| CN109394168B (en) | A kind of iris information measuring system based on light control | |
| JP2003202216A (en) | Method, device, system and program for three-dimensional image processing | |
| CN109151447A (en) | A kind of novel naked eye three-dimensional optical field imaging system | |
| CN103340606B (en) | Light switches human body three-dimensional scanning system and method for work thereof | |
| CN104374374B (en) | 3D environment dubbing system and 3D panoramas display method for drafting based on active panoramic vision | |
| CN113163188B (en) | 3D structured light full-face imaging device combined with optical reflection imaging | |
| US20190223718A1 (en) | Eye examining device | |
| US11579449B2 (en) | Systems and methods for providing mixed-reality experiences under low light conditions | |
| CN109394170B (en) | A kind of iris information measuring system of no-reflection | |
| WO2024098322A1 (en) | Method for synthesizing 3d human face model by using optical reflection | |
| WO2024098321A1 (en) | 3d structured light full-face imaging apparatus combined with optical reflection imaging | |
| CN211485040U (en) | Intraoral three-dimensional scanner | |
| CA2284299A1 (en) | Method and apparatus for a high resolution eye scanner | |
| CN209103318U (en) | A kind of iris shape measurement system based on illumination | |
| KR102091838B1 (en) | Realistic 3D stereoscopic image acquisition device that provides 3D stereoscopic images by acquiring photometric and moire images | |
| CN209203221U (en) | A kind of iris dimensions measuring system and information acquisition system based on light control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A Method for Synthesizing 3D Face Models Using Optical Reflection Effective date of registration: 20230627 Granted publication date: 20220211 Pledgee: Hangzhou United Rural Commercial Bank Co.,Ltd. Beishan Branch Pledgor: HANGZHOU XIAOFU TECHNOLOGY Co.,Ltd. Registration number: Y2023980045844 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20231129 Granted publication date: 20220211 Pledgee: Hangzhou United Rural Commercial Bank Co.,Ltd. Beishan Branch Pledgor: HANGZHOU XIAOFU TECHNOLOGY Co.,Ltd. Registration number: Y2023980045844 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right |