CN108596008B - Face shake compensation method for three-dimensional face measurement - Google Patents

Face shake compensation method for three-dimensional face measurement Download PDF

Info

Publication number
CN108596008B
CN108596008B CN201711317485.7A CN201711317485A CN108596008B CN 108596008 B CN108596008 B CN 108596008B CN 201711317485 A CN201711317485 A CN 201711317485A CN 108596008 B CN108596008 B CN 108596008B
Authority
CN
China
Prior art keywords
phase shift
delta
phase
face
light intensity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711317485.7A
Other languages
Chinese (zh)
Other versions
CN108596008A (en
Inventor
陈钱
冯世杰
左超
陶天阳
胡岩
张良
尹维
顾国华
张玉珍
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN201711317485.7A priority Critical patent/CN108596008B/en
Publication of CN108596008A publication Critical patent/CN108596008A/en
Application granted granted Critical
Publication of CN108596008B publication Critical patent/CN108596008B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a face shake compensation method aiming at three-dimensional face measurement, which comprises the steps of firstly adopting a three-step phase shift method to project three-step phase shift grating images to a measured object, shooting the projected three-step phase shift grating images through a camera, and solving an initial value of a phase shift amount caused by shaking by utilizing the relation among the three-step phase shift grating images; projecting and shooting an additional uniform brightness image to the measured object as reference background light intensity; and optimizing the initial phase shift quantity, enabling the ideal phase shift quantity to accurately reflect the ripple error caused by facial jitter, substituting the ideal phase shift quantity into a phase solving formula of a three-step phase shift method to obtain compensated face phase information, and finally obtaining an accurate three-dimensional face reconstruction result through phase and depth conversion. The invention has the function of motion error compensation and can compensate the measurement error caused by facial shake in the process of three-dimensional face recognition; the three-dimensional profile of the jittered object can be accurately measured by using a low-cost measuring system.

Description

Face shake compensation method for three-dimensional face measurement
Technical Field
The invention belongs to the technical field of optical measurement, and particularly relates to a face shake compensation method for three-dimensional face measurement.
Background
With the rapid development of computer technology and bioengineering technology in recent decades, the use of uniqueness of human biological features to identify the identity of each individual has gained widespread acceptance in international society. Compared with other biological characteristics, the face recognition method has the advantages of non-contact property, complete information, convenience in acquisition, man-machine friendliness and the like, and becomes one of the most ideal technical approaches in identity authentication and identity recognition application. For human beings, face recognition is the ability we have naturally, and each specific face recognition is also a subconscious, natural process. However, the inherent mechanism of face recognition is very complex, and it is a challenging problem to make a computer implement automatic face recognition. Although the traditional two-dimensional face recognition technology based on the apparent characteristics of the face image has been studied for more than ten years, the sensitivity of the two-dimensional face image to changes of illumination, dressing, posture and expression severely limits the accuracy, stability and reliability of the recognition result, and these factors also become the biggest obstacles for the further forward development of the two-dimensional face recognition technology.
In recent years, more and more people turn their eyes from the two-dimensional face recognition technology to the three-dimensional face recognition technology. The three-dimensional face recognition is expected to fundamentally solve the difficult problems of 'illumination, posture and expression' faced by the two-dimensional face recognition, thereby providing new potential and space for further improving the accuracy, stability and reliability of the recognition. However, to realize the three-dimensional face recognition technology, a first problem to be solved is how to obtain high-precision face three-dimensional data. For a three-dimensional contour measurement method, such as a fringe projection method, the method is widely applied to the fields of human body modeling, pattern recognition, industrial detection, reverse engineering and the like by virtue of the advantages of non-contact, high speed, good flexibility, high precision and the like. Therefore, the technology can provide accurate and reliable three-dimensional face data for three-dimensional face recognition.
In principle, fringe projection is achieved by replacing one camera in stereoscopic vision with a light source generator (xiaodan, cheng liang, dry river red. three-dimensional reconstruction based on digital raster phase shift [ J ] application of optoelectronics, 2011,26 (5): 17-20). The light source projects a series of coded images to the measured object to form active three-dimensional shape measurement. The coding pattern is modulated by the surface shape of the object to generate deformation, the structural light with the deformation is shot by the camera at another position, and the three-dimensional appearance of the object can be determined through the position relation between the camera projection light sources and the deformation degree of the structural light. In general, fringe projection is generally used only for three-dimensional measurement of static objects, since it is necessary to acquire a series of coded patterns reflected from the surface of the object to be measured. However, for human face scanning, it is difficult to avoid the problems of unintentional facial shaking and slight head shaking during data acquisition. The overall displacement or movement of the face caused by shaking or shaking will have a serious influence on the accuracy of the reconstructed three-dimensional face data. Although the measurement speed can be increased and the influence of jitter on the measurement can be indirectly reduced by increasing the projection speed (e.g., "Pulse-width modulation in focused three-dimensional project", author "gate", a. ayubi, etc.) or reducing the number of projection patterns (e.g., "Dual-frequency pattern for high-speed-space 3-D shape measurement", author k. liu, etc.), they still cannot completely solve this problem, and in particular, for low-cost three-dimensional measurement systems, the pattern projection apparatus and the pattern collection apparatus used cannot achieve high-speed projection and shooting of the code pattern. Therefore, for the three-dimensional face contour measurement, a face shake compensation technology which does not depend on the improvement of the system measurement speed is still lacked.
Disclosure of Invention
The invention aims to provide a face shake compensation method aiming at three-dimensional face measurement, which realizes error compensation of face shake without depending on system measurement speed.
The technical solution for realizing the purpose of the invention is as follows: a face shake compensation method aiming at three-dimensional face measurement comprises the following steps:
firstly, projecting three-step phase shift grating images to a measured object by adopting a three-step phase shift method, shooting the projected three-step phase shift grating images by a camera, and solving initial values of phase shift caused by jitter by utilizing the relation among the three-step phase shift grating images;
secondly, projecting and shooting an extra uniform brightness image to the measured object as reference background light intensity;
thirdly, optimizing the initial phase shift quantity, namely setting a value range for the phase shift quantity, and traversing the respective value ranges by the phase shift quantity with a fixed step length; calculating corresponding background light intensity for the combination of each group of phase shift quantity, and solving the difference between the reference background light intensity and the calculated background light intensity; for all combinations of phase shift quantities, when the corresponding background light intensity difference is minimum, the phase shift quantity contained in the combination is an ideal phase shift quantity, the ideal phase shift quantity can accurately reflect the ripple error caused by facial jitter, the ripple error is brought into a phase solving formula of a three-step phase shift method, compensated face phase information can be obtained, and an accurate three-dimensional face reconstruction result can be finally obtained through phase and depth conversion.
Compared with the prior art, the invention has the following remarkable advantages: (1) compared with the most common three-step phase shift method in dynamic three-dimensional measurement, the method has the function of motion error compensation, and can compensate the measurement error caused by facial shake in the process of three-dimensional face recognition. (2) Compared with the traditional high-speed fringe projection method and the fringe projection quantity reducing method, the method does not need expensive high-speed image acquisition and projection equipment, and can accurately measure the three-dimensional profile of the jittered object under the condition of using a low-cost measuring system. (3) Compared with the traditional jitter error compensation method (document of Fast 3d scanning with automatic motion compensation, author T.Weise, etc.), the invention has wider application range, and can not only deal with the problem of uniform jitter of the measured object, but also deal with the problem of non-uniform jitter.
The present invention is described in further detail below with reference to the attached drawing figures.
Drawings
Fig. 1 is a flowchart of a facial shake compensation method for three-dimensional face measurement according to the present invention.
Fig. 2 is a schematic diagram of a three-dimensional human face measuring device based on a fringe projection method.
Fig. 3 is an experimental diagram of a first test object: (a) the detected face is the detected face; (b) the face is jittered during the measurement process.
FIG. 4 is a three-step phase-shifted intensity map taken with a uniform brightness image: (a) is l'1(ii) a (b) Is l'2(ii) a (c) Is l'3(ii) a (d) Is Aref。
Fig. 5 is a comparison graph of three-dimensional reconstruction results of a first measured object: (a) side view: reconstructing a three-dimensional face before shake compensation; (b) side view: reconstructing the three-dimensional human face after the shake compensation; (c) front view: reconstructing a three-dimensional face before shake compensation; (d) front view: and (5) reconstructing the three-dimensional human face after the shake compensation.
Fig. 6 is a second experimental plot of the test subjects: (a) the detected face is the detected face; (b) the face is jittered during the measurement process.
Fig. 7 is a comparison of the three-dimensional reconstruction results for a second object under test: (a) side view: reconstructing a three-dimensional face before shake compensation; (b) side view: reconstructing the three-dimensional human face after the shake compensation; (c) front view: reconstructing a three-dimensional face before shake compensation; (d) front view: and (5) reconstructing the three-dimensional human face after the shake compensation.
Detailed Description
The invention relates to a face shake compensation method for three-dimensional face measurement, which mainly solves the problem of reconstructed surface ripple error caused by face shake, and comprises the steps of firstly adopting a three-step phase shift method, projecting three-step phase shift grating images to a measured object, shooting the projected three-step phase shift grating images through a camera, and solving an initial value of a phase shift amount caused by shake by utilizing the relationship among the three-step phase shift grating images;
secondly, projecting and shooting an extra uniform brightness image to the measured object as reference background light intensity;
thirdly, optimizing the initial phase shift quantity, namely setting a value range for the phase shift quantity, and traversing the respective value ranges by the phase shift quantity with a fixed step length; calculating corresponding background light intensity for the combination of each group of phase shift quantity, and solving the difference between the reference background light intensity and the calculated background light intensity; for all combinations of phase shift quantities, when the corresponding background light intensity difference is minimum, the phase shift quantity contained in the combination is an ideal phase shift quantity, the ideal phase shift quantity can accurately reflect the ripple error caused by facial jitter, the ripple error is brought into a phase solving formula of a three-step phase shift method, compensated face phase information can be obtained, and an accurate three-dimensional face reconstruction result can be finally obtained through phase and depth conversion.
The method comprises the following specific steps:
the method comprises the following steps: and solving an initial value of the phase shift quantity. Firstly, a three-dimensional face measurement device based on fringe projection is constructed, and the principle is shown in fig. 2. The basic process is as follows: the projector projects a three-step phase shift grating image to the measured object. The camera takes a three-step phase-shifted grating image reflected from the object under test from another angle. And transmitting the acquired image to a computer for image processing, namely calculating an initial value of the phase shift amount. Projected three-step phase-shifted grating intensity map I1，I2， I3Comprises the following steps:
Figure BDA0001504046150000041
Figure BDA0001504046150000042
Figure BDA0001504046150000043
wherein (x)p,yp) Representing the projector pixel coordinates, ApRepresents a direct current component, BpDenotes the amplitude, fpRepresenting the spatial frequency, w, of the projection gratingpIndicating the lateral resolution of the projector. Generating a three-step phase-shift grating light intensity diagram I1，I2，I3Sequentially projecting to the measured object, shooting the images by a camera, transmitting the images to a computer, and shooting the obtained light intensity (I ') of the raster image'1，I′2，I′3) Can be expressed as
I′1(x,y)＝A(x,y)+B(x,y)cos[φ(x,y)]
I′2(x,y)＝A(x,y)+B(x,y)cos[φ(x,y)-δ′2]
I′3(x,y)＝A(x,y)+B(x,y)cos[φ(x,y)-δ′2-Δδ′]
Wherein (x, y) is the camera pixel coordinate, A isThe background light intensity, B is the light intensity modulation degree, and phi is the phase to be solved. Delta 'of'2And Δ δ' is the unknown amount of phase shift caused by facial jitter during the measurement. The known quantity in the set of equations is l'1、I′2And l'3To solve for the phase shift quantity δ 'from the system of equations'2With Δ δ', the jitter-induced error is compensated for, so the amount of phase shift is solved using the following relationship:
Figure BDA0001504046150000051
Figure BDA0001504046150000052
Figure BDA0001504046150000053
where i, j, k are intermediate variables. The operation symbol | represents an absolute value operation<·>Indicating that the average light intensity value of the whole image is calculated. For each item in i, j, k<|2Bsin(·)|>Since the phase phi has a periodicity of-pi to pi,
Figure BDA0001504046150000054
and
Figure BDA0001504046150000055
can be seen as different starting points for a periodic signal. Thus, from the trigonometric relationship, it is possible to infer
Figure BDA0001504046150000056
Figure BDA0001504046150000057
And
Figure BDA0001504046150000058
approximately equal for the entire image. Therefore, each item can be uniformly represented by a variable c<|2Bsin(·)|>So that the following simplified relationship can be obtained
Figure BDA00015040461500000510
Figure BDA00015040461500000511
Figure BDA00015040461500000512
Using the above three equations, the variable c can be calculated:
Figure BDA00015040461500000513
therefore δ 'can be solved using the following relationship'2And Δ δ':
Figure BDA0001504046150000061
Figure BDA0001504046150000062
amount of phase Shift δ 'obtained at this time'2And Δ δ' are initial values and are respectively recorded as constants
Figure BDA0001504046150000063
And
Figure BDA0001504046150000064
step two: a uniform light intensity map is projected onto the measured object. Projecting a pattern with uniform brightness to an object to be measured by using a projector, wherein the generation method comprises the following steps:
I4(xp,yp)＝Ap(xp,yp)
the scene at this time is photographed with a camera and the image is transmitted to a computer. This image is the fourth image captured by the camera and is denoted as Aref。
Step three: iterative optimization of the initial phase Shift quantity δ'2And Δ δ'. Firstly determining a phase shift quantity delta'2And the iterative optimization range of delta'. Is delta'2Is in the value range of
Figure BDA0001504046150000065
Delta delta' has a value range of
Figure BDA0001504046150000066
And alpha is ∈ [0,1 ]]。δ′2The values of the delta 'and the delta' are respectively taken from the lower limit values of the respective definition domains, and then the values are sequentially increased, wherein the increasing step length is 0.0001 radian until the values are equal to the upper limit values of the respective definition domains. δ 'for each group'2And delta' are calculated, and the corresponding phase phi is calculated by
Figure BDA0001504046150000067
Wherein h (x, y) ═ I'2(x,y)-I′1(x,y)]/[I′3(x,y)-I′1(x,y)]。
Then, utilizing expression I'1，I′2，I′3And the variable phi, delta 'known at this time'2Δ δ', the unknown quantity a is solved by the least squares method (document "Prediction and regulation by linear least-square methods", author p. Subsequently, the difference degree delta of background light intensity is usedAFormula for calculation
Figure BDA0001504046150000068
Solving the A obtained by calculation and the A obtained by shootingrefThe degree of difference between them. Where Σ is the sum of the intensities of all pixels in the entire image. Due to the utilization of each group delta'2And delta' can be calculated to obtain a deltaA. When the degree of difference ΔAWhen the minimum value is obtained, the corresponding phase shift amount
Figure BDA0001504046150000069
And
Figure BDA00015040461500000610
is the ideal phase shift amount after optimization.
Step four: and compensating the ripple error, and calculating the three-dimensional data of the human face. Using the desired amount of phase shift
Figure BDA0001504046150000071
And
Figure BDA0001504046150000072
the phase after ripple compensation is calculated by the following formula
Figure BDA0001504046150000073
Figure BDA0001504046150000074
Then, the phase is expanded by multi-frequency time domain phase expansion method
Figure BDA0001504046150000075
Performing time phase unwrapping to obtain unwrapped phases:
Figure BDA0001504046150000076
wherein phicTo assist the grating phase, fcNINT represents the rounding function for the spatial frequency of the auxiliary grating. For the multi-frequency time domain phase expansion method, reference is made to "Temporal phase unwarping algorithms for fringe projection profile," author c. Finally, the space position relation between the camera and the projector and respective calibration parameters are utilized to remove the packageConverting the phase into space three-dimensional coordinates to obtain the three-dimensional coordinates (X, Y, Z) of the human faceTThe conversion method is to combine the following two equations:
u(x,y,1)T＝Pc(X,Y,Z,1)T
v(xp,yp,1)T＝Pp(Φ)(X,Y,Z,1)T
wherein u and v are respectively space points (X, Y and Z)TProjection scalar in the process of projecting to the plane of the camera and the projector. PcRepresenting a camera matrix, Pp(Φ) represents the projector matrix whose matrix parameters are determined by the unwrapping phase Φ. For the three-dimensional coordinate calculation method, reference is made to "High-speed three-dimensional profiling for multiple objects with complex profiles", author c.
Examples
In order to verify the effectiveness of the method, a set of three-dimensional face measuring device based on the stripe projection method is constructed by using a camera (model acA640-750, Basler), a projector (model Lightcraft 4500, TI) and a computer. The shooting speed of the device is 20 frames per second when the three-dimensional contour of the human face is measured. Fig. 3 (a) shows a first measurement object, and fig. 3 (b) shows a face displacement obtained by the Lucas-Kanade method (reference "Performance of optical flow technologies, International Journal of Computer Vision", author j.l. barron, etc.). It can be seen that there is a slight shaking of the object to the left during the measurement. A group of three-step phase shift grating patterns is generated by using the method for generating the three-step phase shift pattern in the first step of the invention, wherein the parameters are set as Ap(xp,yp)＝Bp(xp,yp)＝127.5，fpIs taken as 80, wpWas taken as 912. The three patterns are projected to a measured object, and an image I 'is acquired by a camera'1，I′2，I′3As shown in fig. 4 (a) to 4 (c). Utilizing the phase shift quantity delta 'of the step one'2Calculating to obtain an initial value with the initial value of delta
Figure BDA0001504046150000081
Arc degree and
Figure BDA0001504046150000082
and (4) radian. Generating a uniform pattern by the uniform pattern generation method in the step two, wherein the parameter is set as Ap(xp,yp) The image A is collected by projecting the pattern to the measured object as 127.5refAs shown in fig. 4 (d). D ', by using the iterative optimization method in the step three'2Optimized for Δ δ'. Since α is 0.5, the ranges of α and δ are 2.05 to 0.5 ≦ δ'2Is less than or equal to 2.05 plus 0.5 and is less than or equal to 2.0558-0.5, and delta' is less than or equal to 2.0558 plus 0.5. Delta 'was set to 0.0001 radian'2Traverse all possible values within the range of values with delta'. δ 'for each group'2Corresponding background light intensity a is calculated, corresponding to delta'. Combined captured image arefCalculating the degree of difference Δ of background light intensityA. The results show that
Figure BDA0001504046150000083
Arc degree and
Figure BDA0001504046150000084
in radian,. DELTA.AThe value is the smallest. Thus, an ideal amount of phase shift is determined
Figure BDA0001504046150000085
Arc degree and
Figure BDA0001504046150000086
and (4) radian. And finally, performing ripple error compensation by using the method in the fourth step, and calculating three-dimensional data of the human face, as shown in (b) and (d) in fig. 5. To compare the effectiveness of the method of the present invention, fig. 5 (a) and (c) show three-dimensional face data obtained before the shake compensation, in which significant moire errors are present, can be easily found. When the method of the invention is used, these jitter errors can be removed altogether. FIG. 6 (a) shows the second object, and FIG. 6 (b) shows the second objectFacial displacement obtained by the Lucas-Kanade method. It can be seen that the human face has a problem of shaking to the right during the measurement process. Fig. 7 (a) and (c) show three-dimensional face reconstruction results before shake compensation, and severe ripples appear in the face reconstruction results because face shakes are not supplemented. When the method of the present invention was used, the results were shown in (b) and (d) of FIG. 7. It can be seen that the method of the invention has the advantages that the reconstruction of the three-dimensional face is accurate, and the shaking errors are completely eliminated.

Claims (4)

1. A face shake compensation method aiming at three-dimensional face measurement is characterized by comprising the following steps:
firstly, projecting three-step phase shift grating images to a measured object by adopting a three-step phase shift method, shooting the projected three-step phase shift grating images by a camera, and solving initial values of phase shift caused by jitter by utilizing the relation among the three-step phase shift grating images;
secondly, projecting and shooting an extra uniform brightness image to the measured object as reference background light intensity;
thirdly, optimizing the initial phase shift quantity, namely setting a value range for the phase shift quantity, and traversing the respective value ranges by the phase shift quantity with a fixed step length; calculating corresponding background light intensity for the combination of each group of phase shift quantity, and solving the difference between the reference background light intensity and the calculated background light intensity; for all combinations of phase shift quantities, when the corresponding background light intensity difference is minimum, the phase shift quantity contained in the combination is an ideal phase shift quantity, the ideal phase shift quantity can accurately reflect the ripple error caused by facial jitter, the ripple error is brought into a phase solving formula of a three-step phase shift method, compensated face phase information can be obtained, and an accurate three-dimensional face reconstruction result can be finally obtained through phase and depth conversion;
the initial value process for solving the phase shift amount is as follows: the three-dimensional face measurement device based on fringe projection is constructed, namely, a projector projects three-step phase shift grating images to a measured object, a camera shoots the three-step phase shift grating images reflected by the measured object from another angle, and the collected images are transmitted to a computer for image processing, namelySolving the initial value of the phase shift quantity, wherein the projected three-step phase shift grating light intensity diagram I1，I2，I3Comprises the following steps:
Figure FDA0003186852910000011
Figure FDA0003186852910000012
Figure FDA0003186852910000013
wherein (x)p，yp) Representing the projector pixel coordinates, ApRepresents a direct current component, BpDenotes the amplitude, fpRepresenting the spatial frequency, w, of the projection gratingpRepresenting the transverse resolution of the projector, and generating a three-step phase-shift grating light intensity diagram I1，I2，I3Sequentially projecting to the measured object, shooting the images by a camera, transmitting the images to a computer, and shooting the obtained light intensity (I ') of the raster image'1，I′2，I′3) Is shown as
I′1(x，y)＝A(x，y)+B(x，y)cos[φ(x，y)]
I′2(x，y)＝A(x，y)+B(x，y)cos[φ(x，y)-δ′2]
I′3(x，y)＝A(x，y)+B(x，y)cos[φ(x，y)-δ′2-Δδ′]
Wherein, (x, y) is the pixel coordinate of the camera, A is the background light intensity, B is the light intensity modulation degree, phi is the phase to be solved; delta 'of'2And Δ δ 'is the unknown amount of phase shift caused by facial jitter during measurement, the known amount in the system of equations is I'1、I′2And l'3To solve for the phase shift quantity δ 'from the system of equations'2Delta delta ' is compensated for the jitter-induced error, and delta ' is solved by the following relationship '2And Δ δ':
Figure FDA0003186852910000021
Figure FDA0003186852910000022
Figure FDA0003186852910000023
wherein i, j, k is intermediate variable, the operation symbol | represents absolute value operation, the operation symbol < > represents the average light intensity value of the whole image, and the variable c is used to represent the value of each item < |2Bsin (·) |), thereby obtaining the following simplified relation
Figure FDA0003186852910000024
Figure FDA0003186852910000025
Figure FDA0003186852910000026
Calculating the variable c:
Figure FDA0003186852910000027
therefore, δ 'is solved by the following relationship'2And Δ δ':
Figure FDA0003186852910000028
Figure FDA0003186852910000029
amount of phase Shift δ 'obtained at this time'2And Δ δ' are initial values, respectively
Figure FDA0003186852910000031
And
Figure FDA0003186852910000032
2. the method of claim 1, wherein the projector is used to project a pattern with uniform brightness to the measured object, and the generation process is as follows:
I4(xp，yp)＝Ap(xp，yp)
shooting the scene by using a camera, transmitting the image to a computer, and recording the image acquired by the camera as Aref。
3. Face jitter compensation method for three-dimensional face measurement according to claim 1, characterized in that the initial phase shift amount δ 'is iteratively optimized'2And Δ δ': firstly determining a phase shift quantity delta'2Iterative optimization Range with Delta 'let delta'2Is in the value range of
Figure FDA0003186852910000033
Delta delta' has a value range of
Figure FDA0003186852910000034
And alpha is ∈ [0,1 ]]，δ′2The values of delta 'and delta' are respectively taken from the lower limit of the respective definition domain; then, the values are sequentially increased, and the increasing step length is 0.0001 radian until the values are equalUpper limit values for the respective domain; δ 'for each group'2In combination with delta ', and calculating the phase phi corresponding to the delta', and the calculation method is
Figure FDA0003186852910000035
Wherein h (x, y) ═ I'2(x，y)-I′1(x，y)]/[I′3(x，y)-I′1(x，y)]；
Finally, utilizing expression I'1、I′2、I′3And the variables φ, δ 'known at this time'2Delta', solving the unknown quantity A by a least square method and utilizing the difference degree delta of the background light intensityAFormula for calculation
Figure FDA0003186852910000036
Solving the A obtained by calculation and the A obtained by shootingrefThe degree of difference between; wherein sigma is the sum of the light intensities of all pixels of the whole image; due to the utilization of each group delta'2And delta' can be calculated to obtain a deltaAWhen the difference is ΔAWhen the minimum value is obtained, the corresponding phase shift amount
Figure FDA0003186852910000037
And
Figure FDA00031868529100000313
is the ideal phase shift amount after optimization.
4. The face shake compensation method for three-dimensional face measurement according to claim 3, wherein the ripple error is compensated, and the three-dimensional data of the face is calculated by using an ideal amount of phase shift
Figure FDA0003186852910000039
And
Figure FDA00031868529100000310
the phase after ripple compensation is calculated by the following formula
Figure FDA00031868529100000311
Figure FDA00031868529100000312
Then, the phase is expanded by using a multi-frequency time domain phase expansion method
Figure FDA0003186852910000041
Performing time phase unwrapping to obtain unwrapped phases:
Figure FDA0003186852910000042
wherein phicTo assist the grating phase, fcNINT represents the rounding function for the spatial frequency of the auxiliary grating;
finally, the unwrapping phase is converted into a space three-dimensional coordinate by utilizing the space position relation between the camera and the projector and respective calibration parameters to obtain a face three-dimensional coordinate (X, Y, Z)TThe conversion method is to combine the following two equations:
u(x，y，1)T＝Pc(X，Y，Z，1)T
v(xp，yp，1)T＝Pp(Φ)(X，Y，Z，1)T
wherein u and v are respectively space points (X, Y and Z)TProjection scalar in projection onto camera and projector planes, PcRepresenting a camera matrix, Pp(Φ) represents a projector matrix.
CN201711317485.7A 2017-12-12 2017-12-12 Face shake compensation method for three-dimensional face measurement Active CN108596008B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711317485.7A CN108596008B (en) 2017-12-12 2017-12-12 Face shake compensation method for three-dimensional face measurement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711317485.7A CN108596008B (en) 2017-12-12 2017-12-12 Face shake compensation method for three-dimensional face measurement

Publications (2)

Publication Number Publication Date
CN108596008A CN108596008A (en) 2018-09-28
CN108596008B true CN108596008B (en) 2021-11-30

Family

ID=63633341

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711317485.7A Active CN108596008B (en) 2017-12-12 2017-12-12 Face shake compensation method for three-dimensional face measurement

Country Status (1)

Country Link
CN (1) CN108596008B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109253708B (en) 2018-09-29 2020-09-11 南京理工大学 Stripe projection time phase unwrapping method based on deep learning
CN109579741B (en) * 2018-11-01 2020-06-26 南京理工大学 Full-automatic multi-mode three-dimensional color measurement method based on multiple visual angles
CN111667441A (en) * 2020-05-15 2020-09-15 成都飞机工业（集团）有限责任公司 Multi-frequency phase shift image sequence vibration detection method
CN111629011B (en) * 2020-07-28 2020-11-13 深圳诚一信科技有限公司 Instant video communication method, equipment, system and readable storage medium
CN112508778B (en) * 2020-12-18 2024-04-12 咪咕文化科技有限公司 3D face prop mapping method, terminal and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102175179A (en) * 2011-02-23 2011-09-07 东南大学 Method and device for three-dimensionally reestablishing surface contour of human body
US20110298891A1 (en) * 2010-06-04 2011-12-08 Iowa State University Research Foundation, Inc. Composite phase-shifting algorithm for 3-d shape compression
CN105403172A (en) * 2015-10-27 2016-03-16 华侨大学 Regional Gamma precorrection phase error compensation method in large view field structured light vision measurement
CN105806259A (en) * 2016-04-29 2016-07-27 东南大学 Three-dimensional measuring method based on binary grating out-of-focus projection

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110298891A1 (en) * 2010-06-04 2011-12-08 Iowa State University Research Foundation, Inc. Composite phase-shifting algorithm for 3-d shape compression
CN102175179A (en) * 2011-02-23 2011-09-07 东南大学 Method and device for three-dimensionally reestablishing surface contour of human body
CN105403172A (en) * 2015-10-27 2016-03-16 华侨大学 Regional Gamma precorrection phase error compensation method in large view field structured light vision measurement
CN105806259A (en) * 2016-04-29 2016-07-27 东南大学 Three-dimensional measuring method based on binary grating out-of-focus projection

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Real-time 3-D shape measurement with composite phase-shifting fringes and multi-view system;TIANYANG TAO 等;《OPTICS EXPRESS》;20160229;第24卷(第18期);正文第2，5-13页 *
应用三步相移法的三维面形测量;马晓航 等;《光电技术应用》;20140831;第29卷(第4期);第65-68页 *

Also Published As

Publication number Publication date
CN108596008A (en) 2018-09-28

Similar Documents

Publication Publication Date Title
CN108596008B (en) Face shake compensation method for three-dimensional face measurement
CN109506589B (en) Three-dimensional profile measuring method based on structural light field imaging
Zhang Absolute phase retrieval methods for digital fringe projection profilometry: A review
Zhang High-speed 3D shape measurement with structured light methods: A review
Feng et al. Robust dynamic 3-D measurements with motion-compensated phase-shifting profilometry
Zuo et al. Phase shifting algorithms for fringe projection profilometry: A review
CN110288642B (en) Three-dimensional object rapid reconstruction method based on camera array
CN111351450B (en) Single-frame stripe image three-dimensional measurement method based on deep learning
Liu et al. Real-time 3D surface-shape measurement using background-modulated modified Fourier transform profilometry with geometry-constraint
CN105844633B (en) Single frames structure optical depth acquisition methods based on De sequence and phase code
CN109945802B (en) Structured light three-dimensional measurement method
CN107967697B (en) Three-dimensional measurement method and system based on color random binary coding structure illumination
CN109631798B (en) Three-dimensional surface shape vertical measurement method based on pi phase shift method
CN107990846B (en) Active and passive combination depth information acquisition method based on single-frame structured light
CN113506348B (en) Gray code-assisted three-dimensional coordinate calculation method
Deng et al. High-speed and high-accuracy fringe projection profilometry without phase unwrapping
Zeng et al. Self-unwrapping phase-shifting for fast and accurate 3-D shape measurement
Sui et al. Active stereo 3-d surface reconstruction using multistep matching
Liu et al. A novel phase unwrapping method for binocular structured light 3D reconstruction based on deep learning
CN113008163A (en) Encoding and decoding method based on frequency shift stripes in structured light three-dimensional reconstruction system
Fernandez et al. Absolute phase mapping for one-shot dense pattern projection
CN114252026B (en) Three-dimensional measurement method and system for modulating three-dimensional code on periodic edge
Liu et al. Investigation of phase pattern modulation for digital fringe projection profilometry
CN111815697B (en) Thermal deformation dynamic three-dimensional measurement method
CN113450460A (en) Phase-expansion-free three-dimensional face reconstruction method and system based on face shape space distribution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant