CN113251952B - Three-dimensional measurement system and three-dimensional measurement method for grating translation structured light - Google Patents
Three-dimensional measurement system and three-dimensional measurement method for grating translation structured light Download PDFInfo
- Publication number
- CN113251952B CN113251952B CN202110534572.8A CN202110534572A CN113251952B CN 113251952 B CN113251952 B CN 113251952B CN 202110534572 A CN202110534572 A CN 202110534572A CN 113251952 B CN113251952 B CN 113251952B
- Authority
- CN
- China
- Prior art keywords
- structured light
- grating
- phase
- grating sheet
- dimensional measurement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/2545—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with one projection direction and several detection directions, e.g. stereo
Abstract
The invention relates to the field of optical detection three-dimensional imaging, in particular to a grating translation structured light three-dimensional measurement system and a three-dimensional measurement method. The projection structure of the invention is different from the traditional DMD projection device of an optical machine, and is a simple and easily-controlled optical phase modulation projection device with a spatial structure. The function which can be completed by a complex electronic system can be realized only by accurate mechanical displacement of the grating without depending on a complex stripe display control program, the complexity of the system is greatly simplified, and the volume and the cost of the system are greatly reduced while the measurement precision is ensured.
Description
Technical Field
The invention relates to the field of optical detection three-dimensional imaging, in particular to a grating translation structured light three-dimensional measurement system and a three-dimensional measurement method.
Background
The three-dimensional face recognition has a very wide application prospect in the fields of public safety, finance, transportation and other civilians due to the non-contact, high recognition rate and high anti-counterfeiting performance, and is one of important core technologies for building an intelligent public service platform, a smart city and a peaceful city. As is well known, the premise of the rapid development of three-dimensional face recognition is to provide a three-dimensional face acquisition device with high integration, high speed and high precision.
At present, the main structured light projection device has a two-dimensional coding mode of surface projection and a random speckle mode, and sine structured light is mainly projected by controlling a galvanometer scanning or digitally modulated by an optical projector and then realized by a complex light projection system. The images after the projected structured light is modulated by the measured target are synchronously collected, and the images are subjected to transformation analysis, calculation measurement, three-dimensional reconstruction and the like.
The existing high-precision three-dimensional reconstruction method is mainly based on structured light three-dimensional measurement profilometry, mainly takes the face as a common measurement object based on the principles of active structured light and triangulation, and combines with the precise mapping of a 2D color texture image, so that a vivid 3D face can be obtained at high precision. Among them, the most promising to be considered as the development advantage and market prospect is Fringe Projection Profilometry (FPP), which can be used to measure dynamic objects. Two schemes are most commonly used in FPP, one is Fourier Transform Profilometry (FTP), which is a representative single frame scheme that requires only one frame of stripes to complete a three-dimensional reconstruction, and is well suited for high-speed 3D measurements. However, due to the limitations of band pass filtering and the cautious nature of parameter selection, the FTP approach is more challenging to implement in automated processing in complex dynamic scenes where object shape changes over time. The other is Phase Shift Profilometry (PSP), which is known for its higher accuracy, greater resolution, lower complexity and insensitivity to ambient light compared to FTP. Therefore, the PSP is more suitable for acquiring high-speed and high-precision three-dimensional face data.
However, no matter structured light or binocular stereoscopic vision, in the existing three-dimensional face acquisition and identification system, the structured light projection device has a series of problems of complex system, high control difficulty, high manufacturing cost, difficulty in considering practical engineering and the like in the aspects of system implementation and control mode.
Most of the existing projection devices for three-dimensional face measurement are Digital Light Processing (DLP) projectors or galvanometers. DLP has high resolution, strong contrast, but large volume, which is not beneficial to miniaturization (such as high-precision three-dimensional full-face camera developed by Sichuan large-intelligence-win software corporation); although the galvanometer is small in size, the galvanometer is limited by output power, low in brightness and poor in contrast, so that the measurement precision is low (such as RealSense of Inter and Astra system of optical system in Australia); more importantly, the cost of the two is high, which is not beneficial to large-scale production of engineering. Therefore, the development of a simple, easily-controlled, low-cost and high-speed projected space structure light projection device has important significance for building a high-speed and high-precision three-dimensional human face model.
Therefore, a grating translation structured light three-dimensional measurement system and a three-dimensional measurement method which can reduce the integration volume, reduce the cost and realize high-speed acquisition while ensuring the measurement accuracy are needed nowadays.
Disclosure of Invention
The invention aims to overcome the defect that a high-precision three-dimensional reconstruction system with small size and low cost does not exist in the prior art, and provides a grating translation structure light three-dimensional measurement system and a three-dimensional measurement method.
In order to achieve the above purpose, the invention provides the following technical scheme:
a grating translation structured light three-dimensional measurement system comprises a sine structured light projection module, a binocular acquisition module and a controller,
the binocular acquisition module comprises two IR acquisition cameras and a texture RGB camera; the two IR collecting cameras are respectively arranged on two sides of the system, and the texture RGB camera and the sinusoidal structured light projection module are arranged between the two IR collecting cameras; the texture RGB camera is used for acquiring comprehensive color information and high-frequency details of a target object;
the sine structured light projection module comprises an illumination light source, a translational grating sheet, a micro motor, a displacement detector and a projection lens; the illumination light source is used for providing illumination for the translational grating sheet; the translational grating sheet is arranged in the projection direction of the illumination light source; the micro motor is used for driving the translational grating sheet to translate along a straight line; the displacement detector is used for acquiring displacement data of the translational grating sheet; the projection lens is used for projecting structured light;
the controller is respectively electrically connected with the binocular acquisition module and the sinusoidal structured light projection module. The invention adopts a movable grating illumination projection, namely, a translational grating sheet for determining the fringe frequency is adopted for illumination projection, in the measuring process, the measurement can be carried out by translating for a certain period along the direction vertical to the grating and simultaneously projecting a snapshot modulation pattern, and grating sheet elements with different spatial frequencies can be replaced according to the measurement requirement. The projection structure of the invention is different from the traditional DMD projection device of an optical machine, and is a simple and easily controlled spatial structure optical phase modulation projection device. The function which can be completed by a complex electronic system can be realized only by accurate mechanical displacement of the grating without depending on a complex stripe display control program, and the system complexity is greatly simplified, so that the measurement precision is ensured, and the volume and cost of the system are greatly reduced.
As a preferred embodiment of the present invention, the sinusoidal structured light projection module further includes a collimating lens and a field lens, and the collimating lens and the field lens are sequentially disposed between the translational grating sheet and the illumination light source. The invention optimizes the light collecting system of the light source, adopts the design of the field lens, and the collimating lens and the diaphragm can adjust the illumination angle, thereby improving the light energy utilization rate and the illumination uniformity of the illumination system. Meanwhile, the optical system is optimized by adopting the Kohler illumination system, and the defect of critical illumination is overcome, so that the contrast of the projected stripes is improved, and the acquisition effect is greatly improved.
As a preferable scheme of the present invention, the sinusoidal structured light projection module further includes a fixed grating plate, and the fixed grating plate is disposed between the translational grating plate and the collimating lens. The invention adopts a Ke Lezhao bright system to carry out illumination projection on the double gratings which are stacked in parallel, namely a certain dynamic double grating microstructure is illuminated at the same time, the dynamic grating (namely a translational grating sheet) is dragged and translated through a micro motor, and the moving precision is fed back by a displacement detector so as to improve the moving precision. The projected fringe period varies continuously over a range during the measurement. The fine measurement can be performed according to an algorithmic process.
As a preferred scheme of the present invention, the translational grating sheet is disposed in the projection direction of the illumination light source through two mutually parallel rail brackets, and the two rail brackets are of a bilateral sliding groove type structure. The invention adopts the design of the bilateral sliding groove, and enhances the anti-falling performance and the motion stability of the translational grating sheet.
As a preferable scheme of the invention, the micro motor is a motor GM18168-01 of a stepping belt reduction gearbox.
As a preferable scheme of the invention, the displacement detector adopts an MTE-40 grating detector.
A three-dimensional measurement method of a grating translation structure light three-dimensional measurement system comprises the following steps:
s1: the illumination light source emits light rays, and the light rays pass through the translational grating sheet and then emit structured light to a target object through the projection lens; the texture RGB camera collects texture data of the target object;
s2: the controller controls the micro motor to drive the translational grating sheet to periodically move on the track support, and a structured light image reflected by the target object is synchronously acquired through the binocular acquisition module;
s3: preprocessing and phase resolving are carried out on the structured light image, and phase information of the target object is obtained; the preprocessing comprises edge analysis and image Fourier transform;
s4: and generating point clouds according to the phase information, and splicing according to the texture data to form a space three-dimensional entity.
As a preferable embodiment of the present invention, the step S2 further includes: the controller receives the displacement data acquired by the displacement detector and adjusts the motion rate of the micro motor in real time according to the displacement data; and the device is used for eliminating the shaking of the translational grating sheet and enabling the translational grating sheet to reciprocate on the track support at a constant speed.
As a preferred scheme of the present invention, the binocular acquisition module acquires at least three sets of the structured light images within one movement period of the translational grating sheet, and the acquisition intervals between the three sets of the structured light images are consistent; wherein, the expression of the light intensity spatial distribution of the sine structure light after the modulation of the surface of the target object is as follows:
I n (x,y)＝a(x,y)+b(x,y)×cos[φ(x,y)+Δ n ],(n＝0,1,…,N)，
(x, y) is an image coordinate in the target object obtained from the internal parameters of the IR capturing camera (4), I n (x, y) is the light intensity at the image coordinate, a, b are respectively the background component and the modulation factor, phi (x, y) is the relative phase of the target object to the translation grating, and delta n The phase shift amount of the n time of the structured light image.
As a preferable embodiment of the present invention, the phase calculation in step S3 includes the following steps:
(1): the structured light image comprises multi-frame stripe information; constructing two-two difference images by using four frame stripe images, and generating three difference images in total, wherein the three difference images are shown in the following formula:
wherein, I S0 ，I S1 ，I S3 Three of the difference images are respectively provided,
and establishing an equation set related to the unknown phase shift step length according to the difference image, wherein the equation set is as follows:
(2): solving the system of equations to obtain the unknown phase shift step size delta 1 ，Δ 2 ；
(3): obtaining the relative phase phi (x, y) of the target object for modulating the translation grating sheet according to the following formula;
I n (x，y)＝a(x，y)+b(x，y)×cos[φ(x，y)+Δ n ]，(n＝0，1，…，N)
and performing phase expansion on the relative phase by the following formula to obtain a truncated phase phi (x, y) corresponding to the target object:
Φ(x，y)＝φ(x，y)+2πk(x，y)，
k is a natural number;
(4) Detecting human face characteristic points on the texture image, matching the obtained human face characteristic points with the phase information in the truncation phase, obtaining the stripe level difference K (x, y) corresponding to each pair of human face characteristic points according to the following formula,
where Round is a rounding operation, phi L (x, y) and Φ R (x, y) are respectively the truncated phases corresponding to the left camera and the right camera for the coordinates L (x, y) and R (x, y) of the human face characteristic point on the target surface;
(5): outputting truncated phases Φ 'of the phase-adjusted left and right cameras for corresponding points on the target surface according to the following equation' L (x, y) and Φ' R (x，y)，
Φ' L (x,y)＝Φ L (x,y)
Φ' R (x,y)＝Φ R (x,y)+2Kπ，
And K is the fringe level difference corresponding to the truncation phase. The method estimates the unknown phase shift step length by using the phase difference image between each two multi-frame fringe images, and then calculates the truncation phase by using the least square method and combining the estimated phase shift step length, thereby reducing the phase extraction error caused by inaccurate phase shift. And the starting point of the continuous phase is corrected according to the human face characteristic points, so that the calculation precision of the disparity map is improved.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention adopts a movable grating illumination projection, namely, a translational grating sheet for determining the fringe frequency is adopted for illumination projection, in the measuring process, the measurement can be carried out by translating for a certain period along the direction vertical to the grating and simultaneously projecting a snapshot modulation pattern, and grating sheet elements with different spatial frequencies can be replaced according to the measurement requirement. The projection structure of the invention is different from the traditional DMD projection device of an optical machine, and is a simple and easily controlled spatial structure optical phase modulation projection device. The function which can be completed by a complex electronic system can be realized only by accurate mechanical displacement of the grating without depending on a complex stripe display control program, and the system complexity is greatly simplified, so that the measurement precision is ensured, and the volume and cost of the system are greatly reduced.
2. The invention optimizes the light collecting system of the light source, adopts the design of the field lens, and the collimating lens and the diaphragm can adjust the illumination angle, thereby improving the light energy utilization rate and the illumination uniformity of the illumination system. Meanwhile, the optical system is optimized by adopting the Kohler illumination system, and the defect of critical illumination is overcome, so that the contrast of the projected stripes is improved, and the acquisition effect is greatly improved.
3. The invention adopts a Kohler illumination system to carry out illumination projection on double gratings which are stacked in parallel, namely, a certain dynamic double grating microstructure is illuminated at the same time, a dynamic grating (namely a translational grating sheet) is dragged to translate through a micro motor, and the movement precision is fed back by a displacement detector so as to improve the movement precision. The period of the projected fringes varies continuously over a range during the measurement. The fine measurement can be performed according to an algorithmic process.
4. The invention adopts the design of the bilateral sliding groove, and enhances the anti-falling performance and the motion stability of the translational grating sheet.
5. The method estimates the unknown phase shift step length by utilizing the phase difference image between each two multi-frame fringe images, and then calculates the truncation phase by utilizing the least square method and combining the estimated phase shift step length, thereby reducing the phase extraction error caused by inaccurate phase shift. And the starting point of the continuous phase is corrected according to the human face characteristic points, so that the calculation precision of the disparity map is improved.
Drawings
Fig. 1 is a structural diagram a of a grating translation structured light three-dimensional measurement system according to embodiment 1 of the present invention;
fig. 2 is a structural schematic diagram B of a grating translation structured light three-dimensional measurement system according to embodiment 1 of the present invention;
fig. 3 is a schematic structural diagram C of a grating translation structured light three-dimensional measurement system according to embodiment 1 of the present invention;
fig. 4 is a schematic structural diagram of a sinusoidal structured light projection module in a grating translational structured light three-dimensional measurement system according to embodiment 1 of the present invention;
fig. 5 is a schematic structural diagram of a sinusoidal structured light projection module in a grating translational structured light three-dimensional measurement system according to embodiment 2 of the present invention;
fig. 6 is a schematic structural diagram of a Ke Lezhao illumination system in a grating translation structured light three-dimensional measurement system according to embodiment 3 of the present invention;
fig. 7 is a schematic diagram of a Ke Lezhao illumination system in a grating translation structured light three-dimensional measurement system according to embodiment 3 of the present invention;
fig. 8 is a schematic view of image acquisition results of left and right cameras when a grating translates in a three-dimensional measurement method of structured light with grating translation according to embodiment 5 of the present invention;
fig. 9 is a flow chart of phase solution in a three-dimensional measurement method of grating translation structured light according to embodiment 5 of the present invention;
fig. 10 is a schematic diagram of a result of outputting a truncated phase diagram and a texture diagram and detecting a human face feature point by a random phase shift correction algorithm in the grating translation structured light three-dimensional measurement method according to embodiment 5 of the present invention;
fig. 11 is a schematic diagram of a modeling result in a three-dimensional measurement method of grating translation structured light according to embodiment 5 of the present invention;
the labels in the figure are: the system comprises a projection lens 1, a texture RGB camera 2, a sine structure light projection module 3, an IR collecting camera 4, an illumination light source 5, a translation grating sheet 6, a micro motor 7, a displacement detector 8, a fixed grating sheet 9, a collimating lens 10 and a field lens 11.
Detailed Description
The present invention will be described in further detail with reference to test examples and specific embodiments. It should be understood that the scope of the above-described subject matter is not limited to the following examples, but is intended to include all technical aspects that can be achieved based on the present disclosure.
Example 1
As shown in fig. 1 to 3, a grating translation structured light three-dimensional measurement system includes a binocular acquisition module, a sinusoidal structured light projection module 3, and a controller.
The binocular acquisition module comprises two IR acquisition cameras 4 and a texture RGB camera 2, the two IR acquisition cameras 4 are respectively arranged at the left end and the right end of the system, and the texture RGB camera 2 and the sinusoidal structured light projection module 3 are both arranged between the two IR acquisition cameras 4;
as shown in fig. 4, the sinusoidal structured light projection module 3 includes an illumination light source 5, a translational grating sheet 6, a micro motor 7, a displacement detector 8, and a projection lens 1; the illumination light source 5 is used for providing illumination for the translational grating sheet 6 and adopts an LED illumination light source; the translational grating sheet 6 is arranged above the illumination light source 5 through two parallel track supports (namely, in the projection direction of the illumination light source 5, wherein the two track supports are of a bilateral sliding groove type structure); the micro motor 7 is used for driving the translational grating sheet 6 to move; the displacement detector 8 is used for acquiring displacement data of the translational grating sheet 6.
The controller is respectively electrically connected with the binocular acquisition module and the sinusoidal structured light projection module 3.
Aiming at the fact that the requirement of stripe structure light on displacement accuracy is high, the micro motor 7 is a motor GM18168-01 with a stepping belt reduction gearbox, and has the advantages of being small in size, high in rotating speed and small in single-step displacement, the single-step displacement is 3um after the speed is reduced through the small gearbox, and the requirements of step distance and accuracy of displacement phase shift can be met.
The displacement detector 8 is an MTE-40 grating detector with the resolution up to 0.5 um. The grating detector is mainly applied to the field of industrial measurement, has non-contact type, high reliability, strong adaptability, small volume, the highest measurement speed of 7.2m/s far exceeding the moving speed of the stripe grating sheet, the state indication of a three-color LED, and a control system forms a closed loop through a grating reading head.
The invention adopts a translation grating sheet 6 to singly project sinusoidal stripe light, and only needs the micro motor 7 to drag the translation grating sheet 6 to translate at least three frame frequencies along the direction of vertical grating bars, and simultaneously controls the camera to synchronously collect, thus being capable of measuring three-dimensional entities through projection modulation of structured light generated by grating illumination, and simultaneously, the invention can also replace grating sheet elements with different spatial frequencies according to the measurement requirement.
The specific operation process is as follows:
the sine structured light projection module 3 projects N (N is more than or equal to 3) stripe structured light field sequences with adjustable image quantity to the surface of the face, and simultaneously outputs a synchronous control signal to the binocular acquisition module; the binocular acquisition module works in an external trigger state, shoots a face surface under illumination of a fringe structure light field under the control of a synchronous control signal and transmits the face surface as a modeling image to the controller; the controller controls the sine-structured light projection module 3 to perform projection output by sending a control signal or an enabling signal to the sine-structured light projection module 3, can coordinate the working process of the three-dimensional face modeling system, and completes three-dimensional face modeling based on the modeling image sent by the real-time binocular acquisition module. And the controller is one of a processor, a singlechip or a PC with signal and data processing capacity.
After the sinusoidal structured light projection module 3 receives a control signal sent by the controller, the internal control micro motor 7 moves left or right by a fixed step pitch, the projection grating sheet synchronously illuminates and moves left or right by a determined displacement X, and the first-order small amount delta X of the error of the X is fed back and compensated by the displacement detector 8. The micro motor 7 controls the translation grating sheet 6 to move, simultaneously sends an external trigger control signal to the left and right IR collecting cameras 4, and the IR collecting cameras 4 finish the collection of the modulated face images to form a left image L0 and a right image R0; and then, controlling the micro motor 7 to move left or right by a fixed step distance, and synchronously acquiring by the IR acquisition camera 4 to form a left image L1 and a right image R1. And transmitting the collected images with phase shift spatial modulation to a control end by a USB3.0 data line through a PCIE interface to wait for calculation.
Example 2
As shown in fig. 5, the lighting device further includes a fixed grating sheet 9, and the fixed grating sheet 9 is disposed between the translational grating sheet 6 and the lighting source 5. In the embodiment, a certain dynamic double-grating microstructure is adopted, the dynamic grating is dragged to translate through the micro motor 7, and the movement precision is fed back by the displacement detector 8 so as to improve the movement precision. In the measuring process, the period of the projected stripes is continuously changed within a certain range, and fine measurement can be carried out according to algorithm processing.
Example 3
As shown in fig. 6, the illumination system further includes a collimating lens 10 and a field lens 11, where the collimating lens 10 and the field lens 10 are sequentially disposed between the translational grating 6 and the illumination light source 5, as shown in fig. 7, so as to form a kohler illumination system. The optical system is optimized by adopting the Kohler illumination system, the illumination system overcomes the defect of critical illumination, thereby improving the contrast of the projection stripe, and the image illuminated by the grating is projected on the surface of the measured object by using the lens.
Example 4
A three-dimensional measurement method of the grating translation structure light three-dimensional measurement system comprises the following steps:
s1: the illumination light source 5 emits light rays, and the light rays pass through the translational grating sheet 6 and then emit structured light to a target object through the projection lens 1; the texture RGB camera 2 collects texture data of the target object.
S2: the controller controls the micro motor 7 to drive the translational grating sheet 6 to periodically move on the track support, and the structured light image reflected by the target object is synchronously acquired through the binocular acquisition module.
The controller receives the displacement data acquired by the displacement detector 8 and adjusts the movement rate of the micro motor 7 in real time according to the displacement data; the device is used for eliminating the shaking of the translational grating sheet 6 and enabling the translational grating sheet 6 to move, stop and shoot on the track support; motion-stop-beat; motion-stop-beat; the specific process of the uniform reciprocating motion is as follows:
s21: after the controller receives the starting command, the micro motor 7 is driven to drive the translational grating sheet 6 to move and simultaneously receive the signal of the displacement detector 8.
S22: the controller receives a certain number of pulse signals, controls the micro motor 7 to move, prolongs the time of the last step, increases the electromagnetic force to eliminate jitter (a reduction gearbox can also be used for further eliminating jitter), controls the light source to illuminate the grating stripe projection, synchronously triggers the signal to trigger the IR acquisition camera 4 to take a snapshot, and uploads the picture shot by the IR acquisition camera 4 to an upper computer (external processing equipment, such as a PC terminal).
S23: and the upper computer performs calculation. And repeating the displacement-snapshot for several times, and after the displacement-snapshot is completed, controlling the electric reverse rotation to return to the initial position by the controller, and performing zero return inspection.
S3: preprocessing and phase resolving are carried out on the structured light image, and phase information of the target object is obtained; the preprocessing includes edge analysis and image fourier transform.
S4: and generating point cloud according to the phase information, and splicing to form a space three-dimensional entity according to the texture data.
The mechanical phase shift of the optical projector may cause a large reduction in the phase shift accuracy due to various factors, and under such conditions, if the conventional phase shift method is continuously used for phase resolution of the fringes, the final resolution result may have a large error. In order to overcome the problem, the invention provides a random unknown phase shift fringe phase resolving method. Firstly, an equation set related to a plurality of random unknown phase shift step sizes is established by utilizing difference images between every two multi-frame stripe images, then the equation set is solved, and the unknown phase shift step sizes are estimated. And then, a least square method is utilized, the phase extraction error caused by inaccurate phase shift is reduced by calculating the truncated phase by combining the estimated phase shift step length, the phase expansion is carried out on the truncated phase by utilizing a spatial phase expansion algorithm, and finally the initial point of the continuous phase is corrected according to the human face characteristic point, so that the calculation precision of the disparity map can be further improved. After the correct unfolded phase is obtained by random unknown phase shift fringe phase calculation, the three-dimensional face contour operation based on the face characteristic points is adopted, and the three-dimensional reconstruction can be completed only by adopting an N (N is more than or equal to 3) Zhang Tiaowen image. The algorithm utilizes human face characteristic points as prior knowledge, adopts a phase adjustment algorithm to determine accurate phase level difference K, adjusts relative phase pairs to the same phase reference through a 2K pi phase adjustment relation, and then performs sub-pixel level stereo matching to finally obtain high-precision three-dimensional human face reconstruction. The algorithm solves the problem that the relative phase obtained by the stereo camera through the space phase expansion algorithm cannot be directly used for stereo matching due to different phase references; and because only three phase shift modes are needed, the method has low sensitivity to dynamic scenes and can be applied to measurement of dynamic human faces; in addition, compared with the traditional technology, the method is not limited by complex algorithms such as projection frequency, measurement depth range and the like, such as assistance of multi-frame low-frequency signals, speckle or triangular wave embedding in stripes, assistance of other image information for phase unwrapping and the like. Therefore, the method is matched with the grating structure light modulation system for use, can simultaneously meet the requirements on acquisition speed and measurement accuracy in three-dimensional model application, and is very suitable for engineering application.
Example 5
This embodiment is an actual implementation flow illustration of embodiment 4, and mainly uses a micro motor 7 to drive a moving translational grating sheet 6 to project the spatial modulation of the sinusoidal structured light, and the IR capturing camera 4 performs multi-step movement to synchronously capture the sinusoidal fringe pattern modulated by the target. The method comprises the following specific steps:
1) The illumination light source 5 emits light rays, and the light rays pass through the translational grating sheet 6 and then emit structured light to a target object through the projection lens 1; the texture RGB camera 2 collects texture data of the target object.
2) The motor moves left by a fixed step pitch, the projection grating sheet synchronously illuminates and moves left by a determined displacement X, wherein the error of the X is compensated by the feedback of the displacement detection grating in a first order and a small amount delta X. Synchronously acquiring by using a binocular camera after the projection grating is translated to form left views L0, L1 and L2; multiple ones of the spatial modulation patterns with determined phase shifts for the right views R0, R1, R2 are shown in fig. 8.
3) Optimizing a reconstruction system: and carrying out image processing and analysis on the calibrated optical system framework and the collected pattern, and carrying out spatial calculation on the three-dimensional image. The phase solving flow chart is shown in fig. 9.
Wherein, the three-frame stripe image can be represented as follows:
I n (x,y)＝a(x,y)+b(x,y)×cos[φ(x,y)+Δ n ],(n＝0,1,…,N)
and constructing pairwise difference images by using the four frame stripe images. A total of three difference images can be generated as shown in the following equation.
Establishing an equation set according to the difference image
When the number of stripes in the stripe pattern is greater than 1, the following approximate equation holds.
Namely:
∑b 2 *sin 2 (φ)≈∑b 2 *sin 2 (φ+Δ)
therefore, the following equation set, including 6 equations, can be established from the difference image as follows.
A＝||b*sin(φ)||＝||b*sin(φ+Δ)||
Solving the system of equations
Since the system of equations contains 3 unknowns and 3 equations, it can be solved numerically.
The solving method comprises the following steps:
since 0 < delta 1 ＜Δ 2 ＜2pi
The absolute value sign can be removed on the right side of all expressions.
Equation (3) can be expanded as:
the cross-over, and the square can be given as:
so that it is possible to obtain:
up to this point, the structures shown in equations (6) and (7) can be constructed
The optimal solution can be solved by least squares. And the phase shift amount delta can be estimated by an arcos operator 1 ，Δ 2 (i.e., the amount of phase shift of the sinusoidal structured light due to the translation of the grating plate driven by the motor). And then, the minimum two multiplication is utilized, the estimated phase shift step length is combined, and the truncation phase and the corresponding texture image of the left camera and the right camera are calculated and output. And detecting the human face characteristic points on the obtained texture map. As shown in fig. 10, the method for detecting the face feature points is not limited, and any method for detecting the face feature points may be used.
Obtaining a relative phase phi (x, y) of the target object for modulating the translational grating sheet according to the following formula;
I n (x,y)＝a(x,y)+b(x,y)×cos[φ(x,y)+Δ n ],(n＝0,1,…,N)
and performing phase expansion on the relative phase by the following formula to obtain a truncated phase phi (x, y) corresponding to the target object:
Φ(x，y)＝φ(x，y)+2πk(x，y)，
k is a natural number;
furthermore, a corresponding phase matching pair is found in the truncated phase according to the detected face characteristic points. Extracting the coordinates L (x) of the characteristic points of the human face from the texture image pair i ，y i ) And R (x) i ，y i ) Corresponding to a phase among the cut-off phases of
And &>
i is the number of feature points. Each detected feature point has its own inherent property, and the feature points at corresponding positions on the left and right faces can be regarded as a pair of feature point pairs. Since the acquired modulation image has been subjected to the line-level correction, a phase pair corresponding to the coordinates of the face feature point pair at the truncated phase can be regarded as a phase matching pair. Calculating to obtain a phase matching pair corresponding to the characteristic point according to the formula 1>
And &>
The phase difference between the human face feature points is divided by 2 pi to obtain the stripe level difference K corresponding to each pair of human face feature points i (x，y)。
Through voting strategy at K i And (x, y) obtaining the final fringe order difference K. And adjusting the left and right relative phases to the same phase reference by adopting a phase adjustment algorithm and taking the relative phase of the left camera as a reference according to the formulas (2) and (3) after adding 2K pi to the absolute phase of the right camera, and performing sub-pixel levelAnd phase matching is carried out, and finally high-precision three-dimensional face data is obtained.
Φ′ L (x，y)＝Φ L (x，y) (2)
Φ′ R (x，y)＝Φ R (x，y)+2Kπ (3)
4) And point cloud is generated, and reconstruction information of the space three-dimensional entity can be formed by corresponding texture splicing and the like. The modeling results are shown in fig. 11.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (8)
1. A grating translation structured light three-dimensional measurement system comprises a sine structured light projection module (3), a binocular acquisition module and a controller, and is characterized in that,
the binocular acquisition module comprises two IR acquisition cameras (4) and a texture RGB camera (2); the two IR collecting cameras (4) are respectively arranged at two sides of the system, and the texture RGB camera (2) and the sinusoidal structured light projection module (3) are both arranged between the two IR collecting cameras (4);
the sine structured light projection module (3) comprises an illumination light source (5), a translational grating sheet (6), a micro motor (7), a displacement detector (8) and a projection lens (1); the illumination light source (5) is used for providing illumination for the translational grating sheet (6); the translational grating sheet (6) is arranged in the projection direction of the illumination light source (5); the micro motor (7) is used for driving the translational grating sheet (6) to translate along a straight line; the displacement detector (8) is used for acquiring displacement data of the translational grating sheet (6); the projection lens (1) is used for projecting structured light;
the controller is electrically connected with the binocular acquisition module and the sinusoidal structured light projection module (3) respectively;
the sine-structured light projection module (3) further comprises a collimating lens (10), a field lens (11) and a fixed grating sheet (9), wherein the collimating lens (10) and the field lens (11) are sequentially arranged between the translational grating sheet (6) and the illumination light source (5); the fixed grating sheet (9) is arranged between the translational grating sheet (6) and the collimating lens (10).
2. The grating translation structured light three-dimensional measurement system according to claim 1, wherein the translation grating sheet (6) is disposed in the projection direction of the illumination light source (5) through two mutually parallel rail supports, and the two rail supports are of a bilateral sliding groove type structure.
3. The grating translation structured light three-dimensional measurement system according to claim 1, wherein the micro motor (7) is a stepping motor with a reduction gearbox GM18168-01.
4. A grating translation structured light three-dimensional measurement system according to claim 1, characterized in that the displacement detector (8) is a MTE-40 grating detector.
5. A three-dimensional measurement method of a grating translation structured light three-dimensional measurement system according to claim 1, comprising the steps of:
s1: the illumination light source (5) emits light rays, and the light rays pass through the translational grating sheet (6) and then emit structured light to a target object through the projection lens (1); the texture RGB camera (2) collects texture data of the target object;
s2: the controller controls the micro motor (7) to drive the translational grating sheet (6) to move periodically on the track support, and the IR acquisition camera (4) is used for synchronously acquiring a structured light image reflected by the target object;
s3: preprocessing and phase resolving are carried out on the structured light image, and phase information of the target object is obtained; the preprocessing comprises edge analysis and image Fourier transform;
s4: and generating point cloud according to the phase information, and splicing to form a space three-dimensional entity according to the texture data.
6. The three-dimensional measurement method according to claim 5, wherein the step S2 further comprises: the controller receives the displacement data collected by the displacement detector (8) and adjusts the movement rate of the micro motor (7) in real time according to the displacement data; the device is used for eliminating the shaking of the translational grating sheet (6) and enabling the translational grating sheet (6) to reciprocate on the track support at a constant speed.
7. A three-dimensional measurement method according to claim 5, wherein the IR collecting camera (4) collects three frames of the structured light images in one motion period of the translational grating sheet (6), and the collection intervals between the three frames of the structured light images are consistent; wherein, the expression of the light intensity spatial distribution of the sine structure light after the modulation of the surface of the target object is as follows:
I n (x,y)＝a(x,y)+b(x,y)×cos[φ(x,y)+Δ n ],(n＝0,1,…,N)，
(x, y) are image coordinates in the target object obtained from internal parameters of the IR acquisition camera (4), I n (x, y) is the light intensity at the image coordinate, a, b are respectively the background component and the modulation factor, phi (x, y) is the relative phase of the target object to the translation grating, and delta n The phase shift amount of the n time of the structured light image.
8. The three-dimensional measurement method according to claim 7, wherein the phase calculation in step S3 comprises the steps of:
(1): the structured light image comprises multi-frame stripe information; two difference images are constructed by using four frame stripe images, and three difference images can be generated in total, wherein the following formula is shown:
wherein, I S0 ，I S1 ，I S3 Are respectively threeThe difference image is a difference image of the difference image,
and establishing an equation set related to the unknown phase shift step length according to the difference image, wherein the equation set is as follows:
(2): solving the system of equations to obtain the unknown phase shift step size delta 1 ，Δ 2 ；
(3): obtaining the relative phase phi (x, y) of the target object for modulating the translation grating sheet according to the following formula;
I n (x,y)＝a(x,y)+b(x,y)×cos[φ(x,y)+Δ n ],(n＝0,1,…,N)
and performing phase expansion on the relative phase by the following formula to obtain a truncated phase phi (x, y) corresponding to the target object:
Φ(x，y)＝φ(x)y)+2πk(x，y)
k is a natural number;
(4) Detecting human face characteristic points on the texture image, matching the obtained human face characteristic points with the phase information in the truncation phase, obtaining the stripe level difference K (x, y) corresponding to each pair of human face characteristic points according to the following formula,
wherein Round is a rounding operation, phi L (x, y) and Φ R (x, y) are respectively the truncated phases corresponding to the left camera and the right camera for the coordinates L (x, y) and R (x, y) of the human face characteristic point on the target surface;
(5): outputting truncated phases Φ 'of the phase-adjusted left and right cameras for corresponding points on the target surface according to the following equation' L (x, y) and Φ' R (x,y)，
Φ' L (x,y)＝Φ L (x,y)
Φ' R (x,y)＝Φ R (x,y)+2Kπ，
And K is the fringe level difference corresponding to the truncation phase.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110534572.8A CN113251952B (en) | 2021-05-17 | 2021-05-17 | Three-dimensional measurement system and three-dimensional measurement method for grating translation structured light |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110534572.8A CN113251952B (en) | 2021-05-17 | 2021-05-17 | Three-dimensional measurement system and three-dimensional measurement method for grating translation structured light |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113251952A CN113251952A (en) | 2021-08-13 |
| CN113251952B true CN113251952B (en) | 2023-03-24 |
Family
ID=77182232
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110534572.8A Active CN113251952B (en) | 2021-05-17 | 2021-05-17 | Three-dimensional measurement system and three-dimensional measurement method for grating translation structured light |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113251952B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113763444B (en) * | 2021-09-07 | 2024-04-19 | 四川川大智胜软件股份有限公司 | Phase unwrapped three-dimensional face reconstruction method and system for hierarchical line pairing |
| CN115421349A (en) * | 2022-11-02 | 2022-12-02 | 四川川大智胜软件股份有限公司 | Non-digital optical machine structure light projection module, acquisition device and three-dimensional measurement system |
| CN116592794B (en) * | 2023-07-17 | 2023-11-24 | 南京理工大学 | Rapid three-dimensional reconstruction method based on polarized structured light |
| CN117274352B (en) * | 2023-11-17 | 2024-02-06 | 四川川大智胜软件股份有限公司 | Three-dimensional reconstruction system, phase calculation method and three-dimensional reconstruction method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107063129A (en) * | 2017-05-25 | 2017-08-18 | 西安知象光电科技有限公司 | A kind of array parallel laser projection three-dimensional scan method |
| CN110260822A (en) * | 2019-06-18 | 2019-09-20 | 西安交通大学 | A kind of more mesh structured-light system high-precision calibrating methods |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1133402C (en) * | 1999-02-03 | 2004-01-07 | 中国人民解放军总后勤部军需装备研究所 | Method for measuring sizes of human body |
| CN101726258B (en) * | 2009-12-10 | 2011-01-19 | 华中科技大学 | On-line detection system for hot object |
| CN101825445B (en) * | 2010-05-10 | 2011-11-30 | 华中科技大学 | Three-dimension measuring system for dynamic object |
| CN103175847B (en) * | 2013-03-19 | 2016-08-10 | 哈尔滨理工大学 | Grating surface defect detecting device |
-
2021
- 2021-05-17 CN CN202110534572.8A patent/CN113251952B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107063129A (en) * | 2017-05-25 | 2017-08-18 | 西安知象光电科技有限公司 | A kind of array parallel laser projection three-dimensional scan method |
| CN110260822A (en) * | 2019-06-18 | 2019-09-20 | 西安交通大学 | A kind of more mesh structured-light system high-precision calibrating methods |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113251952A (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113251952B (en) | Three-dimensional measurement system and three-dimensional measurement method for grating translation structured light | |
| CN105953747B (en) | Structured light projection full view 3-D imaging system and method | |
| CN106931910B (en) | A kind of efficient acquiring three-dimensional images method based on multi-modal composite coding and epipolar-line constraint | |
| US20220221270A1 (en) | A calibration method for fringe projection systems based on plane mirrors | |
| WO2016138758A1 (en) | Calibration method of telecentric imaging three-dimensional shape measurement system | |
| CN110692084B (en) | Apparatus and machine-readable storage medium for deriving topology information of a scene | |
| CN104583713B (en) | The method that the body surface of preferably structuring is imaged and the device for imaging | |
| CN103292740B (en) | A kind of 3-D scanning instrument measurement method and device thereof | |
| CN104335005A (en) | 3-D scanning and positioning system | |
| CN108362228B (en) | Double-optical-machine-based optical knife grating hybrid three-dimensional measurement device and measurement method | |
| CN102261896A (en) | Method and system for measuring three-dimensional shape of object based on phase measurement | |
| CN112762859B (en) | High-precision three-dimensional measuring device for sine stripe structured light of non-digital optical machine | |
| CN104408762A (en) | Method for obtaining object image information and three-dimensional model by using monocular unit and two-dimensional platform | |
| CN109307483A (en) | A kind of phase developing method based on structured-light system geometrical constraint | |
| CN104680534B (en) | Object depth information acquisition methods based on single frames composite shuttering | |
| CN104279981A (en) | Mirror surface/mirror-surface-like object absolute surface shape measuring method and device based on stripe reflection | |
| CN109631798A (en) | A kind of 3 d shape vertical measurement method based on π phase shifting method | |
| CN104197837A (en) | Non-contact type optical measuring method and device for measuring volume of objects with complicated surfaces | |
| CN113763540A (en) | Three-dimensional reconstruction method and equipment based on speckle fringe hybrid modulation | |
| CN106546193A (en) | A kind of high reflection body surface three-dimensional measuring method and system | |
| CN108645353A (en) | Three dimensional data collection system and method based on the random binary coding light field of multiframe | |
| CN111462331A (en) | Method for expanding epipolar geometry and calculating three-dimensional point cloud in real time | |
| CN212843399U (en) | Portable three-dimensional measuring equipment | |
| CN112325799A (en) | High-precision three-dimensional face measurement method based on near-infrared light projection | |
| CN112611320A (en) | Multi-frequency heterodyne surface structured light three-dimensional reconstruction method based on gradient light intensity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |