CN113160393A - High-precision three-dimensional reconstruction method and device based on large field depth and related components thereof - Google Patents

Abstract

The invention discloses a high-precision three-dimensional reconstruction method and device based on large depth of field and related components thereof. The method comprises the following steps: carrying out region division on a large depth-of-field measurement scene; performing binocular vision three-dimensional calibration on each divided area by using a three-dimensional measurement system to obtain calibration data; calculating an absolute phase distribution map and three-dimensional information of the flat panel at different positions; establishing a three-dimensional mapping coefficient table of a corresponding area; acquiring a target image of a measured object and calculating an absolute phase distribution map; and acquiring the absolute phase of each pixel point in the absolute phase distribution map of the target image, searching a three-dimensional mapping coefficient in a three-dimensional mapping coefficient table of a corresponding region, and calculating the spatial three-dimensional point coordinate by using the three-dimensional mapping coefficient. According to the invention, the large depth-of-field scene is divided into a plurality of regions for calculation, so that the whole calculation process is more rigorous and accurate, and the space three-dimensional point coordinates of the measured object are acquired more quickly and accurately after the three-dimensional mapping coefficient table is established.

Description

High-precision 3D reconstruction method, device and related components based on large depth of field

technical field

The invention relates to the technical field of three-dimensional imaging, and in particular, to a high-precision three-dimensional reconstruction method and device based on a large depth of field, and related components thereof.

Background technique

The fringe projection 3D measurement in the prior art is a kind of structured illumination method, which has the advantages of high speed, high precision, low cost, easy operation, etc. The principle of fringe projection 3D measurement technology is to project a standard sinusoidal fringe pattern on the measured object, the height of the object modulates the projected fringes, the camera collects the modulated deformation fringe pattern, and then uses the fringe phase demodulation and system calibration technology, combined with phase -The principle of height mapping, reconstructing the three-dimensional information of the object.

Generally, fringe projection uses a digital micromirror array (DMD) based on the principle of optical imaging to generate fringe patterns, and the optical projection lens with a fixed focal length has a limited depth of field range. In particular, to improve image brightness, commercial projectors are designed with large apertures, resulting in a smaller depth of field range. In some large measurement scenarios, the size of the object to be measured is large or the large-span spatial range composed of multiple objects, the projection system cannot meet the requirements. The MEMS (Micro Electro Mechanical System) galvanometer laser scanning projector has a large depth of field imaging range due to the use of a laser light source and a galvanometer scanning projection method. Camera lenses also face the problem of limited depth of field range, and electronic zoom lenses have the ability to continuously change the focal length of the lens.

In recent years, the development of Electronic Tunable Lens (ETL) has provided more options for designing more compact optical systems, especially its precision, speed, convenience and repeatability, etc., have been widely used in Display, microscope, autofocus imaging, laser processing and other fields. For three-dimensional measurement, there are also studies using electronically adjustable lenses to achieve certain functions. One technique is to use electronically tunable lenses to rapidly acquire multiple discrete focused images to obtain three-dimensional information about the entire scene by combining data from different focusing situations. However, the current 3D measurement system has low accuracy when measuring scenes with large depth of field, and the problem of poor reconstruction and restoration effect still cannot be solved.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a high-precision 3D reconstruction method, device and related components based on a large depth of field, aiming to solve the problems of low 3D measurement accuracy and poor 3D reconstruction effect in a large depth of field scene in the prior art.

In a first aspect, an embodiment of the present invention provides a high-precision three-dimensional reconstruction method based on a large depth of field, including:

Regional division of large depth of field measurement scenes;

Use a three-dimensional measurement system to perform binocular vision stereo calibration on each of the divided regions to obtain calibration data; wherein, the three-dimensional measurement system includes an imaging device and a projection device;

In different areas, the projection device is used to project the specified pattern to the flat plate at different positions, and the image of the flat plate at different positions is collected by the imaging device, and the absolute phase distribution of the flat plate at different positions is calculated. Figure, and calculate the three-dimensional information of the flat plate at different positions according to the calibration data;

In each area, the absolute phase of each pixel point of the imaging device in the absolute phase distribution diagram of the plane plate is obtained, and the absolute phase of each pixel point of the plane plate is obtained according to the difference between the three-dimensional information of each plane plate and the absolute phase of the corresponding pixel point. The mapping relationship establishes the three-dimensional mapping coefficient table of the corresponding area;

Use the projection device to project the target pattern to the measured object, and collect the target focal scan image of the measured object through the imaging device, and deblur the target focal scan image to obtain the target image and calculate the absolute phase of the target image Distribution;

Obtain the absolute phase of each pixel of the target image in the absolute phase distribution diagram of the target image, and search for the corresponding three-dimensional mapping coefficient in the three-dimensional mapping coefficient table of the corresponding area according to the region to which the absolute phase belongs, and use the The three-dimensional mapping coefficient is calculated to obtain the corresponding three-dimensional point coordinates in space.

In a second aspect, an embodiment of the present invention provides a high-precision three-dimensional reconstruction device based on a large depth of field, which includes:

The area division unit is used to divide the area of the large depth of field measurement scene;

A calibration data acquisition unit, configured to perform binocular stereo calibration on each of the divided regions by using a 3D measurement system to obtain calibration data; wherein the 3D measurement system includes an imaging device and a projection device;

A three-dimensional information acquisition unit is used to project a specified pattern on a flat plate at different positions by using a projection device in different areas, and collect the flat plate images of the flat plate in different positions through an imaging device, and calculate the flat plate at different positions. The absolute phase distribution diagram at different positions, and the three-dimensional information of the flat plate at different positions is obtained by calculating according to the calibration data;

A three-dimensional mapping coefficient table acquiring unit, configured to acquire, in each region, the absolute phase of each pixel of the imaging device in the absolute phase distribution map of the plane plate, and according to the three-dimensional information of each plane plate A three-dimensional mapping coefficient table of the corresponding area is established with the mapping relationship between the absolute phases of the corresponding pixel points;

The target focal scan image acquisition unit is used to project the target pattern to the measured object by using the projection device, collect the target focal scan image of the measured object through the imaging device, and deblur the target focal scan image to obtain the target image and calculate the absolute phase distribution of the target image;

A spatial three-dimensional point coordinate obtaining unit, used to obtain the absolute phase of each pixel of the target image in the absolute phase distribution map of the target image, and look up the three-dimensional mapping coefficient table of the corresponding area according to the area to which the absolute phase belongs Corresponding three-dimensional mapping coefficients are calculated by using the three-dimensional mapping coefficients to obtain corresponding three-dimensional point coordinates in space.

In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer The program implements the high-precision three-dimensional reconstruction method based on the large depth of field described in the first aspect above.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when executed by a processor, the computer program causes the processor to execute the above-mentioned first step. On the one hand, the high-precision three-dimensional reconstruction method based on a large depth of field is described.

Embodiments of the present invention provide a high-precision three-dimensional reconstruction method, device and related components based on a large depth of field. The method includes: dividing a large depth of field measurement scene into regions; using a three-dimensional measuring system to perform binocular vision stereo calibration on each of the divided regions to obtain calibration data; wherein, the three-dimensional measuring system includes an imaging device and a projection device; In different areas, the projection device is used to project the specified pattern to the flat plate in different positions, and the flat plate image of the flat plate in different positions is collected by the imaging device, and the absolute phase distribution diagram of the flat plate in different positions is calculated and obtained. , and calculate and obtain the three-dimensional information of the flat plate at different positions according to the calibration data; in each area, obtain the absolute phase of each pixel of the imaging device in the absolute phase distribution diagram of the flat plate , and establish a three-dimensional mapping coefficient table of the corresponding area according to the mapping relationship between the three-dimensional information of each said flat plate and the absolute phase of the corresponding pixel point; use the projection device to project the target pattern to the object under test, and collect the object pattern through the imaging device. Measure the target focal scan image of the object, and deblur the target focal scan image to obtain the target image and calculate the absolute phase distribution map of the target image; obtain each pixel of the target image in the target image The absolute phase of the absolute phase distribution map is obtained, and the corresponding three-dimensional mapping coefficient is searched in the three-dimensional mapping coefficient table of the corresponding region according to the region to which the absolute phase belongs, and the corresponding spatial three-dimensional point coordinates are calculated by using the three-dimensional mapping coefficient. In the embodiment of the present invention, the large depth of field scene is divided into regions, and a three-dimensional mapping coefficient table is established for each region, and the three-dimensional mapping coefficient of the corresponding region of the measured object is directly obtained during the three-dimensional reconstruction, so as to calculate the spatial three-dimensional point coordinates. The large depth of field scene is divided into multiple areas for calculation, the entire calculation process is more rigorous and accurate, and the spatial 3D point coordinates of the measured object are obtained more quickly and accurately after the 3D mapping coefficient table is established.

Description of drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention, which are of great significance to the art For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic flowchart of a high-precision three-dimensional reconstruction method based on a large depth of field provided by an embodiment of the present invention;

2 is a simulation diagram of a high-precision three-dimensional reconstruction method based on a large depth of field provided by an embodiment of the present invention;

FIG. 3 is a schematic block diagram of a high-precision three-dimensional reconstruction device based on a large depth of field provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It is to be understood that, when used in this specification and the appended claims, the terms "comprising" and "comprising" indicate the presence of the described features, integers, steps, operations, elements and/or components, but do not exclude one or The presence or addition of a number of other features, integers, steps, operations, elements, components, and/or sets thereof.

It is also to be understood that the terminology used in this specification of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise.

It should further be understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items .

Please refer to FIG. 1 and FIG. 2 . FIG. 1 is a schematic flowchart of a high-precision three-dimensional reconstruction method based on a large depth of field according to an embodiment of the present invention, and the method includes steps S101 to S106 .

S101. Perform regional division on the large depth of field measurement scene;

In this step, since the scene with large depth of field is large, when performing 3D reconstruction, if the scene with large depth of field is directly measured, the accuracy will decrease. The measurement space is divided to obtain several smaller measurement areas.

In one embodiment, after the step S101, including

Use a calibration algorithm to calibrate the positions of the imaging device and the projection device;

Obtain the maximum and minimum values of the total control current measured by the zoom lens of the imaging device to measure the large depth of field measurement scene, and the corresponding area control current values when the zoom lens of the imaging device is focused on the center position of each area, and Record the maximum and minimum values of the total control current and the region control current value corresponding to each region.

In this step, the positions of the imaging device and the projection device are first calibrated, and then according to the measurement depth range of the imaging device when the focus is fixed and the diopter variation range of the zoom lens of the imaging device, the measurement scene is determined. For a suitable depth range, by changing the diopter of the zoom lens, the zoom lens can focus on the center position of different areas, and then record the corresponding control current value when the zoom lens focuses on the center position of different areas, adjust The diopter of the zoom lens enables the imaging device to focus on the large depth of field to measure the maximum depth and the minimum depth of the scene, and record the control current value corresponding to the depth. In this embodiment, the minimum depth of the large depth of field measurement scene is 400mm, and the maximum depth is 1000mm.

S102. Use a three-dimensional measurement system to perform binocular vision stereo calibration on each of the divided regions to obtain calibration data; wherein the three-dimensional measurement system includes an imaging device and a projection device;

In this step, an imaging device with a zoom lens and a projection device with a MEMS galvanometer are used to form a three-dimensional measurement system, and the three-dimensional measurement system is used to perform binocular vision stereo calibration for each area to obtain calibration data. The imaging device may be a zoom camera with a zoom lens, and the projection device may be a projector with a MEMS galvanometer.

In one embodiment, the stereo calibration of binocular vision is performed on each of the divided regions by using a three-dimensional measurement system to obtain calibration data, including:

Taking the optical center of the imaging device as the origin, and taking the optical axis of the imaging device as the Z axis, the imaging device coordinate system is established; taking the optical center of the projection device as the origin, and taking the optical axis of the projection device as the origin For the Z axis, establish the coordinate system of the projection device;

Use the binocular vision stereo calibration algorithm to obtain the conversion relationship between the intrinsic parameters of the imaging device and the imaging device coordinate system and the conversion relationship between the intrinsic parameters of the projection device and the projection coordinate system, and calculate the imaging device coordinates The conversion relationship between the system and the projected coordinate system is used to obtain the calibration data.

In this embodiment, the intrinsic parameters of the imaging device and the projection device are acquired, the intrinsic parameters of the imaging device and the projection device, that is, the intrinsic parameter matrix, including the focal length, the position of the optical center, and the number of pixels per unit distance. The intrinsic parameters of the imaging device and the coordinate system of the imaging device have a corresponding conversion relationship, and the intrinsic parameters of the projection device and the coordinate system of the projection device also have a corresponding conversion relationship. The conversion relationship between the imaging device coordinate system and the projection coordinate system.

Wherein, the calibration process of the imaging device is as follows:

其中，s_x,s_y分别为沿对应图像坐标轴的图像平面单位距离的像素数(pixel/mm)；(u₀,v₀)为成像装置的光轴与图像平面的交点，即光心在图像平面上的投影，称为主点；f_x，f_y分别为沿对应图像坐标轴的等效焦距；R是一个3×3的正交矩阵，T是一个3×1的向量，R和T分别表示世界坐标系转换到成像装置坐标系的旋转和平移变换。上式可以简写为：

其中，s为尺度因子；[R T]为外部参数矩阵；

和

分别为空间三维点P和其像点的齐次坐标；M为投影矩阵；K为内部参数矩阵：

世界坐标系到成像装置坐标系的转换关系为：

由于成像装置在成像的过程中存在偏差，因此需要计算径向畸变和切向畸变，所述径向畸变和切向畸变分别表示为：δ_Rx＝x(k₁r²+k₂r⁴+k₃r⁶)，δ_Ry＝y(k₁r²+k₂r⁴+k₃r⁶)，δ_Tx＝2p₁xy+p₂(r²+2x²)，δ_Ty＝p₁(r²+2y²)+2p₂xy。其中，δ_Rx和δ_Ry分别为x方向和y方向的径向畸变；δ_Tx和δ_Ty分别为x方向和y方向的切向畸变；(x,y)是理想图像坐标；

表示理想像点到主点的距离；k₁、k₂和k₃为径向畸变参数；p₁和p₂为切向畸变参数。考虑这两种畸变误差之后，理想像点(x,y)转化为有畸变像点(x’,y’)的过程可以表示为：x'＝x+δ_Rx+δ_Tx，y'＝y+δ_Ry+δ_Ty。Assuming that there is a point P in a certain area, its coordinates in the world coordinate system and the imaging device coordinate system are (X _W , Y _W , Z _W ) and (X _c , Y _c , Z _c ), respectively, in the imaging device The projection coordinates on the imaging plane are (u, v), then the perspective projection imaging process is:

Among them, s _x , s _y are the number of pixels per unit distance of the image plane along the corresponding image coordinate axis (pixel/mm); (u ₀ , v ₀ ) is the intersection of the optical axis of the imaging device and the image plane, that is, the optical center The projection on the image plane is called the principal point; f _x , f _y are the equivalent focal lengths along the corresponding image coordinate axes; R is a 3×3 orthogonal matrix, T is a 3×1 vector, R and T represent the rotation and translation transformations from the world coordinate system to the imaging device coordinate system, respectively. The above formula can be abbreviated as:

Among them, s is the scale factor; [RT] is the external parameter matrix;

and

are the homogeneous coordinates of the three-dimensional point P in space and its image point respectively; M is the projection matrix; K is the internal parameter matrix:

The conversion relationship from the world coordinate system to the imaging device coordinate system is:

Since there is a deviation in the imaging process of the imaging device, radial distortion and tangential distortion need to be calculated, and the radial distortion and tangential distortion are respectively expressed as: δ _Rx =x(k ₁ r ² +k ₂ r ⁴ + k ₃ r ⁶ ), δ _Ry = y(k ₁ r ² +k ₂ r ⁴ +k ₃ r ⁶ ), δ _Tx = 2p ₁ xy+p ₂ (r ² +2x ² ), δ _Ty = p ₁ ( r ² +2y ² )+2p ₂ xy. Among them, δ _Rx and δ _Ry are the radial distortions in the x and y directions, respectively; δ _Tx and δ _Ty are the tangential distortions in the x and y directions, respectively; (x, y) are the ideal image coordinates;

represents the distance from the ideal image point to the principal point; k ₁ , k ₂ and k ₃ are radial distortion parameters; p ₁ and p ₂ are tangential distortion parameters. After considering these two distortion errors, the process of converting an ideal image point (x, y) into a distorted image point (x', y') can be expressed as: x'=x+δ _Rx +δ _Tx , y'=y +δ _Ry +δ _Ty .

For each area, first use the three-dimensional measurement system to collect target images of the planar target at different positions, and use Zhang Zhengyou's calibration algorithm to calculate the position of the imaging device according to the target images. When collecting the target image, first place the flat target in the middle of a certain area, and then use the imaging device to collect the target image, then change the flat target to another position, and continue to collect the target image, by performing multiple position transformations on the planar target to obtain multiple sets of target images. The overall process of Zhang Zhengyou's calibration algorithm is as follows: first, use an imaging device to shoot a plurality of target images of the planar target at different positions, and then detect the target images to obtain the feature points of the target images. , and then solve the intrinsic and extrinsic parameters of the imaging device under ideal distortion-free conditions and use maximum likelihood estimation to improve the accuracy, and apply least squares to find the actual radial distortion coefficient of the imaging device. Intrinsic parameters, extrinsic parameters and radial distortion coefficients are optimized by using the maximum likelihood method to improve the estimation accuracy.

The calibration process of the projection device is as follows:

其中，N_h为水平相移图案的总条纹数；H为投影装置图像的垂直分辨率。同理，投影装置投射一组垂直条纹的相位移图和格雷码编码图，可以得到同个标定点圆心(u_C,v_C)的垂直方向的绝对相位值Ф_v，其在投影装置图像上对应的坐标值u_P为：

其中，N_v为垂直相移图案的总条纹数；W为投影装置图像的水平分辨率。投影装置标定参数的方法与成像装置标定的相同，投影装置的标定即求出投影装置的内部参数矩阵。During the calibration process of the projection device, the phase shift technique is used to obtain the corresponding relationship between the image of the imaging device and the image of the projection device. The projection device projects a set of phase shift diagrams and Gray code encoding diagrams of horizontal stripes on the flat target, and the imaging device collects the target image. Then, the absolute phase value Φ _h in the horizontal direction of the center of the plane target circle (u _C , v _C ) is calculated by using the phase shift plus Gray code, and a horizontal line corresponding to the image of the projection device is found by the absolute phase value, and its coordinate value v _P is :

Among them, N _h is the total number of stripes of the horizontal phase shift pattern; H is the vertical resolution of the image of the projection device. In the same way, the projection device projects a set of phase shift diagrams and Gray code encoding diagrams of vertical stripes, and the absolute phase value Фv in the vertical direction of the circle center (u _C , _{v C} ) of the same calibration point can be obtained, which is on the image of the projection device. The corresponding coordinate value u _P is:

Among them, N _v is the total number of stripes of the vertical phase shift pattern; W is the horizontal resolution of the image of the projection device. The method of calibrating parameters of the projection device is the same as that of the imaging device. The calibration of the projection device is to obtain the internal parameter matrix of the projection device.

For each area, after the imaging device is calibrated, the projection device is calibrated. The projection device is used to project the pattern to the flat target, the imaging device collects the target image of the flat target with the pattern, and then changes the position of the flat target to continue to project the pattern and collect the target image. The position of the plane target is changed to obtain multiple sets of target images, and then the orthogonal absolute phase distribution diagram of each target image is calculated by using the obtained target images. Specifically, a Gray code combined with a phase shift method can be used to perform phase decoding, so as to calculate the quadrature absolute phase distribution map of each target image. The Gray code plus phase shift method can not only reduce the number of coding bits of the Gray code, speed up the decoding speed, but also make up for the shortcomings of the simple phase shift method and the Gray code method that it is difficult to reconstruct discontinuous positions. The specific encoding method using the combination of Gray code and phase shift method is as follows: first, project a series of gray code black and white stripe patterns to the measured object, and the area with the same code is used as a code cycle, and then the phase shift method is used to project the phase shift in turn. pattern, such that each encoded region is further subdivided consecutively. According to the orthogonal absolute phase distribution diagram of the target image, a pair of points with the same name on the image plane of the imaging device and the image plane of the projection device are found, that is, the image point on the image plane of the imaging device is used as the point to be matched, and the image point on the image plane of the projection device is used as the point to be matched. Find the first sub-pixel point with the same absolute phase as the point to be matched on the surface. Finally, according to the target image, the position of the projection device is calculated by Zhang Zhengyou's calibration algorithm.

S103 , project a specified pattern on the flat plate at different positions by using the projection device in different regions, and collect the flat plate images of the flat plate at different positions through the imaging device, and calculate the absolute value of the flat plate at different positions. phase distribution diagram, and calculate the three-dimensional information of the flat plate at different positions according to the calibration data;

In this step, in each area, the projection device with the MEMS galvanometer is used to project the specified pattern to the flat plate at different positions, and the image of the flat plate is collected by the imaging device, and then calculated according to the flat plate image The absolute phase distribution diagram of the plane plate at different positions, and the three-dimensional information of the plane plate is calculated by using pre-acquired calibration data. The calibration data includes: intrinsic parameters of the imaging device and the projection device, the distortion coefficient of the zoom lens of the imaging device, and the conversion relationship between the two coordinate systems.

The specific acquisition process of the flat plate image of the flat plate is as follows: the flat plate is placed in the measurement area, the projection device projects the quadrature sinusoidal phase-shift fringe pattern and the Gray code coding pattern on the flat plate, and the imaging device captures and collects in the Plate images of the flat plate at different positions, and then changing the position of the flat plate, repeating the process of projection and acquisition, to obtain multiple sets of plate image data.

In an embodiment, the calculating and obtaining the three-dimensional information of the flat plate at different positions according to the calibration data includes:

The three-dimensional information is calculated according to the following formula:

s _C [u _C ,v _C ,1] ^T =K _C M _C [X _W ,Y _W ,Z _W ,1] ^T

s _P [u _P ,v _P ,1] ^T =K _P I[X _W ,Y _W ,Z _W ,1] ^T

Among them, s _C and s _P are the scale factors of the imaging device and the projection device, respectively, K _C and K _P are the internal parameter matrices of the imaging device and the projection device, respectively, _MC is the external parameter matrix of the imaging device, I is the identity matrix, (u _C , v _C ) and (u _P , v _P ) are the image coordinates of the imaging device and the projection device after the distortion parameters of the three-dimensional measurement system are corrected, and T is the transpose of the matrix.

其中，s_C和s_P分别为成像装置和投影装置的尺度因子，K_C和K_P分别为成像装置和投影装置的内部参数矩阵，M_C和M_P分别为成像装置和投影装置的外部参数矩阵。所述三维测量系统的结构参数可表示为：

其中，r为投影装置坐标系转换到成像装置坐标系的旋转向量，t为投影装置坐标系转换到成像装置坐标系的平移向量；把世界坐标系建立在投影装置坐标系下，这时R_P为单位矩阵，T_P为零矩阵，R_C为世界坐标系到成像装置坐标系的旋转向量，T_C为世界坐标系到成像装置坐标系的平移向量，M_P＝I，三维重建过程变换为：

其中，I为单位矩阵；(u_C,v_C)和(u_P,v_P)是系统畸变参数矫正后的成像装置和投影装置的图像坐标。解算出P点三维坐标(X_W,Y_W,Z_W)之后，结合P点的绝对相位值Ф₁，即得到一个区域中一个成像装置像素点的相位三维映射系数表采样数据。In this embodiment, the three-dimensional reconstruction process of point P by the three-dimensional measurement system can be expressed as:

Among them, s _C and s _P are the scale factors of the imaging device and the projection device, respectively, K _C and K _P are the internal parameter matrices of the imaging device and the projection device, respectively, and _MC and _MP are the external parameters of the imaging device and the projection device, respectively. matrix. The structural parameters of the three-dimensional measurement system can be expressed as:

Among them, r is the rotation vector from the coordinate system of the projection device to the coordinate system of the imaging device, t is the translation vector from the coordinate system of the projection device to the coordinate system of the imaging device; the world coordinate system is established in the coordinate system of the projection device, then R _P is the unit matrix, T _P is a zero matrix, R _C is the rotation vector from the world coordinate system to the imaging device coordinate system, T _C is the translation vector from the world coordinate system to the imaging device coordinate system, M _P =I, the three-dimensional reconstruction process is transformed into :

Among them, I is the identity matrix; (u _C , v _C ) and (u _P , v _P ) are the image coordinates of the imaging device and the projection device after the system distortion parameters are corrected. After the three-dimensional coordinates (X _W , Y _W , Z _W ) of point P are solved, combined with the absolute phase value Φ ₁ of point P, the sampling data of the phase three-dimensional mapping coefficient table of a pixel point of an imaging device in an area is obtained.

S104. In each area, obtain the absolute phase of each pixel of the imaging device in the absolute phase distribution diagram of the plane plate, and according to the three-dimensional information of each plane plate and the absolute phase of the corresponding pixel point The mapping relationship between them establishes a three-dimensional mapping coefficient table of the corresponding area;

In this step, the absolute phase corresponding to each pixel of the imaging device in different regions is obtained according to the pre-calculated absolute phase distribution of the flat plate, and then the relationship between each pixel and the corresponding three-dimensional information is obtained. The mapping relationship between them can be used to establish the three-dimensional mapping coefficient table of the corresponding area. In each area, a position information of the flat plate is used as a set of sampling data, that is, a pixel point corresponds to a three-dimensional space point and a phase value. The plane plates at multiple positions provide multiple sets of sampling data, and by fitting the mapping coefficients, a mapping coefficient table for each pixel in an area is obtained.

In one embodiment, the step S104 includes:

According to the three-dimensional information corresponding to the pixel points, the three-dimensional mapping coefficient is calculated according to the following formula, and a three-dimensional mapping coefficient table is established:

Among them, α _n , c _X , b _n , c _Y , c _n and c _Z are three-dimensional mapping coefficients, N is the polynomial order, and Ф is the absolute phase of the corresponding pixel point.

其中，α_n，c_X，b_n，c_Y，c_n和c_Z对应着三个空间维度的映射系数。通过上述公式可计算出每个像素点对应的映射系数{a_n,b_n,c_n}，从而建立每一像素点在每一区域中的绝对相位对应的三维映射系数表。In this embodiment, given that the absolute phase of a certain pixel m _c is Φ _C , and its three-dimensional space point is (X, Y, Z), according to the three-dimensional information of the pixel, it can be deduced that:

Among them, α _n , c _X , b _n , c _Y , c _n and c _Z correspond to the mapping coefficients of the three spatial dimensions. The mapping coefficient {a _n , b _n , c _n } corresponding to each pixel can be calculated by the above formula, thereby establishing a three-dimensional mapping coefficient table corresponding to the absolute phase of each pixel in each region.

S105: Project a target pattern to the object under test by using a projection device, and collect a target focal scan image of the object under test through an imaging device, and perform deblurring on the target focal scan image to obtain a target image and calculate the focal point of the target image. absolute phase distribution map;

In this step, a projection device with a MEMS galvanometer is used to project a pattern to the object under test, the imaging device collects a target focal scan image of the object under test under a single frame exposure condition, and then the target focal scan image is captured by the imaging device. Perform deblurring processing to obtain a target image, and then calculate the absolute phase distribution map of the target image. The imaging device continuously scans the focal plane under a single-frame exposure condition to obtain a target focal scan image, and the single-frame exposure time is the current cycle size of the zoom lens that controls the imaging device. The current cycle changes as a triangular wave with time, and the maximum value and the minimum value of the triangular wave are the maximum value and the minimum value of the current value range that controls the imaging device to continuously focus on the entire large depth of field measurement scene. By controlling the current value of the imaging device, the exposure time of a single frame of the imaging device to collect the focal scan image of the target is controlled.

其中，n为自然数。When collecting the target focal scan pattern of each measured object, the period of the control current of the zoom lens is T, the maximum value is I _H , and the minimum value is the triangular wave current of _IL , and the function of the current changing with time t is :

where n is a natural number.

In one embodiment, the step S105 includes:

Inputting the target focal scan image into the integral point spread function to perform a deconvolution operation to obtain a deblurred target image;

The calculation formula of the integral point spread function is as follows:

Among them, r is the distance from the center of the circle of confusion when the object point is imaged on the sensor plane of the imaging device; b ₀ is the diameter of the circle of confusion when the current value of the control electronic zoom lens is 0, and the object point is imaged on the sensor plane of the imaging device; b ₁ is the diameter of the light spot where the object point is focused on the sensor plane of the imaging device; b ₂ is the diameter of the circle of confusion where the object point is imaged on the sensor plane of the imaging device at the time of half cycle T by controlling the current value of the electronic zoom lens; C ₁ and C ₂ for two constants.

In this embodiment, an integral point spread function is constructed based on the focal scan model of the imaging device, and the target focal scan image is deconvolved by using the integral point spread function to obtain a deblurred target focal scan. image. The calculation formula of the integral point spread function is as described above, and the calculation result is deblurred by Wiener filtering.

S106: Obtain the absolute phase of each pixel of the target image in the absolute phase distribution diagram of the target image, and search for the corresponding three-dimensional mapping coefficient in the three-dimensional mapping coefficient table of the corresponding area according to the region to which the absolute phase belongs, and use The three-dimensional mapping coefficients are calculated to obtain corresponding three-dimensional point coordinates in space.

In this step, first, according to the absolute phase distribution diagram of the target focus scan image, determine the area to which each pixel corresponding to the target focus scan image belongs, and then find out the corresponding three-dimensional mapping coefficient table in the three-dimensional mapping coefficient table of the corresponding area. The three-dimensional mapping coefficient is used to calculate the coordinates of the three-dimensional point in the corresponding space.

In one embodiment, the step S106 includes:

The three-dimensional point coordinates in the space are calculated by the following formula:

Among them, {a _n , b _n , c _n } are the three-dimensional mapping coefficients of the corresponding area, Ф is the absolute phase of the pixel, and N is the polynomial order.

In this embodiment, the absolute phase of each pixel of the imaging device in the target focal scan image is obtained as the target phase, the area to which the target phase belongs is determined, and the corresponding area is found according to the three-dimensional mapping coefficient table of the corresponding area. The three-dimensional mapping coefficients {a _n , b _n , c _n } are calculated according to the absolute phase and the three-dimensional mapping coefficients to obtain the three-dimensional point coordinates in space.

其中，{a_n,b_n,c_n}为区域2对应的三维映射系数。As shown in Figure 2, if the absolute phase of a pixel is Ф, the phase value of the pixel in area 1 ranges from Ф ¹ ₁ to Ф ¹ _n , and the phase value in area 2 ranges from Ф ² ₁ to Ф ² _n , the phase value range in region 3 is Ф ³ ₁ ∼ Ф ³ _n , and the phase value range in region n is Ф ⁿ ₁ ∼Ф ⁿ _n . Determine the area to which the absolute phase belongs. If Φ ² ₁ ≤Φ≤Φ ² _n , then the phase value of the pixel belongs to area 2. Find the three-dimensional mapping coefficient corresponding to the pixel in the three-dimensional mapping coefficient table of area 2, The corresponding three-dimensional point coordinates in space are calculated as follows:

Wherein, {a _n , b _n , c _n } are the three-dimensional mapping coefficients corresponding to the region 2 .

当L≥0时，该像点的相位值属于区域1；当L＜0时，该像点的相位值属于区域2。If Φ ² ₁ ≤Φ≤Φ ² _n and Φ ¹ ₁ ≤Φ≤Φ ¹ _n , the conditions for judging which region Φ belongs to are:

When L≥0, the phase value of the image point belongs to area 1; when L<0, the phase value of the image point belongs to area 2.

Please refer to FIG. 3. FIG. 3 is a schematic block diagram of a high-precision three-dimensional reconstruction device based on a large depth of field provided by an embodiment of the present invention. The high-precision three-dimensional reconstruction device 200 based on a large depth of field includes:

an area division unit 201, configured to perform area division on a large depth of field measurement scene;

The calibration data acquisition unit 202 is configured to perform binocular vision stereo calibration on each of the divided regions by using a 3D measurement system to obtain calibration data; wherein, the 3D measurement system includes an imaging device and a projection device;

The three-dimensional information acquisition unit 203 is used for projecting a specified pattern to the flat plate at different positions by using the projection device in different regions, and collecting the flat plate images of the flat plate in different positions through the imaging device, and calculating the flat plate The absolute phase distribution diagram at different positions, and the three-dimensional information of the flat plate at different positions is obtained by calculating according to the calibration data;

The three-dimensional mapping coefficient table obtaining unit 204 is configured to obtain, in each region, the absolute phase of each pixel of the imaging device in the absolute phase distribution map of the flat plate, and according to the three-dimensional The mapping relationship between the information and the absolute phase of the corresponding pixel point establishes a three-dimensional mapping coefficient table of the corresponding area;

The target focal scan image acquisition unit 205 is used to project the target pattern to the measured object by using the projection device, and collect the target focal scan image of the measured object through the imaging device, and perform deblurring processing on the target focal scan image to obtain the target image and calculate the absolute phase distribution map of the target image;

The spatial three-dimensional point coordinate obtaining unit 206 is used to obtain the absolute phase of each pixel of the target image in the absolute phase distribution map of the target image, and according to the area to which the absolute phase belongs, in the three-dimensional mapping coefficient table of the corresponding area The corresponding three-dimensional mapping coefficients are searched, and the corresponding three-dimensional point coordinates in space are obtained by using the three-dimensional mapping coefficients.

In an embodiment, the area dividing unit 201 includes:

a position calibration unit, configured to use a calibration algorithm to calibrate the positions of the imaging device and the projection device;

The control current value recording unit is used to obtain the maximum and minimum values of the total control current measured by the zoom lens of the imaging device in the measurement scene of the large depth of field, and when the zoom lens of the imaging device is focused on the center of each area The corresponding area control current value, and the maximum and minimum value of the total control current and the area control current value corresponding to each area are recorded.

In one embodiment, the calibration data acquisition unit 202 includes:

The coordinate system establishing unit is used for establishing the coordinate system of the imaging device with the optical center of the imaging device as the origin and the optical axis of the imaging device as the Z axis; with the optical center of the projection device as the origin, and The optical axis of the projection device is the Z axis, and a coordinate system of the projection device is established;

a calibration data calculation unit, configured to obtain the conversion relationship between the intrinsic parameters of the imaging device and the imaging device coordinate system and the conversion relationship between the intrinsic parameters of the projection device and the projection coordinate system by using a binocular vision stereo calibration algorithm, And calculate the transformation relationship between the imaging device coordinate system and the projection coordinate system to obtain calibration data.

In one embodiment, the three-dimensional information acquisition unit 203 includes:

The three-dimensional information formula calculation unit is used to calculate the three-dimensional information according to the following formula:

s _C [u _C ,v _C ,1] ^T =K _C M _C [X _W ,Y _W ,Z _W ,1] ^T

s _P [u _P ,v _P ,1] ^T =K _P I[X _W ,Y _W ,Z _W ,1] ^T

Among them, s _C and s _P are the scale factors of the imaging device and the projection device, respectively, K _C and K _P are the internal parameter matrices of the imaging device and the projection device, respectively, _MC is the external parameter matrix of the imaging device, I is the identity matrix, (u _C , v _C ) and (u _P , v _P ) are the image coordinates of the imaging device and the projection device after the distortion parameters of the three-dimensional measurement system are corrected, and T is the transpose of the matrix.

In one embodiment, the three-dimensional mapping coefficient table obtaining unit 204 includes:

The three-dimensional mapping coefficient calculation unit is used to calculate the three-dimensional mapping coefficient according to the following formula according to the three-dimensional information corresponding to the pixel point, and establish a three-dimensional mapping coefficient table:

Among them, α _n , c _X , b _n , c _Y , c _n and c _Z are three-dimensional mapping coefficients, N is the polynomial order, and Ф is the absolute phase of the corresponding pixel point.

In one embodiment, the target focal scan image acquisition unit 205 includes:

a deblurring processing unit for inputting the target focal scan image into an integral point spread function to perform a deconvolution operation to obtain a deblurred target image;

The integral point spread function calculation unit, the calculation formula used for the integral point spread function is as follows:

Among them, r is the distance from the center of the circle of confusion when the object point is imaged on the sensor plane of the imaging device; b ₀ is the diameter of the circle of confusion when the current value of the control electronic zoom lens is 0, and the object point is imaged on the sensor plane of the imaging device; b ₁ is the diameter of the light spot where the object point is focused on the sensor plane of the imaging device; b ₂ is the diameter of the circle of confusion where the object point is imaged on the sensor plane of the imaging device at the time of half cycle T by controlling the current value of the electronic zoom lens; C ₁ and C ₂ for two constants.

In one embodiment, the spatial three-dimensional point coordinate obtaining unit 206 includes:

A space three-dimensional point coordinate calculation unit, used to calculate the space three-dimensional point coordinates by the following formula:

Among them, {a _n , b _n , c _n } are the three-dimensional mapping coefficients of the corresponding area, Ф is the absolute phase of the pixel, and N is the polynomial order.

An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored on the memory and running on the processor, where the processor implements the above-mentioned computer program when executing the computer program A high-precision 3D reconstruction method based on large depth of field.

Embodiments of the present invention further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the above-mentioned high-precision three-dimensional reconstruction method based on a large depth of field is implemented .

The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities or operations. There is no such actual relationship or sequence between operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article, or device that includes the element.

Claims (10)

1. a high-precision three-dimensional reconstruction method based on large depth of field, is characterized in that, comprises the following steps: Regional division of large depth of field measurement scenes; Use a three-dimensional measurement system to perform binocular vision stereo calibration on each of the divided regions to obtain calibration data; wherein, the three-dimensional measurement system includes an imaging device and a projection device; In different areas, the projection device is used to project the specified pattern to the flat plate at different positions, and the image of the flat plate at different positions is collected by the imaging device, and the absolute phase distribution of the flat plate at different positions is calculated. Figure, and calculate the three-dimensional information of the flat plate at different positions according to the calibration data; In each area, the absolute phase of each pixel point of the imaging device in the absolute phase distribution diagram of the plane plate is obtained, and the absolute phase of each pixel point of the plane plate is obtained according to the difference between the three-dimensional information of each plane plate and the absolute phase of the corresponding pixel point. The mapping relationship establishes the three-dimensional mapping coefficient table of the corresponding area; Use the projection device to project the target pattern to the measured object, and collect the target focal scan image of the measured object through the imaging device, and deblur the target focal scan image to obtain the target image and calculate the absolute phase of the target image Distribution; Obtain the absolute phase of each pixel of the target image in the absolute phase distribution diagram of the target image, and search for the corresponding three-dimensional mapping coefficient in the three-dimensional mapping coefficient table of the corresponding area according to the region to which the absolute phase belongs, and use the The three-dimensional mapping coefficient is calculated to obtain the corresponding three-dimensional point coordinates in space. 2. The high-precision three-dimensional reconstruction method based on a large depth of field according to claim 1, wherein after the large depth of field measurement scene is divided into regions, the method comprises: Use a calibration algorithm to calibrate the positions of the imaging device and the projection device; Obtain the maximum and minimum values of the total control current measured by the zoom lens of the imaging device to measure the large depth of field measurement scene, and the corresponding area control current values when the zoom lens of the imaging device is focused on the center position of each area, and Record the maximum and minimum values of the total control current and the region control current value corresponding to each region. 3. the high-precision three-dimensional reconstruction method based on large depth of field according to claim 1, is characterized in that, described utilizes three-dimensional measurement system to carry out binocular vision stereo calibration to each area that divides, obtains calibration data, comprises: Taking the optical center of the imaging device as the origin, and taking the optical axis of the imaging device as the Z axis, the imaging device coordinate system is established; taking the optical center of the projection device as the origin, and taking the optical axis of the projection device as the origin For the Z axis, establish the coordinate system of the projection device; Use the binocular vision stereo calibration algorithm to obtain the conversion relationship between the intrinsic parameters of the imaging device and the imaging device coordinate system and the conversion relationship between the intrinsic parameters of the projection device and the projection coordinate system, and calculate the imaging device coordinates The conversion relationship between the system and the projected coordinate system is used to obtain the calibration data. 4. The high-precision three-dimensional reconstruction method based on a large depth of field according to claim 1, wherein the three-dimensional information obtained by calculating the plane plate at different positions according to the calibration data, comprising: The three-dimensional information is calculated according to the following formula: s _C [u _C ,v _C ,1] ^T =K _C M _C [X _W ,Y _W ,Z _W ,1] ^T s _P [u _P ,v _P ,1] ^T =K _P I[X _W ,Y _W ,Z _W ,1] ^T Among them, s _C and s _P are the scale factors of the imaging device and the projection device, respectively, K _C and K _P are the internal parameter matrices of the imaging device and the projection device, respectively, _MC is the external parameter matrix of the imaging device, I is the identity matrix, (u _C , v _C ) and (u _P , v _P ) are the image coordinates of the imaging device and the projection device after the distortion parameters of the three-dimensional measurement system are corrected, and T is the transpose of the matrix. 5 . The high-precision three-dimensional reconstruction method based on a large depth of field according to claim 1 , wherein, in each area, the absolute phase distribution map of each pixel of the imaging device on the flat plate is obtained. 6 . The absolute phase in , and the three-dimensional mapping coefficient table of the corresponding area is established according to the mapping relationship between the three-dimensional information of each said flat plate and the absolute phase of the corresponding pixel point, including: According to the three-dimensional information corresponding to the pixel points, the three-dimensional mapping coefficient is calculated according to the following formula, and a three-dimensional mapping coefficient table is established:

Among them, α _n , c _X , b _n , c _Y , c _n and c _Z are three-dimensional mapping coefficients, N is the polynomial order, and Ф is the absolute phase of the corresponding pixel point. 6. The high-precision three-dimensional reconstruction method based on a large depth of field according to claim 1, wherein the projection device is used to project the target pattern to the measured object, and the target focal scan image of the measured object is collected by the imaging device, Deblurring the target focal scan image to obtain a target image and calculate the absolute phase distribution of the target image, including: Inputting the target focal scan image into the integral point spread function to perform a deconvolution operation to obtain a deblurred target image; The calculation formula of the integral point spread function is as follows:

Among them, r is the distance from the center of the circle of confusion when the object point is imaged on the sensor plane of the imaging device; b ₀ is the diameter of the circle of confusion when the current value of the control electronic zoom lens is 0, and the object point is imaged on the sensor plane of the imaging device; b ₁ is the diameter of the light spot where the object point is focused on the sensor plane of the imaging device; b ₂ is the diameter of the circle of confusion where the object point is imaged on the sensor plane of the imaging device at the time of half cycle T by controlling the current value of the electronic zoom lens; C ₁ and C ₂ for two constants. 7. The high-precision three-dimensional reconstruction method based on a large depth of field according to claim 1, wherein the acquiring the absolute phase of each pixel of the target image in the absolute phase distribution map of the target image, and according to The region to which the absolute phase belongs looks up the corresponding three-dimensional mapping coefficient in the three-dimensional mapping coefficient table of the corresponding region, and uses the three-dimensional mapping coefficient to calculate the corresponding three-dimensional point coordinates in space, including: The three-dimensional point coordinates in the space are calculated by the following formula:

Among them, {a _n , b _n , c _n } are the three-dimensional mapping coefficients of the corresponding area, Ф is the absolute phase of the pixel, and N is the polynomial order. 8. A high-precision three-dimensional reconstruction device based on a large depth of field, characterized in that, comprising: The area division unit is used to divide the area of the large depth of field measurement scene; A calibration data acquisition unit, configured to perform binocular stereo calibration on each of the divided regions by using a 3D measurement system to obtain calibration data; wherein the 3D measurement system includes an imaging device and a projection device; A three-dimensional information acquisition unit is used to project a specified pattern on a flat plate at different positions by using a projection device in different areas, and collect the flat plate images of the flat plate in different positions through an imaging device, and calculate the flat plate at different positions. The absolute phase distribution diagram at different positions, and the three-dimensional information of the flat plate at different positions is obtained by calculating according to the calibration data; A three-dimensional mapping coefficient table acquiring unit, configured to acquire, in each region, the absolute phase of each pixel of the imaging device in the absolute phase distribution map of the plane plate, and according to the three-dimensional information of each plane plate A three-dimensional mapping coefficient table of the corresponding area is established with the mapping relationship between the absolute phases of the corresponding pixel points; The target focal scan image acquisition unit is used to project the target pattern to the measured object by using the projection device, collect the target focal scan image of the measured object through the imaging device, and perform deblurring processing on the target focal scan image to obtain the target image and calculate the absolute phase distribution map of the target image; A spatial three-dimensional point coordinate obtaining unit, used to obtain the absolute phase of each pixel of the target image in the absolute phase distribution map of the target image, and look up the three-dimensional mapping coefficient table of the corresponding area according to the area to which the absolute phase belongs Corresponding three-dimensional mapping coefficients are calculated by using the three-dimensional mapping coefficients to obtain corresponding three-dimensional point coordinates in space. 9. A computer device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements the computer program as claimed in the claims The high-precision three-dimensional reconstruction method based on a large depth of field according to any one of 1 to 7. 10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when executed by a processor, the computer program causes the processor to execute any one of claims 1 to 7 The high-precision three-dimensional reconstruction method based on large depth of field described in item.

Images