CN111091599A - A calibration method of multi-camera-projector system based on spherical calibration object - Google Patents

Abstract

The invention discloses a calibration method of a multi-camera-projector system based on a sphere calibration object. The method comprises the following steps: 1) extracting the outline of the sphere imaged under the camera plane and fitting the outline into an elliptic curve; 2) acquiring an absolute phase value of each pixel of the camera by using a structured light method, and acquiring an imaging curve of a sphere under a projector by using a pole-polar line relation; 3) calibrating an internal parameter matrix according to more than 3 curves corresponding to the camera and the projector respectively; 4) and (4) calculating the coordinates of the sphere center under each device coordinate system, and unifying all the devices under the same world coordinate system by using a global registration method. The experimental results of the method in the embodiment prove the feasibility and the accuracy of the method.

Description

A calibration method of multi-camera-projector system based on spherical calibration object

technical field

The invention relates to a simultaneous calibration technology of multi-camera and multi-projector, in particular to a multi-camera-projector system calibration method based on a spherical calibration object, belonging to the technical field of optical precision measurement.

Background technique

With the continuous growth of industrial measurement requirements, optical 3D measurement has attracted more and more attention due to its easy operation and accurate data. Among many optical 3D measurement techniques, Fringe Projection Profilometry (FPP) has been widely studied and applied due to its outstanding advantages of high speed, high precision and large field of view. In general, a structured light measurement system based on the FPP principle differs from traditional stereo vision in that it replaces one of the cameras with a digital projector. The function of the projector is to project the encoded pattern template to the measured object, and the pattern on these objects is captured by the camera to establish the corresponding relationship between the camera pixel and the projector pixel through the corresponding decoding algorithm, and then combine the calibration parameters to use the triangle. method to reconstruct the three-dimensional shape of the target object. For measurement systems of such camera/projector pairs, proper calibration of each component is critical for accurate reconstruction of the 3D shape.

The calibration of such structured light systems is more complicated due to the use of projectors. The more classic method is to regard the projector as an inverse camera, obtain the absolute phase value through the phase shift method to establish the pixel-level mapping between the camera and the projector, and then calibrate the system by the general camera calibration method. However, this method requires a flat calibration board. When multiple cameras and projectors are used, the flat calibration board cannot be guaranteed to be "seen" by the system at multiple angles, so it is difficult to apply to the calibration of multi-camera-projector systems. . In addition, if multiple plane calibration plates are assembled into a cube calibration object, on the one hand, the assembly accuracy is not enough. It is not uniform, so it is difficult to determine the correspondence of the marker points on each surface, resulting in the failure of the calibration.

SUMMARY OF THE INVENTION

In view of the above technical problems, the present invention provides a calibration method for a multi-camera-projector system based on a spherical calibration object. The method takes advantage of the isotropy that the sphere is visible at any angle, and the cameras and projectors in the system are synchronized by the projection profiles of the sphere on the phase planes of the respective cameras and projectors.

The present invention adopts the following technical solutions for solving the above-mentioned technical problems:

The present invention provides a calibration method for a multi-camera-projector system based on a spherical calibration object. In the multi-camera-projector system, a single camera and a single projector are combined into a structured light measurement unit, and the multi-camera-projector system is composed of a single camera and a single projector. Several structured light measurement units in the instrument system are placed in a ring with the measurement area as the center, and the optical axis of each camera and each projector is aligned with the measurement area; each structured light measurement unit works independently, and each structured light measurement unit in The projection format of the projector covers the shooting format of the camera;

The calibration method is specifically:

Step 1, place the spherical calibration object in the measurement area of the multi-camera-projector system, start each structured light measurement unit to perform pattern projection and photography on the spherical calibration object in turn, and obtain the sphere calibration objects under each camera and projector respectively. projection curve;

Step 2, transform the position of the spherical calibration object in the central measurement area and repeat step 1 to obtain at least 3 different projection curves in the respective phase planes of each camera and projector, and obtain each camera and projector from these projection curves. the internal parameters;

Step 3, in each camera/projector coordinate system, use the following formula to find the coordinates of the center of the sphere calibration object in this coordinate system:

Among them, α=tan(θ/2), θ is the cone angle, K is the internal parameter of the camera/projector, r is the radius of the spherical calibration object, and o is the projected coordinate of the spherical center of the spherical calibration object;

Step 4: Use the global matching method to match the spherical center coordinates in each camera/projector coordinate system to obtain the relative rotation matrix and relative translation vector between the cameras/projectors, and finally unify all cameras and projectors into the same system. In a world coordinate system, the calibration is completed.

As a further technical solution of the present invention, in step 1, the method for obtaining the projection curve of the spherical calibration object under each camera and the projector is specifically:

Step 1.1, shoot the spherical calibration object through the camera in each structured light measurement unit, extract the sub-pixel-level outer contour of the spherical calibration object imaged on the camera, and use the least squares method to fit the sub-pixel-level outer contour to an ellipse curve, the elliptic curve is the projection curve of the sphere calibration object under the camera;

Step 1.2, the projector in each structured light measurement unit projects a set of phase-shifted sinusoidal fringes and Gray code patterns to the sphere calibration object in turn, and the corresponding camera takes pictures, and the absolute phase of each pixel of the camera is solved by the phase unwrapping method. value, and then use the eight-point method to calculate the polar geometric relationship between the camera and the projector by phase mapping, and obtain the polar coordinates in the phase plane of the camera and the phase plane of the projector;

Step 1.3, use the absolute phase value of each camera pixel obtained in 1.2 as the gray value of the image to obtain a phase map, and locate the phase-free phase in the elliptic curve fitted in 1.1 due to the occlusion of the projector light in this phase map. value area, extract the outer boundary of the non-phase value area that is biased towards the center of the elliptic curve, and fit the outer boundary into a quadratic curve, the quadratic curve and the elliptic curve intersect at two points, these two points are the apparent Two image points where the intersection of the contour lines is in the camera plane;

In step 1.4, the two image points in _1.3 are corresponding to the _{projector plane} through their absolute phase _values , and two intersection points are obtained. The pole coordinates er in the phase plane and the projection curve C _P of the spherical calibration object under the projector satisfy l=C _P · er , and _{i P1} , _{i P2} are located on C _P , thus the spherical calibration object under the projector is obtained. The projection curve C _P .

As a further technical solution of the present invention, the internal parameter K of the camera/projector in step 2 is:

Among them, f _u , f _v are the focal lengths in the directions of the u and v axes, respectively, s is the offset between the u and v axes, and (u ₀ , v ₀ ) are the coordinates of the principal point of the phase plane.

As a further technical solution of the present invention, in step 2, the double tangent, positive semi-definite programming or orthogonal constraint method is applied to obtain the internal parameters of each camera and projector.

As a further technical solution of the present invention, the value of α in step 3 is uniquely determined by the eigenvalue of the formula K ^T CK, and C is the camera projection curve.

Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: the present invention overcomes the technical bottleneck that the traditional plane calibration method cannot be applied to the calibration of a multi-camera-projector system, so that the cameras and projectors under multiple angles are can be calibrated at the same time. The method is simple to operate and can be calibrated using only spheroid calibrators without additional equipment and steps. In addition, once the cameras and projectors under multiple viewing angles are calibrated, the reconstructed point clouds can be automatically combined without additional algorithms. Therefore, the calibration method proposed in the present invention is of great significance for building a multi-camera-projector fast measurement system. , the fast system calibrated by this method can measure the multi-angle three-dimensional shape of the moving object in real time, and stitch it into a full-view three-dimensional shape in real time.

Description of drawings

Fig. 1 is the imaging schematic diagram of the sphere under the camera-projector system;

2 is a schematic diagram of the geometric relationship of the projected imaging of the intersection of the apparent contour lines in two phase planes under two viewing angles;

FIG. 3 is a schematic diagram of the relationship between poles and poles in the right imaging plane;

Figure 4 shows a system composed of four camera-projector pairs using sphere calibration and its reconstruction results, where (a) is a system composed of four camera-projector pairs, and (b) is a multi-angle image of four point clouds. Rebuild the result.

Detailed ways

Below in conjunction with accompanying drawing, the technical scheme of the present invention is described in further detail:

1. Multi-camera multi-projector system configuration and model description:

In the present invention, a single camera and a single projector are formed into a structured light measurement unit, and the projection width of the projector needs to cover the width captured by the camera. Each measurement unit works independently, enabling 3D reconstruction of the area it covers. A plurality of measurement units are arranged in a ring around the central measurement area, and the optical axis of each device is approximately aligned with the central measurement area.

In the present invention, we regard the projector as a reverse camera, and use the pinhole model to describe the parameter models of the camera and the projector. Therefore, the parameters of each device have been represented by the projection matrix P=K[R|t]. in,

K is an internal parameter of the device, including the focal lengths f _u , f _v in the u and v axes, the offset s between the u and v axes, and the coordinates of the principal point of the phase plane (u ₀ , v ₀ ). R, t represent the rotation matrix and translation vector from the device coordinate system to the world coordinate system, respectively.

2. Spherical projection extraction of the projector plane

Figure 1 is a schematic diagram of the imaging of a sphere in a structured light measurement unit (including a camera and a projector). From the knowledge of projection geometry, it can be known that the sphere on the phase plane of the camera is like an ellipse. The outer contour of the ellipse can be fitted to a quadratic curve, whose matrix representation is denoted as C _C . The outer contour is formed by the projection of the apparent contour line Γ _C of the surface of the sphere through the small hole. Since the imaging of the sphere on the camera can be directly obtained by shooting, the imaging of the sphere on the projector cannot be directly obtained. And as shown in Figure 1, the contour curves C _C , C _P of the imaging of the same ball in the camera and the projector are generated by the projection of different apparent contour lines Γ _C , Γ _P , which cannot be directly established with each other. Therefore, the traditional phase mapping cannot be applied to obtain the imaging curve of the sphere under the projector.

The present invention derives the projection curve of the sphere on the projector plane from the relationship between the epipolar geometry and the imaging curve, and the specific steps are as follows:

Step 2.1: First, shoot the sphere through the camera, extract the subpixel-level outer contour of the sphere imaged on the camera, and use the least squares method to fit it to the elliptic curve C _C .

Step 2.2: A set of phase-shifted sinusoidal fringes and Gray code patterns are sequentially projected onto the sphere using a projector and captured by a camera. The absolute phase value of each pixel of the camera is solved by the phase unwrapping method.

The above phase unwrapping method will calculate the absolute phase value of each pixel, and each pixel has a preset absolute phase value in the projector. Camera pixels and projector pixels have the same phase value indicating that they are corresponding points. Through this phase mapping, the corresponding pixel points between the camera and the projector can be known, and the epipolar geometric relationship can be obtained by further calculation.

Then use the eight-point method to calculate the antipolar geometric relationship between the camera and the projector from the phase mapping, and obtain the coordinates of the poles in the two phase planes.

Step 2.3: The obtained absolute phase value of each camera pixel is used as the gray value of the image to obtain a phase map, and the phase _- free area (camera and There is a certain viewing angle difference, so that a certain area of the sphere captured by the camera cannot be illuminated by the projector light, so this area has no phase value). The outer boundary of the region, which is biased towards the center of the ellipse C _C , is extracted and fitted into a quadratic curve, which intersects the elliptic curve C _C at two points. It can be known from the relevant geometric knowledge that these two points are the image points of the intersection of the apparent contour lines Γ _C , Γ _P in the camera plane, as shown in Fig. 2 .

Step 2.4: Corresponding two image points to the projector plane through their absolute phase values (sub-pixel points are obtained by interpolation of pixel points with valid phase values in the neighborhood), and obtain intersection points i _P1 , i _P2 , as shown in Figure 3 shown. Assuming that the projector is placed on the right side of the camera, the coordinate of the right pole in the projector plane is er , and the two intersection points obtained by mapping are _{i P1} , i _P2 . A straight line l=i _P1 ×i _P2 is determined by two points i _P1 , i _P2 . It is easy to prove that the image C _P and the pole _er of the line and the sphere under the projector satisfy the pole-polar relationship, that is,

l=C _P · _er (2)

Note that the curves, lines and points here are all represented by homogeneous coordinates.

Among them, the line connecting i _P1 , _{i P2} and the pole _{er is tangent to the curve CP, and the tangent point is i P1 ,} i _P2 . Because the curve C _P has 5 unknowns, equation (2) can provide 3 constraints on C _P , and i _P1 and i _P2 are located on C _P to provide two other independent constraints, so the curve C _P can be obtained analytically untie.

3. Multi-camera-projector system calibration

After obtaining the imaging curve of the sphere under the projector according to the above content, the parameters can be calibrated using the traditional sphere method. Generally, the imaging formula of a sphere Q with a radius r whose center is located at the coordinate a can be expressed as,

where C ^* and Q ^* are the dual forms of the camera projection curve C and the sphere Q respectively, which are their respective inverses in expression. Set the world coordinate system to coincide with the camera coordinate system, so the above formula is finally simplified to

βC ^* = KK ^T -oo ^T (4)

The parameter β is the scale factor, K is the parameter matrix in the device, and o is the projection coordinate of the center of the sphere (the projection here refers to the projection of a point in space to a point in the phase plane of the camera/projector, and the point in the phase plane is spatial point projection coordinates). For the solution of the internal parameter matrix in equation (4), methods such as bitangent, positive semi-definite programming or orthogonal constraints can be applied.

The specific steps of calibration are as follows:

Step 3.1: Place the calibration sphere in the measurement area of the multi-camera-projector system, start each measurement unit to perform pattern projection and photography on the sphere in turn, and obtain the spheres under each camera and projector according to the previous steps of sphere projection extraction. Sphere projection curve.

Step 3.2: Perform step 3.1 by changing the position of the calibration ball multiple times. Each camera and projector obtains at least 3 different projection curves in their respective phase planes, and the internal parameters K of their respective devices are obtained from these projection curves using methods such as bitangent, positive semi-definite programming or orthogonal constraints.

Step 3.3: In each device coordinate system, we can use formula (5) to find the coordinates of the center of the sphere in the device coordinates.

Wherein, α=tan(θ/2), and θ is the cone angle in FIG. 1 . The value of α can be uniquely determined by the eigenvalues of the formula K ^T CK. Since multiple positions of the sphere are transformed, a set of spherical center coordinates can be obtained in each device coordinate system.

Step 3.4: Use the global matching method to match the spherical center coordinates under the coordinate system of each device, obtain the relative rotation matrix and relative translation vector between the devices, and finally unify all cameras and projectors into the same world coordinate system, Calibration is complete.

As an example, two embodiments of the present invention for calibrating a single camera-projector system and a plurality of camera-projector systems are as follows:

Example 1: Calibrating a single camera-projector system

Step 1: Mount four matte ceramic balls with a diameter of 50.80mm on the fixing bracket and place them in the common field of view of the camera and projector. In this embodiment, an 8-step phase shift method is adopted, and corresponding fringes and Gray code patterns are projected onto a sphere according to the number of phase shift steps, and a phase map is generated by de-phasing.

Step 2: Use the camera picture to extract the outer contours of the four spheres, and use the least squares method to fit them into 4 elliptic curves C _C respectively; at the same time, apply the phase diagram and the pole-polar line relationship to obtain the four spheres under the projector. Elliptic curve C _P .

Step 3: Apply the orthogonal constraint method to the two sets of curves under the camera and the projector to obtain their respective internal parameter matrices K.

Step 4: Find the positions of the four balls in the camera and projector coordinate systems respectively, use the rigid body transformation method to register the two point sets, obtain the relative rotation matrix and translation vector between the camera and the projector, and complete the calibration .

Example 2: Calibrating a multi-camera-projector system

Step 1: Place 4 structured light measurement units consisting of cameras and projectors around the measurement area.

Step 2: In order to avoid the situation that the spheres may block each other at some viewing angles when using multiple spheres at the same time, in this embodiment, only one matte ceramic sphere with a diameter of 50.80mm is used, and it is placed in the measurement field of view .

Step 3: Start each measurement unit in turn to scan the sphere.

Step 4: Transform the position of the sphere several times, and repeat Step 3 until more than 3 projection curves are obtained for each device.

Step 5: Use the projection curve under each device to calibrate the internal parameters of each device.

Step 4: Find the position of the sphere's center in each camera and projector coordinate system at each position, use the global registration method to register all point sets at the same time, and obtain the relative rotation between all cameras and projectors The matrix and translation vector unify all devices into the same coordinate system to complete the calibration.

The system consisting of 4 camera-projector pairs is calibrated using a sphere as shown in (a) of Figure 4, and the reconstruction results of 4 point clouds under multi-angle as shown in (b) of Figure 4 are obtained.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this, any person familiar with the technology can understand the transformation or replacement that comes to mind within the technical scope disclosed by the present invention, All should be included within the scope of the present invention, therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (5)

1. A calibration method of a multi-camera-projector system based on a sphere calibration object is characterized in that in the multi-camera-projector system, a single camera and a single projector form a structured light measuring unit, a plurality of structured light measuring units in the multi-camera-projector system are annularly arranged by taking a measuring area as a center, and the optical axes of each camera and each projector are aligned with the measuring area; each structured light measuring unit works independently, and the projection breadth of the projector in each structured light measuring unit covers the shooting breadth of the camera;

the calibration method specifically comprises the following steps:

step 1, placing a sphere calibration object in a measurement area of a multi-camera-projector system, starting each structured light measurement unit to sequentially perform pattern projection and shooting on the sphere calibration object, and respectively obtaining projection curves of the sphere calibration object under each camera and each projector;

step 2, changing the position of the sphere calibration object in the central measurement area and repeatedly executing the step 1 to obtain at least 3 different projection curves in respective phase planes of each camera and each projector, and obtaining the internal parameters of each camera and each projector according to the projection curves;

and 3, under each camera/projector coordinate system, calculating the coordinates of the sphere center of the sphere calibration object under the coordinate system by using the following formula:

wherein α is tan (θ/2), θ is a cone angle, K is an internal parameter of the camera/projector, r is a radius of the sphere calibration object, and o is a projection coordinate of a sphere center of the sphere calibration object;

and 4, matching the coordinates of the center of sphere under each camera/projector coordinate system by using a global matching method to obtain a relative rotation matrix and a relative translation vector between the cameras/projectors, and finally unifying all the cameras and the projectors under the same world coordinate system to finish calibration.

2. The calibration method of a multi-camera-projector system based on a sphere calibration object as claimed in claim 1, wherein the method for obtaining the projection curve of the sphere calibration object under each camera and projector in step 1 comprises:

step 1.1, shooting a sphere calibration object through a camera in each structured light measuring unit, extracting a sub-pixel level outer contour of the sphere calibration object imaged on the camera, and fitting the sub-pixel level outer contour into an elliptic curve by using a least square method, wherein the elliptic curve is a projection curve of the sphere calibration object under the camera;

step 1.2, the projector in each structured light measurement unit sequentially projects a group of phase shift sine stripes and Gray code patterns to the sphere calibration object, the phase shift sine stripes and the Gray code patterns are shot by the corresponding camera, the absolute phase value of each pixel of the camera is solved by a phase expansion method, the antipodal geometric relation between the camera and the projector is calculated by phase mapping by using an eight-point method, and the pole coordinates in the phase plane of the camera and the phase plane of the projector are obtained;

step 1.3, the absolute phase value of each camera pixel solved in step 1.2 is used as the gray value of an image to obtain a phase diagram, a phase value-free area caused by the fact that light of a projector is shielded in an elliptic curve fitted in step 1.1 is positioned in the phase diagram, the outer boundary of the phase value-free area, which is deviated to one side of the center of the elliptic curve, is extracted, the outer boundary is fitted into a quadratic curve, the quadratic curve and the elliptic curve are intersected at two points, and the two points are two image points of the intersection point of the apparent contour line in the plane of the camera;

step 1.4, corresponding the two image points in step 1.3 to the plane of the projector through the absolute phase values thereof to obtain two intersection points i_P1,i_P2Is located atLine i ═ i_P1×i_P2Coordinate e of pole in the plane of the projector_rAnd projection curve C of sphere calibration object under projector_PSatisfy l ═ C_P·e_rAnd i is_P1,i_P2Is located at C_PFrom the above, the projection curve C of the sphere calibration object under the projector is obtained_P。

3. The calibration method of multi-camera-projector system based on sphere calibration object as claimed in claim 1, wherein the intrinsic parameters K of the camera/projector in step 2 are:

wherein f is_u,f_vFocal length in the u and v axis directions, s is the offset between the u and v axes, respectively, (u₀,v₀) Is the phase plane principal point coordinate.

4. The calibration method of multi-camera-projector system based on sphere calibration object as claimed in claim 1, wherein the bi-tangential, semi-orthogonal planning or orthogonal constraint method is applied in step 2 to obtain the intrinsic parameters of each camera and projector.

5. The calibration method of a multi-camera-projector system based on sphere calibration object as claimed in claim 1, wherein the value of α in step 3 is represented by the formula K^TThe characteristic value of CK is uniquely determined, and C is a camera projection curve.

Images