CN101329764B - Method of Camera Calibration Using Two Arbitrary Coplanar Circles - Google Patents

Abstract

The invention relates to a camera calibration method using two arbitrary coplanar circles, using two arbitrary coplanar circles with different centers as calibration objects. Use the camera to shoot the calibration object to get a clear image, determine the equations of the two ellipse images in the image, and find the image point coordinates of the virtual circle point. The focal length of the camera is determined by using the image point coordinates of the imaginary circle point and the projection equation of the absolute conic curve. Establish a world coordinate system, the origin is at the center of any one of the two circles, and the Z axis is parallel to the normal direction of the plane where the two circles are located. The rotation matrix is determined according to the relationship among vanishing point, vanishing line and ellipse image. The position of the origin of the world coordinate system in the camera coordinate system is calculated as the translation vector. The invention can easily and conveniently determine all the variable parameters of each calibration of the camera, can realize automatic calibration, and reduces calculation errors caused by manual intervention, and is especially suitable for non-contact industrial detection and vision-based autonomous navigation systems.

Description

Method of Camera Calibration Using Two Arbitrary Coplanar Circles

technical field

The invention relates to a camera calibration method using two arbitrary coplanar circles, which can accurately determine the camera focal length, rotation matrix, translation vector and other parameters. The invention belongs to the field of advanced measurement technology, and is especially suitable for camera calibration in computer stereo vision, non-contact industrial detection and vision-based autonomous navigation system.

Background technique

The purpose of camera calibration is to use specific calibration objects to determine the internal and external parameters of the camera. Camera calibration is the basic requirement for obtaining three-dimensional information from two-dimensional images in the field of computer vision, and it is an essential step to complete many visual tasks. With the popularity of cameras, many non-vision professionals need a simple and flexible calibration method to help them complete vision-related work. Camera calibration is the basis for non-contact measurement and autonomous navigation based on machine vision. Because the circle has the advantages of easy detection and anti-occlusion, it is usually used as a calibration object to calibrate the camera.

A lot of work has been done on using multiple circles for camera calibration with some results. A recently proposed algorithm for calibrating two or more coplanar circles (P.Gurdjos, P.Sturm, and Y.H.Wu, "Euclidean structure from N≥2 parallel circles: theory and algorithms," in Proc.Eur.Conf.on Computer Vision, pp.238-252, IEEE(2006)) requires at least three different calibration object images, which mainly involves the calculation of the internal parameters of the camera, while ignoring the calculation of the external parameters of the camera to a certain extent. However, the external parameters of the camera are very important for computer vision work, such as the need for autonomous navigation and 3D pose determination of objects. In addition, in real applications, the position of the camera is often changed, and the corresponding focal length is adjusted accordingly, while other internal parameters of the camera, such as aspect ratio, tilt factor and principal point, remain almost unchanged. There are also methods to use two arbitrary coplanar circles for camera calibration (Q.Chen, H.Wu, and T.Wada, "Camera calibration with two arbitrary coplanar circles," in Proc.Eur.Conf.on Computer Vision, pp. 521-532, IEEE (2004)), but this is a method based on iterative optimization, the convergence of the algorithm is very dependent on the appropriate initialization conditions, and there is no clear way to initialize the algorithm. In addition, this method starts from the perspective of analytic geometry and requires quite complex equations to describe the oblique cone determined by the optical center and the ellipse image. Therefore, under the condition of known constant parameters: aspect ratio, tilt factor and principal point, a simple and stable method that can fully determine the focal length and external parameters of the camera has great practical application value.

Contents of the invention

The purpose of the present invention is to address the deficiencies in the prior art, and propose a method for camera calibration using two arbitrary coplanar circles, which can easily determine the focal length and external parameters of the camera, and can realize fully automatic calibration without man-machine interact.

The above object of the present invention is achieved through the following technical scheme: the calibration objects used in the present invention are two arbitrary coplanar circles with different centers. Place the calibration object in the field of view of the camera, and use the camera to shoot the calibration object to obtain a clear image; detect the boundary of the two ellipse images in the image, and use the boundary pixels to fit two elliptic equations; use the fitted Two elliptic equations to find the corresponding image point positions of two groups of virtual circle points in the image, so as to obtain the image point coordinates of two different groups of virtual circle points; use the image point coordinates of these two groups of virtual circle points and the absolute quadratic curve The focal length of the camera is obtained from the projection curve equation; the rotation matrix is determined according to the relationship between the vanishing point, the vanishing line and the ellipse image, and finally the translation vector is obtained according to the particularity of the established coordinate system to complete the calibration of the camera parameters.

The measurement method involved in the present invention comprises the following specific steps:

1. First make a calibration object. The calibration objects of the present invention are two arbitrary coplanar circles with different centers.

2. Shoot the calibration object with a camera to obtain a calibration object image with two elliptical images, to ensure that the imaging is clear and each circle is not covered by more than one-third of the area.

3. Use the canny operator to detect the boundary of each ellipse image on the calibration object image respectively, so as to obtain two sets of point sets composed of pixel points on the edge of the ellipse image; use these two sets of point sets respectively to fit two ellipses image equation.

4. Using the fitted two ellipse image equations, solve their intersection coordinates in the complex domain, that is, get the image point coordinates i _n , j _n , n=1, 2 of two groups of virtual circle points, where i ₁ and j ₁ , i ₂ and j ₂ are conjugate pairs.

5. Use the projection equation of the absolute conic curve and the image point coordinates of the imaginary circle to solve the focal length of the camera.

6. Establish a world coordinate system, the origin is at the center of any one of the two circles, and the Z axis is parallel to the normal direction of the plane where the two circles are located. The equation of the vanishing line is obtained from the image point coordinates of the imaginary circle point, and the rotation matrix is determined by using the constraint relationship among the vanishing point, the vanishing line and the ellipse image equation.

7. Solve the coordinates of the origin of the world coordinate system in the camera coordinate system, which is the translation vector. At this point, the calibration of the camera is completed.

Compared with the existing methods, the present invention only needs one image of the calibration object, can easily determine the internal and external parameters that change with the position of the camera, and can realize automatic calibration without human-computer interaction, reducing manual intervention cause measurement errors. The invention is especially suitable for camera calibration in computer stereo vision, non-contact industrial detection and vision-based autonomous navigation systems.

Description of drawings

Figure 1 is a schematic diagram of camera calibration using two arbitrary coplanar circles.

Fig. 2 is a schematic diagram of a calibration object used for camera calibration using two arbitrary coplanar circles.

Detailed ways

In order to better understand the present invention, the technical solutions of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

FIG. 1 is a schematic diagram of the projection of the calibration object proposed by the present invention on the image plane. The two arbitrary circles used for calibration are on the plane π, OXYZ is the camera coordinate system, O _w X _w Y _w Z _w is the world coordinate system, where O _{w is} now on the center of the left circle in the figure (also can be seen in the figure On the center of the right circle), the Z _w axis is parallel to the normal direction of the plane π. Assuming that the internal parameter matrix of the camera is K, then according to the pinhole model, Where (u ₀ , v ₀ ) represents the position of the principal point, f ₁ /f ₂ is the aspect ratio, s is the tilt factor, these are invariant parameters and are known in the present invention. In addition, f ₁ =fm _x , f ₂ =fm _y , where f is the focal length of the camera, m _x and m _y are the number of pixels per unit distance of the image coordinates in the x and y directions, respectively, and m _x and m _y are known parameters .

Fig. 2 is a schematic diagram of calibration objects used in the present invention. Two circles are on the same plane, they can be tangent, partially overlapping, and not intersecting each other, but they do not share the same center.

Describe in detail the implementation steps of the inventive method below:

1. First make a calibration object. Draw two circles arbitrarily on a flat object, and note that the centers of the circles are not at the same point. Take these two arbitrary coplanar circles with different centers as calibration objects.

2. Place the calibration object flatly within the field of view of the camera, and use the camera to shoot the calibration object to obtain a calibration object image with two elliptical images. It must be ensured that the image is clear and that each circle is not covered by more than one-third of its area.

3. Use the canny operator to detect the boundary of each ellipse image in the calibration object image respectively, so as to obtain two sets of point sets composed of ellipse image boundary pixel points. Using these two sets of point sets respectively, the equations of two elliptical images are fitted, represented by the matrix C _n of the elliptical images, where n=1,2. For specific methods, see (AW Fitzgibbon, M. Pilu, and RB Fisher, "Direct Least-Squares Fitting of Ellipses", IEEE Trans. Pattern Analysis and Machine Intelligence, vol.14, no.2, pp.239-256).

4. Using the equations of two elliptical images, solve their intersection point coordinates in the complex domain, that is, get the image point coordinates i _n , j _n , n=1, 2 of two groups of imaginary circle points, where i ₁ and j ₁ , i ₂ and j ₂ are conjugate pairs.

和

其中ω＝(KK^T)^-1＝K^-TK^-1为绝对二次曲线的投影方程。K中只有一个未知参数f，显然可解出摄像机的焦距f。5. Use the projection equation of the absolute conic curve and the image point coordinates of the imaginary circle to solve the focal length of the camera. Since the image point of the imaginary circle point is on the projection of the absolute conic curve, there is

and

Where ω=(KK ^T ) ^-1 =K ^-T K ^-1 is the projection equation of the absolute quadratic curve. There is only one unknown parameter f in K, obviously the focal length f of the camera can be solved.

6. Determine the rotation matrix: rotation matrix R=[r ₁ , r ₂ , r ₃ ], where r _n , n=1, 2, 3 is the nth column vector of the rotation matrix. In order to eliminate the uncertainty of the direction of the world coordinate system, it is assumed that

(0 0 1)·r ₃ ≥0, (1 0 0)·r ₁ ≥0. (1)

Firstly, the equation l _∞ of the vanishing line is obtained from the image point coordinates of the imaginary circle point, and then the constraint relationship between the vanishing point, the vanishing line and the ellipse image equation in the normal direction of the plane π is used (R.Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge Univ. Press (2000)) Kr ₃ ∝ω ^* l _∞ , can determine

${\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\frac{{\mathrm{<span\; class="notranslate">\; K</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}^{<span\; class="notranslate">\; *</span>}{\mathrm{<span\; class="notranslate">\; l</span>}}_{<span\; class="notranslate">\; \&infin;</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; K</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}^{<span\; class="notranslate">\; *</span>}{\mathrm{<span\; class="notranslate">\; l</span>}}_{<span\; class="notranslate">\; \&infin;</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

where s ₁ is an undetermined sign, + or -.

Use relational

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="p"\; filenames="H2008100412167E00011.png"\; state="\{\{state\}\}">p</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="Kt"\; filenames="H2008100412167E00011.png"\; state="\{\{state\}\}">Kt</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&Proportional;</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="C"\; filenames="H2008100412167E00011.png"\; state="\{\{state\}\}">C</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; *</span>}{\mathrm{<span\; class="notranslate">\; l</span>}}_{<span\; class="notranslate">\; \&infin;</span>}\\ {\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; Kt</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \&Proportional;</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{<span\; class="notranslate">\; *</span>}{\mathrm{<span\; class="notranslate">\; l</span>}}_{<span\; class="notranslate">\; \&infin;</span>}\end{array}\right.<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, t _n , n=1,2 is the coordinates of the center of circle Q _n , n=1,2 in the camera coordinate system, p _n , n=1,2 is the center of circle Q _n , n=1,2 The coordinates of the image point in the camera coordinate system.

Use the formula Kr ₁ ∝l×l _∞ , where l=p ₁ ×p ₂ is the symmetry axis of the image.

Available $r_{}$${}_1={\mathrm{the\; s}}_2\frac{K^{-1}(l\&times;l_{\&infin;})}{\left|\right|K^{-1}(l\&times;l_{\&infin;})\left|\right|}---\left(4\right)$

where s ₂ is an undetermined symbol, + or -.

Combining formulas (1), (2) and (4), s ₁ and s ₂ can be uniquely determined. Therefore, the rotation matrix R=[r ₁ r ₃ ×r ₁ r ₃ ] can be calculated.

7. Determine the translation vector: Since the center of any one of the two circles is taken as the origin of the world coordinate system, the coordinates of the center of the circle in the camera coordinate system are solved to be the translation vector.

Specific steps are as follows:

其中r₃₃为r₃的第三个元素。Normalize i _n , j _n , that is, the fourth element of the i _n , j _n vector is 1. The degenerate quadratic envelope can be obtained

Where r ₃₃ is the third element of r ₃ .

其中R₁和R₂分别为圆Q₁和Q₂的半径，C₁ ^*和₂ ^*分别为圆Q₁和Q₂的椭圆影像的对偶二次曲线。Use relational

Where R ₁ and R ₂ are the radii of circles Q ₁ and Q ₂ respectively, and C ₁ ^* and ₂ ^* are the dual quadratic curves of the ellipse images of circles Q ₁ and Q ₂ respectively.

Calculating the generalized eigenvalue of (C _∞ ′ ^* , C ₁ ^* ), the only non-zero eigenvalue can be obtained, which is k ₁ , and k ₂ can be obtained in the same way. Knowing that C _∞ ′ ^* -k ₁ C ₁ ^* and C _∞ ′ ^* -k ₂ C ₂ ^* have a rank of 1, so we have

$\left\{\begin{array}{c}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; K</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{<span\; class="notranslate">\; \&infin;</span>}^{<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="C"\; filenames="H2008100412167E00011.png"\; state="\{\{state\}\}">C</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; K</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; T</span>}}\\ <span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; K</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}^{<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; K</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; T</span>}}\end{array}\right.$

Obviously, t ₁ /R ₁ and t ₂ /R ₂ can be easily obtained from the above formula. Also t ₂ =Dr ₁ +t ₁ , where D is the distance between two circle centers. Reuse the following relationship

$\frac{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}\frac{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; D.</span>}}{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="r"\; filenames="H2008100412167E00011.png"\; state="\{\{state\}\}">r</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\frac{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="R"\; filenames="H2008100412167E00011.png"\; state="\{\{state\}\}">R</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}$

It is easy to estimate R ₂ /R ₁ and D/R ₁ . Obviously, if the radius of any circle or the distance between two circle centers is known, the translation vector _t1 or _t2 can be obtained (when the origin of the world coordinate system is _O1 , then it is _t1 ; the origin of the world coordinate system is O ₂ , then t ₂ ).

So far, the camera parameters have been calibrated.

Claims (1)

1. A method for camera calibration using two arbitrary coplanar circles, characterized in that it comprises the following specific steps: 1) Two arbitrary coplanar circles with different centers are used as calibration objects; 2) Shoot the calibration object with a camera to obtain a calibration object image with two elliptical images to ensure that the imaging is clear and each circle is not covered by more than one-third of the area; 3) Use the canny operator to detect the boundary of each ellipse image on the calibration object image respectively, so as to obtain two sets of point sets composed of pixel points on the edge of the ellipse image, and use these two sets of point sets to fit two ellipses the equation of the image; 4) Using the equations of two elliptical images, solve their intersection point coordinates in the complex domain, that is, get the image point coordinates i _n , j _n , n=1, 2 of two groups of imaginary circle points, where i ₁ and j ₁ , i ₂ and j ₂ are conjugate pairs; 5) Utilize the projection equation of the absolute quadratic curve and the image point coordinates of the imaginary circle point to solve the focal length of the camera; 6) Establish a world coordinate system, the origin is at the center of any one of the two circles, and the Z axis is parallel to the normal direction of the plane where the two circles are located; assuming conditions (0 0 1) r ₃ ≥ 0, (1 0 0) r ₁ ≥ 0; calculate the equation l _∞ of the vanishing line from the image point coordinates of the imaginary circle point, and then use the constraint relation Kr ₃ ∝ω ^* l _∞ , determine

Use relational

and Kr ₁ ∝l×l _∞ , we get

And r ₂ =r ₃ ×r ₁ , thus obtaining the rotation matrix R=[r ₁ , r ₂ , r ₃ ]; Among them, s ₁ and s ₂ are symbols + or ^- ; K is the internal parameter matrix of the camera, ω ^* =KK ^T is the dual image of the absolute conic curve; t _n , n=1, 2 is the circle Q _n , n=1 , the coordinates of the center of 2 in the camera coordinate system, p _n , n=1, 2 is the circle Q _n , n=1, the coordinates of the image point of the center of 2 in the camera coordinate system, l=p ₁ ×p ₂ is the axis of symmetry of the image; 7) Solving the coordinates of the origin of the world coordinate system in the camera coordinate system is the translation vector; so far, the calibration of the camera is completed.

Images