CN103530880B - Based on the camera marking method of projection Gaussian network pattern - Google Patents

Abstract

The camera marking method that the present invention is based on projection Gaussian network pattern belongs to image procossing and Computer Vision Detection field, the field calibration method of the inside and outside parameter of video camera in particularly large forgings Size Measuring System.Camera marking method utilizes horizontal, vertical striation gray scale in the direction of the width in Gaussian network pattern to be the characteristic of Gaussian distribution, the image coordinate of the point on light stripe centric line can be obtained accurately by fitted Gaussian curve, and then simulate the center line equation of horizontal stroke, vertical striation, the intersection point of horizontal, vertical light stripe centric line is feature point for calibration, according to the image coordinate of taking the feature point for calibration provided in the image of the Gaussian network pattern obtained, substep obtains the inside and outside parameter of video camera.The present invention has high real-time, robustness and higher stated accuracy, and substep is demarcated can obtain high-precision camera parameters, avoids coupling problem when simultaneously being solved by all camera parameters, is applicable to forging scene and carries out on-line proving to video camera.

Description

Camera calibration method based on projection Gauss grid pattern

Technical Field

The invention belongs to the field of image processing and computer vision detection, and particularly relates to a field calibration method for internal and external parameters of a camera in a large forging dimension measurement system.

Background

One of the basic tasks of computer vision processing is to recover the three-dimensional geometric information of an object from two-dimensional image information. To perform the task of finding the corresponding surface points of the spatial object using the image points, a geometric model of the camera imaging is determined, the parameters of which are referred to as camera parameters. The camera parameters are divided into internal and external parameters, the internal parameters are parameters of the camera, which are related to geometric and optical characteristics, and the external parameters are three-dimensional positions and directions of the camera relative to a certain world coordinate system. The process of determining the internal and external parameters of the camera is called camera calibration, and the precision of the calibration method directly influences the precision of computer vision measurement. Therefore, the research of quickly, simply and accurately calibrating the camera is undoubtedly of great significance.

The traditional camera calibration methods can be classified into 3D-based three-dimensional target calibration methods, 2D-based planar target calibration methods (represented by checkerboard target calibration methods proposed by zhangzhengyou) and 1D-based calibration methods according to the differences of calibration references. In the traditional calibration methods, a calibration reference object is needed, and for the calibration of a large-field-of-view camera, whether the characteristic points of the calibration reference object are uniformly distributed in the whole field of view directly influences the calibration precision. On one hand, the manufacturing of the high-precision large-size calibration target is expensive and difficult to maintain. On the other hand, the method is not suitable for the occasions which are not suitable for being used on line and cannot use the calibration reference object. Under the high-temperature environment of a forging workshop, the method for marking the block, the plate and the target point can not be applied. Therefore, the traditional camera calibration method cannot meet the requirement of online size parameter measurement of the large forging. In addition, although the self-calibration method does not use any calibration object, the self-calibration method can estimate the intrinsic parameters of the camera according to the corresponding relation of the image points between the images by only using the constraint existing in the intrinsic parameters of the camera. The method is flexible in operation, but the precision is not high, and the robustness is not enough.

The problem can be solved by adopting a projector to project a target, the projected characteristic pattern theoretically has a step change at the edge, but actually, due to diffusion effect, the pattern edge has a gradual change trend, and the edge can shift to the black background side. Taking a projector projecting a circular characteristic light spot array as an example, the center of a circular light spot is a characteristic point for calibration, and because each light spot can diffuse to the periphery to different degrees, it is difficult to obtain the accurate center of the circular spot by adopting a centroid method according to an image after binarization, and it is not easy to extract the boundary of the circular characteristic and perform circular (or elliptical) fitting by using a related algorithm to obtain the center of the circular spot with high precision. Similarly, a projector is used for projecting a general light bar combination pattern, and the center of the light bar is extracted as a characteristic line, so that the precision is difficult to guarantee.

Disclosure of Invention

The invention aims to solve the technical problems of the prior art, and aims to solve the problems that the traditional calibration method has low precision, non-real time and even can not be applied, and the self-calibration method has low precision, insufficient robustness and the like in a forging site.

The technical scheme adopted by the invention is a camera calibration method based on a projected Gaussian grid pattern, which is characterized in that the characteristic that the gray scales of horizontal and vertical light bars in the Gaussian grid pattern in the width direction are in Gaussian distribution is utilized, the image coordinates of points on the central line of the light bars can be obtained at high precision by fitting a Gaussian curve, further the central line equation of the horizontal and vertical light bars is fitted, the intersection point of the central lines of the horizontal and vertical light bars is a calibration characteristic point, and the internal and external parameters of the camera are obtained step by step according to the image coordinates of the calibration characteristic point provided in the shot image of the Gaussian grid pattern; the method comprises the following specific steps:

step 1: and (5) building a camera calibration system. A left four-dimensional electric control platform 2a, a right four-dimensional electric control platform 2b and a projector 3 are installed on the table top of the platform 1, a left camera 4a is fixed on the left four-dimensional electric control platform 2a, and a right camera 4b is fixed on the right four-dimensional electric control platform 2 b.

Step 2: and projecting a Gaussian grid pattern, shooting and acquiring intersection point coordinates. Projecting a Gaussian grid pattern 6 consisting of a plurality of parallel transverse light bars and a plurality of parallel longitudinal light bars onto a smooth flat plate or wall surface 5 in a factory building through a projector 3, wherein the gray scales of all the light bars in the width direction are in Gaussian distribution, and the intersection point A of the transverse light bars and the longitudinal light bars is_i,jTo mark the feature points, i is the number of the horizontal light bars in the order from top to bottom, and j is the number of the vertical light bars in the order from left to right. Due to the superposition of light intensity at the intersection of the horizontal and vertical light bars, after the binarization processing is performed on the images shot by the left camera 4a and the right camera 4b, only bright spots at the intersection of the grids, namely isolated connected regions, are left in the obtained images. Centroid coordinates of connected regions can be obtained by centroid method (u 0)_i,j,v0_i,j) As a feature point A_i,jThe coarse position of (2). A circular area with the rough position as the center and a radius of delta pixels is used as a search range, and then the search range is [ u0 ]_i,j-Δ,u0_i,j+Δ]The transverse light bars are searched once in the width direction every delta/n within the range, fitting is carried out according to Gaussian distribution characteristics, and Gaussian distribution peak points are used as points on the central lines of the transverse light bars, so that points P on 2n +1 central lines can be obtained_i,j,sSubscript s is 1,2,3, …,2n +1, and a straight line l is fitted_h,i,j. Similarly, in [ v0_i,j-Δ,v0_i,j+Δ]The longitudinal light bars are searched once in the width direction every delta/n within the range, fitting is carried out according to Gaussian distribution characteristics, Gaussian distribution peak points are used as points on the central line of the longitudinal light bars, and points Q on 2n +1 central lines can be obtained_i,j,tSubscript t is 1,2,3, …,2n +1, and then a straight line l is fitted_v,i,j. Finally, the intersection point of two intersecting straight lines in the same search range is obtained as a calibration characteristic point A_i,jThe coordinates of which are (u)_i,j,v_i,j)。

And step 3: the rough coordinates of the principal point are obtained. Shooting the same projected Gaussian grid pattern 6, characteristic point A, by using the left camera 4a or the right camera 4b under two different focal lengths_i,jRespectively, are (u 1)_i,j,v1_i,j) And (u 2)_i,j,v2_i,j) The principal point coordinate is (u)₀,v₀) Then, there are:

$\frac{u{2}_{i,j}-u_0}{u{1}_{i,j}-u_0}=\frac{v{2}_{i,j}-v_0}{v{1}_{i,j}-v_0}---\left(2\right)$

the rough position of the principal point being determined by the above equationCoordinates (u)₀,v₀)。

And 4, step 4: and solving the distortion coefficient and the optimized principal point coordinate. The intersection point A of actual shooting can be deduced according to the distortion model_i,jCoordinate p of_i,j＝(u_i,j,v_i,j,1)^TCoordinates q of the intersection with the ideal_i,j＝(u'_i,j,v'_i,j,1)^TThe conversion relationship of (1) is as follows:

$\left[\begin{array}{c}{u^{\′}}_{i,j}\\ {v^{\′}}_{i,j}\end{array}\right]=\left[\begin{array}{c}{u}_{i,j}\\ v_{i,j}\end{array}\right]+\left[\begin{array}{c}{\tilde{u}}_{i,j}(k_1{r}_{i,j}^2+k_2{r}_{i,j}^4)\mathrm{+2}{p}_1{\tilde{u}}_{i,j}{\tilde{v}}_{i,j}+p_2(r_{i,j}^2+2{\tilde{u}}_{i,j}^2)\\ {\tilde{v}}_{i,j}(k_1{r}_{i,j}^2+k_2{r}_{i,j}^4)+p_1(r_{i,j}^2+2{\tilde{v}}_{i,j}^2)+2p_2{\tilde{u}}_{i,j}{\tilde{v}}_{i,j}\end{array}\right]---\left(3\right)$

wherein, ${\tilde{u}}_{i,j}=u_{i,j}-u_0,{\tilde{v}}_{i,j}=v_{i,j}-v_0,r_{i,j}=\sqrt{{\tilde{u}}_{i,j}^2+{\tilde{v}}_{i,j}^2},$ k₁and k is₂As radial distortion coefficient, p₁And p₂Is the tangential distortion coefficient.

In addition, taking the grid pattern with the same number of horizontal and vertical bars as an example, the total number of intersections is num, and then there is one row for each rowThe intersection points, according to the linear fidelity, that is, the property of collinear points on the same light bar, and combining the essential conditions of collinear three points, can list the optimization objective function as follows:

$\begin{array}{cc}\min & \underset{i=1}{\overset{\sqrt{num}}{\mathrm{\Σ}}}\left(\underset{j=\sqrt{num}(i-1)+1}{\overset{\sqrt{num}(i-1)+\sqrt{num}-2}{\mathrm{\Σ}}}\right|q_{i,j+1}^T{\[q_{i,j}\]}_{\×}{q}_{i,j+2}\left|\right)---\left(4\right)\end{array}$

wherein,is three points A_i,j，A_i,j+1，A_i,j+2The requirement of collinearity. [ q ] of_i,j]_×Represents q_i,jThe antisymmetric matrix of (a), namely:

${\[q_{i,j}\]}_{\×}=\left[\begin{array}{ccc}0& -1& -{v^{\′}}_{i,j}\\ 1& 0& -{u^{\′}}_{i,j}\\ -{v^{\′}}_{i,j}& {u^{\′}}_{i,j}& 0\end{array}\right]---\left(5\right)$

optimizing by a Levenberg-Marquardt nonlinear optimization algorithm to minimize the value of the target function of the formula (4), and acquiring the distortion coefficient k₁、k₂、p₁And p₂And optimized principal point coordinates (u'₀,v'₀). Then, the coordinates of all the intersections are corrected to ideal coordinates by equation (3).

And 5: the remaining internal parameters of the camera are evaluated. And (3) taking the corrected ideal intersection point as a characteristic point, driving the left camera 4a to do two groups of orthogonal motions by using the left four-dimensional electronic control platform 2a by adopting an active vision method, respectively shooting an image of a projected Gaussian grid pattern 6 at three start and end positions of each group of orthogonal motions, and finally shooting by using the left camera 4a to obtain 6 images.

The parallel straight line and the infinite plane intersect at the same infinite point, namely a vanishing point. And a group of orthogonal motion comprises two translations, the connecting line of corresponding intersection points on two images shot at the start and end positions of one translation motion is a group of spatial parallel lines, and the two translations are mutually vertical, so that a group of orthogonal vanishing point pairs e can be obtained_i1(u_i1,v_i1) And e_i2(u_i2,v_i2) Where i is 1,2, stands for the order of orthogonal movements and has Oe_i1·Oe_i2Using two sets of orthogonal pairs of vanishing points, the scale factor α in the intrinsic parameter matrix K for the left camera 4a can be solved separately by solving the following set of equations_xAnd α_y：

$\left\{\begin{array}{c}(u_{11}-{u^{\′}}_0)(u_{12}-{u^{\′}}_0)/{\mathrm{\α}}_x^2+(v_{11}-{v^{\′}}_0)(v_{12}-{v^{\′}}_0)/{\mathrm{\α}}_y^2+1=0\\ (u_{21}-{u^{\′}}_0)(u_{22}-{u^{\′}}_0)/{\mathrm{\α}}_x^2+(v_{21}-{v^{\′}}_0)(v_{22}-{v^{\′}}_0)/{\mathrm{\α}}_y+1=0\end{array}\right.---\left(6\right)$

Likewise, the scale factor α in the intrinsic parameter matrix K of the right camera 4b may be obtained_xAnd α_y。

Step 6: external parameters of the camera are acquired. The left camera 4a and the right camera 4b capture the same projected gaussian grid pattern 6, a world coordinate system is established on the camera coordinate system of the left camera 4a, and the basic matrix F is calculated using the corrected matching points of the images captured by the left and right cameras 4a, 4 b. The essential matrix E can be calculated with the internal parameters and the basic matrix being determined with a difference of a scaling factor s. After decomposition of the intrinsic matrix E, the extrinsic parameters (rotation matrix R 'and translation vector t') can be determined with a difference of one scaling factor.

Projecting parallel Gauss light bar to actual length L by projector 3₀On the accurately measured gauge block, the central line of sub-pixel light bar is fitted by using the Gaussian characteristic of the light bar, the point with abrupt gray scale change is used as the boundary point of the gauge block, and the length L 'of the gauge block is reconstructed according to the obtained internal and external parameters'₀The scale factor can be obtained as: s ═ L₀/L'₀. Thus, the actual camera extrinsic parameters (rotation matrix R 'and translation vector t s t') can be obtained. Thus, the calibration process of the camera is completed.

The method has the advantages that the intersection points of the Gaussian grid patterns projected by the projector are used as the calibration characteristic points, so that the use of calibration blocks, calibration plates and pasting marker points is avoided, and the real-time calibration of the camera under complex environments such as a forging site is convenient to realize. The method can determine the position information of the calibration characteristic points with high precision according to the characteristic that the gray scale in the width direction of the light strip is in Gaussian distribution, has high robustness, can obtain high-precision camera parameters through step-by-step calibration, and can simultaneously avoid the problem of coupling when all the camera parameters are simultaneously solved.

Drawings

FIG. 1 is a schematic diagram of a calibration system of the present invention. Wherein: 1-shock insulation platform, 2 a-left side four-dimensional electric control platform, 2 b-right side four-dimensional electric control platform, 3-projector, 4 a-left side camera, 4 b-right side camera, 5-smooth flat plate or wall surface, 6-Gaussian grid pattern.

Fig. 2 is an image of a grid pattern taken by a camera according to the present invention.

Fig. 3 illustrates the present invention performing binarization processing on an image of a grid pattern and acquiring rough positions of intersections.

FIG. 4 is a scale factor for the reconstructed slab size of a Gaussian light bar array of the present invention.

Detailed Description

The following describes the embodiments of the present invention in further detail with reference to the drawings and technical solutions.

Camera calibration typically employs a classical pinhole imaging model, whose expression is as follows:

wherein (X)_w,Y_w,Z_w,1)^TIs the homogeneous coordinate of a space point in a world coordinate system, (u, v,1)^TFor corresponding image pixel coordinate system o₀Homogeneous coordinates in uv, α_xF/dx is o₀Scale factor on the u-axis in the uv coordinate system, α_yF/dy is o₀The scale factor on the v axis in the uv coordinate system, f is the focal length of the camera lens, dx and dy are the horizontal and vertical physical dimensions of the pixel respectively, (u₀,v₀) As principal point coordinates, p_cFor the scale factor, K is the camera internal parameter matrix, [ R | t]Is the external parameter matrix of the camera, wherein R is the rotation matrix and t is the translation vector.

The camera internal parameters include principal point coordinates (u)₀,v₀) Scale factor α_x、α_yCoefficient of radial distortion k₁、k₂And tangential distortion coefficient p₁、p₂. The camera external parameters are the orientation of the camera coordinate system relative to the world coordinate system, and comprise a rotation matrix R and a translation vector t.

Step 1: and (5) building a camera calibration system. A left four-dimensional electric control platform 2a, a right four-dimensional electric control platform 2b and a projector 3 are installed on the table top of the platform 1, a left camera 4a is fixed on the left four-dimensional electric control platform 2a, and a right camera 4b is fixed on the right four-dimensional electric control platform 2b, as shown in fig. 1.

Step 2: and projecting a Gaussian grid pattern, shooting and acquiring intersection point coordinates. Projecting a Gaussian grid pattern 6 on a smooth flat plate or a wall surface 5 in a factory building through a projector 3, wherein the Gaussian grid pattern 6 consists of a plurality of parallel transverse light bars and a plurality of parallel longitudinal light bars, the gray scale of each light bar is in Gaussian distribution in the width direction, and the intersection point A of the transverse light bar and the longitudinal light bar_i,jTo mark the feature points, i is the number of the horizontal light bars in the order from top to bottom, and j is the number of the vertical light bars in the order from left to right. An image of the projected gaussian grid pattern captured by the left camera 4a or the right camera 4b is shown in fig. 2. Due to the superposition of light intensity at the intersection of the horizontal and vertical light bars, after the binarization processing is performed on the images captured by the left camera 4a and the right camera 4b, only bright spots at the intersection of the grids, i.e., isolated connected regions, remain in the obtained images, as shown in fig. 3. Centroid coordinates of connected regions can be obtained by centroid method (u 0)_i,j,v0_i,j) As a feature point A_i,jThe coarse position of (2). A circular area with the rough position as the center and a radius of delta pixels is used as a search range, and then the search range is [ u0 ]_i,j-Δ,u0_i,j+Δ]The transverse light bars are searched once in the width direction every delta/n within the range, fitting is carried out according to Gaussian distribution characteristics, and Gaussian distribution peak points are used as points on the central lines of the transverse light bars, so that points P on 2n +1 central lines can be obtained_i,j,sSubscript s is 1,2,3, …,2n +1, and a straight line l is fitted_h,i,j. Similarly, in [ v0_i,j-Δ,v0_i,j+Δ]Searching the longitudinal light bars once every delta/n along the width direction within the range, fitting according to the Gaussian distribution characteristic, and fitting the Gaussian distribution characteristicThe peak point of the distribution is used as the point on the central line of the longitudinal light bar, and the points Q on 2n +1 central lines can be obtained_i,j,tSubscript t is 1,2,3, …,2n +1, and then a straight line l is fitted_v,i,j. Finally, the intersection point of two intersecting straight lines in the same search range is obtained as a calibration characteristic point A_i,jThe coordinates of which are (u)_i,j,v_i,j)。

And step 3: the rough coordinates of the principal point are obtained. The principal point is obtained by using a zoom method, and the left side camera 4a or the right side camera 4b shoots the same projection Gaussian grid pattern 6 under two different focal lengths, namely a characteristic point A_i,jRespectively, are (u 1)_i,j,v1_i,j) And (u 2)_i,j,v2_i,j) The principal point coordinate is (u)₀,v₀) Then, there are:

$\frac{u{2}_{i,j}-u_0}{u{1}_{i,j}-u_0}=\frac{v{2}_{i,j}-v_0}{v{1}_{i,j}-v_0}---\left(2\right)$

once the zoom center of the lens is regarded as the principal point, the coordinate (u) of the rough position of the principal point can be obtained by the above expression₀,v₀)。

And 4, step 4: and solving the distortion coefficient and the optimized principal point coordinate. The intersection point A of actual shooting can be deduced according to the distortion model_i,jCoordinate p of_i,j＝(u_i,j,v_i,j,1)^TCoordinates q of the intersection with the ideal_i,j＝(u'_i,j,v'_i,j,1)^TThe conversion relationship of (1) is as follows:

$\left[\begin{array}{c}{u^{\′}}_{i,j}\\ {v^{\′}}_{i,j}\end{array}\right]=\left[\begin{array}{c}{u}_{i,j}\\ v_{i,j}\end{array}\right]+\left[\begin{array}{c}{\tilde{u}}_{i,j}(k_1{r}_{i,j}^2+k_2{r}_{i,j}^4)\mathrm{+2}{p}_1{\tilde{u}}_{i,j}{\tilde{v}}_{i,j}+p_2(r_{i,j}^2+2{\tilde{u}}_{i,j}^2)\\ {\tilde{v}}_{i,j}(k_1{r}_{i,j}^2+k_2{r}_{i,j}^4)+p_1(r_{i,j}^2+2{\tilde{v}}_{i,j}^2)+2p_2{\tilde{u}}_{i,j}{\tilde{v}}_{i,j}\end{array}\right]---\left(3\right)$

wherein, ${\tilde{u}}_{i,j}=u_{i,j}-u_0,{\tilde{v}}_{i,j}=v_{i,j}-v_0,r_{i,j}=\sqrt{{\tilde{u}}_{i,j}^2+{\tilde{v}}_{i,j}^2},$ k₁and k is₂As radial distortion coefficient, p₁And p₂Is the tangential distortion coefficient.

In addition, taking the grid pattern with the same number of horizontal and vertical bars as an example, the total number of intersections is num, and then there is one row for each rowOne crossingThe following can be listed as the optimization objective function according to the principle condition of the collinear three points, i.e. the property of the collinear straight line, i.e. the collinear points on the same light bar:

$\begin{array}{cc}\min & \underset{i=1}{\overset{\sqrt{num}}{\mathrm{\Σ}}}\left(\underset{j=\sqrt{num}(i-1)+1}{\overset{\sqrt{num}(i-1)+\sqrt{num}-2}{\mathrm{\Σ}}}\right|q_{i,j+1}^T{\[q_{i,j}\]}_{\×}{q}_{i,j+2}\left|\right)---\left(4\right)\end{array}$

wherein,is three points A_i,j，A_i,j+1，A_i,j+2The requirement of collinearity. [ q ] of_i,j]_×Represents q_i,jThe antisymmetric matrix of (a), namely:

${\[q_{i,j}\]}_{\×}=\left[\begin{array}{ccc}0& -1& -{v^{\′}}_{i,j}\\ 1& 0& -{u^{\′}}_{i,j}\\ -{v^{\′}}_{i,j}& {u^{\′}}_{i,j}& 0\end{array}\right]---\left(5\right)$

optimizing by a Levenberg-Marquardt nonlinear optimization algorithm to minimize the value of the target function of the formula (4), and acquiring the distortion coefficient k₁、k₂、p₁And p₂And optimized principal point coordinates (u'₀,v'₀). Then, the coordinates of all the intersections are corrected to ideal coordinates by equation (3).

And 5: the remaining internal parameters of the camera are evaluated. And (3) taking the corrected ideal intersection point as a characteristic point, driving the left camera 4a to do two groups of orthogonal motions by using the left four-dimensional electronic control platform 2a by adopting an active vision method, respectively shooting an image of a projection Gaussian grid pattern 6 at three start and end positions of each group of orthogonal motions, and finally shooting by using the left camera 4a to obtain 6 images. The specific process is as follows: (1) adjusting the left four-dimensional electric control platform 2a to a proper position, starting a first group of orthogonal motions, and respectively shooting an image of a projection Gaussian grid pattern 6 at three initial and final positions of the orthogonal motions; (2) and (3) the left side camera 4a is enabled to look down at a certain angle by using the left side four-dimensional electric control platform 2a, a second group of orthogonal motions are started, and images of the projected Gaussian grid pattern 6 are respectively shot at three start and end positions of the orthogonal motions.

The image of an infinitely distant point on a straight line is called the vanishing point of the straight line. Because the parallel straight line intersects with the infinite plane at the same infinite point, namely a vanishing point. A group of orthogonal motion comprises two translations, the connecting line of the corresponding intersection points on the two images shot at the start and end positions of one translation motion is a group of space parallel lines, and the two translations are mutually perpendicular, so that a group of positive lines can be obtainedHidden and vanished point pair e of intersection_i1(u_i1,v_i1) And e_i2(u_i2,v_i2) The index i being 1,2, representing the order of orthogonal movements, and having Oe_i1·Oe_i2Using two sets of orthogonal pairs of vanishing points, the scale factor α in the intrinsic parameter matrix K for the left camera 4a can be solved separately by solving the following set of equations_xAnd α_y：

$\left\{\begin{array}{c}(u_{11}-{u^{\′}}_0)(u_{12}-{u^{\′}}_0)/{\mathrm{\α}}_x^2+(v_{11}-{v^{\′}}_0)(v_{12}-{v^{\′}}_0)/{\mathrm{\α}}_y^2+1=0\\ (u_{21}-{u^{\′}}_0)(u_{22}-{u^{\′}}_0)/{\mathrm{\α}}_x^2+(v_{21}-{v^{\′}}_0)(v_{22}-{v^{\′}}_0)/{\mathrm{\α}}_y+1=0\end{array}\right.---\left(6\right)$

Likewise, the scale factor α in the intrinsic parameter matrix K of the right camera 4b may be obtained_xAnd α_y。

Step 6: and acquiring external parameters. The left camera 4a and the right camera 4b capture the same projected gaussian grid pattern 6, a world coordinate system is established on the camera coordinate system of the left camera 4a, and the fundamental matrix F is calculated using the corrected matching points of the images captured by the left and right cameras 4a, 4 b. The intrinsic matrix E can be calculated with the use of the intrinsic parameters and the fundamental matrix with a difference of a scale factor s. After decomposition of the intrinsic matrix E, the extrinsic parameters (rotation matrix R 'and translation vector t') can be determined with a difference of one scaling factor.

As shown in fig. 4, a parallel bar of gaussian light is projected to a substantial length L by a projector 3₀On the accurately measured gauge block, the central line of sub-pixel light bar is fitted by using the Gaussian characteristic of the light bar, the point with abrupt gray scale change is used as the boundary point of the gauge block, and the length L 'of the gauge block is reconstructed according to the obtained internal and external parameters'₀The scale factor can be obtained as: s ═ L₀/L'₀. Thus, the actual camera extrinsic parameters (rotation matrix R 'and translation vector t s t') can be obtained. Thus, the calibration process of the camera is completed.

The camera calibration method provided by the invention has good real-time performance, robustness and higher calibration precision, and can be used for online calibration of a large-view-field camera in complex environments such as a forging site and the like.

Claims (1)

1. A camera calibration method based on a projected Gaussian grid pattern is characterized in that the camera calibration method utilizes the characteristic that the gray scales of horizontal and vertical light bars in the Gaussian grid pattern in the width direction are in Gaussian distribution, image coordinates of points on the center line of the light bars can be obtained at high precision by fitting a Gaussian curve, further a center line equation of the horizontal and vertical light bars is fitted, the intersection point of the center lines of the horizontal and vertical light bars is a calibration feature point, and internal and external parameters of a camera are obtained step by step according to image coordinates of the calibration feature point provided in an image of the photographed Gaussian grid pattern; the method comprises the following specific steps:

step 1: building a camera calibration system; a left four-dimensional electric control platform (2a), a right four-dimensional electric control platform (2b) and a projector (3) are arranged on the table top of a platform (1), a left camera (4a) is fixed on the left four-dimensional electric control platform (2a), and a right camera (4b) is fixed on the right four-dimensional electric control platform (2 b);

step 2: projecting a Gaussian grid pattern, shooting and acquiring intersection point coordinates; a Gauss grid pattern (6) consisting of a plurality of parallel transverse light bars and a plurality of parallel longitudinal light bars is projected to a smooth flat plate or a wall surface (5) in a factory building through a projector (3), the gray scales of all the light bars in the width direction are in Gaussian distribution, and the intersection point A of the transverse light bars and the longitudinal light bars is_i,jFor calibrating the characteristic points, i is the number of the transverse light bars in the sequence from top to bottom, and j is the number of the longitudinal light bars in the sequence from left to right; because the light intensity at the intersection of the horizontal and vertical light bars is superposed, after the binarization processing is carried out on the images shot by the left camera (4a) and the right camera (4b), only bright spots at the intersection of grids, namely isolated connected regions, are left in the obtained images; centroid coordinates of connected regions can be obtained by centroid method (u 0)_i,j,v0_i,j) As a feature point A_i,jA coarse position of (a); a circular area with the rough position as the center and a radius of delta pixels is used as a search range, and then the search range is [ u0 ]_i,j-Δ,u0_i,j+Δ]The transverse light bars are searched once in the width direction every delta/n within the range, fitting is carried out according to Gaussian distribution characteristics, and Gaussian distribution peak points are used as points on the central lines of the transverse light bars, so that points P on 2n +1 central lines can be obtained_i,j,sSubscript s is 1,2,3, …,2n +1, and a straight line l is fitted_h,i,j(ii) a Similarly, in [ v0_i,j-Δ,v0_i,j+Δ]The longitudinal light bars are searched once in the width direction every delta/n within the range, fitting is carried out according to Gaussian distribution characteristics, Gaussian distribution peak points are used as points on the central line of the longitudinal light bars, and points Q on 2n +1 central lines can be obtained_i,j,tSubscript t is 1,2,3, …,2n +1, and then a straight line l is fitted_v,i,j(ii) a Finally, the intersection point of two intersecting straight lines in the same search range is obtained as a calibration characteristic point A_i,jThe coordinates of which are (u)_i,j,v_i,j)；

And step 3: acquiring a rough coordinate of a principal point; shooting the same projected Gaussian grid pattern (6) by using a left camera (4a) or a right camera (4b) at two different focal lengths, wherein the characteristic point A is_i,jRespectively, are (u 1)_i,j,v1_i,j) And (u 2)_i,j,v2_i,j) The principal point coordinate is (u)₀,v₀) Then, there are:

$\frac{u{2}_{i,j}-u_0}{u{1}_{i,j}-u_0}=\frac{v{2}_{i,j}-v_0}{v{1}_{i,j}-v_0}---\left(2\right)$

the coordinates (u) of the rough position of the principal point can be found by the above equation₀,v₀)；

Step (ii) of4: solving a distortion coefficient and an optimized principal point coordinate; the intersection point A of actual shooting can be deduced according to the distortion model_i,jCoordinate p of_i,j＝(u_i,j,v_i,j,1)^TCoordinates q of the intersection with the ideal_i,j＝(u'_i,j,v'_i,j,1)^TThe conversion relationship of (1) is as follows:

$\left[\begin{array}{c}{u^{\′}}_{i,j}\\ {v^{\′}}_{i,j}\end{array}\right]=\left[\begin{array}{c}{u}_{i,j}\\ v_{i,j}\end{array}\right]+\left[\begin{array}{c}{\tilde{u}}_{i,j}\left(k_1{r}_{i,j}^2+k_2{r}_{i,j}^4\right)+2p_1{\tilde{u}}_{i,j}{\tilde{v}}_{i,j}+p_2(r_{i,j}^2+2{\tilde{u}}_{i,j}^2)\\ {\tilde{v}}_{i,j}\left(k_1{r}_{i,j}^2+k_2{r}_{i,j}^4\right)+p_1(r_{i,j}^2+2{\tilde{v}}_{i,j}^2)+2p_2{\tilde{u}}_{i,j}{\tilde{v}}_{i,j}\end{array}\right]---\left(3\right)$

wherein, $\begin{array}{ccc}{\tilde{u}}_{i,j}=u_{i,j}-u_0,& {\tilde{v}}_{i,j}=v_{i,j}-v_0,& r_{i,j}=\sqrt{{\tilde{u}}_{i,j}^2+{\tilde{v}}_{i,j}^2}\end{array},$ k₁and k is₂As radial distortion coefficient, p₁And p₂Is a tangential distortion coefficient;

in addition, taking the grid pattern with the same number of horizontal and vertical bars as an example, the total number of intersections is num, and then there is one row for each rowThe intersection points, according to the linear fidelity, that is, the property of collinear points on the same light bar, and combining the essential conditions of collinear three points, can list the optimization objective function as follows:

$\begin{array}{cc}\min & \underset{i=1}{\overset{\sqrt{num}}{\Σ}}\left(\underset{j=\sqrt{num}\left(i-1\right)+1}{\overset{\sqrt{num}\left(i-1\right)+\sqrt{num}-2}{\Σ}}\left|q_{i,j+1}^T{\[q_{i,j}\]}_{\×}{q}_{i,j+2}\right|\right)\end{array}---\left(4\right)$

wherein,is three points A_i,j，A_i,j+1，A_i,j+2A sufficient condition for co-linearity; [ q ] of_i,j]_×Represents q_i,jThe antisymmetric matrix of (a), namely:

${\[q_{i,j}\]}_{\×}=\left[\begin{array}{ccc}0& -1& {v^{\′}}_{i,j}\\ 1& 0& -{u^{\′}}_{i,j}\\ -{v^{\′}}_{i,j}& {u^{\′}}_{i,j}& 0\end{array}\right]---\left(5\right)$

optimizing by a Levenberg-Marquardt nonlinear optimization algorithm to minimize the value of the target function of the formula (4), and acquiring the distortion coefficient k₁、k₂、p₁And p₂And optimized principal point coordinates (u'₀,v'₀) (ii) a Then, correcting all the intersection point coordinates into ideal coordinates by using a formula (3);

and 5: obtaining other internal parameters of the camera; the corrected ideal intersection points are used as characteristic points, an active vision method is adopted, the left side four-dimensional electric control platform (2a) is used for driving the left side camera (4a) to do two groups of orthogonal motions, images of a projected Gaussian grid pattern (6) are respectively shot at three start and end positions of each group of orthogonal motions, and finally the left side camera (4a) shoots to obtain 6 images;

the parallel straight line and the infinite plane are intersected at the same infinite point, namely a vanishing point; and a group of orthogonal motion comprises two translations, the connecting line of corresponding intersection points on two images shot at the start and end positions of one translation motion is a group of spatial parallel lines, and the two translations are mutually vertical, so that a group of orthogonal vanishing point pairs e can be obtained_i1(u_i1,v_i1) And e_i2(u_i2,v_i2) The index i being 1,2, representing the order of orthogonal movements, and having Oe_i1·Oe_i20, wherein O ═ u'₀,v'₀) Is the principal point of the camera, and by using two orthogonal pairs of vanishing points, the scale factor α in the internal parameter matrix K of the left camera (4a) can be solved respectively by solving the following equations_xAnd α_y：

$\left\{\begin{array}{c}\left(u_{11}-{u^{\′}}_0\right)\left(u_{12}-{u^{\′}}_0\right)/{\mathrm{\α}}_x^2+\left(v_{11}-{v^{\′}}_0\right)\left(v_{12}-{v^{\′}}_0\right)/{\mathrm{\α}}_y^2+1=0\\ \left(u_{21}-{u^{\′}}_0\right)\left(u_{22}-{u^{\′}}_0\right)/{\mathrm{\α}}_x^2+\left(v_{21}-{v^{\′}}_0\right)\left(v_{22}-{v^{\′}}_0\right)/{\mathrm{\α}}_y^2+1=0\end{array}\right.---\left(6\right)$

Likewise, the scale factor α in the intrinsic parameter matrix K of the right camera (4b) can be obtained_xAnd α_y；

Step 6: acquiring external parameters of a camera; the left camera (4a) and the right camera (4b) shoot Gaussian grid patterns (6) with the same projection, a world coordinate system is established on a camera coordinate system of the left camera (4a), and a basic matrix F is calculated by using the corrected matching points of the images shot by the left camera (4a), the right camera (4a) and the right camera (4 b); calculating an essential matrix E by using the solved internal parameters and the basic matrix under the condition of a difference of a scale factor s; after the intrinsic matrix E is decomposed, the external parameters (the rotation matrix R 'and the translation vector t') can be determined under the condition of a difference of a scale factor;

projecting parallel Gaussian light bars to actual length L by using projector (3)₀On the accurately measured gauge block, the central line of sub-pixel light bar is fitted by using the Gaussian characteristic of the light bar, the point with abrupt gray scale change is used as the boundary point of the gauge block, and the length L 'of the gauge block is reconstructed according to the obtained internal and external parameters'₀The scale factor can be obtained as: s ═ L₀/L'₀(ii) a Therefore, the actual external parameters of the camera (the rotation matrix R 'and the translation vector t s t') can be obtained; thus, the calibration process of the camera is completed.