CN106780628A - High Precision Camera Calibration method based on mixing distortion model - Google Patents

Abstract

Based on the High Precision Camera Calibration method of mixing distortion model, comprise the following steps：S1. distortion of camera model is set up；S2. pattern is demarcated to choose, demarcate feature extraction and characteristic point automatic numbering；S3. camera calibration.The present invention considers various distortion comprehensively, and pseudo-random numbers generation is filtered based on the criterion for proposing, then realizes being automatically positioned for index point based on the center of circle automatic numbering method for proposing.

Description

High-precision camera calibration method based on mixed distortion model

Technical Field

The invention belongs to the field of signal processing, and relates to a high-precision camera calibration method for a solder paste measurement system in the technical field of calibration.

Background

The structured light three-dimensional measurement system establishes the three-dimensional morphology of a corresponding object on the basis of two-dimensional information by acquiring the two-dimensional image information of the object, and the accuracy of the three-dimensional morphology is based on the standard and accurate two-dimensional image information. Therefore, the specification of computer vision system parameters is very important for the two-dimensional image acquisition in the early stage of structured light projection three-dimensional measurement and the reconstruction of the three-dimensional morphology in the later stage. In most cases, these parameters must be obtained through experimentation and calculation, a process known as camera calibration. The camera calibration is one of the main contents of machine vision, and is a precondition and a key step for realizing the machine vision. With the continuous popularization of machine vision technology, the camera calibration technology has important significance in deep research, and is highly valued and widely concerned by scholars at home and abroad. In the past twenty years, the calibration technology of the camera has been greatly developed, and many scholars propose different camera models and calibration methods according to actual application specific needs, and a series of achievements are obtained based on different starting points and ideas. At present, academic research ideas in the field of camera calibration are very active, and new technical and new methods are emerging.

There are many researches on calibration of cameras, which are mainly classified into three types, namely a traditional camera calibration method, a self-calibration method and a camera calibration method based on active vision. The most widely used camera calibration methods are the Tsai method and the zhangnyou method. The method of Tsai relies on accurate 3-D measurements of an external 3-D calibrant that determines a good reference coordinate system. All position and orientation measurements are relative to this reference frame. This approach has been widely used in multi-camera systems. The sheet method uses a planar calibration object. The advantage of this method is that planar calibrators are easier to construct than 3-D calibrators, and the extrinsic (camera-to-world or camera-to-camera geometry) and intrinsic (intrinsic) parameters of the camera can be calibrated in the same system. Similar methods are also proposed by Sturm and Maybank, and the singularity of the calibration result is discussed. Brown proposed an 8-parameter model to compensate for lens distortion, and later researchers developed 10-parameter models and multi-parameter models. The models used by Zhangyippe and Tsai are simpler parametric models. 2001. Luhmann and Hastedt et al proposed a finite element-based mixed distortion model in 2002. In 2004 von willebrand professor et al proposed a digital distortion model based on two-dimensional direct linear transformation.

The camera calibration technology is not only researched by foreign researchers, but also the researchers in China pay great attention to and pay extensive attention to the camera calibration technology. In recent years, masonde et al, the institute of automation of the academy of sciences in china, has marked insights into camera calibration technology, and represents the research level of the camera calibration technology in our country. Although the international application field has no influence, the method shows that the talents and technical conditions for developing the research of the camera calibration field are provided in China, and plays a model role in promoting the research and application of the camera calibration technology. In addition, many key laboratories in China also make extensive research on camera calibration, such as the intelligent technology and system state key laboratory of the Qinghua university, the pattern recognition state key laboratory of the Chinese academy of sciences, and the precision testing technology and instrument state key laboratory of the Tianjin university.

How to improve the precision and efficiency of system calibration has been a hot spot and a difficult point studied by many scholars. Although a number of researchers have proposed different camera calibration methods. However, most of the distortion models adopted by the existing camera calibration methods are relatively simple distortion models, and some scholars directly adopt linear models without considering distortion at all, and few people use relatively complex distortion models to calibrate the camera.

Disclosure of Invention

The invention aims to solve the problem of camera calibration of a high-precision solder paste measuring system close to physical reality conditions, and provides a high-precision camera calibration method based on a mixed distortion model, which adopts the following technical scheme:

a high-precision camera calibration method based on a mixed distortion model comprises the following steps:

s1, establishing a camera distortion model;

s2, selecting a calibration pattern, extracting calibration characteristics and automatically numbering characteristic points;

and S3, calibrating the camera.

Further, the establishing of the camera distortion model comprises

1) Pinhole imaging model: four are established in the modelA base coordinate system comprising: world coordinate system O_wX_wY_wZ_wCamera coordinate system O_cX_cY_cZ_cImage coordinate system xy, pixel coordinate system uv, camera coordinate system (X)_c,Y_c,Z_c) With the optical center O of the camera lens_cIs the origin of coordinates, X_c,Y_cThe axis being parallel to the image plane, Z_cThe axis is perpendicular to the image plane, and the coordinate of the intersection point on the image plane on the image coordinate system is (u)₀,v₀) The distance between the optical center of the camera lens and the principal point of the camera is the focal length f;

based on the coordinate system, a point P (X) in space can be obtained_w,Y_w,Z_w) Can be expressed in the camera coordinate system as:

whereinIs a 3 × 3 orthogonal rotation matrix,is a 3 × 1 translation matrix;

obtaining camera coordinates (X) based on pinhole imaging model_c,Y_c,Z_c,1)^TCan be expressed as:

wherein, (u, v,1)^TAs image pixel coordinates, p_x,ρ_yThe number of pixels in the x, y directions in the image plane, respectively, (u)₀,v₀) Is the coordinate of the principal point of the camera, which is the pixel coordinate of the center of the image plane, α is the drawingA skew factor of the pixel coordinate axis vertical error;

obtaining a relational expression of the image coordinates and the world coordinates according to the two expressions:

wherein,as a camera intrinsic parameter matrix, E ═ R T]Is an external parameter matrix;

2) constructing a camera distortion model: a mixed distortion model based on a rectangular finite element method is used, and is described as follows:

wherein dx and dy are the deviation of the x coordinate and the y coordinate of the image point respectively;

wherein,x_i,y_irespectively, the ideal distortion-free image plane coordinate, K, conforming to the pinhole imaging_c1,K_c2,K_c3Third order radial distortion coefficients;

wherein, P_c1,P_c2Is a second order tangential distortion coefficient;

wherein S is_c1,S_c2Second order thin prism distortion coefficients;

wherein, x and y are the length ratio of the image point in the x and y directions of the rectangular grid respectively.

Further, in the step S2, the calibration pattern is selected, the calibration board is an aluminum calibration board, the camera is calibrated by using the aluminum calibration board, the calibration board has 9 × 11 circular calibration points, wherein the small circle has a diameter of 4mm, the large circle has a diameter of 8mm, and five large circles are used for identifying the direction of the calibration board in the calibration process.

Further, the calibration feature extraction comprises the following steps: image acquisition, image graying, image filtering, image binarization, contour extraction, pseudo point filtering, ellipse fitting and circle center extraction;

the method comprises the steps of directly reading a color image into a gray image by using a function in an OpenCV function library, filtering the obtained gray image, realizing image filtering by using a cvSmooth function, setting a threshold value as 100 through experiments in image binarization, and extracting a contour by using an iteration-based improved Harris corner detection method, wherein the method is described as follows: considering any point p in the neighborhood of the corner q, # i (p) represents the image gradient vector at point p, then the gradient property yields:

▽I(p)(q-p)＝0

considering all the p points in the corner q neighborhood, an overdetermined equation set is formed, and therefore the Minimum Mean Square Error (MMSE) criterion is utilized for solving;

after extracting the contour, eliminating the pseudo feature points by using the following rule:

area criterion: and (3) determining the range of the area of the extracted feature object according to the system configuration:

S_min＜S＜S_max

roundness criterion: the proximity of the shape to the circle is measured by:

wherein T is the circumference of the circle,

error criteria: taking the algebraic distance from the edge contour points to the fitting ellipse as an error, and further filtering out circular feature points with larger errors by adopting the following criteria:

whereinThe average error of the contour points is represented,_maxwhich is indicative of the maximum allowable error,_meanindicating an allowable average error;

further, after eliminating the pseudo characteristic points, performing contour ellipse fitting by adopting a weighted least square method to extract circle center coordinates of sub-pixel level precision.

The feature points are automatically numbered in step S2: utilizing 5 solid circles on the calibration plate, wherein each solid circle corresponds to a determined invariable number, and carrying out the following steps:

(1) selecting four corner points corresponding to a certain rectangular area in space on an image by using a mouse, acquiring image coordinates of the four corner points and performing sub-pixel processing;

(2) and automatically counting the circles of the selected area, and generating a grid map according to the counting result.

The camera calibration method comprises the following steps:

1) homography matrix solving

Assuming that the template is located at the position of the world coordinate system Z being 0, obtaining the template through a pinhole model

sm＝HM

Where s is a scale factor, M and M are points on the image plane and the corresponding template, respectively, and H is K_c[R T]；

Based on the space coordinates of each corner point of the plane template, the estimated value of H is obtained by solving the following MMSE estimation,

wherein, is the ith row vector of the H matrix;

solving by adopting an improved Leverberg-Marquardt method, solving by using a nonlinear optimization method requires a proper initial value for iteration, and obtaining by solving the following equation:

given n > 6 points, forming an over-determined equation set, and solving through Singular Value Decomposition (SVD);

2) solution of internal and external parameters

Based on the property that the internal parameter matrix R is an orthogonal matrix, the internal and external parameters of the camera are obtained by solving the following over-determined equation set:

Vb＝0

the equation set is the superposition of the following equations obtained by observing n is more than or equal to 3 times:

in the formula v_ij＝[h_1ih_1j,h_1ih_2j+h_2ih_1j,h_2ih_2j,h_3ih_1j+h_1ih_3j,h_3ih_2j+h_2ih_3j,h_3ih_3j]^T，b＝[B₁₁,B₁₂,B₂₂,B₁₃,B₂₃,B₃₃]^T，

Then, the solution is refined through maximum likelihood estimation to obtain a better optimized solution, and the likelihood estimation problem is expressed as follows:

in the formula, m_ijThe j image point on the ith image is taken;as a spatial point M_jA projection point of the ith image;

3) solving distortion parameters;

4) solving distortion parameters of the finite element model;

5) the LM algorithm is improved.

Has the advantages that:

the invention provides a camera calibration method based on a mixed distortion model, aiming at the problem that the distortion model adopted by the existing calibration algorithm is simpler, so that high-precision calibration cannot be realized. The method comprises the steps of firstly establishing an imaging model based on pinhole imaging, and then establishing a mixed distortion model aiming at three kinds of distortion and plane unequal physical errors. In order to calibrate the internal and external parameters and distortion parameters of the camera, the invention determines a calibration template capable of realizing automatic positioning, and then realizes the sub-pixel positioning of the mark points based on the template. Under the condition of obtaining the accurate positioning of the mark points, the initial solution of the homography matrix is obtained by solving the over-determined equation set, and then the high-efficiency optimization is carried out by utilizing the improved LM method; based on the homography matrix, internal and external parameters of the camera can be obtained, and further optimization can be realized by utilizing maximum likelihood estimation; after the internal and external parameters are obtained, three distortion parameters and finite element distortion parameters of the camera can be respectively obtained by solving an over-determined equation and then further optimizing by using an improved LM method. Compared with the traditional calibration method, the method comprehensively considers various distortions, filters the pseudo characteristic points based on the proposed criterion, realizes the automatic positioning of the mark points based on the proposed automatic circle center numbering method, and optimizes based on the improved LM algorithm in the process of solving the calibration parameters, thereby obviously accelerating the convergence speed and improving the calibration precision. Based on the above discussion, the method provided by the invention can provide a solid theoretical and implementation basis for high-precision camera calibration in engineering applications such as machine vision, three-dimensional measurement and the like.

Drawings

FIG. 1 is a flow chart of an implementation of the present invention;

FIG. 2 is a schematic diagram of pinhole imaging;

FIG. 3 is a calibration pattern employed in the present invention;

FIG. 4 is a flow chart of feature point extraction;

FIG. 5 is a calibration image taken according to the present invention;

FIG. 6 is a flow chart of edge detection;

FIG. 7 is a plot of marker point numbers;

FIG. 8 is a result of automatic feature point numbering;

FIG. 9 is a graph of camera calibration error distribution.

Detailed Description

Example 1:

a high-precision camera calibration method based on a mixed distortion model comprises the steps of firstly solving an overdetermined equation set to obtain an initial solution of a homography matrix, and then carrying out efficient optimization by using the proposed improved LM method; based on the homography matrix, internal and external parameters of the camera can be obtained, and further optimization can be realized by utilizing maximum likelihood estimation; after the internal and external parameters are obtained, three distortion parameters and finite element distortion parameters of the camera can be respectively obtained by solving an over-determined equation and then further optimizing by using an improved LM method. Therefore, accurate estimation of the mixed distortion parameters of the camera is obtained, and high-precision calibration of the camera is further realized.

The basic idea for realizing the method is to establish an imaging model based on pinhole imaging, and then establish a mixed distortion model aiming at three kinds of distortion and plane unequal physical errors. In order to calibrate the internal and external parameters and distortion parameters of the camera, the invention determines a calibration template capable of realizing automatic positioning, and then realizes the sub-pixel positioning of the mark points based on the template. Under the condition of obtaining the accurate positioning of the mark points, the initial solution of the homography matrix is obtained by solving the over-determined equation set, and then the high-efficiency optimization is carried out by utilizing the improved LM method; based on the homography matrix, internal and external parameters of the camera can be obtained, and further optimization can be realized by utilizing maximum likelihood estimation; after the internal and external parameters are obtained, three distortion parameters and finite element distortion parameters of the camera can be respectively obtained by solving an over-determined equation and then further optimizing by using an improved LM method. Compared with the traditional calibration method, the method comprehensively considers various distortions, filters the pseudo characteristic points based on the proposed criterion, realizes the automatic positioning of the mark points based on the proposed automatic circle center numbering method, and optimizes based on the improved LM algorithm in the process of solving the calibration parameters, thereby obviously accelerating the convergence speed and improving the calibration precision. The method comprises the following specific steps:

1. establishing a camera distortion model

1) Constructing pinhole imaging models

The camera model is a simplification of the optical imaging geometry, and many camera imaging geometry models are obtained according to the pinhole imaging principle. The pinhole imaging model becomes the simplest camera model due to the simpler principle of pinhole imaging, and is the basic model of the camera calibration algorithm. If the distortion factor of the lens is properly considered, the precision required by many application occasions can be met. The pinhole model in the ideal situation shown in fig. 2 is adopted in the patent, and the imaging process can be reflected more accurately by considering the influence of lens distortion on the basis. World coordinate system (X)_w,Y_w,Z_w) The reference coordinate system is selected in the environment and used for describing the position of the camera, and the reference coordinate system can be selected according to the principles of convenience in description and calculation and the like. Four basic coordinate systems are established in the model, including: world coordinate system O_wX_wY_wZ_wCamera coordinate system O_cX_cY_cZ_cImage coordinate system xy, pixel coordinate system uv. The imaging process from the three-dimensional coordinates of the object points in space to the image is the process of the step-by-step transformation of these several coordinate systems. Camera coordinate system (X)_c,Y_c,Z_c) With the optical center O of the camera lens_cIs the origin of coordinates, X_c,Y_cThe axis is parallel to the image plane,Z_cthe axis is perpendicular to the image plane, and the coordinate of the intersection point on the image plane on the image coordinate system is (u)₀,v₀) I.e. the main point of the camera. It should be noted that this point is generally located at the center of the image plane, but sometimes deviates due to the manufacturing of the camera, so the camera principal point coordinates are also generally two parameters that need to be calibrated. The distance between the optical center of the camera lens and the principal point is the focal length f.

Based on the coordinate system, a point P (X) in space can be obtained_w,Y_w,Z_w) Can be expressed in the camera coordinate system as:

whereinIs a 3 × 3 orthogonal rotation matrix,is a 3 × 1 translation matrix.

Camera coordinates (X) can be obtained based on camera pinhole imaging model_c,Y_c,Z_c,1)^TCan be expressed as:

wherein, (u, v,1)^TAs image pixel coordinates, p_x,ρ_yThe number of pixels in the x, y directions in the image plane, respectively, (u)₀,v₀) For the camera principal point coordinates, i.e., the pixel coordinates of the center of the image plane, α is a skew factor that describes the vertical error of the pixel coordinate axes.

From the above two formulas, the relational expression of the image coordinate and the world coordinate can be obtained,

wherein,as a camera intrinsic parameter matrix, E ═ R T]Is an extrinsic parameter matrix.

2) Camera distortion model

The last part of camera models are linear models without considering lens distortion, but due to the influence of factors such as lens design, manufacture and assembly, an actual imaging system cannot strictly meet the pinhole imaging principle, and light rays are slightly deviated due to distortion. And a large number of experiments show that the camera pinhole model given in the previous section cannot accurately describe the imaging geometric relationship of the camera, and particularly, when a wide-angle lens is used, an image point far away from the center of the image in the image has great distortion. The nonlinear optical distortion of the camera lens mainly includes three types: radial Distortion, eccentric Distortion, and thin prism Distortion. Wherein the radial distortion causes the image point to move radially inward or outward along the principal point of the image, the tangential distortion causes the image point to move tangentially, and the eccentric distortion and the thin prism distortion cause distortion at both the radial position and the tangential position.

The parametric model has very accurate description on radial distortion, tangential distortion and thin prism distortion, has few unknown parameters, is relatively simple to solve, and cannot describe distortion caused by factors such as uneven image plane. Finite element models can describe various distortions, but the unknown parameters to be considered are many, and the solution is relatively difficult. The finite element model integrates the advantages of the two models, can accurately solve three large distortions which have large influence on the distortion, and can also compensate the distortions such as an image plane and the like. Thus, the present invention employs a hybrid distortion model based on a rectangular finite element method, which can be described as follows:

wherein dx and dy are the deviation of the x coordinate and the y coordinate of the image point respectively;

wherein,x_i,y_irespectively, the ideal distortion-free image plane coordinate, K, conforming to the pinhole imaging_c1,K_c2,K_c3Third order radial distortion coefficients;

wherein, P_c1,P_c2Is a second order tangential distortion coefficient;

wherein S is_c1,S_c2Second order thin prism distortion coefficients;

wherein, x and y are the length ratio of the image point in the x and y directions of the rectangular grid respectively.

2. Calibration pattern selection, calibration feature extraction and feature point automatic numbering

1) Calibration pattern selection

In general, the design of a two-dimensional planar calibration object should satisfy the following basic requirements: firstly, the image characteristics of the calibration object should be easy to identify, that is, the mark points on the image should have sharp differences between the color and the background and should be as clear and uniform as possible; secondly, the mark points on the calibration object need to be easy to extract when the image is processed, and a high-quality contour can be provided; thirdly, the calibration object should have high stability, i.e. its mark point should not generate large-scale distortion with the change of the camera position, and it should be easy to measure; fourthly, to realize automatic calibration, the designed mark points are easy to realize automatic coding.

Based on the above requirements, this patent adopts the aluminium system high accuracy calibration board that fig. 3 shows to carry out parameter calibration to the camera, has 9X 11 circular calibration points on the calibration board, and wherein the small circle diameter is 4mm, and the major circle diameter is 8mm, and five great circles are used for marking the calibration board direction in the calibration process to can realize demarcating the automation.

2) Calibration feature extraction

To accurately extract the coordinates of the center of circle in the calibration plate image, image processing techniques are indispensable. The structural block diagram of the circle center extraction process is shown in fig. 4. The method mainly comprises the following steps: the method comprises the steps of image acquisition, image graying, image filtering, image binarization, contour extraction, pseudo point filtering, ellipse fitting and circle center extraction.

This patent utilizes the camera to fix a plate at different positions and angle acquisition 12 images, as shown in fig. 5.

This patent utilizes the function in the OpenCV function library directly to become the grey image with the color image read-in, carries out the filtering to the grey image that obtains immediately, and function cvSmooth can effectual realization image filtering. In image binarization, the threshold value can be set to 100 through experiments.

The contour extraction can adopt an iteration-based improved Harris corner detection method proposed by the patent, and the method can be described as follows: considering any point p in the neighborhood of the corner q, # I (p) represents the gradient vector of the image at point p, the gradient property can be derived

▽I(p)(q-p)＝0

Considering all p points in the neighborhood of the corner point q, an overdetermined system of equations can be formed, so that the solution can be performed by using Minimum Mean Square Error (MMSE) criterion.

Let q be^k＝[x_k,y_k]^TFor the initial estimation of points for the corner points, then

Solving the optimization problem, the method can be obtained

q^k+1＝F-¹W

Wherein, x, Y are first order gray gradients, w_u,vAre gaussian filter coefficients.

By repeating the iteration, q can be updated^k+1Up to | q^k+1-q^kI is no longer reduced. The above-mentioned edge detection method is shown in the flowchart of fig. 6.

After extracting the contour, the pseudo feature points can be eliminated by the following rules:

1. criterion of area

Since the imaging size is only related to the resolution of the camera and the measured distance, the area from which the feature objects are extracted can also be range-determined according to the system configuration, i.e. the imaging size is determined by the system configuration

S_min＜S＜S_max

2. Criterion of roundness

The proximity of a shape to a circle can be measured by:

where T is the circumference and a larger C indicates a greater difference in shape from a circle. The roundness criterion of the patent is

3. Error criterion

The algebraic distance from the edge contour points to the fitting ellipse is used as an error, and the circular feature points with larger errors are further filtered by adopting the following criteria.

WhereinThe average error of the contour points is represented,_maxwhich is indicative of the maximum allowable error,_meanindicating the allowable average error. This patent sets up the error correlation value as follows:_max＝0.6pixel，_mean＝0.3pixel。

after the pseudo characteristic points are eliminated, the invention adopts a weighted least square method to carry out contour ellipse fitting, thereby extracting the circle center coordinates of sub-pixel level precision.

3) Automatic numbering of feature points

The operation of the front part has extracted the coordinates of the center of the ellipse of interest, but cannot correspond the coordinates of the center of the ellipse to the coordinates of the center of the circle of the circular mark point on the calibration plate. To realize camera calibration, it is necessary to correspond the corresponding ellipse center coordinates to the mark point circle center, and the automatic numbering of the circular mark points is the key point for realizing the automatic calibration of the camera.

In order to realize automatic numbering of the circular mark points, 5 large solid circles on the calibration plate can be utilized, the numbers of all solid circles in the calibration plate are shown in fig. 7, and each solid circle corresponds to a certain invariable number.

To realize correct automatic numbering of all circular mark points, the following steps are required:

(3) selecting four corner points corresponding to a certain rectangular area in space on an image by using a mouse, acquiring image coordinates of the four corner points and performing sub-pixel processing. To make more use of the image information, the area should be as large as possible to contain more control points. Assuming that the homogeneous coordinates of the points on the spatial plane template corresponding to the four points are (0,0,1), (1,0,1), (1,1,1), (0,1,1), this is actually a coordinate transformation of the points on the spatial plane template. This step simultaneously establishes the world coordinate system. The method for establishing the world coordinate system is that the selected first angular point is used as the origin of the world coordinate system, the Z axis is perpendicular to the plane of the square grids and points to the outside of the plane template, and after the Z axis is determined, the X axis and the Y axis are determined by taking the right-hand system as a standard.

(4) And automatically counting the circles of the selected area, and generating a grid map according to the counting result. The automatic counting method comprises the following steps: a) positioning a great circle: 1. extracting 5 great circles: firstly, screening outline information, only retaining ellipse information, then calculating the area and the perimeter of an ellipse according to ellipse parameters obtained by ellipse fitting, and selecting 5 with the largest areas. 2. The distance of the 5 great circles from each other is calculated. 3. Grouping the great circles: the nearest is group A26, 27, the farthest is group B46, 52, and the rest is group 71. 4. The great circle is numbered: the sum of the distances between the centers of two circles in the group B and the group A is calculated, wherein the larger is No. 52, the other is No. 46, and the larger is No. 27 and the other is No. 26 in the same way. b) Positioning the small circle: on the basis of the positioning of the big circle, the positioning of the small circle can be carried out. 1. Defining an area: the actual coordinates of the centers of the 27 # and 71 # and 46 # and 52 # great circles are obtained, so that the two straight-line tracks can be calculated and then expanded up and down along the tracks to form local images (the up and down expansion is 3 pixels larger than that of the great circles). 2. Calculating a gray curve: and calculating the sum of the gray levels of each pixel point in the area. 3. Noise filtering: filtering may be performed to prevent jitter at zero crossings of the curve. 3. According to the prior knowledge of the small circle radius, the system configuration and the like, the automatic numbering of the small circle can be realized by combining the gray curve. The numbering result is shown in fig. 8, in which the cross symbols indicate the coordinates of the center of the ellipse.

3. Camera calibration

Because what this patent adopted is mixed distortion model, therefore the distortion of camera mainly includes two parts: various distortions described by using parameters and unequal distortions of an image plane described by using a finite element model. As the parameter model can accurately describe radial distortion, tangential distortion and thin prism distortion, and the finite element model describes the distortion of each image point in the x and y directions caused by the factors of image plane inequality and the like, the calibration process can be divided into the general solution of the parameter model and the solution of the finite element model.

1) Homography matrix solving

Assuming that the template is located at the position where the world coordinate system Z is 0, the pinhole model can be used to obtain the template

sm＝HM

Where s is a scale factor, M and M are points on the image plane and the corresponding template, respectively, and H is K_c[R T]。

Based on the spatial coordinates of each corner point of the plane template, the estimated value of H can be obtained by solving the following MMSE estimation, namely

Wherein, is the ith row vector of the H matrix.

The problem is a nonlinear least squares problem and can be solved by various nonlinear optimization methods such as a gradient method, a gauss-newton method or a Leverberg-Marquardt method. The improved Leverberg-Marquardt solution proposed by this patent (see details 5)) solution by non-linear optimization requires a suitable initial value for iteration, which can be obtained by solving the following equation:

given n > 6 pairs of points, an overdetermined system of equations may be formed, which may then be solved by Singular Value Decomposition (SVD).

2) Solution of internal and external parameters

Based on the property that the internal parameter matrix R is an orthogonal matrix, the internal and external parameters of the camera can be obtained by solving the following over-determined equation set:

Vb＝0

the equation set is the superposition of the following equations obtained by observing n is more than or equal to 3 times:

in the formula v_ij＝[h_1ih_1j,h_1ih_2j+h_2ih_1j,h_2ih_2j,h_3ih_1j+h_1ih_3j,h_3ih_2j+h_2ih_3j,h_3ih_3j]^T，b＝[B₁₁,B₁₂,B₂₂,B₁₃,B₂₃,B₃₃]^T，

This solution can then be refined by maximum likelihood estimation to get a better optimized solution, and the likelihood estimation problem can be expressed as follows:

in the formula, m_ijThe j image point on the ith image is taken;as a spatial point M_jAt the projection point of the ith image.

3) Solution of distortion parameters

From the above distortion model, we can obtain:

in the formula (x)_r,y_r) As actual image plane coordinates, (x)_i,y_i) Is ideal distortion-free image plane coordinates.

In addition, as is known in the art,

in the formula (u)_r,v_r) As actual image coordinates, (u)_i,v_i) Ideal undistorted image coordinates.

If each image has m marker points and n images are collected, based on the above two equations, the following overdetermined system of equations can be obtained:

GC＝D

wherein,

the vector C can be found by means of the least squares method (LS):

C＝(G^TG)^-1G^TD

after the initial solution is obtained, the initial solution can be further optimized by a nonlinear method, so that an accurate solution of the distortion parameter can be obtained.

4) Finite element model distortion parameter solution

According to the finite element-based distortion model, an overdetermined equation set can be obtained by using the obtained data, so that an initial solution of the finite element model distortion parameters can be obtained by using an LS method, and then nonlinear optimization is performed by using an improved Levenberg-Marquardt algorithm.

5) Improved LM algorithm

There are three types of nonlinear least squares methods commonly used today: steepest descent method, gauss-newton method and LM method. In the LM method, an iteration increment formula is as follows:

Δθ＝(J^TJ+μI)^-1J^T _i

where μ is the damping factor, J is the partial derivative matrix,_ito estimate the error. Wherein, the value of mu greatly determines the convergence rate of the LM method and the accuracy of the solution. However, in practical applications, the optimal value of μ cannot be determined. To improveThe invention provides an improved factor which can change along with the change of iteration times, so that an initial value can be widened in the initial iteration period, the convergence speed can be increased in the later period, the LM algorithm has the steepest descent method in the initial iteration period and has the advantages of a Gaussian-Newton method near an optimal solution, and the descent factor is as follows:

1-(1-μ)ⁱ⁺¹

the incremental iterative formula can thus be re-expressed as:

Δθ＝(J^TJ+(1-(1-μ)ⁱ⁺¹)I)^-1J^T _i

example 2:

a high-precision camera calibration method based on a mixed distortion model comprises the following steps: comprises that

1. Establishing a camera distortion model

1) Constructing pinhole imaging models

The camera model is a simplification of the optical imaging geometry, and many camera imaging geometry models are obtained according to the pinhole imaging principle. The pinhole imaging model becomes the simplest camera model due to the simpler principle of pinhole imaging, and is the basic model of the camera calibration algorithm. If the distortion factor of the lens is properly considered, the precision required by many application occasions can be met. The pinhole model in the ideal situation shown in fig. 2 is adopted in the patent, and the imaging process can be reflected more accurately by considering the influence of lens distortion on the basis. World coordinate system (X)_w,Y_w,Z_w) The reference coordinate system is selected in the environment and used for describing the position of the camera, and the reference coordinate system can be selected according to the principles of convenience in description and calculation and the like. Four basic coordinate systems are established in the model, including: world coordinate system O_wX_wY_wZ_wCamera coordinate system O_cX_cY_cZ_cImage coordinate system xy, pixel coordinate system uv. The imaging process from the three-dimensional coordinates of the object points in space to the image is the process of the step-by-step transformation of these several coordinate systems. Camera coordinate system (X)_c,Y_c,Z_c) With the optical center O of the camera lens_cIs the origin of coordinates, X_c,Y_cThe axis being parallel to the image plane, Z_cThe axis is perpendicular to the image plane, and the coordinate of the intersection point on the image plane on the image coordinate system is (u)₀,v₀) I.e. the main point of the camera. It should be noted that this point is generally located at the center of the image plane, but sometimes deviates due to the manufacturing of the camera, so the camera principal point coordinates are also generally two parameters that need to be calibrated. The distance between the optical center of the camera lens and the principal point is the focal length f.

Based on the coordinate system, a point P (X) in space can be obtained_w,Y_w,Z_w) Can be expressed in the camera coordinate system as:

whereinIs a 3 × 3 orthogonal rotation matrix,is a 3 × 1 translation matrix.

Camera coordinates (X) can be obtained based on camera pinhole imaging model_c,Y_c,Z_c,1)^TCan be expressed as:

wherein, (u, v,1)^TAs image pixel coordinates, p_x,ρ_yThe number of pixels in the x, y directions in the image plane, respectively, (u)₀,v₀) For the camera principal point coordinates, i.e., the pixel coordinates of the center of the image plane, α is a skew factor that describes the vertical error of the pixel coordinate axes.

From the above two formulas, the relational expression of the image coordinate and the world coordinate can be obtained,

wherein,as a camera intrinsic parameter matrix, E ═ R T]Is an extrinsic parameter matrix.

2) Camera distortion model

The last part of camera models are linear models without considering lens distortion, but due to the influence of factors such as lens design, manufacture and assembly, an actual imaging system cannot strictly meet the pinhole imaging principle, and light rays are slightly deviated due to distortion. And a large number of experiments show that the camera pinhole model given in the previous section cannot accurately describe the imaging geometric relationship of the camera, and particularly, when a wide-angle lens is used, an image point far away from the center of the image in the image has great distortion. The nonlinear optical distortion of the camera lens mainly includes three types: radial Distortion (Radial Distortion), eccentric Distortion (Decentering Distortion), and thin Prism Distortion (Prism Distortion). Wherein the radial distortion causes the image point to move radially inward or outward along the principal point of the image, the tangential distortion causes the image point to move tangentially, and the eccentric distortion and the thin prism distortion cause distortion at both the radial position and the tangential position.

The parametric model has very accurate description on radial distortion, tangential distortion and thin prism distortion, has few unknown parameters, is relatively simple to solve, and cannot describe distortion caused by factors such as uneven image plane. Finite element models can describe various distortions, but the unknown parameters to be considered are many, and the solution is relatively difficult. The finite element model integrates the advantages of the two models, can accurately solve three large distortions which have large influence on the distortion, and can also compensate the distortions such as an image plane and the like. Thus, the present invention employs a hybrid distortion model based on a rectangular finite element method, which can be described as follows:

wherein dx and dy are the deviation of the x coordinate and the y coordinate of the image point respectively;

wherein,x_i,y_irespectively, the ideal distortion-free image plane coordinate, K, conforming to the pinhole imaging_c1,K_c2,K_c3Third order radial distortion coefficients;

wherein, P_c1,P_c2Is a second order tangential distortion coefficient;

wherein S is_c1,S_c2Second order thin prism distortion coefficients;

wherein, x and y are the length ratio of the image point in the x and y directions of the rectangular grid respectively.

2. Calibration pattern selection, calibration feature extraction and feature point automatic numbering

1) Calibration pattern selection

In general, the design of a two-dimensional planar calibration object should satisfy the following basic requirements: firstly, the image characteristics of the calibration object should be easy to identify, that is, the mark points on the image should have sharp differences between the color and the background and should be as clear and uniform as possible; secondly, the mark points on the calibration object need to be easy to extract when the image is processed, and a high-quality contour can be provided; thirdly, the calibration object should have high stability, i.e. its mark point should not generate large-scale distortion with the change of the camera position, and it should be easy to measure; fourthly, to realize automatic calibration, the designed mark points are easy to realize automatic coding.

Based on the above requirements, this patent adopts the aluminium system high accuracy calibration board that fig. 3 shows to carry out parameter calibration to the camera, has 9X 11 circular calibration points on the calibration board, and wherein the small circle diameter is 4mm, and the major circle diameter is 8mm, and five great circles are used for marking the calibration board direction in the calibration process to can realize demarcating the automation.

2) Calibration feature extraction

To accurately extract the coordinates of the center of circle in the calibration plate image, image processing techniques are indispensable. The structural block diagram of the circle center extraction process is shown in fig. 4. The method mainly comprises the following steps: the method comprises the steps of image acquisition, image graying, image filtering, image binarization, contour extraction, pseudo point filtering, ellipse fitting and circle center extraction.

This patent utilizes the camera to fix a plate at different positions and angle acquisition 12 images, as shown in fig. 5.

This patent utilizes the function in the OpenCV function library directly to become the grey image with the color image read-in, carries out the filtering to the grey image that obtains immediately, and function cvSmooth can effectual realization image filtering. In image binarization, the threshold value can be set to 100 through experiments.

The contour extraction can adopt an iteration-based improved Harris corner detection method proposed by the patent, and the method can be described as follows: considering any point p in the neighborhood of the corner q, # I (p) represents the gradient vector of the image at point p, the gradient property can be derived

▽I(p)(q-p)＝0

Considering all p points in the neighborhood of the corner point q, an overdetermined system of equations can be formed, so that the solution can be performed by using Minimum Mean Square Error (MMSE) criterion.

Let q be^k＝[x_k,y_k]^TFor the initial estimation of points for the corner points, then

Solving the optimization problem, the method can be obtained

q^k+1＝F-¹W

Wherein, x, Y are first order gray gradients, w_u,vAre gaussian filter coefficients.

By repeating the iteration, q can be updated^k+1Up to | q^k+1-q^kI is no longer reduced. The above-mentioned edge detection method is shown in the flowchart of fig. 6.

After extracting the contour, the pseudo feature points can be eliminated by the following rules:

4. criterion of area

Since the imaging size is only related to the resolution of the camera and the measured distance, the area from which the feature objects are extracted can also be range-determined according to the system configuration, i.e. the imaging size is determined by the system configuration

S_min＜S＜S_max

5. Criterion of roundness

The proximity of a shape to a circle can be measured by:

where T is the circumference and a larger C indicates a greater difference in shape from a circle. The roundness criterion of the patent is

6. Error criterion

The algebraic distance from the edge contour points to the fitting ellipse is used as an error, and the circular feature points with larger errors are further filtered by adopting the following criteria.

WhereinThe average error of the contour points is represented,_maxwhich is indicative of the maximum allowable error,_meanindicating the allowable average error. This patent sets up the error correlation value as follows:_max＝0.6pixel，_mean＝0.3pixel。

after the pseudo characteristic points are eliminated, the invention adopts a weighted least square method to carry out contour ellipse fitting, thereby extracting the circle center coordinates of sub-pixel level precision.

3) Automatic numbering of feature points

The operation of the front part has extracted the coordinates of the center of the ellipse of interest, but cannot correspond the coordinates of the center of the ellipse to the coordinates of the center of the circle of the circular mark point on the calibration plate. To realize camera calibration, it is necessary to correspond the corresponding ellipse center coordinates to the mark point circle center, and the automatic numbering of the circular mark points is the key point for realizing the automatic calibration of the camera.

In order to realize automatic numbering of the circular mark points, 5 large solid circles on the calibration plate can be utilized, the numbers of all solid circles in the calibration plate are shown in fig. 7, and each solid circle corresponds to a certain invariable number.

To realize correct automatic numbering of all circular mark points, the following steps are required:

(5) selecting four corner points corresponding to a certain rectangular area in space on an image by using a mouse, acquiring image coordinates of the four corner points and performing sub-pixel processing. To make more use of the image information, the area should be as large as possible to contain more control points. Assuming that the homogeneous coordinates of the points on the spatial plane template corresponding to the four points are (0,0,1), (1,0,1), (1,1,1), (0,1,1), this is actually a coordinate transformation of the points on the spatial plane template. This step simultaneously establishes the world coordinate system. The method for establishing the world coordinate system is that the selected first angular point is used as the origin of the world coordinate system, the Z axis is perpendicular to the plane of the square grids and points to the outside of the plane template, and after the Z axis is determined, the X axis and the Y axis are determined by taking the right-hand system as a standard.

(6) And automatically counting the circles of the selected area, and generating a grid map according to the counting result. The automatic counting method comprises the following steps: a) positioning a great circle: 1. extracting 5 great circles: firstly, screening outline information, only retaining ellipse information, then calculating the area and the perimeter of an ellipse according to ellipse parameters obtained by ellipse fitting, and selecting 5 with the largest areas. 2. The distance of the 5 great circles from each other is calculated. 3. Grouping the great circles: the nearest is group A26, 27, the farthest is group B46, 52, and the rest is group 71. 4. The great circle is numbered: the sum of the distances between the centers of two circles in the group B and the group A is calculated, wherein the larger is No. 52, the other is No. 46, and the larger is No. 27 and the other is No. 26 in the same way. b) Positioning the small circle: on the basis of the positioning of the big circle, the positioning of the small circle can be carried out. 1. Defining an area: the actual coordinates of the centers of the 27 # and 71 # and 46 # and 52 # great circles are obtained, so that the two straight-line tracks can be calculated and then expanded up and down along the tracks to form local images (the up and down expansion is 3 pixels larger than that of the great circles). 2. Calculating a gray curve: and calculating the sum of the gray levels of each pixel point in the area. 3. Noise filtering: filtering may be performed to prevent jitter at zero crossings of the curve. 3. According to the prior knowledge of the small circle radius, the system configuration and the like, the automatic numbering of the small circle can be realized by combining the gray curve. The numbering result is shown in fig. 8, in which the cross symbols indicate the coordinates of the center of the ellipse.

3. Camera calibration

Because what this patent adopted is mixed distortion model, therefore the distortion of camera mainly includes two parts: various distortions described by using parameters and unequal distortions of an image plane described by using a finite element model. As the parameter model can accurately describe radial distortion, tangential distortion and thin prism distortion, and the finite element model describes the distortion of each image point in the x and y directions caused by the factors of image plane inequality and the like, the calibration process can be divided into the general solution of the parameter model and the solution of the finite element model.

1) Homography matrix solving

Assuming that the template is located at the position where the world coordinate system Z is 0, the pinhole model can be used to obtain the template

sm＝HM

Where s is a scale factor, M and M are points on the image plane and the corresponding template, respectively, and H is K_c[R T]。

Based on the spatial coordinates of each corner point of the plane template, the estimated value of H can be obtained by solving the following MMSE estimation, namely

Wherein, is the ith row vector of the H matrix.

The problem is a nonlinear least squares problem and can be solved by various nonlinear optimization methods such as a gradient method, a gauss-newton method or a Leverberg-Marquardt method. The improved Leverberg-Marquardt solution proposed by this patent (see details 5)) solution by non-linear optimization requires a suitable initial value for iteration, which can be obtained by solving the following equation:

given n > 6 pairs of points, an overdetermined system of equations may be formed, which may then be solved by Singular Value Decomposition (SVD).

2) Solution of internal and external parameters

Based on the property that the internal parameter matrix R is an orthogonal matrix, the internal and external parameters of the camera can be obtained by solving the following over-determined equation set:

Vb＝0

the equation set is the superposition of the following equations obtained by observing n is more than or equal to 3 times:

in the formula v_ij＝[h_1ih_1j,h_1ih_2j+h_2ih_1j,h_2ih_2j,h_3ih_1j+h_1ih_3j,h_3ih_2j+h_2ih_3j,h_3ih_3j]^T，b＝[B₁₁,B₁₂,B₂₂,B₁₃,B₂₃,B₃₃]^T，

This solution can then be refined by maximum likelihood estimation to get a better optimized solution, and the likelihood estimation problem can be expressed as follows:

in the formula, m_ijThe j image point on the ith image is taken;as a spatial point M_jAt the projection point of the ith image.

3) Solution of distortion parameters

From the above distortion model, we can obtain:

in the formula (x)_r,y_r) As actual image plane coordinates, (x)_i,y_i) Is ideal distortion-free image plane coordinates.

In addition, as is known in the art,

in the formula (u)_r,v_r) As actual image coordinates, (u)_i,v_i) Ideal undistorted image coordinates.

If each image has m marker points and n images are collected, based on the above two equations, the following overdetermined system of equations can be obtained:

GC＝D

wherein,

C＝[K_c1,K_c2,K_c3,P_c1,P_c2,S_c1,S_c2]^T，D＝[d₁d₂…d_mn]^T。

the vector C can be found by means of the least squares method (LS):

C＝(G^TG)^-1G^TD

after the initial solution is obtained, the initial solution can be further optimized by a nonlinear method, so that an accurate solution of the distortion parameter can be obtained.

4) Finite element model distortion parameter solution

According to the finite element-based distortion model, an overdetermined equation set can be obtained by using the obtained data, so that an initial solution of the finite element model distortion parameters can be obtained by using an LS method, and then nonlinear optimization is performed by using an improved Levenberg-Marquardt algorithm.

5) Improved LM algorithm

There are three types of nonlinear least squares methods commonly used today: steepest descent method, gauss-newton method and LM method. In the LM method, an iteration increment formula is as follows:

Δθ＝(J^TJ+μI)^-1J^T _i

where μ is the damping factor, J is the partial derivative matrix,_ito estimate the error. Wherein, the value of mu greatly determines the convergence rate of the LM method and the accuracy of the solution. However, in practical applications, the optimal value of μ cannot be determined. In order to improve the convergence rate and the solving accuracy of the LM algorithm, the invention provides an improvement factor which can change along with the change of iteration times, so that the initial value can be relaxed at the initial stage of the iteration, the convergence rate can be increased at the later stage, and the LM algorithm has the steepest descent method at the initial stage of the iteration and has the advantages of the Gaussian-Newton method near the optimal solution, wherein the descent factor is as follows:

1-(1-μ)ⁱ⁺¹

the incremental iterative formula can thus be re-expressed as:

Δθ＝(J^TJ+(1-(1-μ)ⁱ⁺¹)I)^-1J^T _i

the effects of the present invention can be further illustrated by the following simulations:

simulation conditions are as follows:

based on the calibration algorithm, the camera is calibrated by utilizing the acquired images and the extraction and automatic numbering results of the calibration patterns. The resulting calibration results have standard deviations of [0.04356,0.06792] in pixels. The error distribution of the calibration result is shown in fig. 9, and it can be known from the graph that the calibration result is stable and reliable, and the error distribution is uniform.

In summary, the invention provides a camera calibration method based on a mixed distortion model, aiming at the problem that the distortion model adopted by the existing calibration algorithm is simple, so that high-precision calibration cannot be realized. The method comprises the steps of firstly establishing an imaging model based on pinhole imaging, and then establishing a mixed distortion model aiming at three kinds of distortion and plane unequal physical errors. In order to calibrate the internal and external parameters and distortion parameters of the camera, the invention determines a calibration template capable of realizing automatic positioning, and then realizes the sub-pixel positioning of the mark points based on the template. Under the condition of obtaining the accurate positioning of the mark points, the initial solution of the homography matrix is obtained by solving the over-determined equation set, and then the high-efficiency optimization is carried out by utilizing the improved LM method; based on the homography matrix, internal and external parameters of the camera can be obtained, and further optimization can be realized by utilizing maximum likelihood estimation; after the internal and external parameters are obtained, three distortion parameters and finite element distortion parameters of the camera can be respectively obtained by solving an over-determined equation and then further optimizing by using an improved LM method. Compared with the traditional calibration method, the method comprehensively considers various distortions, filters the pseudo characteristic points based on the proposed criterion, realizes the automatic positioning of the mark points based on the proposed automatic circle center numbering method, and optimizes based on the improved LM algorithm in the process of solving the calibration parameters, thereby obviously accelerating the convergence speed and improving the calibration precision. Based on the above discussion, the method provided by the invention can provide a solid theoretical and implementation basis for high-precision camera calibration in engineering applications such as machine vision, three-dimensional measurement and the like.

The above description is only for the purpose of creating a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can substitute or change the technical solution and the inventive concept of the present invention within the technical scope of the present invention.

Claims (6)

1. A high-precision camera calibration method based on a mixed distortion model comprises the following steps:

s1, establishing a camera distortion model;

s2, selecting a calibration pattern, extracting calibration characteristics and automatically numbering characteristic points;

and S3, calibrating the camera.

2. The method for calibrating a high-precision camera based on a hybrid distortion model as claimed in claim 1, wherein said establishing the distortion model of the camera comprises

1) Pinhole imaging model: four basic coordinate systems are established in the model, including: world coordinate system O_wX_wY_wZ_wCamera coordinate system O_cX_cY_cZ_cImage coordinate system xy, pixel coordinate system uv, camera coordinate system (X)_c,Y_c,Z_c) With the optical center O of the camera lens_cIs the origin of coordinates, X_c,Y_cThe axis being parallel to the image plane, Z_cThe axis is perpendicular to the image plane, and the coordinate of the intersection point on the image plane on the image coordinate system is (u)₀,v₀) The distance between the optical center of the camera lens and the principal point of the camera is the focal length f;

based on the coordinate system, a point P (X) in space can be obtained_w,Y_w,Z_w) Can be expressed in the camera coordinate system as:

$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=\left[\begin{array}{cc}R& T\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]$

whereinIs a 3 × 3 orthogonal rotation matrix,is a 3 × 1 translation matrix;

obtaining camera coordinates (X) based on pinhole imaging model_c,Y_c,Z_c,1)^TCan be expressed as:

$\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}{\mathrm{\ρ}}_x& {\mathrm{\α\ρ}}_x& u_0\\ 0& {\mathrm{\ρ}}_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\\ 1\end{array}\right]$

wherein, (u, v,1)^TAs image pixel coordinates, p_x,ρ_yThe number of pixels in the x, y directions in the image plane, respectively, (u)₀,v₀) Is the camera principal point coordinate, which is the pixel coordinate of the center of the image plane, α is the skew factor describing the vertical error of the pixel coordinate axis;

obtaining a relational expression of the image coordinates and the world coordinates according to the two expressions:

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_{\mathrm{\α}}& {\mathrm{\αf}}_{cx}& u_0\\ 0& f_{cy}& v_0\\ 0& 0& 1\end{array}\right]\[\begin{array}{cc}R& T\end{array}\]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=K_{c}E\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$

wherein,as a camera intrinsic parameter matrix, E ═ R T]Is an external parameter matrix;

2) constructing a camera distortion model: a mixed distortion model based on a rectangular finite element method is used, and is described as follows:

$\left\{\begin{array}{c}dx={\mathrm{dx}}_r+{\mathrm{dx}}_d+{\mathrm{dx}}_p+{\mathrm{dx}}_u\\ dy={\mathrm{dy}}_r+{\mathrm{dy}}_d+{\mathrm{dy}}_p+{\mathrm{dy}}_u\end{array}\right.$

wherein dx and dy are the deviation of the x coordinate and the y coordinate of the image point respectively;

$\left\{\begin{array}{c}d{x}_r=x_i\[K_{c1}{R}^2+K_{c2}{R}^4+K_{c3}{R}^6\]\\ d{y}_r=y_i\[K_{c1}{R}^2+K_{c2}{R}^4+K_{c3}{R}^6\]\end{array}\right.$

wherein,x_i,y_irespectively, the ideal distortion-free image plane coordinate, K, conforming to the pinhole imaging_c1,K_c2,K_c3Third order radial distortion coefficients;

$\left\{\begin{array}{c}{\mathrm{dx}}_d=P_{c1}(R^2+2x_i^2)+2P_{c2}{x}_i{y}_i\\ {\mathrm{dy}}_d=P_{c2}(R^2+2y_i^2)+2P_{c1}{x}_i{y}_i\end{array}\right.$

wherein, P_c1,P_c2Is a second order tangential distortion coefficient;

$\left\{\begin{array}{c}{\mathrm{dx}}_p=S_{c1}(x_i^2+y_i^2)=S_{c1}{R}^2\\ {\mathrm{dy}}_p=S_{c2}(x_i^2+y_i^2)=S_{c2}{R}^2\end{array}\right.$

wherein S is_c1,S_c2Second order thin prism distortion coefficients;

$\left\{\begin{array}{c}{\mathrm{dx}}_u=(1-\mathrm{\δ}x)(1-\mathrm{\δ}y){\mathrm{dx}}_{i,j}+\mathrm{\δ}x(1-\mathrm{\δ}y){\mathrm{dx}}_{i,j+1}+(1-\mathrm{\δ}x){\mathrm{\δydx}}_{i+1,j}+{\mathrm{\δx\δydx}}_{i+1,j+1}\\ {\mathrm{dy}}_u=(1-\mathrm{\δ}x)(1-\mathrm{\δ}y){\mathrm{dx}}_{i,j}+\mathrm{\δ}x(1-\mathrm{\δ}y){\mathrm{dx}}_{i,j+1}+(1-\mathrm{\δ}x){\mathrm{\δydx}}_{i+1,j}+{\mathrm{\δx\δydx}}_{i+1,j+1}\end{array}\right.$

wherein, x and y are the length ratio of the image point in the x and y directions of the rectangular grid respectively.

3. The method for calibrating a high-precision camera based on a hybrid distortion model of claim 1, wherein the calibration pattern in step S2 is selected by using an aluminum calibration plate, and using the aluminum calibration plate to perform parameter calibration on the camera, wherein the calibration plate has 9 × 11 circular calibration points, wherein the small circle has a diameter of 4mm, the large circle has a diameter of 8mm, and five large circles are used to identify the orientation of the calibration plate during the calibration process.

4. The method for calibrating a high-precision camera based on a mixed distortion model as claimed in claim 1, wherein the calibration feature extraction comprises the following steps: image acquisition, image graying, image filtering, image binarization, contour extraction, pseudo point filtering, ellipse fitting and circle center extraction;

the method comprises the steps of directly reading a color image into a gray image by using a function in an OpenCV function library, filtering the obtained gray image, realizing image filtering by using a cvSmooth function, setting a threshold value as 100 through experiments in image binarization, and extracting a contour by using an iteration-based improved Harris corner detection method, wherein the method is described as follows: considering any point p in the neighborhood of the corner q, # i (p) represents the image gradient vector at point p, then the gradient property yields:

▽I(p)(q-p)＝0

considering all the p points in the corner q neighborhood, an overdetermined equation set is formed, and therefore the Minimum Mean Square Error (MMSE) criterion is utilized for solving;

after extracting the contour, eliminating the pseudo feature points by using the following rule:

area criterion: and (3) determining the range of the area of the extracted feature object according to the system configuration:

S_min＜S＜S_max

roundness criterion: the proximity of the shape to the circle is measured by:

$C=\frac{T^2}{S}$

wherein T is the circumference of the circle,

error criteria: taking the algebraic distance from the edge contour points to the fitting ellipse as an error, and further filtering out circular feature points with larger errors by adopting the following criteria:

$\left\{\begin{array}{c}\mathrm{\&epsiv;}<{\mathrm{\&epsiv;}}_{\max }\\ \stackrel{\&OverBar;}{\mathrm{\&epsiv;}}<{\mathrm{\&epsiv;}}_{mean}\end{array}\right.$

whereinThe average error of the contour points is represented,_maxwhich is indicative of the maximum allowable error,_meanindicating an allowable average error;

after eliminating the pseudo characteristic points, carrying out contour ellipse fitting by adopting a weighted least square method to extract circle center coordinates with sub-pixel level precision.

5. The method for calibrating a high-precision camera based on a mixed distortion model according to claim 1, wherein the feature points in step S2 are automatically numbered: utilizing 5 solid circles on the calibration plate, wherein each solid circle corresponds to a determined invariable number, and carrying out the following steps:

(1) selecting four corner points corresponding to a certain rectangular area in space on an image by using a mouse, acquiring image coordinates of the four corner points and performing sub-pixel processing;

(2) and automatically counting the circles of the selected area, and generating a grid map according to the counting result.

6. The method for calibrating a high-precision camera based on a mixed distortion model according to claim 2, wherein the method for calibrating the camera is as follows:

1) homography matrix solving

Assuming that the template is located at the position of the world coordinate system Z being 0, obtaining the template through a pinhole model

sm＝HM

Where s is a scale factor, M and M are points on the image plane and the corresponding template, respectively, and H is K_c[R T]；

Based on the space coordinates of each corner point of the plane template, the estimated value of H is obtained by solving the following MMSE estimation,

$\hat{H}=\arg cmin\underset{i}{\&Sigma;}\left|\right|m_i-{\hat{m}}_i|{|}^2$

wherein, is the ith row vector of the H matrix;

solving by adopting an improved Leverberg-Marquardt method, solving by using a nonlinear optimization method requires a proper initial value for iteration, and obtaining by solving the following equation:

$\left[\begin{array}{ccc}{\hat{M}}^T& 0^T& -u{\hat{M}}^T\\ 0^T& {\hat{M}}^T& -u{\hat{M}}^T\end{array}\right]\left[\begin{array}{c}{h}_1^T\\ h_2^T\\ h_2^T\end{array}\right]=0$

given n > 6 points, forming an over-determined equation set, and solving through Singular Value Decomposition (SVD);

2) solution of internal and external parameters

Based on the property that the internal parameter matrix R is an orthogonal matrix, the internal and external parameters of the camera are obtained by solving the following over-determined equation set:

Vb＝0

the equation set is the superposition of the following equations obtained by observing n is more than or equal to 3 times:

$\left[\begin{array}{c}{v}_{12}^T\\ v_{11}^T-v_{22}^T\end{array}\right]b=0$

in the formula v_ij＝[h_1ih_1j,h_1ih_2j+h_2ih_1j,h_2ih_2j,h_3ih_1j+h_1ih_3j,h_3ih_2j+h_2ih_3j,h_3ih_3j]^T，b＝[B₁₁,B₁₂,B₂₂,B₁₃,B₂₃,B₃₃]^T，

Then, the solution is refined through maximum likelihood estimation to obtain a better optimized solution, and the likelihood estimation problem is expressed as follows:

$\underset{i=1}{\overset{n}{\&Sigma;}}\underset{j=1}{\overset{m_i}{\&Sigma;}}\left|\right|m_{ij}-\hat{m}(K_c,R_i,T_i,M_j)|{|}^2$

in the formula, m_ijThe j image point on the ith image is taken;as a spatial point M_jA projection point of the ith image;

3) solving distortion parameters;

4) solving distortion parameters of the finite element model;

5) the LM algorithm is improved.