RU2808083C1 - Photogrammetric calibration method for estimating the ratios of radial and tangential lens distortion and the matrix of internal parameters of the camera - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: used to estimate the ratios of radial and tangential lens distortion and the matrix of internal parameters of the camera. The claimed photogrammetric calibration method for estimating the ratios of radial and tangential lens distortion and the matrix of internal parameters of the camera is based on shooting a test site of nodal points located at the intersections of uniformly distributed N_o concentric circles and N_p radial directions. Based on the results of shooting in frames from the camera, the pixel coordinates of the nodal points of the site are estimated with subpixel accuracy and the ratios of the distortion mathematical model and the elements of the matrix of internal parameters of the camera are calculated from them. The test site is surveyed m≥2 times and before each subsequent exposure, it is rotated in roll by an angleδϕ≈Δϕ/m, where Δϕ =360°/Ν _p is the angular size of the site's radial sector. In this case, the site is centered along the lines of the frame center markers vertically and horizontally and is controlled by the point of intersection of the markers in the center of the circle of the smallest radius on the image from the camera, and the initial angular position is controlled by the horizontal position of the line of the frame center marker in the radial sectors of the site image, adjacent to its horizontal axis of symmetry corresponding to the radial branches of the site with the numbers “0-N_p/2”.

EFFECT: solution of an overdetermined system of equations of projective geometry for a priori known in the coordinate system of the calibration site of the spatial coordinates of its nodal points.

1 cl, 2 dwg

Description

The invention relates to computer technology and can be used to estimate the coefficients of radial and tangential distortion of the lens and the matrix of internal camera parameters.

There is a known method of photogrammetric calibration (Tyuflin Yu.S., Stepanyants D.G., Knyaz V.A., Zheltov S.Yu. Pre-calculation of the accuracy of determining the coordinates of object points in close-range photogrammetry // Geodesy and Cartography. 2004. No. 11. P. 29-32), including determining the coordinates of the points of a flat test object, photographing the test object, measuring the coordinates of the points of the test object in the images, calculating the elements of internal orientation and parameters of photogrammetric distortion. This method is based on photographing a test object twice with one camera from one point in space. After the first photograph, the test object is moved along the optical axis of the camera to a certain distance, and the amount of movement is recorded using a reading device with high accuracy. Next, the coordinates of the points of the test object are measured on the resulting images. Elements of internal orientation are calculated together with elements of external orientation of images from the solution of collinearity equations compiled for each image of a test object point. The parameters of photogrammetric distortion are determined by the residual discrepancies between the calculated and measured coordinates of the points of the test object, taking into account the found elements of the external and internal orientation of the images.

The disadvantage of this method is the need to use a reading device to measure the amount of movement of the test object with high accuracy.

A method for photogrammetric calibration of cameras is known from the prior art (patent RU 2308001, published 10.10.2007, IPC: G01C 11/02 (2006.01)). The method includes determining the coordinates of points of a flat test object, photographing the test object, measuring the coordinates of points of the test object in the images, calculating elements of internal orientation and parameters of photogrammetric distortion. In this case, a flat test object is created on the surface of the earth, photography is performed from an airplane from two photographing heights, the height difference is determined using an on-board satellite navigation system (GPS), when calculating internal orientation elements, the coordinates of the main point are taken equal to the coordinates of the central point of the frame, and the focal point the distance is calculated from points of the test object located symmetrically relative to the center of the frame.

The disadvantage of this method is its applicability only to cases of calibration of cameras installed on aircraft.

There is a known method for calibrating video sensors of a multispectral technical vision system (patent RU 2692970, published on June 28, 2019, IPC: G06T 7/80 (2017.01)), in which a test object is photographed from different angles, and special points are found on images of the test object taken from different angles , evaluate their coordinates with sub-pixel accuracy, evaluate matrices of internal parameters of cameras, evaluate vectors of distortion coefficients of camera lenses, and evaluate matrices of external parameters that determine the relative spatial position of cameras. Simultaneous calibration of several cameras - both visible and infrared ranges - is carried out using a test object with a calibration template with cells in the form of dark n-gons on a light field, while at the initialization stage of the calibration algorithm, the values of the parameters of the test calibration template are entered: cell size , number of cells horizontally and vertically. When n=4, the calibration template is a checkerboard field with angles known a priori in the template’s coordinate system, which makes it possible to use the classical Zhang technique for calibration (Zhang Z. A flexible new technique for camera calibration // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2000 Vol. 22, No. 11, pp. 1330-1334). At the same time, the use of universal test templates, contrasting in different spectral ranges (patent RU 2672466, published on November 14, 2018, IPC: G06T 7/80 (2017.01)), to implement the method, imposes a number of restrictions associated, for example, with the need for high-precision applying a polymer film to form a chessboard field.

The closest combination of characteristics to the method of compensating for lens distortion was chosen as a prototype (patent RU 2790055, published 02/14/2023, IPC: G01M 11/02 (2006.01)). In the prototype method for correcting geometric image distortions, the coefficients of radial and tangential distortion are determined using a calibration template in the form of a polygon of nodal points, ordered along concentric circles and with a constant angular step. At the same time, the Cartesian pixel coordinates of the polygon nodes are determined, they are converted into polar coordinates, the coefficients of the mathematical model of distortion are calculated, which is an expansion into series - a power series and two Fourier series, and the compensation of radial distortion and the periodic components of radial and tangential distortion are sequentially performed.

The disadvantages of the prototype method include the following.

1) The need to center the Cartesian coordinates of the nodal points to ensure that the center of the photomatrix coincides with the center of the polygon image.

2) To eliminate projective distortions of the calibration range image, which can be interpreted as lens distortion during calibration, it is necessary to ensure that the plane of the range is strictly normal in relation to the optical axis of the lens of the camera being calibrated. A similar remark concerns the angular adjustment of the calibration range for roll to ensure alignment of the zero line of the photomatrix with the image of the horizontal axis of the range.

For this reason, to implement the method, precision bench equipment is required that ensures mutual high-precision positioning of the camera and the calibration site.

3) As follows from the description of the prototype, compensation for distortion after calculating the calibration coefficients using this method will first require a transition to polar coordinates, i.e. calculating the inverse trigonometric function, then calculating trigonometric functions to compensate for distortion, and calculating trigonometric functions to return to Cartesian coordinates when rendering a distortion-compensated image. Thus, distortion compensation using the prototype method has greater computational complexity compared to the well-known approach based on on the numerical solution of the nonlinear equation for the Brown-Conradie distortion model (Brown D.C. Close-Range Camera Calibration // Photogrammetric Engineering. 1971. Vol. 37, No. 8. P. 855-866.) without calculating trigonometric functions: both direct and and reverse.

4) It is known that radial distortion is most pronounced at the corners of the frame. However, the radial-circular placement of nodal points of the calibration polygon of the prototype method leads to the fact that it is at the corners of the frame that their number per unit frame area is the smallest, which can be critical when estimating the coefficients characterizing radial distortion using numerical methods.

The technical problem solved by the creation of the claimed invention is to develop a calibration method that does not require:

- centering the image of the calibration area,

- high-precision angular positioning of the camera’s optical axis collinearly normal to the polygon plane,

- calculations of computationally intensive direct and inverse trigonometric functions in the procedure for compensating for radial and tangential distortion,

and also providing a high density of nodal points at the edges of the frame during the calibration process.

The technical result of the invention consists in solving an overdetermined system of equations of projective geometry for the spatial coordinates of its nodal points known a priori in the coordinate system of the calibration polygon.

The technical result is achieved by the fact that, by analogy with (Zhang Z. A flexible new technique for camera calibration // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2000. Vol. 22, No. 11. P. 1330-1334), calibration is not performed from a single shooting angle of the calibration radial-circular polygon of the prototype method, but from several, but not less than two. In this case, the mutual positioning of the calibrated camera and the calibration polygon does not require precise centering and the fulfillment of the condition of aligning the zero row of the photomatrix with the image of the horizontal axis of the polygon: only the full inclusion of the polygon in the field of view of the camera and its approximately symmetrical location relative to the central row and column of the camera frame are controlled. The symmetry of the location can be visually controlled by the operator by displaying service markers in the form of linear worlds (Fig. 1) on top of the central row and column of the frame image from the camera (RU 2703492, IPC F41G 3/06 (2006.01), G02B 23/12 (2006.01), G01B 11/27 (2006.01), publ. 10/17/2019). The criterion for an approximately symmetrical location of the calibration polygon can be that the center of the marker crosshairs falls inside the image of the polygon circle with the smallest radius (see Fig. 1).

The initial angular position of the roll calibration area can also be controlled using markers. In this case, the criterion for setting the range to its initial position with a small roll can be the horizontal marker entering the radial sectors “0-1”, “15-16”, “16-17” or “31-0”. Thus, the installation of the initial angular position is controlled by the horizontal frame center marker hitting the radial sectors of the polygon adjacent to its horizontal symmetry axis “0-N _p /2”, where N _p is the number of radial directions of the polygon (for the polygon from the prototype method N _p =32 and the axis of symmetry corresponds to the radial branches “0-16”).

The normal to the polygon plane during the calibration process may not be collinear to the axis of the camera being calibrated. When shooting a polygon, this will lead to geometric distortions characteristic of any projective camera.

After shooting the calibration range in its initial position, it is rotated along the roll clockwise or counterclockwise by an angle

where m≥2 is the total number of calibration positions, and Δϕ is the angular size of the radial sector of the polygon (for the prototype method Δϕ=360°/N _p =360°/32=11.25°), and the next frame is taken.

Due to the angular movement of the polygon, an increase in the total number of nodal points near the corners of the frame is achieved. In fig. Figure 2 illustrates as an example the change in the image of the calibration range for the second calibration frame with a total number of shooting angles m=3 and a roll rotation in the counterclockwise direction. Linear worlds allow the operator to visually control the fulfillment of condition (1).

For the case of automated calibration, for example, using devices for automated movement and rotation of the camera relative to the calibration range (patent RU 2645432, IPC: G01C 11/00 (2006.01), published 02.21.2018; patent RU 2749363, IPC: G06T 7/80 ( 2017.01), published 06/09/2021), it is possible to ensure the equality of the left and right parts of expression (1) and to abandon the stage of monitoring the angular position by roll using markers.

After shooting m frames of the polygon, in each of which its angular roll position changes compared to the previous one according to (1), with subpixel accuracy (Shortis MR, Clarke T.A., Short T. Comparison of some techniques for the subpixel location of discrete target images // Photonics for Industrial Applications. 1994. Vol. 2350. P. 239-250) 2D coordinates of nodal points X _jik are estimated in each k-th frame, k = 1, 2, …m, where j is the radius number concentric circle of the polygon, i - number of the radial branch; for fig. 1 and fig. 2 numbering j={1, 2,…7}, i={0, 1,…,31}.

Next, the correspondence of each nodal point of the polygon is established with spatial (3D) coordinates in the polygon coordinate system X _jik , where j is the number of the radius of the concentric circle of the polygon, i is the number of the radial branch, and its 2D projection onto the camera image plane x _jik . To find a one-to-one correspondence between the 3D coordinates of the nodal points of a polygon of an a priori known configuration and their 2D projections in the plane of the camera frame, the image of at least one of the nodal points of the polygon, selected as a reference, must differ from the others: for example, have a larger radius. In fig. 1 and 2 is point X ₇₀ , the radius of which is chosen to be twice the radius of the remaining nodal points of the polygon.

After establishing a correspondence between the 3D coordinates of the polygon nodes X _jik and their 2D projections x _jik for a set of m frames using a numerical method, for example, the Levenberg-Marquardt algorithm (More J. The Levenberg-Marquardt algorithm: implementation and theory // Numerical Analysis. Lecture Notes in Mathematics. 1978. Vol. 630. P. 105-116), a search is made for a pseudo-solution {K, k _d } of the system of equations of projective geometry, minimizing the square of the reprojection error (Zhang Z. A flexible new technique for camera calibration // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2000. Vol. 22, No. 11. P. 1330-1334; Hartley R., Zisserman A. Multiple view geometry in computer vision: 2nd edition. Cambridge: Cambridge University Press, 2003. 656 p):

taking into account the Brown-Conradie distortion model:

where N _o and N _p are the number of circles and radial directions of the calibration area, respectively, K is the matrix of internal camera parameters, k _d = [k ₁ , k ₂ , k ₃ , p ₁ , p ₂ ] ^T is the vector of distortion coefficients according to the Brown model -Conradi, in which k _t , t=1...3, are the radial distortion coefficients, p _t , t=1.2, are the tangential distortion coefficients, (x _jiknd , y _jiknd ). and (х _jikн , y _jikн ) - respectively normalized pixel coordinates of nodal points with and without distortion, R _k and t _k - respectively the rotation matrix and the vector of parallel translation (translation) of the coordinate system of the calibrated camera relative to the coordinate system of the calibration polygon in the k-th position calibration site, k=1…m. For the case of using a calibration template according to the prototype method N _o =7 and N _p =32 (see Fig. 1).

Since as a result of roll rotation the plane of location of the polygon's nodal points does not change, then the evaluation of the matrix of internal parameters assumes the acceptance of the following restrictions (Zhang Z., Matsushita Y., Ma Y. Camera calibration with lens distortion from low-rank textures // Proc. of Conf. on Computer Vision and Pattern Recognition CVPR'2011. - Colorado Springs, 2011. - P. 2321-2328):

1) equality of focal lengths ƒ _x and ƒ _y : ƒ _x =ƒ _y =ƒ;

2) the coordinates of the main point (u ₀ ,ν ₀ ) are known a priori: as a rule, they accept the hypothesis (u ₀ ,ν ₀ )=(0.5W,0.5H), where W and H are, respectively, the width and height of the camera frame in pixels.

These limitations are approximately valid for most cameras produced, and if the viewpoint is not degenerate, then calibration using polygon points lying in the same plane correctly restores the lens distortion parameters k _d and focal length ƒ. As shown in (Zhang Z., Matsushita Y., Ma Y. Camera calibration with lens distortion from low-rank textures // Proc. of Conf. on Computer Vision and Pattern Recognition CVPR'2011. - Colorado Springs, 2011. - P . 2321-2328), the accepted restrictions are not an obstacle to successful distortion compensation.

The pseudo-solution (2) found as a result of calibration, i.e. set of parameters {ƒ, k _d }, allows you to further compensate for the distortion of images captured by the camera by numerically solving (3) for each pixel of the frame (Hartley R., Zisserman A. Multiple view geometry in computer vision: 2nd edition. Cambridge: Cambridge University Press, 2003. 656 p). In this case, trigonometric functions are not calculated in the process of searching for a solution (3).

The search for a pseudo-solution (2) provides (Zhang Z. A flexible new technique for camera calibration // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2000. Vol. 22, No. 11. P. 1330-1334) attribution of projective geometric distortions caused by by setting the plane of the calibration area, in the general case, not perpendicular to the camera’s line of sight, to the parameters {R _k , t _k }, and not to the distortion coefficients k _d .

Claims (1)

A photogrammetric calibration method for estimating the coefficients of radial and tangential distortion of the lens and the matrix of internal parameters of the camera, in which a test area of nodal points located at the intersections of uniformly distributed N _o concentric circles and N _p radial directions is photographed, and the pixel coordinates of the nodal points of the polygon are assessed with subpixel accuracy, calculate the coefficients of the mathematical model of distortion and the elements of the matrix of internal parameters of the camera, characterized in that the test site is photographed m≥2 times, and before each subsequent exposure of the test site, it is rotated along the roll by an angle δϕ≈Δϕ/m, where Δϕ= 360°/N _p - the angular size of the radial sector of the polygon, while the centering of the calibration polygon is performed along the lines of the frame center markers vertically and horizontally and is controlled by the point of intersection of the markers hitting the center of the circle of the smallest radius in its image from the camera, and setting the initial angular position is controlled by the horizontal position of the frame center marker line in the radial sectors of the polygon image adjacent to its horizontal axis of symmetry, corresponding to the radial branches of the polygon with numbers “0-N _p /2”.

Images