CN117036498A - Binocular camera calibration method and device and electronic equipment - Google Patents
Binocular camera calibration method and device and electronic equipment Download PDFInfo
- Publication number
- CN117036498A CN117036498A CN202310986672.3A CN202310986672A CN117036498A CN 117036498 A CN117036498 A CN 117036498A CN 202310986672 A CN202310986672 A CN 202310986672A CN 117036498 A CN117036498 A CN 117036498A
- Authority
- CN
- China
- Prior art keywords
- matrix
- distortion
- equation
- basic matrix
- constructing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 239000011159 matrix material Substances 0.000 claims abstract description 261
- 238000005457 optimization Methods 0.000 claims abstract description 29
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 14
- 239000013598 vector Substances 0.000 claims description 60
- 238000003384 imaging method Methods 0.000 claims description 44
- 230000006870 function Effects 0.000 claims description 34
- 238000000354 decomposition reaction Methods 0.000 claims description 29
- 230000003287 optical effect Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The application provides a binocular camera calibration method and device and electronic equipment. According to the method, a distortion basic matrix is constructed through lens distortion information of the binocular camera to be calibrated and the basic matrix, and the distortion basic matrix is the quantity to be solved. And acquiring a first binocular image by using the binocular camera to be calibrated, and extracting the coordinates of the same name points to construct a first equation together with the distortion basic matrix equation. And calculating an estimated value for calibrating the distortion basic matrix of the binocular camera to be calibrated through a first equation so as to construct an objective function for calculating the optimal distortion basic matrix. And optimizing the objective function by using a least squares optimization algorithm to obtain an optimal distortion base matrix. The method is divided into two stages of preliminary estimation and optimization in the process of solving the optimal distortion basic matrix, and the problem that the nonlinear optimization method is difficult to obtain results is solved. Meanwhile, description of radial distortion parameters is added in the process of solving the optimal distortion basic matrix, and the calibration efficiency and accuracy of the binocular camera to be calibrated are improved.
Description
Technical Field
The present application relates to the field of machine vision technologies, and in particular, to a binocular camera calibration method and apparatus, and an electronic device.
Background
The binocular camera applied to the machine vision field needs to be calibrated before use so as to determine model parameters of an imaging model of the binocular camera, further more accurately establish a mapping relation between a camera image and a physical world, and improve the imaging accuracy of the binocular camera.
In combination with actual needs, part of binocular cameras adopt short-focal-length lenses with large lens visual angles and wide visual fields. Such binocular cameras can generate radial distortion when acquiring images, and thus can easily generate large errors during image processing. Therefore, when calibrating a binocular camera employing a short focal length lens, it is necessary to calibrate the radial distortion parameters to correct the image during image processing.
In the process of calibrating the binocular camera, the base matrix and the radial distortion of the binocular camera are required to be calibrated, so that the accuracy of the binocular camera to be calibrated is improved. When the nonlinear optimization method is used for calibrating the basic matrix, the calculation result is not easy to converge, and calibration is difficult. And when the calibration of the basic matrix and the calibration of the radial distortion are mutually independent, the calibration efficiency is lower.
Disclosure of Invention
The application provides a binocular camera calibration method and device and electronic equipment, and aims to solve the problem of low calibration efficiency when a base matrix and radial distortion of a binocular camera are calibrated step by step.
In a first aspect, the present application provides a binocular camera calibration method, including:
determining a distortion basic matrix equation according to lens distortion information and a basic matrix of the binocular camera to be calibrated;
acquiring a first binocular image by using the binocular camera to be calibrated, and extracting a plurality of pairs of homonymous point coordinates from the first binocular image;
constructing a first equation based on the basic matrix equation and a plurality of pairs of homonymous point coordinates to calculate an estimated value of the basic matrix;
constructing an objective function for calculating a distortion optimal basis matrix according to the first equation and the estimated value of the distortion basis matrix;
and optimizing the objective function by using a least squares optimization algorithm to obtain an optimal distortion base matrix.
In some possible embodiments, constructing a first equation based on the basis equation and a plurality of pairs of homonymous point coordinates includes:
constructing a first matrix according to the plurality of pairs of homonymous point coordinates, and constructing a first vector according to undetermined parameters used for representing lens distortion information and the basic matrix in the basic matrix equation;
and obtaining a first equation based on the first matrix and the first vector.
In some possible embodiments, constructing a first equation based on the distortion base matrix equation and a plurality of pairs of homonymous point coordinates to calculate an estimate of the distortion base matrix, further comprising:
performing a singular value decomposition on the first matrix, and extracting a first singular value from candidate singular values obtained from the singular value decomposition;
and calculating a first singular vector corresponding to the first singular value.
In some possible embodiments, the method further comprises:
constructing a second matrix based on the first singular vectors;
performing a singular value decomposition on the second matrix, and extracting a second singular value from candidate singular values resulting from the singular value decomposition.
In some possible embodiments, the method further comprises:
constructing a third matrix based on the second singular values;
and calculating a left zero space solution set and a right zero space solution set of the third matrix according to a matrix zero space algorithm.
In some possible embodiments, the method further comprises:
establishing an imaging plane coordinate system on an imaging plane of the binocular camera to be calibrated; the imaging plane coordinate system comprises a first imaging plane coordinate system corresponding to the left zero space solution set and a second imaging plane coordinate system corresponding to the right zero space solution set;
a first intersection of the left set of zero-space solutions and a first imaging plane coordinate system is calculated, and a second intersection of the right set of zero-space solutions and a second imaging plane coordinate system is calculated.
In some possible embodiments, the method further comprises:
determining a first optical axis on the first imaging plane coordinate system and a second optical axis on the second imaging plane coordinate system;
calculating a first shortest distance point between the first optical axis and a left zero space solution set, and calculating a second shortest distance point between the second optical axis and a right zero space solution set;
and calculating a first radial distortion parameter according to the first shortest distance point and the radial distortion vector, and calculating a second radial distortion parameter according to the second shortest distance point and the radial distortion vector.
In some possible embodiments, the method further comprises:
selecting a first coefficient to be determined and a second coefficient to be determined, and constructing a fourth matrix;
calculating a specific value of the first parameter to be determined according to the coordinates of the first intersection point and the coordinates of the second intersection point;
based on the first radial distortion parameter, the second radial distortion parameter and the first parameter to be determined, obtaining a linear equation set for constructing a second equation;
constructing a second equation consisting of a fifth matrix and a second vector according to the linear equation set; the fifth matrix comprises known amounts; the second vector includes an unknown quantity, and the second vector includes a second pending parameter.
In some possible embodiments, the method further comprises:
performing singular value decomposition on the fifth matrix to extract a fifth minimum singular value;
calculating a singular vector corresponding to the minimum singular value of the fifth matrix based on the fifth matrix and the fifth singular value to determine a specific value of a second undetermined parameter;
and determining an estimated value of the distortion base matrix based on the first radial distortion parameter, the second radial distortion parameter, the specific value of the first parameter to be determined and the specific value of the second parameter to be determined.
In a second aspect, the present application provides a binocular camera calibration apparatus comprising: the device comprises a preprocessing module, an acquisition module, an operation module and an optimization module;
the preprocessing module is used for constructing a distortion basic matrix equation according to lens distortion information of the binocular camera to be calibrated and the basic matrix;
the acquisition module is used for acquiring a first binocular image by using the binocular camera to be calibrated and extracting a plurality of pairs of homonymous point coordinates from the first binocular image;
the operation module constructs a first equation based on the basic matrix equation and a plurality of pairs of homonymous point coordinates so as to calculate an estimated value of the distorted basic matrix;
the operation module is also used for constructing an objective function for calculating the optimal distortion basic matrix according to the first equation and the estimated value of the distortion basic matrix;
the optimization module is used for optimizing the objective function by using a least square optimization algorithm to obtain an optimal distortion basic matrix.
In a third aspect, the application provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to perform a method of calibrating a binocular camera distortion base matrix as described in the above technical solution.
As can be seen from the above technical content, the present application provides a binocular camera calibration method, a binocular camera calibration device and an electronic apparatus. According to the method, a distortion basic matrix is constructed through lens distortion information and a basic matrix of the binocular camera to be calibrated to serve as a calibration basis. And acquiring a first binocular image by using the binocular camera to be calibrated, and extracting the coordinates of the same name points to construct a first equation together with the distortion basic matrix equation. And calculating an estimated value for calibrating the distortion basic matrix of the binocular camera to be calibrated through a first equation so as to construct an objective function for calculating the optimal distortion basic matrix. And optimizing the objective function by using a least squares optimization algorithm to obtain an optimal distortion base matrix. The method is divided into two stages of preliminary estimation and optimization in the process of solving the optimal distortion basic matrix, and the problem that the nonlinear optimization method is difficult to obtain results is solved. Meanwhile, description of radial distortion parameters is added in the process of solving the optimal distortion basic matrix, and the calibration efficiency and accuracy of the binocular camera to be calibrated are improved.
Drawings
In order to more clearly illustrate the technical solution of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic flow chart of calibrating a binocular camera distortion base matrix according to an embodiment of the present application;
FIG. 2 is an exploded view of a first embodiment of the present application;
FIG. 3 is a schematic diagram illustrating a second matrix decomposition according to an embodiment of the present application;
FIG. 4 is a schematic diagram of the relationship among imaging plane, optical axis and null space according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a fourth matrix decomposition provided by an embodiment of the present application;
fig. 6 is a schematic diagram of a fifth matrix decomposition according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The embodiments described in the examples below do not represent all embodiments consistent with the application. Merely exemplary of systems and methods consistent with aspects of the application as set forth in the claims.
Before the binocular camera is put into use, model parameters of an imaging model are required to be calibrated, so that a mapping relation between a camera image and the physical world is established more accurately, and the imaging accuracy of the binocular camera is improved. Because of the adoption of the short-focal-length lens, radial distortion can be generated in the operation process of the partial binocular camera, so that the accuracy of acquiring the image is reduced. The calibration of such binocular cameras therefore includes calibration of the basis matrix, as well as calibration of the radial distortion parameters, to improve their accuracy.
In the calibration process, the calibration of the basic matrix and the calibration of the radial distortion parameters are independent, and the calibration is performed simultaneously, so that the calibration efficiency is low. In addition, the nonlinear optimization method is directly used for solving the basic matrix, and parameters for describing the basic matrix are easily not obtained due to the characteristics of the algorithm and calibration data.
In view of the foregoing, as shown in fig. 1, some embodiments of the present application provide a binocular camera calibration method, including:
s100: determining a distortion basic matrix equation according to lens distortion information and a basic matrix of the binocular camera to be calibrated;
in some embodiments, the basis matrix equation for a binocular camera describes both the effect of lens distortion and binocular camera position on image acquisition accuracy. Therefore, the distortion basic matrix equation can be solved according to the lens distortion information and the configuration information of the binocular camera to serve as a basis for subsequent calibration.
S200: acquiring a first binocular image by using the binocular camera to be calibrated, and extracting a plurality of pairs of homonymous point coordinates from the first binocular image;
the number of the first binocular images is at least 1, and the homonymous point coordinates extracted from the first binocular images are at least 15 pairs for providing calibration data. It can be understood that when the data size of the calibration data is larger, the data size can be realized by increasing the acquisition number of the first binocular image and the extraction number of the homonymous point coordinates. Otherwise, the collection number of the first binocular image and the extraction number of the homonymous point coordinates can be reduced. It should be noted that, in some embodiments, calibration of the binocular camera may be achieved by acquiring a binocular image. Wherein 15 pairs of homonymous point coordinates can be used to construct a 4x4 matrix.
It should be noted that, when constructing a matrix according to the coordinates of the same name points, it is necessary to detect whether radial distortion exists in the binocular camera to be calibrated. In some embodiments, if the lens of the binocular camera is free of radial distortion, the binocular camera conforms to the small bore imaging model. With a group of homonymous point coordinates x u ,y u For example, F u Is equal to x u ,y u A corresponding basis matrix, and can be represented by n u Representing a straight line on the imaging plane of the binocular camera,the straight line is the projection of a straight line in three-dimensional space on the imaging plane of the camera. Based on the characteristics of binocular camera, the projection is located on which plane, which can be used separately>The left imaging plane and the right imaging plane are distinguished.
In other embodiments, if there is lens distortion in the binocular camera to be calibrated, taking a set of homonymous coordinates x, y as an example, F is a distortion base matrix corresponding to x, y, and the distortion base matrix contains a parameter λ describing radial distortion because there is lens distortion L ,λ R . Wherein x, y, x u ,y u Are used to represent the coordinate vectors of a pair of homonymous points in the left and right imaging planes, where x u ,y u The angle sign u of (2) is used for distinguishing the homonymous point coordinates without radial distortion phenomenon from the homonymous point coordinates with mirror image distortion phenomenon.
It can be understood that the embodiment of the application aims to establish an association relation based on a basic matrix without radial distortion phenomenon and a distortion basic matrix with radial distortion phenomenon so as to obtain an estimated value of the distortion basic matrix of the binocular camera to be calibrated with radial distortion phenomenon, and optimize the distortion basic matrix according to the estimated value so as to obtain an optimal distortion basic matrix, thereby completing the calibration of the binocular camera to be calibrated.
In order to introduce distortion parameters into the basic matrix, the embodiment of the application provides a mathematical model for describing radial distortion of a lens:
x u ＝δ -1 (x)＝(wx,wy,w 2 +λ(x 2 +y 2 )) T
wherein lambda is a distortion parameter introduced by the model, lambda L ,λ R Distortion parameters for representing two lenses of the binocular camera, respectively; w is a scaling factor, and x and y are the coordinates of the vector on a plane respectively; t is used to represent the transpose operation of the matrix; delta (x) is the distortion function, delta -1 (x) Is the inverse of the distortion function.
According to the positional relation equation of points and lines in space, when x u In straight line n u When above, it can be expressed as:
(n u ) T ·x u ＝0
at this time, the lens radial distortion model is substituted into the above equation, and it can be obtained:
the formula conforms to the general expression of a circle on a plane, and thus can be further put into:
a(x 2 +y 2 )+d·wx+e·wy+f·w 2 ＝0
thus, when there is a straight line in the three-dimensional space, after being projected onto the imaging surface of the binocular camera by the lens with distortion, the resulting image is a circle, which can be expressed as:
wherein,are four-dimensional vectors. According to->Can define a matrix:
i.e.Will->Substituted into the above formula to obtain +.>The homonymous point x is known without taking lens distortion into account u ,y u And camera base matrix F u The relationship between them corresponds to:
(y u ) T F u x u ＝0
wherein,is a 3x3 matrix. Will->Substitution (y) u ) T F u x u =0, the expression of the distortion base matrix containing distortion in four-dimensional space can be obtained:
the further adjustment is as follows:
wherein,the estimated value of the distortion basic matrix of the binocular camera to be calibrated, which takes lens distortion into consideration, is obtained. The 16 elements contained in the matrix can be used as the estimated values of the distortion basic matrix for subsequent optimization only when a certain constraint condition is met. Let the estimation value of the lifting matrix +.>The specific constraints are as follows:
based on the constraints, a special solution of the basis matrix can be obtained,it should be noted that, in the embodiment of the present application, the special solution vector D may be described by a radial distortion vector, and may be used to calculate a radial distortion parameter. The radial distortion vector is located at the optical axis +.>On (I)>
Based on the basic matrix equation and the homonymous point coordinates to be calibrated and collected, the method can solveThe values of 16 elements in the matrix are used for determining the estimated value of the distortion basic matrix. In some embodiments, a pair of images acquired by the binocular camera to be calibrated may be acquired using (I L ,I R ) Representing that at least 15 pairs of homonymous points (x k ,y k ) Wherein k=1, 2, …, K is not less than 15. At least 15 pairs of homonymous point coordinates can be constructed into a four-dimensional homogeneous coordinate point pair +.>The distortion basic matrix is aligned with the estimated value expression dimension of the distortion basic matrix, so that the subsequent calculation of the estimated value is facilitated.
S300: constructing a first equation based on the distortion basic matrix equation and a plurality of pairs of homonymous point coordinates to calculate an estimated value of the distortion basic matrix;
the first equation can be arranged into a form of the product of the first matrix and the first vector, singular value decomposition is gradually carried out on the first matrix, undetermined parameters contained in the first vector are solved, and finally an estimated value of the distortion basic matrix is obtained. In some embodiments, constructing a first equation based on the basis equation and a plurality of pairs of homonymous point coordinates comprises:
constructing a first matrix according to the plurality of pairs of homonymous point coordinates, and constructing a first vector according to undetermined parameters used for representing lens distortion information and a basic matrix in the distortion basic matrix equation;
and obtaining a first equation based on the first matrix and the first vector.
To four-dimensional homogeneous point pairValue substitution +.>Can be arranged into A.f 1 Form =0, where a is an m×n square matrix, m++15, n=15, the elements of the square matrix being based on +.>Value is obtained, f 1 Is a 15 x 1 column vector comprising the matrix +.>Of 15 pending parameters, i.e. f 1 ＝(c 11, ...,c 43 ) Here, agree about->It has been normalized that,i.e. c 44 ＝1。
The first equation may form a correspondence with the basis matrix containing the distortion information, and thus each element in the distortion basis matrix may be indirectly obtained by obtaining the first matrix and the first vector in the first equation. As shown in fig. 2, in some embodiments, constructing a first equation based on the distortion base matrix equation and the plurality of pairs of homonymous point coordinates to calculate an estimated value of the distortion base matrix, further includes:
performing a singular value decomposition on the first matrix, and extracting a first minimum singular value from candidate singular values obtained from the singular value decomposition;
and calculating a first singular vector corresponding to the first minimum singular value.
It will be appreciated that, in order to describe the correspondence between the singular values and the matrices, in the embodiment of the present application, the minimum singular value obtained by the first matrix decomposition is referred to as a first singular value, the singular value obtained by the second matrix decomposition is referred to as a second singular value, and the minimum singular value obtained by the fifth matrix decomposition is referred to as a fifth singular value.
Singular value decomposition is performed on the first matrix to obtain a 15×15 matrix W 1 And a 15 x 15 diagonal matrix Σ 1 . Wherein Σ is 1 The matrix A comprises 15 singular values corresponding to the matrix A, and the first singular value is the smallest singular value in the 15 singular values. The first singular vector f corresponding to the minimum singular value can also be determined based on the minimum singular value 1 . The first singular vector contains 15 elements and can be used to construct the second matrix. In the singular value decomposition process, each obtained singular value corresponds to the weight of the matrix corresponding to the singular value, so that the matrix corresponding to the smaller singular value has smaller weight for the image data, and therefore can be regarded as noise data, and the minimum singular value corresponds to the denoising process for the image data, so that the calibration precision can be improved.
As shown in FIG. 3, in some embodiments, a first singular vector f is found 1 Thereafter, the method further comprises:
constructing a second matrix based on the first singular vectors;
performing a singular value decomposition on the second matrix, and extracting a second singular value from candidate singular values resulting from the singular value decomposition.
Based on 15 elements contained in the first singular vector, a constant 1 is supplemented as the 16 th element, so that a new 4×4 matrix, namely a second matrix B, can be constructed. Singular value decomposition is performed on the second matrix B to obtain a 4x4 square matrix W 2 A 4x4 diagonal matrix Σ 2 Wherein Σ is 2 Including 4 singular values corresponding to matrix B. The largest two singular values are selected from the 4 singular values as the second singular value, and it is understood that the largest two singular values may be equal or larger of the 4 singular values. The singular value decomposition is performed on the second matrix, which is a dimension-reducing operation, so that the constraint condition of the calculation of the estimated value can be met when the estimated value of the basic matrix is calculated subsequently.
After extracting the second singular values, the matrix may be further established to solve for the estimated value of the base matrix according to the second singular values, and in some embodiments, further includes:
constructing a third matrix based on the second singular values and the singular matrix;
and calculating a left zero space solution set and a right zero space solution set of the third matrix according to a matrix zero space algorithm.
Selecting the largest two singular values from the 4 singular values as second singular values, and constructing a 4×4 diagonal matrix Σ 'by using the two second singular values and two zero elements' 2 Based on a diagonal matrix Σ' 2 A third matrix can be obtainedIt will be appreciated that the third matrix is of rank 2 and therefore contains vectors that also have two free dimensions.
The combined binocular camera has the characteristics of two types of lenses, the lenses of which correspond to 2 different zero spaces, which are represented by a left zero space and a right zero space, respectively, in the embodiment of the application. Wherein the left null spaceSatisfy equation->Is set of all vectors v, right zero space +.>Satisfy equation->Is defined as the set of all vectors v. It will be appreciated that vector v has two free dimensions, corresponding, left zero space +.>The end-point trajectories of all vectors in (a) form a straight line, and similarly, the right zero space is +.>Which also corresponds to a straight line in space.
It will be appreciated that the left null spaceRight zero space->May be collectively referred to as zero space->Zero space->Can be used to determine a pair of four-dimensional vectors (q, u), and the four-dimensional vectors (q, u) can be used to represent the zero space +.>Wherein q= (q 0 ,q 1 ,q 2 1) represents one direction in the null spaceQuantity (which may be any vector), u= (u) 0 ,u 1 ,u 2 1) a direction vector representing the straight line. Based on the vector representation of the null space, the left null space, the right null space and the imaging plane can be calculated>To obtain an estimated value of the base matrix from the intersection points. In some embodiments, the method further comprises:
establishing an imaging plane coordinate system on an imaging plane of the binocular camera to be calibrated; the imaging plane coordinate system comprises a first imaging plane coordinate system corresponding to the left zero space solution set and a second imaging plane coordinate system corresponding to the right zero space solution set;
a first intersection of the left set of zero-space solutions and a first imaging plane coordinate system is calculated, and a second intersection of the right set of zero-space solutions and a second imaging plane coordinate system is calculated.
Left zero spaceThe corresponding imaging plane can be denoted +.>Its intersection point is->Right zero space->The corresponding imaging plane can be denoted +.>Its intersection point is->According to the definition of poles->Is the right of the base matrixThe pole is arranged at the position of the pole,is the left pole of the base matrix. The left pole and the right pole can be used to find the radial distortion parameters in the base matrix.
In some embodiments, after calculating the left pole and the right pole, the method further comprises:
determining a first optical axis on the first imaging plane coordinate system and a second optical axis on the second imaging plane coordinate system;
calculating a first shortest distance point between the first optical axis and a left zero space solution set, and calculating a second shortest distance point between the second optical axis and a right zero space solution set;
and calculating a first radial distortion parameter according to the first shortest distance point and the radial distortion vector, and calculating a second radial distortion parameter according to the second shortest distance point and the radial distortion vector.
The imaging surfaces corresponding to the left zero space and the right zero space can respectively determine the optical axis and the straight line where the optical axis is located. And combining the radial distortion vectors obtained in the above steps to obtain the radial distortion parameters of the binocular camera to be calibrated.
As shown in fig. 4, taking the left zero space as an example, the method for calculating the right pole of the base matrix is described: the optical axis of the imaging plane can be calculated according to the equation of the position relationship between the straight lines in the spaceOn the straight line, and->Nearest coordinates:
bonding ofAnd radial distortion vector->Radial distortion parameters can be obtained:
similarly, another radial distortion parameter may be calculated, and will not be described in detail herein. The first radial distortion parameter λ is used hereinafter for the 2 radial distortion parameters mentioned in this embodiment L And a second radial distortion parameter lambda R Description.
After the first radial distortion parameter and the second radial distortion parameter are obtained through calculation, the radial distortion parameter needs to be substituted into a distortion basic matrix, so that the basic matrix F in ideal conditions (no radial distortion exists in a lens) needs to be combined u As shown in fig. 5, in some embodiments, the step of calculating the estimated value of the distortion base matrix further comprises:
selecting a first coefficient to be determined and a second coefficient to be determined, and constructing a fourth matrix, wherein the fourth matrix comprises a first parameter to be determined and a second parameter to be determined;
calculating a specific value of the first parameter to be determined according to the coordinates of the first intersection point and the coordinates of the second intersection point;
based on the first radial distortion parameter, the second radial distortion parameter and the first parameter to be determined, obtaining a linear equation set for constructing a second equation;
constructing a second equation consisting of a fifth matrix and a second vector according to the linear equation set; the fifth matrix comprises known amounts; the second vector includes an unknown quantity, and the second vector includes a second pending parameter.
Basis matrix F u Comprises 9 elements and is known toThus can pass through parametersChemical basic matrix F u In the way of parameterizing the estimate of the distortion base matrix +.>And further a specific estimated value is obtained. Wherein H is R To get lambda R Substituting the matrix obtained in the defined matrix H, a +.>To get lambda L And carrying into the matrix obtained in the defined matrix H.
For example, matrix F u In the form of a fourth matrix, which can be expressed as,
equation F can be derived from the expression of the fourth matrix u One particular solution for v=0 is ζ 1 ＝(α,β,-1) T Vector ζ according to the definition of the basis matrix 1 I.e. matrix F u Thus by the right pole of (2)The values of α, β can be known. Similarly, equation v T ·F u One particular solution for =0 is ζ 2 ＝(α′,β′,-1) T Vector xi 2 Is a matrix F u Left pole of (2), thus by->The values of α ', β' are known. By the calculation mode, the specific value of the first coefficient to be determined in the fourth matrix can be obtained.
On the basis of obtaining the specific values of the first radial distortion parameter, the second radial distortion parameter and the first coefficient to be determined, the fourth matrix and the radial distortion parameter can be substituted into the distortion basic matrixAnd in the matrix expression corresponding to the estimated value, obtaining an expression of the estimated value of the distortion basic matrix containing the undetermined coefficient and the radial distortion parameter. Wherein the first coefficient to be determined is a known coefficient and the second coefficient to be determined is an unknown coefficient. Further, an estimated value expression of the distortion base matrix containing the undetermined coefficient, and the homonymous point coordinate value may be substituted intoA set of linear equations is obtained. The linear equation set can be arranged as C.f 2 The form of =0, the second equation. Wherein C is a fifth matrix, f 2 Is a second vector, consisting of unknown second undetermined coefficients.
As shown in fig. 6, the singular value decomposition is continuously performed on the fifth matrix, that is, a specific value for obtaining the second undetermined coefficient in the second vector is obtained through the decomposition result, and the steps further include:
performing singular value decomposition on the fifth matrix to extract fifth singular values;
calculating eigenvectors of the fifth matrix based on the fifth matrix and the fifth singular values to determine specific values of the second undetermined parameter;
and determining an estimated value of the distortion base matrix based on the first radial distortion parameter, the second radial distortion parameter, the specific value of the first parameter to be determined and the specific value of the second parameter to be determined.
Singular value decomposition is performed on the fifth matrix to obtain a 4x4 square matrix W 3 A 4x4 diagonal matrix Σ 3 . Wherein Σ is 3 The method comprises the steps of containing 4 singular values corresponding to a matrix C, and selecting the smallest singular value as a fifth singular value. The singular vector f corresponding to the fifth singular value can be calculated based on the fifth singular value 3 Singular vector f 3 Comprising elements and a second vector f 2 And (3) comparing the contained elements to obtain the specific values of the second predetermined coefficients (a, b, c and d).
After obtaining the specific value of the second undetermined coefficient, the second undetermined coefficient is used for parameterizing the estimated value of the distortion basic matrixIs defined by the parameters: first radial distortion parameter lambda L Second radial distortion parameter lambda R The first coefficient to be determined (α, β, α ', β'), the second coefficient to be determined (a, b, c, d) are all known, and thus the known parameters are substituted into F u The expression of (2) can be found with respect to F u Further substituting the parameterized concrete matrix into +.>An estimated value of the distortion basic matrix can be obtained>
S400: constructing an objective function for calculating a distortion optimal basis matrix according to the first equation and the estimated value of the distortion basis matrix;
constructing an objective function for calculating an optimal distortion base matrix according to the first equation and the estimated value of the distortion base matrix, wherein the objective function comprises the following steps:
collecting a second binocular image by using the binocular camera to be calibrated, and extracting a plurality of pairs of homonymous point coordinates from the second binocular image;
constructing a first matrix in the objective function based on a plurality of pairs of homonymous point coordinates extracted from the second binocular image to obtain the objective function; and the initial value of the distortion basic matrix in the objective function is the estimated value of the distortion basic matrix.
From the first equation, and the basis matrix, a reprojection error function may be defined: h (F) =f T ·A1 T ·A1·F。
It should be noted that, the second binocular image and the first binocular image are two different images acquired, and the extracted coordinates of the plurality of pairs of homonymous points are also different. The first binocular image is used to construct a first equation that calculates the distorted base matrix estimate, while the second binocular image is used to construct a matrix in the re-projection error function. The number of second binocular images is at least one pair, and the number of homonymous point coordinates extracted from the second binocular images is at least 15 pairs.
After extracting enough homonymous point coordinates from the second binocular image, a matrix A1 can be constructed and a reprojection error function is formed with the estimated value of the base matrix. It can be understood that the initial value of the reprojection error function in the embodiment of the present application is an estimated value of the base matrix, so that the base matrix can be optimized to obtain an optimal base matrix by using the estimated value as the initial value.
S500: and optimizing the objective function by using a least squares optimization algorithm to obtain an optimal distortion base matrix.
In some embodiments, the objective function, i.e., the re-projection error function, is optimized using a least squares method. In the optimization process, the undetermined parameters contained in the distortion basic matrix are iterated to realize the optimization of the distortion basic matrix. So that the re-projection error value is minimized in the sense of the A1 matrix. The part of optimizing the reprojection error function by using the least square method is not the key content of the description of the present application, and therefore, will not be described in detail. The embodiment of the application aims to complete the calibration of radial distortion parameters of a lens of a binocular camera to be calibrated and the calibration of a basic matrix in one calibration process. The basic matrix is parameterized, the radial distortion parameter is used as one of the parameters of the distortion basic matrix, and the least square optimization algorithm is used for iterative optimization on the basis of taking the estimated value of the distortion basic matrix as an optimization initial value, so that the radial distortion parameter and the basic matrix are calibrated at the same time, and the calibration efficiency is improved.
Some embodiments of the present application further provide a binocular camera calibration apparatus, including: the device comprises a preprocessing module, an acquisition module, an operation module and an optimization module;
the preprocessing module is used for constructing a distortion basic matrix equation according to lens distortion information of the binocular camera to be calibrated and the basic matrix;
the acquisition module is used for acquiring a first binocular image by using the binocular camera to be calibrated and extracting a plurality of pairs of homonymous point coordinates from the first binocular image;
the operation module constructs a first equation based on the distortion basic matrix equation and a plurality of pairs of homonymous point coordinates so as to calculate an estimated value of the distortion basic matrix;
the operation module is also used for constructing an objective function for calculating the optimal distortion basic matrix according to the first equation and the estimated value of the distortion basic matrix;
the optimization module is used for optimizing the objective function by using a least square optimization algorithm so as to obtain an optimal basic matrix.
The embodiment of the application also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to execute the binocular camera calibration method as described in the technical content.
The technical content can be that the application provides a binocular camera calibration method, a binocular camera calibration device and electronic equipment. According to the method, a distortion basic matrix equation is constructed through lens distortion information and a basic matrix of the binocular camera to be calibrated to serve as a calibration basis. And acquiring a first binocular image by using the binocular camera to be calibrated, and extracting the coordinates of the same name points to construct a first equation together with the distortion basic matrix equation. And calculating an estimated value for calibrating the distortion basic matrix of the binocular camera to be calibrated through a first equation so as to construct an objective function for calculating the optimal distortion basic matrix. And optimizing the objective function by using a least squares optimization algorithm to obtain an optimal distortion base matrix. The method is divided into two stages of preliminary estimation and optimization in the process of solving the optimal distortion basic matrix, and the problem that the nonlinear optimization method is difficult to obtain results is solved. Meanwhile, the description of radial distortion parameters is added in the process of solving the optimal distortion basic matrix, so that the calibration efficiency and accuracy of the binocular camera to be calibrated are improved.
The above-provided detailed description is merely a few examples under the general inventive concept and does not limit the scope of the present application. Any other embodiments which are extended according to the solution of the application without inventive effort fall within the scope of protection of the application for a person skilled in the art.
Claims (12)
1. A binocular camera calibration method, comprising:
constructing a distortion basic matrix according to lens distortion information and the basic matrix of the binocular camera to be calibrated;
acquiring a first binocular image by using the binocular camera to be calibrated, and extracting a plurality of pairs of homonymous point coordinates from the first binocular image;
constructing a first equation based on the distorted basic matrix equation and a plurality of pairs of homonymous point coordinates to calculate an estimated value of the basic matrix;
constructing an objective function for calculating an optimal distortion basic matrix according to the first equation and the estimated value of the distortion basic matrix;
and optimizing the objective function by using a least squares optimization algorithm to obtain an optimal distortion base matrix.
2. The method of claim 1, wherein constructing a first equation based on the basis equation and a plurality of pairs of homonymous point coordinates comprises:
constructing a first matrix according to the plurality of pairs of homonymous point coordinates, and constructing a first vector according to undetermined parameters used for representing lens distortion information and a basic matrix in the distortion basic matrix equation;
and obtaining a first equation based on the first matrix and the first vector.
3. The method of claim 2, wherein constructing a first equation based on the distortion base matrix equation and pairs of homonymous point coordinates to calculate an estimate of the distortion base matrix, further comprising:
performing a singular value decomposition on the first matrix, and extracting a first singular value from candidate singular values obtained from the singular value decomposition;
and calculating a first singular vector corresponding to the first singular value.
4. A method according to claim 3, further comprising:
constructing a second matrix based on the first singular vectors;
performing a singular value decomposition on the second matrix, and extracting a second singular value from candidate singular values resulting from the singular value decomposition.
5. The method as recited in claim 4, further comprising:
constructing a third matrix based on the second singular values;
and calculating a left zero space solution set and a right zero space solution set of the third matrix according to a matrix zero space algorithm.
6. The method as recited in claim 5, further comprising:
establishing an imaging plane coordinate system on an imaging plane of the binocular camera to be calibrated; the imaging plane coordinate system comprises a first imaging plane coordinate system corresponding to the left zero space solution set and a second imaging plane coordinate system corresponding to the right zero space solution set;
a first intersection of the left set of zero-space solutions and a first imaging plane coordinate system is calculated, and a second intersection of the right set of zero-space solutions and a second imaging plane coordinate system is calculated.
7. The method as recited in claim 6, further comprising:
determining a first optical axis on the first imaging plane coordinate system and a second optical axis on the second imaging plane coordinate system;
calculating a first shortest distance point between the first optical axis and a left zero space solution set, and calculating a second shortest distance point between the second optical axis and a right zero space solution set;
and calculating a first radial distortion parameter according to the first shortest distance point and the radial distortion vector, and calculating a second radial distortion parameter according to the second shortest distance point and the radial distortion vector.
8. The method as recited in claim 7, further comprising:
selecting a first coefficient to be determined and a second coefficient to be determined, and constructing a fourth matrix;
calculating a specific value of the first parameter to be determined according to the coordinates of the first intersection point and the coordinates of the second intersection point;
based on the first radial distortion parameter, the second radial distortion parameter and the first parameter to be determined, obtaining a linear equation set for constructing a second equation;
constructing a second equation consisting of a fifth matrix and a second vector according to the linear equation set; the fifth matrix comprises known amounts; the second vector includes an unknown quantity, and the second vector includes a second pending parameter.
9. The method as recited in claim 8, further comprising:
performing singular value decomposition on the fifth matrix to extract fifth singular values;
calculating eigenvectors of the fifth matrix based on the fifth matrix and the fifth singular values to determine specific values of the second undetermined parameter;
and determining an estimated value of the distortion base matrix based on the first radial distortion parameter, the second radial distortion parameter, the specific value of the first parameter to be determined and the specific value of the second parameter to be determined.
10. The method of claim 9, wherein constructing an objective function for computing an optimal distortion base matrix based on the first equation and the estimate of the distortion base matrix comprises:
collecting a second binocular image by using the binocular camera to be calibrated, and extracting a plurality of pairs of homonymous point coordinates from the second binocular image;
constructing a first matrix in the objective function based on a plurality of pairs of homonymous point coordinates extracted from the second binocular image to obtain the objective function; and the initial value of the distortion basic matrix in the objective function is the estimated value of the distortion basic matrix.
11. A binocular camera calibration apparatus, comprising: the device comprises a preprocessing module, an acquisition module, an operation module and an optimization module;
the preprocessing module is used for determining a distortion basic matrix equation according to lens distortion information and a basic matrix of the binocular camera to be calibrated;
the acquisition module is used for acquiring a first binocular image by using the binocular camera to be calibrated and extracting a plurality of pairs of homonymous point coordinates from the first binocular image;
the operation module constructs a first equation based on the basic matrix equation and a plurality of pairs of homonymous point coordinates so as to calculate an estimated value of the distorted basic matrix;
the operation module is also used for constructing an objective function for calculating the optimal distortion basic matrix according to the first equation and the estimated value of the distortion basic matrix;
the optimization module is used for optimizing the objective function by using a least square optimization algorithm to obtain an optimal distortion basic matrix.
12. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program to perform the binocular camera calibration method of any of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310986672.3A CN117036498A (en) | 2023-08-07 | 2023-08-07 | Binocular camera calibration method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310986672.3A CN117036498A (en) | 2023-08-07 | 2023-08-07 | Binocular camera calibration method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117036498A true CN117036498A (en) | 2023-11-10 |
Family
ID=88601584
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310986672.3A Pending CN117036498A (en) | 2023-08-07 | 2023-08-07 | Binocular camera calibration method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117036498A (en) |
-
2023
- 2023-08-07 CN CN202310986672.3A patent/CN117036498A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106558080B (en) | Monocular camera external parameter online calibration method | |
| CN108510551B (en) | Method and system for calibrating camera parameters under long-distance large-field-of-view condition | |
| Hartley | In defense of the eight-point algorithm | |
| Zhang | Determining the epipolar geometry and its uncertainty: A review | |
| Zhang et al. | Robust and efficient pose estimation from line correspondences | |
| KR100855657B1 (en) | System for estimating self-position of the mobile robot using monocular zoom-camara and method therefor | |
| CN110264528B (en) | Rapid self-calibration method for binocular camera with fish-eye lens | |
| CN112435193B (en) | Method and device for denoising point cloud data, storage medium and electronic equipment | |
| KR100951309B1 (en) | New Calibration Method of Multi-view Camera for a Optical Motion Capture System | |
| CN112465877B (en) | Kalman filtering visual tracking stabilization method based on motion state estimation | |
| Hajder et al. | Relative planar motion for vehicle-mounted cameras from a single affine correspondence | |
| Zheng et al. | Minimal solvers for 3d geometry from satellite imagery | |
| Perdigoto et al. | Calibration of mirror position and extrinsic parameters in axial non-central catadioptric systems | |
| CN110838146A (en) | Homonymy point matching method, system, device and medium for coplanar cross-ratio constraint | |
| Stachel | Descriptive geometry meets computer vision–the geometry of two images | |
| CN111127560B (en) | Calibration method and system for three-dimensional reconstruction binocular vision system | |
| CN111553954B (en) | Online luminosity calibration method based on direct method monocular SLAM | |
| Jia et al. | Low-rank matrix fitting based on subspace perturbation analysis with applications to structure from motion | |
| CN108416811B (en) | Camera self-calibration method and device | |
| CN117036498A (en) | Binocular camera calibration method and device and electronic equipment | |
| Bartoli | On the non-linear optimization of projective motion using minimal parameters | |
| CN115018922A (en) | Distortion parameter calibration method, electronic device and computer readable storage medium | |
| Georgiev et al. | A fast and accurate re-calibration technique for misaligned stereo cameras | |
| CN113048985A (en) | Camera relative motion estimation method under known relative rotation angle condition | |
| Fitzgibbon et al. | Learning priors for calibrating families of stereo cameras |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |