CN109859272B - Automatic focusing binocular camera calibration method and device - Google Patents
Automatic focusing binocular camera calibration method and device Download PDFInfo
- Publication number
- CN109859272B CN109859272B CN201811550433.9A CN201811550433A CN109859272B CN 109859272 B CN109859272 B CN 109859272B CN 201811550433 A CN201811550433 A CN 201811550433A CN 109859272 B CN109859272 B CN 109859272B
- Authority
- CN
- China
- Prior art keywords
- calibration
- camera
- parameters
- distance
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an automatic focusing binocular camera calibration method and device and a depth calculation method, wherein the calibration method comprises the following steps: a1, acquiring calibration template images at corresponding distances shot at a plurality of calibration distances, wherein each calibration distance corresponds to a single Zhang Biaoding template image; the calibration template is provided with a calibration pattern with known size; a2, analyzing and processing a single Zhang Biaoding template image corresponding to each calibration distance, and calculating to obtain a group of binocular camera calibration parameters corresponding to the distance; and corresponding to the plurality of calibration distances, and obtaining the calibration parameters of the plurality of groups of binocular cameras in total. The invention can be applied to camera (or camera) calibration with automatic focusing function, and has the advantages of simple and efficient method. In addition, camera calibration at a plurality of distances is provided, so that the difference of calibration parameters caused by lens position change can be better compensated, and the subsequent depth calculation is more accurate.
Description
Technical Field
The invention relates to the fields of computer vision, optical measurement and camera manufacturing, in particular to an automatic focusing binocular camera calibration method and device.
Background
The automatic focusing refers to a function of automatically completing focusing on a shot subject and making an image clear by an electronic and mechanical device built in a camera. Due to the characteristics of accurate focusing and convenient operation, cameras with automatic focusing function currently have wide application in more and more industries, including the fields of smart phones, unmanned aerial vehicles, video monitoring and the like.
Camera applications, on the other hand, are moving from 2D into the 3D era based on binocular, multi-camera. For example, in the mobile phone industry, an optical zoom function is realized through the combination of a wide-angle camera and a long-focus camera, or image quality is improved through the combination of a black-white camera and a color camera, or an infrared structured light camera module is used for capturing the 3D point cloud of a human face, so that the human face recognition is realized. In the unmanned aerial vehicle industry, 3D environment perception is realized through a binocular camera to realize obstacle avoidance of the unmanned aerial vehicle in flight. In the automobile industry, distance measurement and 3D environment sensing are completed through a binocular camera to assist driving and support automatic driving.
Calibration of the camera is key to achieving the binocular camera 3D application. In camera-based measurement and vision applications, the relationship between the three-dimensional position of an object point in space and its corresponding two-dimensional pixel point position in an image is mathematically described by a geometric projection model. The parameters of the model are generally obtained by photographing, image processing and calculating a calibration pattern (such as a solid circular array or a black-and-white checkerboard) with known dimensions, and the process of determining the parameters of the projection model of the camera is called calibration. These parameters include: internal parameters refer to a principal point position c during imaging by a camera x 、c y Focal length f x 、f y Such parameters are only relevant to the camera itself; the external parameters refer to the position of the camera in space, and generally refer to a rotation vector R and a translation vector T of the camera in a certain reference coordinate system; distortion parameter, which refers to the deviation between the actual corresponding pixel position of the object point in the image and the theoretical projection point calculated based on the imaging model during the photographing process of the camera, is generally defined by a radial distortion parameter k 1 、k 2 、k 3 Tangential distortion parameter p 1 、p 2 To describe.
There are two limitations to the current calibration solutions common to camera mass production lines. First, because the calibration algorithm used requires taking images of multiple calibration templates from different angles, either rotating the 2D planar calibration template on the current production line or using a calibration stereo template that is stitched by multiple planar calibration templates (with fixed angles to each other). The device for rotating the calibration template is complex in mechanism, and the measurement time is prolonged by rotating, so that the yield per unit time is reduced, and the cost is increased. Besides the cost increase, the three-dimensional template spliced by the plurality of plane calibration templates is required to be accurately fixed at a certain angle or the three-dimensional template is required to be measured with high precision before use.
Cameras with auto-focus function typically have their lenses fixed to an electronic and mechanical motion mechanism, such as a VCM motor (voice coil motor). The lens is pushed back and forth along the optical axis by a motion mechanism in the lens barrel, and the position is changed. The distance from the lens to the surface of the imaging chip is changed (namely, the image distance is changed), and the depth of field of the camera for focusing is also changed, so that the focusing function aiming at different depths of field can be realized. Limitations for this function: for each depth of field of the camera with automatic focusing, the lens has a corresponding position in the lens barrel, and the calibration parameter values of different positions are different. Theoretically, the camera needs to be recalibrated at each lens position, i.e. for each depth of field. The current calibration scheme does not consider the calibration parameter change caused by the lens position change in the lens barrel.
Disclosure of Invention
The invention aims to provide an automatic focusing binocular camera calibration method and device and a depth calculation method, which are simple, efficient and accurate.
For this purpose, the automatic focusing binocular camera calibration method of the invention comprises the following steps: a1, acquiring calibration template images at corresponding distances shot at a plurality of calibration distances, wherein each calibration distance corresponds to a single Zhang Biaoding template image; the calibration template is provided with a calibration pattern with known size; a2, analyzing and processing a single Zhang Biaoding template image corresponding to each calibration distance, and calculating to obtain a group of binocular camera calibration parameters corresponding to the distance; a3, corresponding to a plurality of calibration distances, and obtaining calibration parameters of a plurality of groups of binocular cameras in total.
In some embodiments, the present invention further includes the following technical features:
in the step A1, the calibration pattern is a solid circle array, the solid circle array consists of C rows and L columns of solid circles, and C and L are natural numbers; each solid circle has the same size and the same radius, and the circle center distances in the horizontal direction and the vertical direction are the same; the calibration patterns of each distance are consistent and consist of C multiplied by L solid circles; the circle radius and the circle center distance of the calibration pattern of each distance are in direct proportion to the distance from the binocular camera to the calibration template.
In the step Al, when the calibration template image is shot, the optical axes of the left camera and the right camera are perpendicular to the calibration template, so that the calibration image is focused and no pattern deformation caused by shooting angles exists.
In step A2, analyzing and processing a single Zhang Biaoding template image corresponding to each calibration distance specifically includes the following steps: s2, extracting the pixel position of the solid circle center from the calibration template image shot by the left camera and the right camera through an image processing algorithm; s3, calculating the focal length of each lens of the binocular camera based on the calibration template image, and acquiring initial values of internal parameters, external parameters and distortion parameters of each camera; s4, optimizing internal parameters, external parameters and distortion parameters of the binocular camera based on a Levenberg-Marquardt optimization algorithm to obtain accurate calibration parameters of the binocular camera; s5, calculating the relative position parameter of the binocular camera when the distance is calculated.
In step S2, a solid circle region is detected based on the image gray value, and homogeneous coordinates [ x y 1] of the center of the circle region centroid as the center of the circle are obtained.
In step S3, two internal parameters, the focal length f, are solved by the following set of equations x And f y Initial value of:
h 11 h 12 ·B 11 +(h 31 h 12 +h 11 h 32 )·B 13 +h 21 h 22 ·B 22 +(h 31 h 22 +h 21 h 32 )·
B 23 +h 31 h 32 ·B 33 ＝0
wherein h is ij Homography matrix H matrix elements, which are the ith row and jth column (i, j=1, 2, 3), are known;
B ij determined by the following formula:
in the above formula, K is an internal parameter of the camera, which is defined as
c x And c y Is the principal point of the image, and the principal point c is preset x And c y The initial value of (2) is the center of the image.
In step S4, during optimization, the sum of squares of the re-projection errors of all solid circles centers on the standard template is minimized, namely:
wherein M is the position of the center of a solid circle on the calibration template, M is the position of the center point corresponding to the actual pixel in the image,
the position of the pixel calculated as the position of the heavy projection, i.e. the centre of the solid circle after the central projection, shifted by the distortion of the image (+)>
Is->
Contains distortion offset), K is the internal parameters of the camera, the rotation vector R and the translation vector T are the external parameters of the camera, k= [ K ] 1 k 2 k 3 ]As radial distortion parameter, p= [ p ] 1 p 2 ]Is a tangential distortion parameter.
In step S5, the relative positions of the two cameras are as follows
Wherein R is l 、T l Is the spatial position of the left camera,R r 、T r the spatial position of the right camera;
if a plurality of cameras are provided, calculation is performed between every two cameras.
The invention also comprises an automatic focusing binocular camera calibration device which is characterized by adopting the calibration method.
The invention also comprises a depth measurement method, which is characterized in that the calibration method is adopted to obtain calibration parameters of a plurality of groups of binocular cameras corresponding to a plurality of calibration distances, and when in depth test, a group of binocular camera calibration parameters corresponding to the calibration distance closest to the lens position in the current image shooting process are selected to calculate the depth.
The invention can be applied to camera (or camera) calibration with automatic focusing function, and because only one calibration template image is adopted for parameter calibration, compared with the existing general calibration method, the invention has the advantages of simple method and high efficiency. In addition, the device provides camera calibration at a plurality of distances, so that the difference of calibration parameters caused by lens position change can be better compensated, and the subsequent depth calculation is more accurate.
Drawings
Fig. 1 is a block diagram of a binocular camera calibration apparatus according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of a calibration pattern of a calibration template according to an embodiment of the present invention.
FIG. 3 is a schematic diagram of the steps in the process of calibrating a binocular camera module by the calibration apparatus according to the embodiment of the present invention.
Fig. 4 is a schematic flow chart of a calibration method of the binocular camera module in the embodiment of the invention at each distance calibration.
Fig. 5 is a schematic diagram illustrating calculation of depth information by taking a point P on the object surface of a scene according to an embodiment of the invention as an example.
Detailed Description
The calibration method and device of the binocular camera with the automatic focusing function are illustrated by specific embodiments. When the method is adopted for calibration, the production line of the binocular camera is required to be provided with N distances (N is an integer and can beProperly selecting the requirements of the unit time output of the production line, the size requirement of the production line equipment and the like according to the requirements of the subsequent binocular camera application; four distances are taken as examples for illustration, 0.5,1.0,1.5,2.0 meters are respectively used for the application, and the application is the same as the following), and a 2D plane calibration template is respectively placed; each calibration distance only captures an image of the calibration template at that distance, and the calibration template does not need to be rotated: the preset principal point c of the invention x And c y The initial value of (a) is the center of the image, so that f can be obtained by one image x And f y The initial values of the two unknowns do not need to rotate, and the step of coupling and assembling the optical lens and the imaging chip requires that the main point of the image does not exceed a certain pixel range from the center point of the imaging chip in the actual production process of the camera, so that the use effect of the imaging chip in the actual production is not influenced by the presetting; analyzing and processing a single Zhang Biaoding template image shot at the distance, and calculating to obtain a binocular camera calibration parameter corresponding to the distance; corresponding to a plurality of (four) calibration distances, obtaining calibration parameters of a plurality of groups (four groups) of binocular cameras in total; in depth testing, a set of calibration parameters nearest to the lens position at the current shot is selected to calculate depth.
Embodiment one:
FIG. 1 is a block diagram of a calibration device on a binocular camera production line. The calibration distance can be properly selected according to the requirements of precision, productivity, size and the like, and here, taking four distances as examples, a 2D plane calibration template is placed at each of the four distances: 0.5,1.0,1.5,2.0 meters.
The calibration template is shown in fig. 2. The calibration scheme adopts a solid circle array as a calibration template pattern, wherein the solid circle array consists of C rows and L columns of solid circles, and C and L are natural numbers. The solid circles are identical in size and radius, and the circle center distances in the horizontal direction and the vertical direction are identical. The calibration patterns of each distance are consistent and consist of C multiplied by L solid circles. The circle radius and the circle center distance of the calibration pattern of each distance are in direct proportion to the distance from the binocular camera to the calibration template.
The middle ring in fig. 1 represents a rotating mechanism, and the binocular camera module is fixed on the rotating mechanism and sequentially rotates to a position opposite to the calibration template, i.e. the optical axis of the binocular camera is perpendicular to the calibration template. After the camera reaches the position, the rotating mechanism stops moving, the binocular camera static shooting is performed, and the calibration algorithm calculates camera calibration parameters. Simultaneously, the rotating structure starts to act, and the camera rotates to the next target position.
The steps of calibrating the binocular camera module by the calibrating device disclosed by the invention are as follows, as shown in fig. 3:
the first step: feeding a module to be tested;
and a second step of: rotating to 1.5m, photographing a 1.5m calibration template, and calculating a 1.5m calibration parameter by a calibration program;
and a third step of: rotating to 0.5m, photographing a 0.5m calibration template, and calculating a calibration parameter at 0.5m by a calibration program;
fourth step: rotating to the position of 2.0m, photographing a 2.0m calibration template, and calculating a calibration parameter at the position of 2.0m by a calibration program;
fifth step: rotating to 1.0m, photographing a 1.0m calibration template, and calculating a calibration parameter at 1.0m by a calibration program;
sixth step: and blanking the module to be tested.
The flow of the calibration method is as follows when each distance is calibrated, as shown in fig. 4:
s1, acquiring an image of a fixed template for the distance time scale;
s2, extracting the pixel position of the solid circle center from the calibration template image shot by the left camera and the right camera by an image processing algorithm;
s3, calculating the focal length of the binocular camera lens based on the calibration image, and acquiring initial values of internal parameters, external parameters and distortion parameters of the camera;
s4, optimizing internal parameters, external parameters and distortion parameters of the binocular camera based on a Levenberg-Marquardt optimization algorithm to obtain accurate calibration parameters of the binocular camera;
s5, calculating the relative position parameter of the binocular camera when the distance is calculated.
Accordingly, the device for the scaling method comprises:
the image acquisition unit is used for acquiring a calibration template image obtained by shooting the calibration template;
the extraction unit is used for detecting a solid circle in the calibration template image so as to extract the circle center pixel position;
the calibration unit calculates or sets initial values of internal parameters, external parameters (rotation vectors and translation vectors) and distortion parameters of the binocular camera, and prepares for the next step of calibration parameter optimization;
and the optimization unit is used for optimizing the inner parameters, the outer parameters and the distortion parameters of the binocular camera by using a Levenberg-Marquardt algorithm based on the minimization of the sum of squares of the re-projection errors of all the center points, so as to obtain an accurate calibration result.
And the position determining unit is used for determining the relative positions of the binocular cameras based on the spatial positions of the two cameras obtained by the previous calibration relative to the same calibration template.
The flow of the calibration method is described in detail as follows when each distance is calibrated:
1. acquiring a calibration template image obtained by shooting a calibration template; (corresponding to S1)
The calibration templates are typically repetitive patterns with a fixed pitch, such as black and white checkerboard calibration templates, equally spaced solid circular array calibration templates, and the like.
As shown in fig. 2, the calibration scheme uses a solid circle array as a calibration template pattern, where the solid circle array is composed of 8 rows×11 columns of solid circles. The solid circles are identical in size and radius, and the circle center distances in the horizontal direction and the vertical direction are identical. The calibration patterns of each distance are consistent and consist of 8×11 solid circles. The circle radius and the circle center distance of the calibration pattern of each distance are in direct proportion to the distance from the binocular camera to the calibration template.
In the example, only one calibration template image needs to be shot for each calibration distance, and the optical axes of the left camera and the right camera are perpendicular to the calibration template during shooting, so that the calibration image patterns are clear and no pattern deformation caused by shooting angles exists. In addition, as the calibration template is arranged on the LED panel lamp, the calibration pattern is strongly compared with the white background, and the edge of the calibration pattern is strongly compared and is easy to extract. Based on the above two advantages, it is appropriate to calibrate the template using a solid circular array in this embodiment.
According to the number of solid circles in the horizontal direction and the vertical direction of the calibration template and the circle center distance, the distribution of the solid circle array in the coordinate system of the calibration template can be determined, and the homogeneous coordinate [ X Y Z1 ] of the circle center is obtained.
2. Performing solid circle detection on the calibration template image to extract a circle center point; (corresponding to S2)
In the field of computer vision such as three-dimensional scene reconstruction, repeated solid circles are often utilized to construct calibration patterns, and the size of a calibration template is determined through fixed circle radius and circle center distance. The solid circle center has the advantages of easy detection, high position precision, reliable matching, real-time processing and the like. The current circle center detection algorithm comprises the following steps: circle center detection based on Blob area analysis, circle center detection based on edge extraction, circle center detection based on Hough transformation, and the like. In this example, after the calibration image capturing is completed, a solid circle region is detected based on the image gray value, and the homogeneous coordinates [ x y ] of the center of the circle region centroid as the center of the circle pixel position are obtained. The image processing step is simple in calculation and high in instantaneity.
3. Calculating or setting initial values of internal parameters, external parameters (rotation vectors and translation vectors) and distortion parameters of the binocular camera; (corresponding to S3)
In computer vision, the interrelation of a point on a spatial object with its projected position on the image plane through an imaging system is generally described by a geometric projection model of a video (or camera) system. A common projection model is central projection in optics based on the principle of aperture imaging. In this model, a point on the object passes through the center of projection, i.e., the optical center of the lens, and is projected on the imaging chip along a straight line.
The homogeneous coordinate of the center of the solid circle in the reference coordinate system is [ X Y Z1 ], and the homogeneous coordinate of the pixel obtained by photographing the point on the camera is [ X Y ] on the assumption. According to the projection model based on the pinhole imaging, the circle center [ X Y ] of the solid circle of the calibration template is projected onto the image according to the following relation to obtain a corresponding imaging pixel [ x y ] (in this case, for the planar calibration template, the Z coordinate is assumed to be 0)
Where σ is the scale factor. The rotation vector R and the translation vector T are external parameters of the camera, and describe the spatial position of the camera in a calibration template coordinate system. K is the internal parameter of the camera, which is defined as
Wherein f x And f y Focal length in horizontal and vertical directions, c x C y Is the principal point of the image.
Based on the image of each shot calibration template, the corresponding homography matrix can be calculated
Wherein h is j Is the column vector of the j-th column (j=1, 2, 3), h ij Is the H matrix element of the ith row and jth column (i, j=1, 2, 3). The definition according to homography matrix is:
[h 1 h 2 h 3 ]＝K[r 1 r 2 T] (4)
according to the property of the rotation matrix, r 1 And r 2 Is an orthogonal unit vector, so that there are:
wherein the method comprises the steps of
From (5) and (6), respectively
h 11 h 12 ·B 11 +(h 31 h 12 +h 11 h 32 )·B 13 +h 21 h 22 ·B 22 +(h 31 h 22 +h 21 h 32 )·
B 23 +h 31 h 32 ·B 33 ＝0 (8)
In the production process of the camera, when the optical lens and the imaging chip are coupled and assembled, the main point of the image is required not to exceed a certain pixel range from the center point of the imaging chip. Thus presetting a principal point c x And c y The initial value of (2) is the center of the image. (8) In the expression (9), the homography matrix H can be obtained through the shot calibration template image, and the element H of the homography matrix H ij (i, j=1, 2, 3) is known; c x And c y Is the center point of the image, which is known, B ij Is the focal length f of the internal parameter of the camera x 、f y And principal point c x 、c y Is calculated by calculating the intermediate quantity appearing in the process, B ij Of only two unknowns f x And f y ) So that the two sets of equations (8), (9) can solve for two internal parameters: focal length f x And f y Is set to be a constant value. The initial value of the spatial position of the binocular camera in the coordinate system of the calibration template can be estimated according to the position of the camera relative to the calibration template and the type of the binocular camera. Taking a binocular camera module of a mobile phone as an example, since the optical axes of the two cameras are approximately parallel, an initial value of a relative rotation matrix R is set as an identity matrix
The actual projected pixel will typically have a small offset in the image due to the optical distortion of the lens. The main causes of image distortion are as follows: lens surface machining errors result in defects in the radial curvature; the optical centers of each lens cannot be kept exactly collinear, resulting in decentration errors; due to tolerances in lens design, production and camera assembly, the lens is not parallel to the imaging chip and is tilted. The above errors result in the image being distorted both radially and tangentially. Radial distortion means that the actual image point moves radially inward or outward in its ideal position and optical center line. Tangential distortion means that the actual image point is shifted in a direction perpendicular to the radial direction, i.e. tangential.
Theoretical pixel position [ x y ] based on center projection model]Is affected by distortion and shifts, and its actual projection position
The following relationship was used to simulate
Wherein [ k ] 1 k 2 k 3 ]Is a radial distortion parameter [ p ] 1 p 2 ]As tangential distortion parameter, r 2 ＝x 2 +y 2 . The distortion of the lens used in cameras is typically small, so the radial and tangential distortion parameters [ k ] 1 k 2 k 3 ]And [ p ] 1 p 2 ]Is typically set to zero.
4. Optimizing the internal parameters, the external parameters and the distortion parameters of the binocular camera to obtain an accurate calibration result; (corresponding to S4)
And generating a corresponding image pixel point in the image when the circle center of each solid circle is photographed. According to the projection model based on the pinhole imaging principle and the distortion model, a theoretical imaging position (including the offset generated by distortion) can be calculated for each circle center. The deviation of the actual image point from the theoretical pixel position is called the reprojection error. The parameters of the geometric projection model, namely the calibration parameters of the camera, should be such that the sum of squares of the re-projection errors of all solid circles on the calibration template is minimum, and the projection model describes the optical imaging projection process of the camera with depth of field most accurately
Wherein M is the position of the center of a solid circle on the calibration template, M is the position of the center point corresponding to the actual pixel in the image,
the position of the pixel calculated as the position of the heavy projection, i.e. the centre of the solid circle after the central projection, shifted by the distortion of the image (+)>
Is->
Contains distortion offset), K is the internal parameters of the camera, the rotation vector R and the translation vector T are the external parameters of the camera, k= [ K ] 1 k 2 k 3 ]As radial distortion parameter, p= [ p ] 1 p 2 ]Is a tangential distortion parameter. The equation is optimized by using a Levenberg-Marquardt algorithm, after a plurality of iterations, when the error of the iteration is smaller than a preset threshold value, the optimization iteration is ended, and the obtained results K, R, T, K and p are respectively the internal parameter, the external parameter and the distortion parameter corresponding to the calibration distance. The binocular camera calibration parameters obtained through the nonlinear optimization are more accurate.
5. Determining the relative position of the binocular camera; (corresponding to S5)
When the binocular cameras are calibrated, after the left camera and the right camera respectively calibrate the monocular cameras, the positions of the two cameras relative to the same plane calibration template are already calibratedThe spatial position of the left camera is known as R l 、T l The spatial position of the right camera is R r 、T r Thereby the relative position of the two cameras can be calculated as
6. Depth information calculation
After the calibration of the binocular cameras is completed at four distances of 0.5,1.0,1.5,2.0 meters, the calibration parameters of the four groups of binocular cameras are obtained in total. And during depth test, recording the VCM motor position during current mapping, comparing the current VCM motor position with the VCM motor positions corresponding to the four groups of calibration parameters, and selecting a group of calibration parameters with the nearest VCM motor position distance to calculate the depth.
As shown in fig. 5, taking a point P on the surface of a scene object as an example, the point depth information is calculated as follows: in the left camera coordinate system, its optical center C 1 The coordinates are the origin, and the point P is at the imaging corresponding point m of the left camera 1 Is [ x ] 1 y 1 f 1 ]. In the right camera coordinate system, its optical center C 2 The coordinates are the origin, and the point P is at the imaging corresponding point m of the right camera 2 Is [ x ] 2 y 2 f 2 ]。C 1 M 1 The coordinates in the right camera coordinate system are respectively
Is->
The P point coordinate is C 1 、m 1 Connecting wire with C 2 、m 2 And the depth information of the intersection point of the connecting line under the right camera coordinate system is the Z-axis coordinate of the intersection point.
Compared with the prior art, the embodiment has the advantages and effects that:
the advantages are as follows: only one planar calibration template is used per calibration distance and the calibration template does not need to be rotated. Compared with the current general calibration method requiring multiple angle shots, the method saves the calibration time of the production line, improves the yield per unit time and reduces the production cost.
The advantages are as follows: and calibrating the binocular cameras at a plurality of calibration distances respectively to obtain a plurality of groups of binocular camera calibration parameters. The multiple groups of calibration parameters effectively compensate parameter differences caused by lens position change in the photographing process of the automatic focusing camera, so that the accuracy of depth calculation is improved, the 3D application effect of the camera can be optimized, the user experience is improved, and the product competitiveness is enhanced.
The parameter calibration method provided by the embodiment can be applied to camera (or camera) calibration with an automatic focusing function, and has the advantages of simplicity and high efficiency compared with the existing general calibration method because only one calibration template image is adopted for parameter calibration. In addition, the device provides camera calibration at four distances, so that the difference of calibration parameters caused by lens position change can be better compensated, and the subsequent depth calculation is more accurate.
The method is also suitable for calibrating the multi-camera, and the depth calculation principle is based on the binocular principle, so that the multi-camera is regarded as the binocular camera, and the method belongs to the protection scope of the invention.
Claims (7)
1. The automatic focusing binocular camera calibration method on the production line is characterized by comprising the following steps of:
a1, acquiring calibration template images at corresponding distances shot at a plurality of calibration distances, wherein each calibration distance corresponds to a single Zhang Biaoding template image; the calibration template is provided with a calibration pattern with known size;
a2, analyzing and processing a single Zhang Biaoding template image corresponding to each calibration distance, and calculating to obtain a group of binocular camera calibration parameters corresponding to the distance;
a3, corresponding to a plurality of calibration distances, and obtaining calibration parameters of a plurality of groups of binocular cameras in total;
in the step A1, the calibration pattern is a solid circle array, the solid circle array consists of C rows and L columns of solid circles, and C and L are natural numbers; each solid circle has the same size and the same radius, and the circle center distances in the horizontal direction and the vertical direction are the same; the calibration patterns of each distance are consistent and consist of C multiplied by L solid circles; the calibration templates used in each distance and the calibration patterns thereof are different, wherein the circle radius and the circle center distance of the calibration pattern in each distance are in direct proportion to the distance from the binocular camera to the calibration template; thus, at each different calibration distance, calibration template images having different sizes of circle radii and circle center distances corresponding to each different calibration distance are used;
in step A2, analyzing and processing a single Zhang Biaoding template image corresponding to each calibration distance specifically includes the following steps:
s2, extracting the pixel position of the solid circle center from the calibration template image shot by the left camera and the right camera through an image processing algorithm;
s3, calculating the focal length of each lens of the binocular camera based on the calibration template image, and acquiring initial values of internal parameters, external parameters and distortion parameters of each camera;
s4, optimizing internal parameters, external parameters and distortion parameters of the binocular camera based on a Levenberg-Marquardt optimization algorithm to obtain accurate calibration parameters of the binocular camera;
s5, calculating the relative position parameters of the binocular camera when the distance is calculated;
in step S3, two internal parameters, the focal length f, are solved by the following set of equations x And f y Initial value of:
h 11 h 12 ·B 11 +(h 31 h 12 +h 11 h 32 )·B 13 +h 21 h 22 h 22 +(h 31 h 22 +h 21 h 32 )·B 23 +h 31 h 32 ·B 33 ＝0
wherein h is ij Homography matrix H matrix elements, which are the ith row and jth column (i, j=1, 2, 3), are known;
B ij determined by the following formula:
in the above formula, K is an internal parameter of the camera, which is defined as
c x And c y Is the principal point of the image, and the principal point c is preset x And c y The initial value of (2) is the center of the image.
2. The method according to claim 1, wherein in step A1, the optical axes of the left and right cameras are perpendicular to the calibration template when the calibration template image is photographed, so that the calibration image is focused and there is no pattern distortion due to photographing angle.
3. The method for calibrating an auto-focus binocular camera on a production line according to claim 1, wherein in step S2, a solid circle region is detected based on the gray level value of the image, and homogeneous coordinates [ x y ] of the center of the circle region as the center of the circle are obtained.
4. The method for calibrating an auto-focusing binocular camera on a production line according to claim 1, wherein in step S4, the sum of squares of the re-projection errors of all solid circle centers on the calibration template is minimized during the optimization, namely:
wherein M is the position of the center of a solid circle on the calibration template, M is the position of the center point corresponding to the actual pixel in the image,
for heavy projection position, i.e. pixel position calculated by shifting the centre of the solid circle after central projection due to image distortion, +.>
Is->
The distortion offset is included, K is the internal parameter of the camera, the rotation vector R and the translation vector T are the external parameters of the camera, and k= [ K ] 1 k 2 k 3 ]As radial distortion parameter, p= [ p ] 1 p 2 ]Is a tangential distortion parameter.
5. The method for calibrating an auto-focus binocular camera on a production line according to claim 1, wherein in step S5, the relative positions of the two cameras are as follows
Wherein R is l 、T l Is the space position of the left camera, R r 、T r The spatial position of the right camera;
if a plurality of cameras are provided, calculation is performed between every two cameras.
6. An automatic focusing binocular camera calibration device on a production line, characterized in that a calibration method as claimed in any one of claims 1 to 5 is adopted.
7. A depth measurement method, characterized in that the calibration method according to any one of claims 1 to 5 is adopted to obtain calibration parameters of a plurality of groups of binocular cameras corresponding to a plurality of calibration distances, and in the depth test, a group of binocular camera calibration parameters corresponding to a calibration distance closest to the lens position in the current image capturing process is selected to calculate the depth.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811550433.9A CN109859272B (en) | 2018-12-18 | 2018-12-18 | Automatic focusing binocular camera calibration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811550433.9A CN109859272B (en) | 2018-12-18 | 2018-12-18 | Automatic focusing binocular camera calibration method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109859272A CN109859272A (en) | 2019-06-07 |
| CN109859272B true CN109859272B (en) | 2023-05-19 |
Family
ID=66891545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811550433.9A Active CN109859272B (en) | 2018-12-18 | 2018-12-18 | Automatic focusing binocular camera calibration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109859272B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111080705B (en) * | 2019-05-07 | 2021-06-04 | 像工场(深圳)科技有限公司 | Calibration method and device for automatic focusing binocular camera |
| CN110345921B (en) * | 2019-06-12 | 2020-06-26 | 中国农业大学 | Stereo visual field vision measurement and vertical axis aberration and axial aberration correction method and system |
| CN110533728A (en) * | 2019-07-25 | 2019-12-03 | 长沙行深智能科技有限公司 | Scaling method, device and the medium of binocular stereo vision camera based on Zhang Zhengyou calibration |
| CN112862895B (en) * | 2019-11-27 | 2023-10-10 | 杭州海康威视数字技术股份有限公司 | Fisheye camera calibration method, device and system |
| CN111314602B (en) * | 2020-02-17 | 2021-09-17 | 浙江大华技术股份有限公司 | Target object focusing method, target object focusing device, storage medium and electronic device |
| CN111986512B (en) * | 2020-07-16 | 2022-04-05 | 华为技术有限公司 | Target distance determination method and device |
| CN111932636B (en) * | 2020-08-19 | 2023-03-24 | 展讯通信（上海）有限公司 | Calibration and image correction method and device for binocular camera, storage medium, terminal and intelligent equipment |
| CN114095657B (en) * | 2021-11-22 | 2024-02-27 | 成都天翼空间科技有限公司 | Automatic calibration method and system based on newly added camera |
| CN114205483B (en) * | 2022-02-17 | 2022-07-29 | 杭州思看科技有限公司 | Scanner precision calibration method and device and computer equipment |
| CN115170674B (en) * | 2022-07-20 | 2023-04-14 | 禾多科技（北京）有限公司 | Camera principal point calibration method, device, equipment and medium based on single image |
| CN115272491A (en) * | 2022-08-12 | 2022-11-01 | 哈尔滨工业大学 | Binocular PTZ camera dynamic self-calibration method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101231750A (en) * | 2008-02-21 | 2008-07-30 | 南京航空航天大学 | Calibrating method of binocular three-dimensional measuring system |
| CN101447073A (en) * | 2007-11-26 | 2009-06-03 | 新奥特（北京）视频技术有限公司 | Zoom lens calibration method |
| CN103544699A (en) * | 2013-10-11 | 2014-01-29 | 江西省电力公司检修分公司 | Method for calibrating cameras on basis of single-picture three-circle template |
| CN103559708A (en) * | 2013-10-30 | 2014-02-05 | 同济大学 | Industrial fixed-focus camera parameter calibration device based on square target model |
| CN104155470A (en) * | 2014-07-15 | 2014-11-19 | 华南理工大学 | Detecting method and system based on binocular camera for real-time vehicle speed |
| CN106131408A (en) * | 2016-07-11 | 2016-11-16 | 深圳市金立通信设备有限公司 | A kind of image processing method and terminal |
| CN106846415A (en) * | 2017-01-24 | 2017-06-13 | 长沙全度影像科技有限公司 | A kind of multichannel fisheye camera binocular calibration device and method |
-
2018
- 2018-12-18 CN CN201811550433.9A patent/CN109859272B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101447073A (en) * | 2007-11-26 | 2009-06-03 | 新奥特（北京）视频技术有限公司 | Zoom lens calibration method |
| CN101231750A (en) * | 2008-02-21 | 2008-07-30 | 南京航空航天大学 | Calibrating method of binocular three-dimensional measuring system |
| CN103544699A (en) * | 2013-10-11 | 2014-01-29 | 江西省电力公司检修分公司 | Method for calibrating cameras on basis of single-picture three-circle template |
| CN103559708A (en) * | 2013-10-30 | 2014-02-05 | 同济大学 | Industrial fixed-focus camera parameter calibration device based on square target model |
| CN104155470A (en) * | 2014-07-15 | 2014-11-19 | 华南理工大学 | Detecting method and system based on binocular camera for real-time vehicle speed |
| CN106131408A (en) * | 2016-07-11 | 2016-11-16 | 深圳市金立通信设备有限公司 | A kind of image processing method and terminal |
| CN106846415A (en) * | 2017-01-24 | 2017-06-13 | 长沙全度影像科技有限公司 | A kind of multichannel fisheye camera binocular calibration device and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109859272A (en) | 2019-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| CN109767476B (en) | Automatic focusing binocular camera calibration and depth calculation method | |
| CN110276808B (en) | Method for measuring unevenness of glass plate by combining single camera with two-dimensional code | |
| CN105716542B (en) | A kind of three-dimensional data joining method based on flexible characteristic point | |
| US8934721B2 (en) | Microscopic vision measurement method based on adaptive positioning of camera coordinate frame | |
| CN110517325B (en) | Coordinate transformation and method and system for positioning objects around vehicle body through coordinate transformation | |
| JP2018179981A (en) | Camera calibration method, camera calibration program and camera calibration device | |
| CN111080705B (en) | Calibration method and device for automatic focusing binocular camera | |
| CN109272555B (en) | External parameter obtaining and calibrating method for RGB-D camera | |
| CN113034612B (en) | Calibration device, method and depth camera | |
| CN110874854B (en) | Camera binocular photogrammetry method based on small baseline condition | |
| WO2021129437A1 (en) | Method and system for light calibration field camera without requiring white image | |
| CN112489137A (en) | RGBD camera calibration method and system | |
| CN107534715B (en) | Camera production method and advanced driving assistance system | |
| Ding et al. | A robust detection method of control points for calibration and measurement with defocused images | |
| CN110827360B (en) | Photometric stereo measurement system and method for calibrating light source direction thereof | |
| CN113119129A (en) | Monocular distance measurement positioning method based on standard ball | |
| CN107063644B (en) | Finite object distance distortion measuring method and system | |
| CN116625258A (en) | Chain spacing measuring system and chain spacing measuring method | |
| CN112229323A (en) | Six-degree-of-freedom measurement method of checkerboard cooperative target based on monocular vision of mobile phone and application of six-degree-of-freedom measurement method | |
| Zhang et al. | Improved Camera Calibration Method and Accuracy Analysis for Binocular Vision | |
| CN112489141B (en) | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera | |
| Kumar et al. | On the equivalence of moving entrance pupil and radial distortion for camera calibration | |
| Fasogbon et al. | Calibration of fisheye camera using entrance pupil | |
| CN110503683B (en) | Narrow parking space reversing and warehousing guiding method and system based on monocular ranging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |