CN112489141B - Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera - Google Patents
Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera Download PDFInfo
- Publication number
- CN112489141B CN112489141B CN202011521340.0A CN202011521340A CN112489141B CN 112489141 B CN112489141 B CN 112489141B CN 202011521340 A CN202011521340 A CN 202011521340A CN 112489141 B CN112489141 B CN 112489141B
- Authority
- CN
- China
- Prior art keywords
- calibration
- vehicle
- mounted camera
- relay lens
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 39
- 239000007787 solid Substances 0.000 claims abstract description 88
- 239000011159 matrix material Substances 0.000 claims description 24
- 230000003287 optical effect Effects 0.000 claims description 24
- 238000005457 optimization Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 description 18
- 239000013598 vector Substances 0.000 description 18
- 230000008569 process Effects 0.000 description 10
- 238000001514 detection method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000013519 translation Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012634 optical imaging Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 239000012482 calibration solution Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000003754 machining Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a production line calibration method and device for a single-board single-image strip relay lens of a vehicle-mounted camera, wherein the method comprises the following steps: s1, a virtual image formed by a calibration template through a relay lens is obtained by a vehicle-mounted camera to serve as a calibration template image; s2, extracting pixel position coordinates of the circle center of the solid circle in the calibration template image; s3, calculating an initial value of a calibration parameter of the vehicle-mounted camera; and S4, optimizing the initial value of the calibration parameter to obtain the accurate calibration parameter of the vehicle-mounted camera. The device comprises a relay lens, a calibration template and a vehicle-mounted camera to be calibrated, wherein the relay lens is arranged between a lens of the vehicle-mounted camera and the calibration template. The invention saves the calibration time of the production line, improves the output per unit time, and can simulate different focusing distances, thereby realizing accurate and efficient calibration on the production line of different types of cameras and having higher compatibility.
Description
Technical Field
The invention relates to the field of camera calibration methods, in particular to a production line calibration method and device for a single-board single-image strip relay lens of a vehicle-mounted camera.
Background
With the popularization of driving assistance and automatic driving technologies, automobiles are increasingly using cameras, such as ranging, 3D environment sensing, etc., through binocular cameras. Calibration of the camera is a key for realizing the 3D application of the vehicle-mounted camera. In camera-based measurement and vision applications, the relationship between the three-dimensional position of an object point in space and its corresponding two-dimensional pixel point position in an image is mathematically described by a geometric projection model. The parameters of the model are typically obtained by photographing, image processing and calculation of a calibration pattern of known size (e.g. a solid circular array or a black and white checkerboard),this process of determining camera projection model parameters is known as calibration. These parameters include: internal parameters refer to a principal point position c during imaging by a camera x 、c y Lens focal length f x 、f y Such parameters are only relevant to the camera itself; the external parameters refer to the position of the camera in space, and generally refer to a rotation vector R and a translation vector T of the camera in a certain reference coordinate system; distortion, which is the deviation between the actual corresponding pixel position of the object point in the image and the theoretical projection point calculated based on the imaging model, in the photographing process of the camera, is generally defined by a radial distortion parameter k 1 、k 2 、k 3 Tangential distortion parameter p 1 、p 2 It is described that it is mainly caused by design, manufacturing and assembly errors of the lens and camera module.
There are two limitations to the current camera universal calibration solutions. First, because the calibration algorithm used requires taking images of multiple calibration templates from different angles, either rotating the 2D planar calibration template on the current production line or using a calibration stereo template that is stitched by multiple planar calibration templates (with fixed angles to each other). The device for rotating the calibration template is complex in mechanism, and the measurement time is prolonged by rotating, so that the yield per unit time is reduced, and the cost is increased. Besides the cost increase, the three-dimensional template spliced by the plurality of plane calibration templates is required to be accurately fixed at a certain angle or the three-dimensional template is required to be measured with high precision before use.
The prior patent CN 209640928U (shenzhen city popularizing technology limited, filing date 2019.03.31) describes a calibration device on a camera production line, the device comprising: the device comprises a support frame, a mechanical arm and a calibration template fixed on the mechanical arm. The device completes camera calibration by controlling the mechanical arm to move the calibration template to a plurality of positions and analyzing and processing images shot at each calibration position. The patent needs to adjust the position of the mechanical arm movable calibration template for multiple times to perform multi-image calibration, and has the advantages of long time consumption, complex operation and incapability of accurately calibrating cameras with far focusing distances.
Patent CN 110490940A (beijing migwei science and technology limited, application date 2019.08.15) describes a method and a device for calibrating a camera based on a single checkerboard image, and the method can realize rapid calibration of camera parameters. The patent mainly obtains internal parameters of a camera through an absolute conic equation according to two vanishing points corresponding to two groups of parallel lines acquired on a checkerboard image by utilizing pixel coordinates of the two vanishing points. The camera with a far focusing distance cannot be accurately calibrated.
Patent CN 207123866U (changdong meyobo science and technology limited liability company, application date 2017.07.26) describes a calibration system based on single frame image calibration, which can complete camera calibration based on one calibration image. The calibration template used in the patent consists of a square array with fixed spacing and at least eight spherical calibration pieces positioned at the outer sides of four corners of an array image, and has a complex structure, and the patent cannot accurately calibrate a camera with a far focusing distance.
Patent CN 106570907B (sea information group limited, filing date 2016.11.22) describes a camera calibration method and apparatus. According to the method, the rotation angles of the cameras along two axis directions of the world coordinate system respectively are solved through a simple linear relation, and then other parameters of the cameras are solved. The patent does not relate to calibration of distortion parameters, only the focal length and external parameters of a camera are calibrated, and the patent cannot accurately calibrate a camera with a far focusing distance.
Patent CN 110033491A (south kyo engineering institute, filing date 2019.04.15) describes a camera calibration method. The method comprises the steps of constructing a multi-dimensional vector based on camera internal parameters and lens distortion parameters, constructing a new multi-dimensional vector after performing de-distortion processing on a calibration image by using the lens distortion parameters, and performing loop iteration until Euclidean distance between two adjacent multi-dimensional vectors is smaller than a set value, so as to obtain the camera internal parameters and the camera external parameters. In the patent, the Euclidean distance of a multidimensional vector formed by two adjacent internal parameters and two adjacent external parameters is used as a cost function, iterative optimization is performed on numerical values, and the accurate calibration cannot be performed on a camera with a far focusing distance.
The acquisition of the coordinate positions of the characteristic points (such as the circle centers of solid circles and the like) of the calibration patterns in the shot calibration images is the premise and the basis for camera calibration. Imaging definition can directly influence the accuracy of feature point position extraction, and indirectly influences the accuracy of camera calibration. Therefore, the camera needs to take a picture in focus at the target distance, so that the accurate parameter value of the camera is obtained based on the clear calibration template image. For a vehicle-mounted camera, the focusing distance is generally several meters, tens of meters and hundreds of meters, and the size of the calibration module is proportional to the focusing distance, so that the required calibration equipment is too large in size, and the production line of the vehicle-mounted camera in a clean room is high in cost and even can not be realized. Therefore, there is a need for a method and a calibration device for calibrating a vehicle-mounted camera in a short distance on a production line.
Disclosure of Invention
The invention aims to provide a production line calibration method and device for a single-board single-image strip relay lens of a vehicle-mounted camera, which are used for solving the problem that the prior art cannot accurately and efficiently calibrate the vehicle-mounted camera on a production line.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
the method for calibrating the production line of the single-image belt relay lens of the vehicle-mounted camera comprises the steps of constructing a calibration system comprising a calibration template, a relay lens and the vehicle-mounted camera to be calibrated, and adjusting the distance between the relay lens and the calibration template to be suitable for the vehicle-mounted cameras with different focal lengths, wherein the calibration template is provided with a solid circular array, and the vehicle-mounted camera shoots the calibration template through the relay lens, and comprises the following specific processes:
s1, acquiring a virtual image formed by a calibration template through a relay lens by a vehicle-mounted camera as a calibration template image;
s2, extracting pixel position coordinates of the circle center of the solid circle in the calibration template image by using an image processing algorithm;
s3, calculating initial values of calibration parameters of the vehicle-mounted camera, including internal parameters, external parameters and distortion parameters, based on the calibration template image and combining a homography matrix and solid circle center position coordinates of the calibration template image;
and S4, optimizing the initial values of the internal and external parameters and the distortion parameters of the vehicle-mounted camera obtained in the step S3 to obtain accurate calibration parameters of the vehicle-mounted camera.
In the calibration method, firstly, a calibration template image in a virtual image form at a remote position formed by simulating a calibration template through a relay lens is shot; and then analyzing and processing a single calibration template image corresponding to the simulated calibration distance, and calculating to obtain corresponding camera calibration parameters.
The calibration pattern on the surface of the calibration template is a solid circle array, the solid circle array consists of C rows and L columns of solid circles, and C and L are natural numbers; the solid circles are identical in size and radius, and the circle center distances in the horizontal direction and the vertical direction are identical. The calibration template image shot by the relay lens is consistent with the original pattern and also consists of C multiplied by L solid circles; the circle radius and the circle center distance of the simulated remote calibration template image are related to the distance from the relay lens to the calibration template. When the image of the calibration template is shot, the optical axis of the vehicle-mounted camera is parallel and concentric with the optical axis of the relay lens as much as possible, and the optical axis of the vehicle-mounted camera and the optical axis of the relay lens are perpendicular to the calibration template, so that the image of the calibration template is focused and pattern deformation caused by shooting angles does not exist.
In the invention, the process of analyzing and processing the single calibration template image corresponding to the calibration distance comprises the following steps:
1. extracting the pixel position of the solid circle center from the shot calibration template image through an image processing algorithm; and extracting the center of the solid circle of the calibration template image by using a binarization and median filtering method to obtain the pixel position coordinates of the center of the solid circle in the calibration template image, or detecting the solid circle area based on the gray value of the image, and obtaining the center of mass of the circle area as the coordinates of the center of the circle.
2. Based on the calibration template image, according to the homography matrix of the calibration template image, the initial values of the internal parameters, the external parameters and the distortion parameters of the camera are calculated and obtained by combining the pixel positions of the main points and the circle centers of the solid circles of the calibration template image. The central coordinate position of the image can be preset as a principal point c x And c y According to the initial value of the main point and the solid circle center position, the focal length f is solved based on the homography matrix of the single calibration template image x And f y Is set to be a constant value. The initial value of the spatial position T of the vehicle-mounted camera in the coordinate system of the calibration template can be estimated according to the position of the camera relative to the calibration template and the type of the vehicle-mounted camera. Because the optical axis of the vehicle-mounted camera is approximately perpendicular to the calibration template, the initial value of the rotation matrix R is set as the identity matrix. Radial and tangential distortion parameters of camera lens in general [ k ] 1 k 2 k 3 ]And [ p ] 1 p 2 ]Is set to zero.
3. Based on the principle of minimizing the sum of squares of the re-projection errors of the solid circle centers of the calibration template images, the inner parameters, the outer parameters and the distortion parameters of the camera are optimized by using a Levenberg-Marquardt optimization algorithm, so that accurate calibration parameters of the camera are obtained.
The utility model provides a production line calibration device of on-vehicle camera veneer list picture area relay mirror, includes relay mirror, calibration template and the on-vehicle camera of waiting to mark, and relay mirror locates between the camera lens of on-vehicle camera and the calibration template, and relay mirror, on-vehicle camera optical axis coincidence set up, and the optical axis of relay mirror and on-vehicle camera all is perpendicular to the calibration template surface. The distance between the relay lens and the calibration template is adjustable, the calibration template pattern is a solid circle array, the solid circle array is a row-column array formed by a plurality of solid circles, the radius of each solid circle is the same, and the circle center distances of the adjacent solid circles in the horizontal direction and the vertical direction are the same.
The device also comprises an extraction unit, a calibration unit and an optimization unit which are constructed in the computer, wherein the extraction unit extracts the pixel position coordinates of the center of the solid circle in the calibration template image, the calibration unit calculates the initial values of the calibration parameters of the vehicle-mounted camera, including the internal parameter, the external parameter and the distortion parameter, and the optimization unit optimizes the internal parameter, the external parameter and the distortion parameter of the vehicle-mounted camera to obtain the accurate calibration parameters of the vehicle-mounted camera.
The invention can achieve the calibration of the vehicle-mounted camera by acquiring one frame of calibration template image. Based on the imaging characteristics of the relay lens, the distance from the relay lens to the calibration template is changed to simulate a virtual calibration image with a long distance, so that the device is suitable for calibrating vehicle-mounted cameras with different focusing distances (generally long distances). Based on a single board with a single picture and a scheme with a relay lens, the method reduces the calibration time, can rapidly complete the calibration of the vehicle-mounted camera, has small volume of calibration equipment, occupies less production line space of a clean room, and has the advantages of simplicity and convenience in operation, good universality, low cost and the like.
The calibration scheme of the invention has the advantages of short time consumption, simple operation, and capability of ensuring that a clearly focused calibration image can be obtained when the camera is calibrated at a long distance by introducing the relay lens between the camera and the calibration template, accurately calibrating the vehicle-mounted camera with a long focusing distance, and calibrating the internal and external parameters and distortion parameters of the vehicle-mounted camera. Meanwhile, the invention can also optimize the internal and external parameters, and find the optimal solution based on the minimization of the re-projection error during optimization.
Compared with the prior art, the invention has the advantages and effects that:
1. on the production line, each camera is calibrated at a distance, only one plane calibration template is used, and the calibration template does not need to rotate. Compared with the current general calibration method requiring multiple angle shots, the method saves the calibration time of the production line, improves the yield per unit time and reduces the production cost.
2. According to the invention, the remote distance is simulated based on the relay lens, the vehicle-mounted camera is focused on the virtual calibration template for calibration, and the accurate calibration parameters of the vehicle-mounted camera during remote distance focusing are obtained. The relay lens simulates a long distance, so that the calibration of the vehicle-mounted camera in a small space is realized, the sizes of camera calibration equipment and a production line are reduced, the space of a clean room is saved, the production cost of the vehicle-mounted camera is reduced, and the competitiveness of products is enhanced. And different focusing distances can be simulated by changing the distance between the relay lens and the physical calibration template, so that different types of cameras can be accurately calibrated, and the compatibility is high.
3. The method is also suitable for calibrating the multi-camera, and the calibration parameters of each camera are calculated based on the principle, so that the method can be regarded as repeated calibration of the monocular camera (although the simulation distances of relay lenses of different cameras can be different), and the method still belongs to the protection scope of the invention.
Drawings
FIG. 1 is a flow chart of a method of an embodiment of the present invention.
FIG. 2 is a schematic diagram of a calibration device according to an embodiment of the present invention.
FIG. 3 is a schematic block diagram of a complete calibration system according to an embodiment of the present invention.
FIG. 4 is a schematic diagram of a calibration template pattern according to an embodiment of the present invention.
FIG. 5 is a schematic diagram of a relay lens simulating a long-distance imaging relationship according to an embodiment of the present invention.
Detailed Description
The invention will be further described with reference to the drawings and examples.
As shown in fig. 1, the production line calibration method of the single-board single-diagram with the relay lens of the vehicle-mounted camera constructs a calibration system comprising a calibration template, the relay lens and the vehicle-mounted camera to be calibrated, the distance between the relay lens and the calibration template is adjusted to be suitable for vehicle-mounted cameras with different focal lengths, the calibration template patterns are solid circle arrays, and the vehicle-mounted cameras shoot the calibration template through the relay lens, and the specific process is as follows:
s1, acquiring a virtual image formed by a calibration template through a relay lens by a vehicle-mounted camera to serve as a calibration template image.
S2, extracting pixel position coordinates of the circle center of the solid circle in the calibration template image by using an image processing algorithm; extracting solid circle center characteristic points of the calibration template image by a binarization and median filtering method to obtain pixel position coordinates of the solid circle center in the calibration template image; or detecting a solid circle region in the calibration template image through the image gray value, and calculating the mass center of the region to obtain the pixel position coordinates of the circle center of the solid circle in the calibration template image.
S3, calculating initial values of calibration parameters of the vehicle-mounted camera, including internal parameters, external parameters and distortion parameters, based on the calibration template image and combining a homography matrix and solid circle center position coordinates of the calibration template image, wherein:
and according to the homography matrix of the calibration template image, combining the pixel positions of the main points and the solid circle centers of the calibration template image, and calculating to obtain the internal parameter initial value of the vehicle-mounted camera. The initial value of the external parameter rotation matrix of the vehicle-mounted camera is an identity matrix. The initial value of the distortion parameter of the vehicle-mounted camera is zero.
S4, optimizing the initial values of the internal and external parameters and the distortion parameters of the vehicle-mounted camera obtained in the step S3 to obtain accurate calibration parameters of the vehicle-mounted camera, wherein the initial values of the internal and external parameters and the distortion parameters of the vehicle-mounted camera are optimized by using a Levenberg-Marquardt optimization algorithm based on the principle that the sum of squares of the re-projection errors of the solid circle centers of the calibration template images is minimum.
As shown in FIG. 2, the production line calibration device with the relay lens on the single board of the vehicle-mounted camera comprises a relay lens 1, a calibration template 2 and a vehicle-mounted camera 3 to be calibrated, wherein the relay lens 1 is arranged between a lens of the vehicle-mounted camera 3 and the calibration template 2, the distance between the relay lens 1 and the calibration template 2 is adjustable, and a solid circular array is arranged on the surface of the calibration template 2.
The optical axes of the relay lens 1 and the vehicle-mounted camera 3 are overlapped, and the optical axes of the relay lens 1 and the vehicle-mounted camera 3 are perpendicular to the calibration template 2. The solid circle array on the surface of the calibration template 2 is a row-column array formed by a plurality of solid circles, the radiuses of the solid circles are the same, and the circle center distances of the adjacent solid circles in the horizontal direction and the vertical direction are the same.
The calibration device for the single board single diagram of the vehicle-mounted camera comprises a single plane 2D calibration template, a relay lens and a vehicle-mounted camera to be calibrated. The complete calibration system is shown in fig. 3, and is realized by an optical system (comprising a graph capturing unit) and an algorithm software system (comprising an extraction unit, a calibration unit and an optimization unit) arranged in a computer, wherein:
the image acquisition unit is a calibration system optical system comprising a relay lens 1, a calibration template 2 and a vehicle-mounted camera 3 to be calibrated, and is used for acquiring an image of a simulated remote calibration template obtained by shooting the calibration template through the relay lens.
The extraction unit is used for detecting a solid circle in the calibration template image so as to extract the circle center pixel position;
the calibration unit calculates or sets initial values of internal parameters, external parameters (rotation vectors and translation vectors) and distortion parameters of the vehicle-mounted camera, and prepares for the next step of calibration parameter optimization;
and the optimization unit is used for optimizing the internal parameters, the external parameters and the distortion parameters of the vehicle-mounted camera by using a Levenberg-Marquardt algorithm based on the minimization of the sum of squares of the re-projection errors of all the center points, so as to obtain an accurate calibration result.
The concrete explanation is as follows:
1. drawing unit
The calibration template is typically a repeating pattern with a fixed pitch, such as a black and white checkerboard calibration template, an equally spaced solid circular array calibration template, or the like (as shown in fig. 4). The calibration scheme adopts a solid circle array as a calibration pattern, wherein the solid circle array consists of C rows and L columns of solid circles, and C and L are natural numbers. The solid circles are identical in size and radius, the circle center distances in the horizontal direction and the vertical direction are identical, and the radius and the circle center distance of the solid circles are determined by the number C of rows and the number L of columns of the solid circle array and the size of the standard template. The calibration pattern shot by the relay lens is consistent with the original pattern and also consists of C multiplied by L solid circles; the radius of the circle and the center distance of the circle simulating the remote calibration pattern are related to the distance from the relay lens to the calibration template.
The relay lens is positioned between the vehicle-mounted camera to be calibrated and the calibration template, and the physical calibration template 2 with a short distance generates an enlarged virtual image (namely a virtual calibration template 4 as shown in fig. 5) at the same side and a long distance. The object at a close distance is shot through the relay lens 1, namely, the acquisition of the virtual image of the calibration template at a long distance is equivalent. According to the size of the physical calibration template 2 and the distance between the relay lens 1 and the physical calibration template 2, the size and the simulation distance of the virtual calibration template 4 can be calculated based on an optical Gaussian formula, so that the designated focusing long distance can be accurately calibrated through the relay lens 1.
From the Gaussian formulaThe distance from the virtual calibration template 4 to the relay lens 1 can be obtained
Size of the virtual calibration template 4
Wherein:
f': the focal length of the relay mirror 1;
l': the distance from the virtual calibration template 4 to the relay lens 1 (simulated focusing distance);
l: the distance from the physical calibration template 2 to the relay lens 1;
y': the vertical dimension of the virtual calibration template 4;
y: the vertical dimension of the physical calibration template 2;
as can be seen from the formulas (1) and (2), the simulated distance between the virtual calibration template 4 and the relay lens 1 is determined by the distance between the physical calibration template 2 and the relay lens 1, so that when the vehicle-mounted cameras with different focal lengths are calibrated, the distance between the relay lens and the physical calibration template 2 can be adjusted based on the gaussian formula according to the target focusing distance of the vehicle-mounted cameras, thereby achieving the effect of accurately simulating the target distance.
2. Extraction unit
Under the distance increasing effect of the relay lens, the calibration object of the camera is a long-distance virtual calibration template, and a frame of calibration image obtained by the camera is used for extracting circle center characteristic points through methods such as binarization, median filtering and the like. Or the solid circle area can be detected based on the gray value of the image, and the center of mass of the circle area is calculated to serve as the coordinate of the center of the circle.
3. Calibration unit
Based on the image of the calibration template shot by a single piece, a corresponding homography matrix can be calculated, and the initial value of the internal parameter of the vehicle-mounted camera is calculated by combining the pixel position of the center of the solid circle in the image of the calibration template. The initial value of the external parameter rotation matrix of the vehicle-mounted camera is an identity matrix, and the initial value of the distortion parameter of the vehicle-mounted camera is zero. The calibration process is described in detail in the examples below.
4. Optimization unit
And generating a corresponding image pixel point in the image when the circle center of each solid circle is photographed. Based on the principle of pinhole imaging and the projection model of image distortion, a theoretical imaging position (including image distortion) can be calculated for each circle center. The deviation of the actual image point from the theoretical pixel position is called the reprojection error. The parameters of the geometric projection model, namely the calibration parameters of the camera, should be such that the sum of squares of the re-projection errors of the centers of the solid circles on all calibration templates is minimum, and the projection model describes the optical imaging projection process of the camera most accurately
Wherein M is the position of the center of a solid circle on the virtual calibration template, M is the position of a pixel corresponding to the center point in the calibrated image for distortion correction,for the heavy projection position, K is the internal parameter of the camera, the rotation vector R and the translation vector T are the external parameters of the camera, and k= [ K ] 1 ，k 2 ，k 3 ]Is a radial distortion parameter>Is a tangential distortion parameter.
The equation was optimized using a Levenberg-Marquardt. After the initial value is input for a plurality of iterations, when the error of the iteration is smaller than a preset threshold value, the optimization iteration is finished, and the obtained results K, R, T, K and p are respectively the inner parameter, the outer parameter and the distortion parameter corresponding to the calibration distance. The camera calibration parameters obtained through the nonlinear optimization are more accurate.
The calibration method and device of the vehicle-mounted camera with the relay lens for the single board single image are illustrated by specific embodiments. When the method is adopted for calibration, a 2D plane calibration template is required to be installed on a production line of the vehicle-mounted camera; a relay lens is placed at a specific distance of the calibration template to generate a remote virtual image of the calibration template, the calibration distance is just one image of the calibration template, and the calibration template does not need to rotate; and analyzing and processing the single Zhang Biaoding template image shot at the distance, and calculating to obtain the calibration parameter of the vehicle-mounted camera corresponding to the distance.
And selecting a proper relay lens according to parameters (focal length, focusing distance and field angle) of the vehicle-mounted camera to be calibrated, including determining a focal length f'. According to the calibration distance l 'which is required to be simulated and the focal length f' of the relay lens, the distance l from the relay lens to the calibration template can be determined based on the formula (1).
The vertical dimension y 'of the virtual calibration template is determined by the field angle of the vehicle-mounted camera to be calibrated and the calibration distance l'. The vertical dimension y of the 2D plane calibration template can be determined based on the foregoing formula (2) according to the simulated calibration distance l' and the distance l from the relay lens to the calibration template.
For the distance calibration, the detailed flow of the calibration method is as follows:
1. obtaining a simulated remote calibration template image obtained by shooting a calibration template through a relay lens; (corresponding to S1)
The calibration templates are typically repetitive patterns with a fixed pitch, such as black and white checkerboard calibration templates, equally spaced solid circular array calibration templates, and the like. As shown in fig. 4, the calibration scheme uses a solid circle array as a calibration template pattern, where the solid circle array is composed of 8 rows×11 columns of solid circles. The solid circles are identical in size and radius, the circle center distances in the horizontal direction and the vertical direction are identical, and the radius and the circle center distance of the solid circles are determined by the number 8 of rows and the number 11 of columns of the solid circle array and the size of the standard template. The calibration pattern shot by the relay lens is consistent with the original pattern and also consists of 8 multiplied by 11 solid circles; the radius of the circle and the center distance of the circle simulating the remote calibration pattern are related to the distance from the relay lens to the calibration template. According to the number and circle center distance of solid circles in the horizontal and vertical directions of the calibration template, the distribution of the solid circle array in the coordinate system of the calibration template can be determined; based on the distance simulated by the relay lens, homogeneous coordinates [ X Y Z1 ] of the circle center can be obtained.
In this example, only one calibration template image needs to be shot for the simulated calibration distance, and the optical axis of the vehicle-mounted camera is parallel to the optical axis of the relay lens, concentric and perpendicular to the calibration template during shooting, so that the calibration image pattern is clear and pattern deformation caused by shooting angles does not exist. In addition, as the calibration template is arranged on the LED panel lamp, the calibration pattern is strongly compared with the white background, and the edge of the calibration pattern is strongly compared and is easy to extract. Based on the above two advantages, it is appropriate to calibrate the template using a solid circular array in this embodiment.
2. Performing solid circle detection on the calibration template image to extract a circle center point; (corresponding to S2)
In the field of computer vision, such as three-dimensional scene reconstruction, repeated solid circles are often utilized to construct a calibration pattern. The solid circle center has the advantages of easy detection, high position precision, reliable matching, real-time processing and the like. The current circle center detection algorithm comprises the following steps: circle center detection based on Blob area analysis, circle center detection based on edge extraction, circle center detection based on Hough transformation, and the like. In this example, after the calibration image capturing is completed, a solid circle region is detected based on the image gray value, and the homogeneous coordinates [ x y ] of the center of the circle region centroid as the center of the circle pixel position are obtained. The image processing step is simple in calculation and high in instantaneity.
3. Calculating or setting initial values of internal parameters, external parameters (rotation vectors and translation vectors) and distortion parameters of the vehicle-mounted camera; (corresponding to S3)
In computer vision, the interrelation of a point on a spatial object with its projected position on the image plane through an imaging system is generally described by a geometric projection model of a video (or camera) system. A common projection model is central projection in optics based on the principle of aperture imaging. In this model, a point on the object passes through the center of projection, i.e., the optical center of the lens, and is projected on the imaging chip along a straight line.
The homogeneous coordinate of the center of the solid circle in the reference coordinate system is [ X Y Z1 ], and the homogeneous coordinate of the pixel obtained by photographing the point on the camera is [ X Y ] on the assumption. According to the projection model based on the pinhole imaging, the circle center [ X Y ] of the solid circle of the calibration template is projected onto the image according to the following relation to obtain a corresponding imaging pixel [ x y ] (in this case, for the planar calibration template, the Z coordinate is assumed to be 0)
Where σ is the scale factor. The rotation vector R and the translation vector T are external parameters of the camera, and describe the spatial position of the camera in a calibration template coordinate system. K is the internal parameter of the camera, which is defined as
Wherein f x And f y C is the focal length of the lens in the horizontal and vertical directions x C y Is the principal point of the image.
Based on the image of each shot calibration template, the corresponding homography matrix can be calculated
Wherein h is j Is the column vector of the j-th column (j=1, 2, 3), h ij Is the H matrix element of the ith row and jth column (i, j=1, 2, 3). The definition according to homography matrix is:
according to the property of the rotation matrix, r 1 And r 2 Is an orthogonal unit vector, so that there are:
wherein:
from formulas (8) and (9)
h 11 h 12 ·B 11 +(h 31 h 12 +h 11 h 32 )·B 13 +h 21 h 22 ·B 22 +(h 31 h 22 +h 21 h 32 )·B 23 +h 31 h 32 ·B 33 ＝0 (11)
In the production process of the camera, when the optical lens and the imaging chip are coupled and assembled, the main point of the image is required not to exceed a certain pixel range from the center point of the imaging chip. Thus presetting a principal point c x And c y The initial value of (2) is the center of the image. (11) In the expression (12), the homography matrix H can be obtained through the shot calibration template image, and the element H of the homography matrix H ij (i, j=1, 2, 3) is known; c x And c y Is the center point of the image, which is known, B ij Is the focal length f of the internal parameter of the camera x 、f y And principal point c x 、c y Is the intermediate quantity appearing in the calculation process, B ij Of only two unknowns f x And f y So that the two sets of equations (11), (12) can solve for two internal parameters: focal length f x And f y Is set to be a constant value. The initial value of the spatial position T of the vehicle-mounted camera in the calibration template coordinate system canAnd estimating according to the position of the camera relative to the calibration template and the type of the vehicle-mounted camera. Because the optical axis of the vehicle-mounted camera is approximately perpendicular to the calibration template, the initial value of the rotation matrix R is set as an identity matrix
The actual projected pixel will typically have a small offset in the image due to the optical distortion of the lens. The main causes of image distortion are as follows: lens surface machining errors result in defects in the radial curvature; the optical centers of each lens cannot be kept exactly collinear, resulting in decentration errors; due to tolerances in lens design, production and camera assembly, the lens is not parallel to the imaging chip and is tilted. The above errors result in the image being distorted both radially and tangentially. Radial distortion means that the actual image point moves radially inward or outward in its ideal position and optical center line. Tangential distortion means that the actual image point is shifted in a direction perpendicular to the radial direction, i.e. tangential.
Theoretical pixel position [ x y ] based on center projection model]Is affected by distortion and shifts, and its actual projection positionThe following relationship was used to simulate
Wherein [ k ] 1 k 2 k 3 ]Is a radial distortion parameter [ p ] 1 p 2 ]As tangential distortion parameter, r 2 ＝x 2 +y 2 . Radial and tangential distortion parameters of camera lens in general [ k ] 1 k 2 k 3 ]And [ p ] 1 p 2 ]Is set to zero.
4. Optimizing the internal parameters, the external parameters and the distortion parameters of the vehicle-mounted camera to obtain an accurate calibration result; (corresponding to S4)
And generating a corresponding image pixel point in the image when the circle center of each solid circle is photographed. According to the projection model based on the pinhole imaging principle and the distortion model, a theoretical imaging position (including the offset generated by distortion) can be calculated for each circle center. The deviation of the actual image point from the theoretical pixel position is called the reprojection error. The parameters of the geometric projection model, namely the calibration parameters of the camera, should be the least square sum of the re-projection errors of all solid circle centers on the calibration template. The projection model now most accurately describes the optical imaging projection process of the camera at this depth of field, where the imaging projection process is as shown in equation (3) above.
Wherein M is the position of the center of a solid circle in a virtual image simulated by the calibration template through the relay lens, M is the position of the center point corresponding to an actual pixel in the image,the position of the pixel calculated as the position of the heavy projection, i.e. the centre of the solid circle after the central projection, shifted by the distortion of the image (+)>Is->Contains distortion offset), K is the internal parameters of the camera, the rotation vector R and the translation vector T are the external parameters of the camera, k= [ K ] 1 k 2 k 3 ]As radial distortion parameter, p= [ p ] 1 p 2 ]Is a tangential distortion parameter. The equation is optimized by using a Levenberg-Marquardt algorithm, after a plurality of iterations, when the error of the iteration is smaller than a preset threshold value, the optimization iteration is ended, and the obtained results K, R, T, K and p are respectively the internal parameter, the external parameter and the distortion parameter corresponding to the calibration distance. Through the non-passing throughThe calibration parameters of the vehicle-mounted camera obtained through linear optimization are more accurate.
The embodiments of the present invention are merely described in terms of preferred embodiments of the present invention, and are not intended to limit the spirit and scope of the present invention, and various modifications and improvements made by those skilled in the art to the technical solutions of the present invention should fall within the protection scope of the present invention, and the technical content of the present invention as claimed is fully described in the claims.
Claims (13)
1. The production line calibration method for the single-image strip relay lens of the vehicle-mounted camera is characterized by constructing a calibration system comprising a calibration template, a relay lens and a vehicle-mounted camera to be calibrated, wherein the calibration system is suitable for vehicle-mounted cameras with different focal lengths by adjusting the distance between the relay lens and the calibration template, the calibration template is provided with a solid circular array, and the vehicle-mounted camera shoots the calibration template through the relay lens, and the method comprises the following specific steps of:
s1, acquiring a virtual image formed by a calibration template through a relay lens by a vehicle-mounted camera as a calibration template image; the distance from the relay lens to the calibration template is set according to the target focusing distance to be simulated by the vehicle-mounted camera and the focal length of the relay lens;
s2, extracting pixel position coordinates of the circle center of the solid circle in the calibration template image by using an image processing algorithm;
s3, calculating initial values of calibration parameters of the vehicle-mounted camera, including internal parameters, external parameters and distortion parameters, based on the calibration template image and combining a homography matrix and solid circle center position coordinates of the calibration template image;
and S4, optimizing the initial values of the internal and external parameters and the distortion parameters of the vehicle-mounted camera obtained in the step S3 to obtain accurate calibration parameters of the vehicle-mounted camera.
2. The production line calibration method of the single-board single-image strip relay lens of the vehicle-mounted camera according to claim 1, wherein in the step S1, the optical axes of the vehicle-mounted camera and the relay lens are kept parallel and concentric, and the optical axes of the vehicle-mounted camera and the relay lens are kept perpendicular to the calibration template.
3. The production line calibration method of the single-image strip relay lens of the vehicle-mounted camera single-plate of claim 1, wherein in step S2, solid circle center feature points of the calibration template image are extracted through binarization and median filtering methods, and pixel position coordinates of solid circle centers in the calibration template image are obtained.
4. The production line calibration method of the single-image strip relay lens of the vehicle-mounted camera single-plate of claim 1, wherein in step S2, a solid circle area in the calibration template image is detected through an image gray value, and the center of mass of the area is calculated to obtain the pixel position coordinates of the center of the solid circle in the calibration template image.
5. The production line calibration method of the single-board single-image strip relay lens for the vehicle-mounted camera according to claim 1, wherein in the step S3, according to the homography matrix of the calibration template image, the pixel position of the center of a solid circle in the calibration template image is combined, and an internal parameter initial value of the vehicle-mounted camera is calculated.
6. The production line calibration method of the single-board single-image strip relay lens for the vehicle-mounted camera according to claim 1, wherein in step S3, an initial value of an external parameter rotation matrix of the vehicle-mounted camera is an identity matrix.
7. The production line calibration method of the single-board single-image strip relay lens for the vehicle-mounted camera according to claim 1, wherein in step S3, an initial value of a distortion parameter of the vehicle-mounted camera is zero.
8. The production line calibration method of the single-board single-image strip relay lens for the vehicle-mounted camera according to claim 1, wherein in the step S4, initial values of internal parameters, external parameters and distortion parameters of the vehicle-mounted camera are optimized based on a principle that the sum of squares of re-projection errors of solid circle centers of the calibration template images is minimized.
9. The production line calibration method of the single-image strip relay lens of the vehicle-mounted camera according to claim 1 or 8, wherein in step S4, initial values of internal and external parameters and distortion parameters of the vehicle-mounted camera are optimized by using a Levenberg-Marquardt optimization algorithm.
10. The production line calibration device with the relay lens on the single board of the vehicle-mounted camera is characterized by comprising the relay lens, a calibration template and the vehicle-mounted camera to be calibrated, wherein the relay lens is arranged between a lens of the vehicle-mounted camera and the calibration template, the distance between the relay lens and the calibration template is adjustable, and the calibration template pattern is a solid circle array; and setting the distance from the relay lens to the calibration template according to the target focusing distance to be simulated by the vehicle-mounted camera and the focal length of the relay lens.
11. The production line calibration device for the single-board single-image strip relay lens of the vehicle-mounted camera according to claim 10, wherein the optical axes of the relay lens and the vehicle-mounted camera are arranged in a superposition mode, and the optical axes of the relay lens and the vehicle-mounted camera are perpendicular to the surface of the calibration template.
12. The production line calibration device for the single-image strip relay lens of the vehicle-mounted camera single plate of claim 10, wherein the solid circle array on the surface of the calibration template is a row-column array formed by a plurality of solid circles, the radiuses of the solid circles are the same, and the circle center distances of the adjacent solid circles in the horizontal and vertical directions are the same.
13. The production line calibration device with the relay lens for the single board of the vehicle-mounted camera according to claim 10, further comprising an extraction unit, a calibration unit and an optimization unit which are constructed in a computer, wherein the extraction unit extracts pixel position coordinates of the center of a solid circle in the calibration template image, the calibration unit calculates initial values of calibration parameters of the vehicle-mounted camera including internal parameters, external parameters and distortion parameters, and the optimization unit optimizes the internal parameters, the external parameters and the distortion parameters of the vehicle-mounted camera to obtain accurate calibration parameters of the vehicle-mounted camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011521340.0A CN112489141B (en) | 2020-12-21 | 2020-12-21 | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011521340.0A CN112489141B (en) | 2020-12-21 | 2020-12-21 | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112489141A CN112489141A (en) | 2021-03-12 |
| CN112489141B true CN112489141B (en) | 2024-01-30 |
Family
ID=74915173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011521340.0A Active CN112489141B (en) | 2020-12-21 | 2020-12-21 | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112489141B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115170674B (en) * | 2022-07-20 | 2023-04-14 | 禾多科技（北京）有限公司 | Camera principal point calibration method, device, equipment and medium based on single image |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109360245A (en) * | 2018-10-26 | 2019-02-19 | 魔视智能科技(上海)有限公司 | The external parameters calibration method of automatic driving vehicle multicamera system |
| CN109543660A (en) * | 2018-12-20 | 2019-03-29 | 深圳奥比中光科技有限公司 | A kind of focus control and focus adjustment method |
| CN109767476A (en) * | 2019-01-08 | 2019-05-17 | 像工场(深圳)科技有限公司 | A kind of calibration of auto-focusing binocular camera and depth computing method |
| CN110274752A (en) * | 2018-03-14 | 2019-09-24 | 深圳市隆测技术有限公司 | The Multifunctional test card and its test method of relay lens image quality |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109816733B (en) * | 2019-01-14 | 2023-08-18 | 京东方科技集团股份有限公司 | Camera parameter initialization method and device, camera parameter calibration method and device and image acquisition system |
-
2020
- 2020-12-21 CN CN202011521340.0A patent/CN112489141B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110274752A (en) * | 2018-03-14 | 2019-09-24 | 深圳市隆测技术有限公司 | The Multifunctional test card and its test method of relay lens image quality |
| CN109360245A (en) * | 2018-10-26 | 2019-02-19 | 魔视智能科技(上海)有限公司 | The external parameters calibration method of automatic driving vehicle multicamera system |
| CN109543660A (en) * | 2018-12-20 | 2019-03-29 | 深圳奥比中光科技有限公司 | A kind of focus control and focus adjustment method |
| CN109767476A (en) * | 2019-01-08 | 2019-05-17 | 像工场(深圳)科技有限公司 | A kind of calibration of auto-focusing binocular camera and depth computing method |
Non-Patent Citations (1)
| Title |
|---|
| 基于单平面模板的摄像机定标研究;杨敏, 沈春林;数据采集与处理(01);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112489141A (en) | 2021-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| CN109767476B (en) | Automatic focusing binocular camera calibration and depth calculation method | |
| CN110276808B (en) | Method for measuring unevenness of glass plate by combining single camera with two-dimensional code | |
| CN109598762B (en) | High-precision binocular camera calibration method | |
| CN105716542B (en) | A kind of three-dimensional data joining method based on flexible characteristic point | |
| CN110517325B (en) | Coordinate transformation and method and system for positioning objects around vehicle body through coordinate transformation | |
| CN111080705B (en) | Calibration method and device for automatic focusing binocular camera | |
| CN111667536A (en) | Parameter calibration method based on zoom camera depth estimation | |
| CN102567989A (en) | Space positioning method based on binocular stereo vision | |
| CN109272555B (en) | External parameter obtaining and calibrating method for RGB-D camera | |
| CN110874854B (en) | Camera binocular photogrammetry method based on small baseline condition | |
| CN110827360B (en) | Photometric stereo measurement system and method for calibrating light source direction thereof | |
| CN109974618B (en) | Global calibration method of multi-sensor vision measurement system | |
| CN112489137A (en) | RGBD camera calibration method and system | |
| CN113119129A (en) | Monocular distance measurement positioning method based on standard ball | |
| CN110763204A (en) | Planar coding target and pose measurement method thereof | |
| CN109544642B (en) | N-type target-based TDI-CCD camera parameter calibration method | |
| CN114359405A (en) | Calibration method of off-axis Samm 3D line laser camera | |
| CN116740187A (en) | Multi-camera combined calibration method without overlapping view fields | |
| CN107067441B (en) | Camera calibration method and device | |
| CN113963067B (en) | Calibration method for calibrating large-view-field visual sensor by using small target | |
| CN112489141B (en) | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera | |
| CN113781581B (en) | Depth of field distortion model calibration method based on target loose attitude constraint | |
| CN109242911A (en) | One kind being based on subregional binocular camera fundamental matrix calculation method | |
| CN115841517A (en) | Structural light calibration method and device based on DIC double-circle cross ratio |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |