CN114998447A - Multi-view vision calibration method and system - Google Patents
Multi-view vision calibration method and system Download PDFInfo
- Publication number
- CN114998447A CN114998447A CN202210612289.7A CN202210612289A CN114998447A CN 114998447 A CN114998447 A CN 114998447A CN 202210612289 A CN202210612289 A CN 202210612289A CN 114998447 A CN114998447 A CN 114998447A
- Authority
- CN
- China
- Prior art keywords
- camera
- calibration
- cameras
- pose
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Abstract
The invention discloses a multi-view vision calibration method and a multi-view vision calibration system, which solve the problem that a multi-camera system visual field cannot cover a two-dimensional calibration object; the method numbers the cameras to be 0-N, and the calibration steps are as follows: firstly, the camera 0 and the camera 1 observe a complete calibration plate at the same time, and the relative pose of the two cameras is calculated by extracting the mark points to obtain a transformation matrix of the relative pose. Secondly, the camera 1 and the camera 2 acquire complete calibration plate pictures, and the relative poses of the camera 1 and the camera 2 are calculated to obtain a relative pose matrix of the other camera. And calculating the rigid transformation relation of the two pose matrixes to obtain the relative pose transformation relation of the camera 2 and the camera 0. By analogy, the pose relationship of the cameras 0-N can be obtained. The invention can simplify the calibration method of multi-view vision, and carry out calibration when the camera observes different calibration objects; and calculating a pose matrix of the plurality of cameras according to the relative poses, and unifying the pose matrix under a unified coordinate system.
Description
Technical Field
The invention relates to the field of multi-view vision, in particular to a multi-view vision calibration method and system.
Background
With the continuous development of three-dimensional measurement technology and multi-view vision technology, the traditional single-camera measurement can not meet the requirements of industrial production, and the multi-view vision measurement becomes a research hotspot. The method is characterized in that multi-view vision is used for three-dimensional measurement, the first step is calibration of a camera, and only by calibrating the camera, the camera can be connected with a world coordinate system, so that three-dimensional measurement is carried out.
The existing camera calibration mostly uses a two-dimensional calibration object for calibration, for a multi-view vision system, the two-dimensional calibration object can not be observed by all cameras at times, the internal and external parameters of all cameras are calculated by generally adopting a method of calibrating the cameras one by one, and along with the increase of the number of the cameras, calibration pictures needing to be collected are multiplied, and the workload is overlarge.
Disclosure of Invention
The invention aims to provide a multi-view vision calibration method and a multi-view vision calibration system.
The technical solution for realizing the purpose of the invention is as follows: in a first aspect, the present invention provides a multi-view vision calibration method, including:
building a multi-camera system, using a calibration plate with known parameters of the mark points, and enabling the visual fields of at least two cameras to simultaneously cover one calibration plate image;
finding out the pixel coordinates of the mark points through an image morphological correlation algorithm, transforming the pose of the calibration plate, acquiring different calibration images to solve an optimal solution, and solving internal and external parameters of the camera, including a focal length, a distortion coefficient, a scale factor and a rigid transformation matrix of relative pose;
and setting a reference camera by utilizing the solved rigid transformation matrix, and unifying the cameras into the same coordinate system through calculation.
In one of the embodiments, the single camera field of view of the multi-camera system can cover the entire calibration plate.
In one embodiment, in the multi-camera system, adjacent cameras at least observe a calibration image at the same time, and the relative poses of the adjacent cameras are guaranteed to be calculated through the calibration image.
In one embodiment, the pixel coordinates of the mark points are found out through an image morphology correlation algorithm, the position and the posture of the calibration plate are transformed, different calibration images are collected so as to solve the optimal solution, and internal and external parameters of the camera, including focal length, distortion coefficient, scale factor and a rigid transformation matrix of relative position and posture, are solved; the method comprises the following specific steps:
step S21, extracting the mark point and calculating its pixel coordinate, according to the formula:
wherein (X) c ,Y c ,Z c ) Denotes the pixel coordinate, (X) w ,Y w ,Z w ) Representing world coordinates, R being a 3x3 orthogonal rotation matrix, t being a three-dimensional translation vector, O T Calculating the values of R and t for 0 vector matrixes with corresponding sizes;
step S22: carrying out image acquisition for a plurality of times on the calibration plate, solving the optimal solution d (theta) of the camera parameters through the following formula,
wherein the coordinate of the center of the dot obtained by the calibration plate is M j Extracting the center coordinates of the camera calibration image, and recording the first camera as m j,k,1 And the second camera is marked as m j,k,2 The Nth camera is denoted as m j,k,l The parameters of the two cameras, represented simultaneously by θ, include the internal reference and relative position of the two cameras, where π i Representing the projection of a certain landmark point of the calibration plate onto the image plane coordinate system, theta i,l Is a subset of the camera parameters in camera i that affect this projection; pi r Indicating a correction of a point in the image to the image plane coordinate system, theta r,l Represents a subset of the parameters in camera i that affect this correction; v. of j,k,l Indicating whether this point is valid; n is 0 Obtaining the number of the calibration pictures; n is m The number of the mark points;
step S23: setting any one camera as a reference camera, namely the reference camera is a coordinate origin, and calculating internal and external parameters of the camera;
in one embodiment, a reference camera is set by using a solved rigid transformation matrix, and the cameras are unified into the same coordinate system through calculation, specifically:
step S31: in step S22, the adjacent cameras observe the world coordinate point P of the calibration board in the same pose w (x,y,z,1) T Is then adjacent toThe transformation relationship of the two cameras is as follows:
wherein
And
solving related parameters for the world coordinate systems of the calibration plates of different cameras observing the same pose to obtain a transformation relation of the relative poses, wherein R is a rotation matrix, and t is a translation increment;
step S32: and further calculating the position and posture relations of all the cameras in the system according to the calculated relative position and posture relations of the different cameras in the S31.
In a second aspect, the present invention provides a multi-view vision calibration system, comprising:
the system comprises a multi-camera system, cameras are numbered from 0 to N, a calibration plate with known parameters of mark points is used, and the visual fields of at least two cameras can simultaneously cover an image of the calibration plate;
the calibration module finds out the pixel coordinates of the mark points through an image morphology correlation algorithm, transforms the position and the posture of a calibration plate, acquires different calibration images to solve an optimal solution, and solves the internal and external parameters of the camera, including a focal length, a distortion coefficient, a scale factor and a rigid transformation matrix of relative position and posture; and setting a reference camera by utilizing the solved rigid transformation matrix, and unifying the cameras into the same coordinate system through calculation.
In a third aspect, the present invention provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of the first aspect when executing the program.
In a fourth aspect, the invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of the first aspect.
Compared with the prior art, the invention has the following remarkable advantages:
(1) the invention can simplify the calibration method of multi-view vision, and carry out calibration when the camera observes different calibration objects; (2) calculating a pose matrix of the cameras according to the relative poses, and unifying the pose matrix under a unified coordinate system;
drawings
Fig. 1 is a diagram of the calibration relationship of a multi-camera system to which the present invention is directed.
Fig. 2 is a flowchart of the multi-view vision calibration method of the present invention.
Fig. 3 is a calibration plate according to an embodiment of the present invention, which is a dot matrix coplanar circle calibration plate.
Detailed Description
The method of the present invention will be described in detail below with reference to fig. 1 to 2. The described method is part of a multi-camera system, not all. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a diagram of calibration of a multi-camera system, taking a multi-view vision system composed of five cameras as an example, the cameras are numbered 0-4, and solid arrows indicate directions in which the fields of view of the cameras point.
S1, a multi-camera system is built, the number of the cameras is 0-N, a calibration plate with obvious mark points and known parameters is used, and meanwhile, the visual fields of at least two cameras can cover one calibration plate image at the same time, so that the cameras are guaranteed to have the position relation shown in the figure 1. The multi-camera system satisfies the following conditions:
1) the calibration plate must have distinct marking points, the position coordinates of which are known. Such as a spandrel calibration plate or a coplanar circular calibration plate. The calibration plate in this embodiment is shown in fig. 3.
2) The single camera view of the multi-camera system is constructed to cover the entire calibration plate while the marker points are clearly visible in the view.
3) After the image is collected, image preprocessing is carried out, and the mark point position can be extracted through an image morphology method in the subsequent process.
4) For a multi-camera system, adjacent cameras at least observe a calibration image at the same time, and the relative pose of the adjacent cameras is guaranteed to be calculated through the calibration image.
In the present invention, the multi-camera system is constructed to have a "chain" relationship, wherein the chain relationship means that at least two adjacent cameras can observe the calibration plate at the same time. For example, cameras 0-2 in fig. 1, three cameras simultaneously take a calibration picture, wherein the number of mark points of the calibration picture is greater than or equal to 6, the mark points are obvious, the center position of a dot in the calibration picture, the size of the calibration plate and the position of the dot are known, and the plane coordinate of the calibration plate in a world coordinate system is (X) coordinate w ,Y w 1), the pixel coordinates of the captured image are (u, v, 1).
S2, finding out the pixel coordinates of the mark points through an image morphology correlation algorithm, transforming the position and posture of the calibration plate, acquiring different calibration images to obtain the optimal solution, and solving the internal and external parameters of the camera, including focal length, distortion coefficient, scale factor, rigid transformation matrix of relative position and posture and the like.
In the invention, the cameras 0-2 in fig. 1 can simultaneously acquire an image of a certain pose of the calibration plate, that is, the calibration plate with complete and clear mark points is in the image of the cameras 0-2, and a formula is provided according to the relationship between the pixel coordinate system and the world coordinate system:
wherein:
r 1 ,r 2 for a rotation matrix, T is a translation variable, Z c To calibrate the distance between the plane and the camera, a x ,a y Scale factors for the horizontal and vertical axes of the image, respectively. u. of 0 、v 0 For image coordinate system origin at pixel locationCoordinates in the coordinate system. The parameters of K only include focal length, principal point coordinates, etc., and are determined by the internal structure of the camera, so K is called an internal parameter matrix. H is a homography matrix formed by the product of the internal parameter and the external parameter. Simultaneously, r is calculated according to the relation of the internal and external parameters of the camera 1 ,r 2 And T, i.e., the relative pose matrix of the camera.
Let θ be (f, k, s) the camera parameter extracted from the picture x ,s y ,c x ,c y ,α,β,γ,t x ,t y ,t z ) Wherein f, k, s x ,s y ,c x ,c y The parameters are respectively focal length, distortion coefficient, x-axis scaling factor, y-axis scaling factor, distortion center x-axis coordinate and distortion center y-axis coordinate. α, β, γ, t x ,t y ,t z The parameters are respectively an x-axis rotation angle, a y-axis rotation angle, a z-axis rotation angle, an x-axis translation component, a y-axis translation component and a z-axis translation component. Let us assume here the projection of the world coordinate system onto the image coordinate system to be π i (M j ,θ i ) Wherein M is j Representing three-dimensional coordinates in the world coordinate system, let m j Coordinates of the center point (index point) of the solid circle of the plate in the image coordinate system are calibrated. By calculating m j And pi calculated by projection i (M j ,θ i ) The optimal solution of the distance between the two to determine the parameters of the camera.
Wherein n is m To scale the number of marked points on the board, v i Indicating whether the mark point is visible, which is 1, otherwise, it is 0. The number of the above equations needing optimization is too large, and meanwhile, the initial value setting is not ideal, so that the method easily falls into a local optimal solution. According to the characteristics of the optimal solution, if a better initial value can be set, the optimal solution can be quickly obtained and cannot fall into local optimization. So subsequent calibration requires reading certain parameters, typically f, s, from the camera specifications x s, y Because these three parameters are relatively fixed, they depend on the cameraThe process level of (1).
S3, setting a reference camera by using the rigid transformation matrix solved in S2, and unifying the cameras into the same coordinate system through calculation
In the invention, the coordinate of the center of the mark point in the world coordinate system is set as P w (x,y,z,1) T And the same center coordinates of the mark points are as in camera 0
The coordinates in the camera 1 are
The two are obtained by calibration, and the transformation of the relative poses of the two cameras is still a rigid transformation.
The rigidity of both transforms into:
also, there are:
write the above formula as:
further, the camera parameters, i.e., R and t in the above formula, are known from the previous calibration, and camera 0 is set as the reference coordinate, i.e., R and t in the above formula
Can find out
The pose of the camera 3 with respect to the camera 0 can be found by the above method.
For the camera 2 and the camera 1, the step of S2 is repeated to acquire the transformation relation of the relative pose matrix
Write the above formula
The relative pose relationship of the cameras 1 and 2 can be obtained. And then the relative pose relationship between the camera 2 and the camera 0 is obtained.
The relative position relationship between the camera 4 and the camera 0 can be obtained by repeating the above steps.
So far, all cameras are unified to the coordinate system where the camera 0 is located, and the camera calibration of the example is completed.
It should be understood that the present example illustrates the calibration of 5 cameras, and that calibration for more cameras is equally feasible by this method.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A multi-view vision calibration method is characterized by comprising the following steps:
building a multi-camera system, using a calibration plate with known parameters of the mark points, and enabling the visual fields of at least two cameras to simultaneously cover one calibration plate image;
finding out the pixel coordinates of the mark points through an image morphological correlation algorithm, transforming the pose of the calibration plate, acquiring different calibration images to obtain an optimal solution, and solving internal and external parameters of the camera, including a focal length, a distortion coefficient, a scale factor and a rigid transformation matrix of relative pose;
and setting a reference camera by utilizing the solved rigid transformation matrix, and unifying the cameras into the same coordinate system through calculation.
2. The multi-purpose visual calibration method according to claim 1, wherein a single camera view of the multi-camera system can cover the entire calibration plate.
3. The method for calibrating multi-vision according to claim 1, wherein in the multi-camera system, at least one calibration image is observed by adjacent cameras at the same time, so as to ensure that the relative pose of the adjacent cameras is calculated by the calibration images.
4. The multi-view vision calibration method according to claim 1, wherein the pixel coordinates of the mark points are found out through an image morphology correlation algorithm, the pose of the calibration plate is transformed, different calibration images are collected to solve the optimal solution, and internal and external parameters of the camera, including a focal length, a distortion coefficient, a scale factor and a rigid transformation matrix of the relative pose, are solved; the method comprises the following specific steps:
step S21, extracting the mark point and calculating its pixel coordinate, according to the formula:
wherein (X) c ,Y c ,Z c ) Denotes the pixel coordinate, (X) w ,Y w ,Z w ) Representing world coordinates, R being a 3x3 orthogonal rotation matrix, t being a three-dimensional translation vector, O T Calculating the values of R and t for 0 vector matrixes with corresponding sizes;
step S22: carrying out image acquisition for a plurality of times on the calibration plate, solving the optimal solution d (theta) of the camera parameters through the following formula,
wherein the coordinate of the center of the dot obtained by the calibration plate is M j Extracting camera calibration imagesCenter coordinates, first camera as m j,k,1 And the second camera is marked as m j,k,2 The Nth camera is denoted as m j,k,l The parameters of the two cameras, represented simultaneously by theta, include the internal reference and relative position of the two cameras, where pi i Representing the projection of a certain landmark point of the calibration plate onto the image plane coordinate system, theta i,l Is a subset of the camera parameters in camera i that affect this projection; pi r Representing a correction of a point in the image to the image plane coordinate system, theta r,l Represents a subset of the parameters in camera i that affect this correction; v. of j,k,l Indicating whether this point is valid; n is 0 Obtaining the number of the calibration pictures; n is m The number of the mark points;
step S23: and setting any one camera as a reference camera, namely a coordinate origin, and calculating internal and external parameters of the camera.
5. The multi-view vision calibration method according to claim 4, wherein the solved rigid transformation matrix is used to set a reference camera, and the cameras are unified into the same coordinate system through calculation, specifically:
step S31: in step S22, the adjacent cameras observe the world coordinate point P of the calibration board in the same pose w (x,y,z,1) T Then, the transformation relationship between two adjacent cameras is:
wherein
And
solving related parameters for the world coordinate systems of the calibration plates of different cameras observing the same pose to obtain a transformation relation of the relative pose, wherein R is a rotation matrix, and t is a translation increment;
step S32: and further calculating the position and posture relations of all the cameras in the system according to the calculated relative position and posture relations of the different cameras in the S31.
6. A multi-view vision calibration system, comprising:
the system comprises a multi-camera system, cameras are numbered from 0 to N, a calibration plate with known parameters of mark points is used, and the visual fields of at least two cameras can simultaneously cover an image of the calibration plate;
the calibration module finds out the pixel coordinates of the mark points through an image morphology correlation algorithm, transforms the position and the posture of a calibration plate, acquires different calibration images to solve an optimal solution, and solves the internal and external parameters of the camera, including a focal length, a distortion coefficient, a scale factor and a rigid transformation matrix of relative position and posture; and setting a reference camera by utilizing the solved rigid transformation matrix, and unifying the cameras into the same coordinate system through calculation.
7. The multi-purpose visual calibration system according to claim 6, wherein the single camera view of the multi-camera system can cover the entire calibration plate.
8. The system of claim 6, wherein in the multi-camera system, at least one calibration image is observed by adjacent cameras at the same time, so that the relative pose of the adjacent cameras is calculated from the calibration images.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of claims 1-5 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210612289.7A CN114998447A (en) | 2022-05-31 | 2022-05-31 | Multi-view vision calibration method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210612289.7A CN114998447A (en) | 2022-05-31 | 2022-05-31 | Multi-view vision calibration method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114998447A true CN114998447A (en) | 2022-09-02 |
Family
ID=83031000
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210612289.7A Pending CN114998447A (en) | 2022-05-31 | 2022-05-31 | Multi-view vision calibration method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114998447A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115641382A (en) * | 2022-10-21 | 2023-01-24 | 哈尔滨工业大学 | External parameter calibration method for orthogonal stereoscopic vision structure |
| CN116934871A (en) * | 2023-07-27 | 2023-10-24 | 湖南视比特机器人有限公司 | Multi-objective system calibration method, system and storage medium based on calibration object |
-
2022
- 2022-05-31 CN CN202210612289.7A patent/CN114998447A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115641382A (en) * | 2022-10-21 | 2023-01-24 | 哈尔滨工业大学 | External parameter calibration method for orthogonal stereoscopic vision structure |
| CN115641382B (en) * | 2022-10-21 | 2023-09-08 | 哈尔滨工业大学 | External parameter calibration method for orthogonal stereoscopic vision structure |
| CN116934871A (en) * | 2023-07-27 | 2023-10-24 | 湖南视比特机器人有限公司 | Multi-objective system calibration method, system and storage medium based on calibration object |
| CN116934871B (en) * | 2023-07-27 | 2024-03-26 | 湖南视比特机器人有限公司 | Multi-objective system calibration method, system and storage medium based on calibration object |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI555379B (en) | An image calibrating, composing and depth rebuilding method of a panoramic fish-eye camera and a system thereof | |
| CN206441238U (en) | Scaling board and calibration system | |
| CN107886547B (en) | Fisheye camera calibration method and system | |
| KR101666959B1 (en) | Image processing apparatus having a function for automatically correcting image acquired from the camera and method therefor | |
| WO2021004416A1 (en) | Method and apparatus for establishing beacon map on basis of visual beacons | |
| WO2018209968A1 (en) | Camera calibration method and system | |
| WO2021043213A1 (en) | Calibration method, device, aerial photography device, and storage medium | |
| CN114998447A (en) | Multi-view vision calibration method and system | |
| CN105608671A (en) | Image connection method based on SURF algorithm | |
| CN109920004B (en) | Image processing method, device, calibration object combination, terminal equipment and calibration system | |
| CN104760812B (en) | Product real-time positioning system and method on conveyer belt based on monocular vision | |
| CN110717942A (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN112132908B (en) | Camera external parameter calibration method and device based on intelligent detection technology | |
| CN113329179B (en) | Shooting alignment method, device, equipment and storage medium | |
| CN110779491A (en) | Method, device and equipment for measuring distance of target on horizontal plane and storage medium | |
| CN102156986A (en) | Calibration method of near-infrared camera | |
| CN112132874B (en) | Calibration-plate-free heterogeneous image registration method and device, electronic equipment and storage medium | |
| CN103679693A (en) | Multi-camera single-view calibration device and calibration method thereof | |
| WO2023273108A1 (en) | Monocular distance measurement method and apparatus, and intelligent apparatus | |
| CA3233222A1 (en) | Method, apparatus and device for photogrammetry, and storage medium | |
| CN112470192A (en) | Dual-camera calibration method, electronic device and computer-readable storage medium | |
| CN113554709A (en) | Camera-projector system calibration method based on polarization information | |
| CN113112545B (en) | Handheld mobile printing device positioning method based on computer vision | |
| CN112950528A (en) | Certificate posture determining method, model training method, device, server and medium | |
| CN110223356A (en) | A kind of monocular camera full automatic calibration method based on energy growth |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |