CN110827358B - Camera calibration method applied to automatic driving automobile - Google Patents
Camera calibration method applied to automatic driving automobile Download PDFInfo
- Publication number
- CN110827358B CN110827358B CN201910976445.6A CN201910976445A CN110827358B CN 110827358 B CN110827358 B CN 110827358B CN 201910976445 A CN201910976445 A CN 201910976445A CN 110827358 B CN110827358 B CN 110827358B
- Authority
- CN
- China
- Prior art keywords
- camera
- image
- calibration plate
- laser radar
- corner points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 239000011159 matrix material Substances 0.000 claims abstract description 30
- 230000003068 static effect Effects 0.000 claims description 3
- 230000003287 optical effect Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 7
- 230000004927 fusion Effects 0.000 abstract description 7
- 238000007499 fusion processing Methods 0.000 abstract 1
- 230000009466 transformation Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
A camera calibration method applied to an automatic driving automobile belongs to the field of automatic driving of automobiles. The fusion technology among multiple sensors involved in the existing automobile automatic driving technology is insufficient. The invention relates to a camera calibration method applied to an automatic driving automobile, wherein a calibration plate is arranged in front of the automatic driving automobile, an image of the calibration plate is obtained through a camera, and image coordinates of four corner points of the image of the calibration plate are obtained through lableme software; measuring physical coordinates of corner points of the calibration plate, combining the coordinates of the image, and calculating a rotation translation matrix for projecting the three-dimensional space to the two-dimensional projection position by utilizing a pnp algorithm; and projecting the three-dimensional point cloud space of the laser radar onto a two-dimensional image of the camera by using the calculated rotation translation matrix, so as to realize joint calibration of the camera and the laser radar. The invention has the advantages of simple fusion process and accurate fusion result.
Description
Technical Field
The invention relates to a camera calibration method applied to an automatic driving automobile.
Background
Because of the advantages of reducing traffic accidents, reducing labor cost and the like, the automatic driving automobile is currently a popular research field worldwide. And as various sensor costs decrease and technologies develop, the autopilot technology is also gradually tending to the commercial field. Vehicle-mounted sensors currently in common use in the autopilot field include cameras (cameras), lidar (lidar), millimeter wave radar, and the like. Because of the advantages and disadvantages of single sensors, the mainstream technology generally adopts a multi-sensor fusion scheme in order to improve the reliability and stability of automatic driving.
Before multi-sensor fusion, calibration is performed on each sensor. Calibration is that when the sensors work cooperatively, a unified coordinate system is needed, and external parameters of the sensors, namely a rotation translation transformation matrix, need to be estimated. Aiming at the problem of camera calibration, a calibration method based on a calibration plate (a square plank with the length of 2 meters multiplied by 2 meters) is provided, and the calibration of the camera can be realized by extracting the image coordinates of four corner points of the calibration plate, measuring the physical coordinates of the four corner points in a vehicle body coordinate system, and then performing coordinate transformation by using a pnp algorithm.
Disclosure of Invention
The invention aims to solve the defects of the fusion technology among multiple sensors in the existing automobile automatic driving technology, and provides a camera calibration method applied to an automatic driving automobile.
A camera calibration method for an autopilot vehicle, the method comprising the steps of:
firstly, setting a calibration plate in front of an automatic driving vehicle, acquiring an image of the calibration plate through a camera, and acquiring image coordinates of four corner points of the image of the calibration plate by using lableme software;
measuring physical coordinates of the corner points of the calibration plate in a vehicle body coordinate system, and calculating a rotation translation matrix from the camera coordinate system to the vehicle body coordinate system by utilizing a pnp algorithm in combination with the image coordinates of the first step;
and thirdly, calculating and extracting three-dimensional data of four corner points of a calibration plate in the laser radar, calculating a rotation translation matrix from the camera to the laser radar by utilizing a pnp algorithm under a laser radar coordinate system, and projecting three-dimensional point cloud data of the laser radar onto a two-dimensional image of the camera by utilizing the rotation translation matrix so as to realize joint calibration of the camera and the laser radar.
The beneficial effects of the invention are as follows:
the camera calibration method applied to the automatic driving automobile has the advantages of being simple in operation and accurate in fusion result between sensors. Specifically, by means of a square plank, the physical coordinates of the four corner points in a car body coordinate system are measured by extracting the image coordinates of the four corner points of the calibration plank, and then the pnp algorithm is used for carrying out coordinate conversion, so that the calibration of the camera is realized.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The first embodiment is as follows:
the flow of the method is shown in fig. 1, and the method is realized by the following steps:
firstly, setting a calibration plate in front of an automatic driving vehicle in a static state, acquiring an image of the calibration plate through a camera, and acquiring image coordinates of four corner points of the image of the calibration plate by using lableme software;
measuring physical coordinates of the corner points of the calibration plate in a vehicle body coordinate system, and calculating a rotation translation matrix from the camera coordinate system to the vehicle body coordinate system by utilizing a pnp (permanent-n-Point) algorithm in combination with the image coordinates of the first step;
and thirdly, calculating and extracting three-dimensional data of four corner points of a calibration plate in the laser radar, calculating a rotation translation matrix from the camera to the laser radar by utilizing a pnp algorithm under a laser radar coordinate system, and projecting three-dimensional point cloud data of the laser radar onto a two-dimensional image of the camera by utilizing the rotation translation matrix so as to realize joint calibration of the camera and the laser radar.
The second embodiment is as follows:
in a first difference from the specific embodiment, in the camera calibration method for an automatic driving vehicle according to the present embodiment, a calibration board is disposed in front of the automatic driving vehicle in a stationary state, an image of the calibration board is obtained by the camera, and the process of obtaining the image coordinates of four corner points of the image of the calibration board by using lableme software is that,
step 1, acquiring an image of a calibration plate:
setting a calibration plate perpendicular to the optical axis of the camera in front of the camera of the automatic driving automobile, wherein the distance between the calibration plate and the camera is about 8-15 m, starting the camera to drive and photograph, and obtaining an image of the calibration plate;
step one, 2, obtaining image coordinates of corner points in the calibration plate image:
and (3) sequentially and manually marking four corner points of the calibration plate image by using third-party open source software lableme software, storing, and generating a json file corresponding to the image coordinates containing the corner points by the software through the storage operation, so as to obtain the image coordinates of the corner points in the calibration plate image. The invention acquires the image coordinates of the corner points by using the lableme, so that the image coordinates can be ensured to be more accurate. Under the influence of factors such as illumination and weather, if the angular points are automatically identified by directly using an algorithm, errors are increased, and therefore, the image coordinates of the angular points cannot be accurately acquired.
And a third specific embodiment:
unlike the first or second embodiment, in the camera calibration method for an automatic driving vehicle according to the present embodiment, the step two of calculating the rotational translation matrix from the camera coordinate system to the vehicle body coordinate system by using PnP (transparent-n-Point) algorithm is that,
step two, 1, obtaining physical coordinates of corner points:
measuring and calculating physical coordinates of four corner points of the calibration plate under a world coordinate system by using auxiliary tools such as a tape measure, a plumb line, a laser level meter and the like;
step 2, calculating a rotation translation matrix:
after obtaining the image coordinates of the four corner points of the calibration plate and the physical coordinates under the vehicle body coordinate system, calculating a rotation translation matrix from the camera coordinate system to the vehicle body coordinate system through a library function solvePnP of OpenCV; the working principle of the PnP algorithm is that the PnP algorithm is used for estimating the pose of a camera when n three-dimensional space points and two-dimensional projection positions of the three-dimensional space points are known.
Namely, solving a rotation matrix R and a translation matrix t corresponding to the pixel coordinate point and the physical coordinate point by using a PnP algorithm, and completing camera calibration after R and t are obtained.
The specific embodiment IV is as follows:
unlike the third embodiment, in the third embodiment, a camera calibration method applied to an automatic driving automobile is provided, in the third step, the combined calibration process of the camera and the laser radar is that,
analyzing point cloud data of a calibration plate of the laser radar, and acquiring three-dimensional coordinates of corner points of the calibration plate under a laser radar coordinate system;
acquiring image coordinates of four corner points of a calibration plate by using lableme in the first step, then calling library function solvePnP in OpenCV, and transmitting the image coordinates of a corner point camera and the three-dimensional coordinates of a laser radar into the library function solvePnP of the OpenCV to obtain a rotation translation matrix of the rotation translation matrix;
and finally, projecting the three-dimensional point cloud of the laser radar onto the two-dimensional image of the camera by using a rotation translation matrix, taking the rotation translation matrix as an external parameter among the sensors, projecting the three-dimensional point cloud data of the laser radar onto the two-dimensional image of the camera, and completing the joint calibration of the camera and the laser radar, thereby preparing for fusion among multiple sensors.
The rotation and translation matrix from the camera to the laser radar can be calculated by inputting the coordinates of the corner points extracted by the laser radar into a PNP algorithm, and the rotation and translation matrix is used for projecting the point cloud of the laser radar onto the camera image.
Claims (1)
1. A camera calibration method applied to an automatic driving automobile is characterized in that: the method is realized by the following steps:
firstly, setting a calibration plate in front of an automatic driving vehicle in a static state, acquiring an image of the calibration plate through a camera, and acquiring image coordinates of four corner points of the image of the calibration plate by using lableme software;
measuring physical coordinates of the corner points of the calibration plate in a vehicle body coordinate system, and calculating a rotation translation matrix from the camera coordinate system to the vehicle body coordinate system by utilizing a PnP algorithm in combination with the image coordinates of the first step;
calculating and extracting three-dimensional data of four corner points of a calibration plate in the laser radar, calculating a rotation translation matrix from a camera to the laser radar by using a PnP algorithm under a laser radar coordinate system, and projecting three-dimensional point cloud data of the laser radar onto a two-dimensional image of the camera by using the rotation translation matrix to realize joint calibration of the camera and the laser radar;
setting a calibration plate in front of the automatic driving vehicle in a static state, acquiring an image of the calibration plate through a camera, acquiring image coordinates of four corner points of the image of the calibration plate by using lableme software,
step 1, acquiring an image of a calibration plate:
setting a calibration plate perpendicular to the optical axis of the camera in front of the camera of the automatic driving automobile, wherein the distance between the calibration plate and the camera is 8-15 m, starting the camera to drive and photograph, and obtaining an image of the calibration plate;
step one, 2, obtaining image coordinates of corner points in the calibration plate image:
sequentially and manually marking four corner points of the calibration plate image by using lableme software, storing, and generating a json file containing corresponding image coordinates of the corner points through storage operation to obtain the image coordinates of the corner points in the calibration plate image;
the process of calculating the rotation translation matrix from the camera coordinate system to the vehicle body coordinate system by utilizing the PnP algorithm comprises the following steps of,
step two, 1, obtaining physical coordinates of corner points:
measuring and calculating physical coordinates of four corner points of the calibration plate under a world coordinate system by using auxiliary tools of a tape measure, a plumb line and a laser level;
step 2, calculating a rotation translation matrix:
after obtaining the image coordinates of the four corner points of the calibration plate and the physical coordinates under the vehicle body coordinate system, calculating a rotation translation matrix from the camera coordinate system to the vehicle body coordinate system through a library function solvePnP of OpenCV; the working principle of the PnP algorithm is that the PnP algorithm is used for estimating the pose of a camera when n three-dimensional space points and two-dimensional projection positions of the three-dimensional space points are known;
in the third step, the joint calibration process of the camera and the laser radar is that,
analyzing point cloud data of a calibration plate of the laser radar, and acquiring three-dimensional coordinates of corner points of the calibration plate under a laser radar coordinate system;
acquiring image coordinates of four corner points of a calibration plate by using lableme in the first step, then calling library function solvePnP in OpenCV, and transmitting the image coordinates of a corner point camera and the three-dimensional coordinates of a laser radar into the library function solvePnP of the OpenCV to obtain a rotation translation matrix of the rotation translation matrix;
and finally, projecting the three-dimensional point cloud data of the laser radar onto the two-dimensional image of the camera by using a rotation translation matrix for projecting the three-dimensional point cloud of the laser radar onto the two-dimensional image of the camera, taking the rotation translation matrix as an external parameter between the sensors, and completing the joint calibration of the camera and the laser radar.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910976445.6A CN110827358B (en) | 2019-10-15 | 2019-10-15 | Camera calibration method applied to automatic driving automobile |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910976445.6A CN110827358B (en) | 2019-10-15 | 2019-10-15 | Camera calibration method applied to automatic driving automobile |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110827358A CN110827358A (en) | 2020-02-21 |
| CN110827358B true CN110827358B (en) | 2023-10-31 |
Family
ID=69549421
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910976445.6A Active CN110827358B (en) | 2019-10-15 | 2019-10-15 | Camera calibration method applied to automatic driving automobile |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110827358B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111429521B (en) * | 2020-03-05 | 2021-12-21 | 深圳市镭神智能系统有限公司 | External parameter calibration method, device, medium and electronic equipment for camera and laser radar |
| CN111815717B (en) * | 2020-07-15 | 2022-05-17 | 西北工业大学 | Multi-sensor fusion external parameter combination semi-autonomous calibration method |
| CN112819903B (en) * | 2021-03-02 | 2024-02-20 | 福州视驰科技有限公司 | L-shaped calibration plate-based camera and laser radar combined calibration method |
| CN113436233A (en) * | 2021-06-29 | 2021-09-24 | 阿波罗智能技术(北京)有限公司 | Registration method and device of automatic driving vehicle, electronic equipment and vehicle |
| CN114463439B (en) * | 2022-01-18 | 2023-04-11 | 襄阳达安汽车检测中心有限公司 | Vehicle-mounted camera correction method and device based on image calibration technology |
| CN115018935B (en) * | 2022-08-09 | 2022-10-18 | 季华实验室 | Calibration method and device for camera and vehicle, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107976669A (en) * | 2016-10-21 | 2018-05-01 | 法乐第(北京)网络科技有限公司 | A kind of device of outer parameter between definite camera and laser radar |
| CN108765496A (en) * | 2018-05-24 | 2018-11-06 | 河海大学常州校区 | A kind of multiple views automobile looks around DAS (Driver Assistant System) and method |
| CN109443369A (en) * | 2018-08-20 | 2019-03-08 | 北京主线科技有限公司 | The method for constructing sound state grating map using laser radar and visual sensor |
| CN109993793A (en) * | 2019-03-29 | 2019-07-09 | 北京易达图灵科技有限公司 | Vision positioning method and device |
| CN110009765A (en) * | 2019-04-15 | 2019-07-12 | 合肥工业大学 | A kind of automatic driving vehicle contextual data system and scene format method for transformation |
| CN110246159A (en) * | 2019-06-14 | 2019-09-17 | 湖南大学 | The 3D target motion analysis method of view-based access control model and radar information fusion |
| CN110288656A (en) * | 2019-07-01 | 2019-09-27 | 太原科技大学 | A kind of object localization method based on monocular cam |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2017139612A (en) * | 2016-02-03 | 2017-08-10 | パナソニックＩｐマネジメント株式会社 | On-vehicle camera calibration system |
-
2019
- 2019-10-15 CN CN201910976445.6A patent/CN110827358B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107976669A (en) * | 2016-10-21 | 2018-05-01 | 法乐第(北京)网络科技有限公司 | A kind of device of outer parameter between definite camera and laser radar |
| CN108765496A (en) * | 2018-05-24 | 2018-11-06 | 河海大学常州校区 | A kind of multiple views automobile looks around DAS (Driver Assistant System) and method |
| CN109443369A (en) * | 2018-08-20 | 2019-03-08 | 北京主线科技有限公司 | The method for constructing sound state grating map using laser radar and visual sensor |
| CN109993793A (en) * | 2019-03-29 | 2019-07-09 | 北京易达图灵科技有限公司 | Vision positioning method and device |
| CN110009765A (en) * | 2019-04-15 | 2019-07-12 | 合肥工业大学 | A kind of automatic driving vehicle contextual data system and scene format method for transformation |
| CN110246159A (en) * | 2019-06-14 | 2019-09-17 | 湖南大学 | The 3D target motion analysis method of view-based access control model and radar information fusion |
| CN110288656A (en) * | 2019-07-01 | 2019-09-27 | 太原科技大学 | A kind of object localization method based on monocular cam |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110827358A (en) | 2020-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110827358B (en) | Camera calibration method applied to automatic driving automobile | |
| EP3540464B1 (en) | Ranging method based on laser radar system, device and readable storage medium | |
| JP7073315B2 (en) | Vehicles, vehicle positioning systems, and vehicle positioning methods | |
| CN111208493B (en) | Quick calibration method of vehicle-mounted laser radar in whole vehicle coordinate system | |
| CN109737981B (en) | Unmanned vehicle target searching device and method based on multiple sensors | |
| WO2020151663A1 (en) | Vehicle positioning apparatus, system and method, and vehicle | |
| CN112577517A (en) | Multi-element positioning sensor combined calibration method and system | |
| CN112684432B (en) | Laser radar calibration method, device, equipment and storage medium | |
| CN112967344B (en) | Method, device, storage medium and program product for calibrating camera external parameters | |
| CN113147738A (en) | Automatic parking positioning method and device | |
| CN112284416B (en) | Automatic driving positioning information calibration device, method and storage medium | |
| CN109282813B (en) | Unmanned ship global obstacle identification method | |
| CN112455502B (en) | Train positioning method and device based on laser radar | |
| JP2018055351A (en) | Estimation preparing system for roof painting or waterproof construction | |
| CN110579754A (en) | Method for determining external parameters of a lidar and other sensors of a vehicle | |
| CN110687521A (en) | Vehicle-mounted laser radar calibration method | |
| CN112985374A (en) | Positioning method, positioning assembly and positioning system | |
| CN110044266B (en) | Photogrammetry system based on speckle projection | |
| CN109712198B (en) | Calibration method of advanced driving assistance system | |
| JP2018116004A (en) | Data compression apparatus, control method, program and storage medium | |
| CN113592951A (en) | Method and device for calibrating external parameters of vehicle-road cooperative middle-road side camera and electronic equipment | |
| CN105571491A (en) | Binocular vision-based automobile chassis data measuring system and method thereof | |
| CN113296120A (en) | Obstacle detection method and terminal | |
| CN116907469A (en) | Synchronous positioning and mapping method and system for multi-mode data combined optimization | |
| CN114966793B (en) | Three-dimensional measurement system, method and GNSS system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240328 Address after: Building 3, 7th Floor, Gaosheng Plaza, No. 163 Yingbin Avenue, Xinhua Street, Huadu District, Guangzhou City, Guangdong Province, 510802 Patentee after: Guangzhou Carl Power Technology Co.,Ltd. Country or region after: China Address before: 518118 No.1, Jinlong Avenue, pinghuan community, Malian street, Pingshan District, Shenzhen City, Guangdong Province Patentee before: Shenzhen Shuxiang Technology Co.,Ltd. Country or region before: China |