CN115908589A - Multi-sensor calibration system and method - Google Patents
Multi-sensor calibration system and method Download PDFInfo
- Publication number
- CN115908589A CN115908589A CN202310154047.2A CN202310154047A CN115908589A CN 115908589 A CN115908589 A CN 115908589A CN 202310154047 A CN202310154047 A CN 202310154047A CN 115908589 A CN115908589 A CN 115908589A
- Authority
- CN
- China
- Prior art keywords
- calibration
- camera
- fixed turntable
- fixed
- plates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 230000008569 process Effects 0.000 claims abstract description 55
- 230000033001 locomotion Effects 0.000 claims abstract description 31
- 238000004891 communication Methods 0.000 claims abstract description 4
- 238000004590 computer program Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 239000000758 substrate Substances 0.000 claims 1
- 238000004519 manufacturing process Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000002310 reflectometry Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The invention provides a multi-sensor calibration system and a multi-sensor calibration method, wherein the system comprises a control terminal, a calibration box and a fixed turntable which is arranged in the calibration box and is in communication connection with the control terminal; a plurality of calibration plates are arranged in the calibration box; wherein: the control terminal is used for presetting the motion track of the fixed rotary table and transmitting a control signal to the fixed rotary table; the fixed turntable is used for fixing the camera; based on the control of the control terminal, the camera calibration, the camera and IMU combined calibration process and the camera and LIDAR combined calibration process are realized on a plurality of calibration boards. Above-mentioned system only needs control terminal to control fixed revolving stage, and the cooperation is markd a plurality of calibration boards that the incasement set up, alright in order to realize demarcating the camera, and camera and IMU jointly mark the process and camera and LIDAR jointly mark the process, has realized demarcating the multisensor respectively on same system, and its easy operation has effectively improved efficiency and the reliability of demarcation, is applicable to the demarcation process of volume production scheme.
Description
Technical Field
The invention relates to the technical field of machine vision, in particular to a multi-sensor calibration system and a multi-sensor calibration method.
Background
With the progress of the technology, a multi-sensor fusion scheme becomes a main stream direction of an unmanned or synchronous positioning And Mapping technology, namely an SLAM (Simultaneous Localization And Mapping) technology, so that the multi-sensor calibration is also an essential important link.
In the existing multi-sensor calibration technology, only calibration of a single or multiple cameras can be performed through a single calibration plate or calibration of a single or multiple cameras at a time through multiple calibration plates, but the calibration needs to be performed statically, and the calibration requirement is high, while a combined calibration process of a camera and an Inertial Measurement Unit (IMU) and a combined calibration process of a camera and a laser radar (LIDAR), i.e., a single calibration system obviously cannot meet the existing multi-sensor calibration requirement, and multiple systems are required to be performed respectively, so the calibration efficiency is low, and the reliability of the final calibration result is not good due to the difference in equipment among different systems.
Disclosure of Invention
The invention aims to provide a multi-sensor calibration system and a multi-sensor calibration method, so as to solve the technical problems, realize the respective calibration of multiple sensors on the same system, effectively improve the calibration efficiency and reliability, and be suitable for the calibration process of a mass production scheme.
In order to solve the technical problem, the invention provides a multi-sensor calibration system which comprises a control terminal, a calibration box and a fixed rotary table, wherein the fixed rotary table is arranged in the calibration box and is in communication connection with the control terminal; a plurality of calibration plates are arranged in the calibration box; wherein:
the control terminal is used for presetting the motion track of the fixed rotary table and transmitting a control signal to the fixed rotary table;
the fixed turntable is used for fixing the camera;
based on the control of the control terminal, the camera calibration, the camera and IMU combined calibration process and the camera and LIDAR combined calibration process are realized on a plurality of calibration boards.
Above-mentioned system only needs control terminal to control fixed revolving stage, and the cooperation is markd a plurality of calibration boards that the incasement set up, alright in order to realize demarcating the camera, and camera and IMU jointly demarcate the process and camera and LIDAR jointly demarcate the process, has realized demarcating the multisensor respectively on same system, and its easy operation has effectively improved efficiency and the reliability of demarcation, is applicable to the demarcation process of volume production scheme.
In the scheme, the control terminal controls the camera to move through the preset motion trail, on one hand, the combined calibration process of the camera and the IMU can be rapidly completed, and on the other hand, the IMU is sufficiently excited, so that the calibration result is more reliable.
Further, the calibration box is of a closed structure.
Among the above-mentioned scheme, because mark the case and be enclosed construction, consequently fix the camera on fixed revolving stage and will fall and mark incasement portion and outside isolated, to the camera in big field of vision, it can not lose the field of vision at the shooting in-process, and it all can see the calibration board in whole field of vision, greatly reduced the degree of difficulty of demarcation, improved efficiency and the maneuverability of demarcation.
Furthermore, the plurality of calibration plates comprise base plates and characteristic patterns, and the characteristic patterns are checkerboards, two-dimensional codes or custom codes and decoders.
Among the above-mentioned scheme, a plurality of calibration board of demarcation case can realize the quick demarcation to the camera, need not to remove alright realization to the camera under general demarcation condition. Meanwhile, the characteristic patterns of a plurality of calibration plates can be set into various types, so that the compatibility of the system is improved.
Furthermore, the plurality of calibration plates are all backlight calibration plates, and the control ends of the calibration plates are electrically connected with the control terminal; the control terminal is used for controlling the backlight conditions of the backlight calibration plates so as to form characteristic patterns required by calibration on the corresponding backlight calibration plates.
In the above scheme, the calibration board is set as the backlight calibration board, and the characteristic patterns required by calibration can be directly formed on the backlight calibration board through the control of the control terminal, so that the complicated process that the calibration board needs to be manually replaced in the calibration process is avoided, and the calibration efficiency is improved. Secondly, the existence of the backlight can enable the imaging quality of the camera to be higher, can better reflect the line, the surface and the reflectivity of the characteristic pattern, and is beneficial to the implementation of the joint calibration process of the camera and the LIDAR.
The invention provides a multi-sensor calibration method, which is realized based on a multi-sensor calibration system and comprises the following steps:
determining a calibration requirement;
based on the calibration requirement, the fixed turntable is controlled according to a preset motion track, and the calibration of the camera fixed on the fixed turntable is realized on a plurality of preset calibration plates;
the calibration requirements include calibration of the camera, a camera and IMU joint calibration process, and a camera and LIDAR joint calibration process.
In the above scheme, only need control terminal to control fixed revolving stage, a plurality of calibration boards that set up in the case are markd in the cooperation, alright in order to realize demarcating the camera, and camera and IMU jointly mark the process and camera and LIDAR jointly mark the process, realized demarcating the multisensor respectively on same system, its easy operation has effectively improved efficiency and the reliability of demarcation, is applicable to the demarcation process of volume production scheme.
Further, if the calibration requirement is the calibration of the camera, the fixed turntable is controlled according to a preset motion trajectory, and the calibration of the camera fixed on the fixed turntable is realized on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire a calibration plate image;
determining a boundary line of a calibration plate based on a calibration plate image acquired by a camera, and segmenting different calibration plate images;
coding and decoding the divided calibration plate images to obtain control points of each calibration plate;
and calibrating the camera internal parameters based on the control point coordinates and the corresponding world coordinates.
Further, if the calibration requirement is a camera and IMU combined calibration process, the fixed turntable is controlled according to a preset motion trajectory, and calibration of the camera fixed on the fixed turntable is realized on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire a calibration plate image;
determining a boundary line of a calibration plate based on a calibration plate image acquired by a camera, and segmenting different calibration plate images;
coding and decoding the segmented calibration plate images to obtain control points of each calibration plate and calculate external parameters of the camera;
determining the camera pose according to the motion track and initializing the node of the B spline;
and establishing an observation model cost function based on the nodes of the B-spline, optimizing to obtain external parameters of the camera and the IMU, and realizing the combined calibration process of the camera and the IMU.
Further, if the calibration requirement is a camera and LIDAR combined calibration process, the fixed turntable is controlled according to a preset motion track, and calibration of the camera fixed on the fixed turntable is realized on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire the calibration plate image;
determining a calibration plate plane and a boundary line based on a calibration plate image acquired by a camera, and segmenting different calibration plate planes;
extracting a point cloud plane and an intersection line of the point cloud plane based on point cloud data acquired by a laser radar;
and realizing joint optimization of point-to-line and point-to-surface of the camera and the LIDAR based on the calibration plate plane, the boundary line, the point cloud plane and the intersection line of the planes, and realizing the joint calibration process of the camera and the LIDAR.
The invention provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the multi-sensor calibration method.
The invention provides a computer program product, which enables a terminal device to perform the multi-sensor calibration method when the computer program product runs on the terminal device.
Drawings
Fig. 1 is an architecture diagram of a multi-sensor calibration system according to an embodiment of the present invention;
FIG. 2 is a flowchart of a multi-sensor calibration method according to an embodiment of the present invention;
wherein: 1. a control terminal; 2. a calibration box; 3. the turntable is fixed.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment provides a multi-sensor calibration system, which comprises a control terminal 1, a calibration box 2 and a fixed turntable 3, wherein the fixed turntable 3 is arranged in the calibration box 2 and is in communication connection with the control terminal 1; a plurality of calibration plates are arranged in the calibration box 2; wherein:
the control terminal 1 is used for presetting a motion track of the fixed rotary table 3 and transmitting a control signal to the fixed rotary table 3;
the fixed turntable 3 is used for fixing a camera;
based on the control of the control terminal 1, the camera calibration, the camera and IMU combined calibration process and the camera and LIDAR combined calibration process are realized on a plurality of calibration boards.
This embodiment only needs control terminal 1 to control fixed revolving stage 3, and the cooperation is markd a plurality of calibration boards that set up in the case 2, alright in order to realize demarcating the camera, and camera and IMU jointly mark the process and camera and LIDAR jointly mark the process, has realized demarcating the multisensor respectively on same system, and its easy operation has effectively improved the efficiency and the reliability of demarcation, is applicable to the demarcation process of volume production scheme.
In this embodiment, the control terminal 1 controls the camera to move through the preset motion trajectory, on one hand, the camera and IMU combined calibration process can be quickly completed, and on the other hand, the IMU is sufficiently excited, so that the calibration result is more reliable. If the IMU of the camera is fully excited due to the existence of the fixed turntable 3 during the IMU calibration, the FOV of the camera cannot be lost, and the pose can be resolved for each frame of picture.
It should be noted that the system can also perform single or combined calibration of other sensors according to requirements, integrates multi-sensor calibration, can flexibly and efficiently realize multi-sensor calibration, and can efficiently finish the calibration of internal and external parameters of any camera. Moreover, each inner surface of the calibration box 2 is provided with a corresponding calibration plate, so that the calibration box 2 is ensured to have 270-degree calibration space, and the calibration reliability of the large FOV camera can be improved.
It should be noted that the fixed turntable 3 can be implemented by a three-degree-of-freedom three-axis turntable, and the control terminal 1 can be implemented by a PC computer.
Further, the calibration box 2 is of a closed structure.
In this embodiment, since the calibration box 2 is a closed structure, the camera fixed on the fixed turntable 3 is isolated from the inside and the outside of the calibration box 2, and for a camera with a large visual field, the visual field cannot be lost in the shooting process, for example, after the camera with the angle of more than 120 degrees is fixed in the calibration box 2, the calibration plate can be seen in the whole visual field, so that the calibration difficulty is greatly reduced, and the calibration efficiency and operability are improved.
Furthermore, the plurality of calibration plates comprise base plates and characteristic patterns, and the characteristic patterns are checkerboards, two-dimensional codes or custom codes and decoders.
In this embodiment, mark a plurality of calibration plate of case 2 and can realize the quick demarcation to the camera, need not to remove alright realization to the camera under general demarcation condition, if can carry out camera demarcation to the inside photo of 2 inside marks of case of 2 back camera of fixing in mark case 2. Meanwhile, the characteristic patterns of a plurality of calibration plates can be set into various types, and the compatibility of the system is improved.
Furthermore, the plurality of calibration plates are all backlight calibration plates, and the control ends of the calibration plates are electrically connected with the control terminal 1; the control terminal 1 is used for controlling the backlight condition of a plurality of backlight calibration plates so as to form characteristic patterns required by calibration on the corresponding backlight calibration plates.
In this embodiment, the calibration board is set as a backlight calibration board, and the characteristic patterns required for calibration can be directly formed on the backlight calibration board through the control of the control terminal 1, so that a cumbersome process that the calibration board needs to be manually replaced in the calibration process is avoided, and the calibration efficiency is improved. Secondly, the existence of backlight can make the imaging quality of the camera higher, and because the interior of the calibration box 2 is formed by a surface structure, the characteristic pattern is generally formed by different colors at intervals, the line, the surface and the reflectivity of the characteristic pattern can be better reflected, the joint calibration process of the camera and the LIDAR can be favorably carried out, and different surface constraints and line constraints can be obviously provided due to the calibration box structure when the camera and the laser radar are calibrated.
In order to better embody the technical gist of the present invention, the present embodiment compares and analyzes the technical gist of the present invention with the prior art to highlight the technical advantages of the present invention. The method specifically comprises the following steps:
in the prior art, when a camera is calibrated through a single calibration plate, at least three pictures need to be taken at three different poses, and the multi-sensor calibration system provided by the embodiment can acquire multiple calibration plate pictures in one-time shooting, so that the calibration efficiency is greatly improved, and the calibration steps are simplified.
In the prior art, when the camera IMU is jointly calibrated through a single calibration board, the movement space of the camera and the IMU is limited, and the camera can only move in the space of ensuring that the camera can see the whole calibration board, which easily causes insufficient excitation of the system.
In the prior art, when the camera LIDAR is jointly calibrated through a single calibration plate, because effective constraint of only one surface can be provided at each time, the provided edge line characteristic constraint is inaccurate, data acquisition needs to be shot for multiple times to be completed, and the calibration box 2 provides line characteristic constraint of three surfaces and three surface intersection lines at each time, the constraint is more efficient and reliable, the calibration efficiency is greatly improved, and the calibration steps are simplified.
Referring to fig. 2, the present embodiment provides a multi-sensor calibration method implemented based on a multi-sensor calibration system, including:
s1: determining a calibration requirement;
s2: based on the calibration requirement, the fixed turntable is controlled according to a preset motion track, and the calibration of the camera fixed on the fixed turntable is realized on a plurality of preset calibration plates;
the calibration requirements include calibration of the camera, a camera and IMU joint calibration process, and a camera and LIDAR joint calibration process.
In this embodiment, only need control terminal to control fixed revolving stage, a plurality of calibration boards that set up in the case are markd in the cooperation, alright in order to realize demarcating the camera, and camera and IMU jointly mark the process and camera and LIDAR jointly mark the process, realized demarcating the multisensor respectively on same system, its easy operation has effectively improved efficiency and the reliability of demarcation, is applicable to the demarcation process of volume production scheme.
Further, if the calibration requirement is the calibration of the camera, the fixed turntable is controlled according to a preset motion trajectory, and the calibration of the camera fixed on the fixed turntable is realized on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire a calibration plate image;
determining a boundary line of a calibration plate based on a calibration plate image acquired by a camera, and segmenting different calibration plate images;
coding and decoding the divided calibration plate images to obtain control points of each calibration plate;
and calibrating the camera internal parameters based on the control point coordinates and the corresponding world coordinates.
It should be noted that before calibrating the internal reference of the camera, external reference calibration of the calibration plates needs to be performed, that is, external reference between the calibration plates is recovered through a large number of shot pictures, and then is used as a calibration configuration parameter for calibration.
Further, the boundary line of the calibration plate is determined based on the calibration plate image acquired by the camera, and different calibration plate images are segmented, specifically:
performing local binarization operation on a calibration plate image acquired by a camera (calculating by adopting a window with the size of 20x20, counting inter-class variance of pixel values, distinguishing a background and a foreground, and performing binarization processing according to foreground and background threshold values) so as to ensure local characteristics of the calibration plate image so as to facilitate image segmentation and calibration plate decoding; and performing linear detection according to the binary image to respectively obtain boundary lines of the calibration plates, separating each photographed calibration plate according to intersection points of the boundary lines, and segmenting different calibration plate images.
Further, calibrating camera internal parameters based on the control point coordinates and the corresponding world coordinates, specifically:
initializing and optimizing camera internal parameters by using control point coordinates and corresponding world coordinates, wherein the process mainly comprises the steps of taking a single divided calibration plate as one independent observation, calibrating four calibration plates which are four independent observations by using four independent observations and a first camera, and defining the first calibration as local calibration; according to the calibration result of the first camera, establishing a world coordinate system by using a 0 th calibration plate, transferring the world coordinates of all the calibration plates to the 0 th calibration plate by using external parameters between the calibration plates, carrying out 2 nd calibration under the condition of constraint to obtain the external parameters of images shot each time and the internal parameters of the camera, and defining the 2 nd calibration as global calibration; and performing inverse perspective transformation by utilizing perspective transformation and external parameters of pictures shot each time to change the images into a front view angle, repeating the calibration process for multiple times, and finally completing the calibration of the internal parameters of the camera.
Further, if the calibration requirement is a camera and IMU combined calibration process, the fixed turntable is controlled according to a preset motion trajectory, and calibration of the camera fixed on the fixed turntable is realized on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire a calibration plate image;
determining a boundary line of a calibration plate based on a calibration plate image acquired by a camera, and segmenting different calibration plate images;
coding and decoding the segmented calibration plate images to obtain control points of each calibration plate and calculate external parameters of the camera;
determining the pose of the camera according to the motion trail and initializing the node of the B spline;
and establishing an observation model cost function based on the nodes of the B-spline, optimizing to obtain external parameters of the camera and the IMU, and realizing the combined calibration process of the camera and the IMU.
Further, if the calibration requirement is a camera and LIDAR combined calibration process, the fixed turntable is controlled according to a preset motion track, and calibration of the camera fixed on the fixed turntable is realized on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire a calibration plate image;
determining a calibration plate plane and a boundary line based on a calibration plate image acquired by a camera, and segmenting different calibration plate planes;
extracting a point cloud plane and an intersection line of the point cloud plane based on point cloud data acquired by a laser radar;
and realizing joint optimization of point-to-line and point-to-surface of the camera and the LIDAR based on the calibration plate plane, the boundary line, the point cloud plane and the intersection line of the planes, and realizing the joint calibration process of the camera and the LIDAR.
The embodiment meets the condition that multiple sensor calibration tasks are completed in one system in an actual production line, has greater applicability to different sensors, and greatly saves the space and cost required by calibration.
The embodiment provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the multi-sensor calibration method described above.
The embodiment provides a computer program product, which when running on a terminal device, enables the terminal device to perform the above-mentioned multi-sensor calibration method.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
Claims (10)
1. A multi-sensor calibration system comprises a control terminal, a calibration box and a fixed turntable, wherein the fixed turntable is arranged in the calibration box and is in communication connection with the control terminal; the device is characterized in that a plurality of calibration plates are arranged in the calibration box; wherein:
the control terminal is used for presetting the motion track of the fixed turntable and transmitting a control signal to the fixed turntable;
the fixed turntable is used for fixing the camera;
based on the control of the control terminal, the camera calibration, the camera and IMU combined calibration process and the camera and LIDAR combined calibration process are realized on a plurality of calibration boards.
2. A multi-sensor calibration system according to claim 1, wherein said calibration box is a closed structure.
3. The multi-sensor calibration system as claimed in claim 1, wherein each of the calibration plates comprises a substrate and a feature pattern, and the feature pattern is a checkerboard, a two-dimensional code or a custom codec.
4. The multi-sensor calibration system according to any one of claims 1~3, wherein said calibration plates are backlight calibration plates, and control terminals thereof are electrically connected to said control terminal; the control terminal is used for controlling the backlight conditions of the backlight calibration plates so as to form characteristic patterns required by calibration on the corresponding backlight calibration plates.
5. A multi-sensor calibration method, implemented based on the multi-sensor calibration system as claimed in any one of claims 1~4, comprising:
determining a calibration requirement;
based on the calibration requirement, controlling the fixed turntable according to a preset motion track, and calibrating the camera fixed on the fixed turntable on a plurality of preset calibration plates;
the calibration requirements include calibration of the camera, a camera and IMU joint calibration process, and a camera and LIDAR joint calibration process.
6. The multi-sensor calibration method according to claim 5, wherein if the calibration requirement is calibration of a camera, the fixed turntable is controlled according to a preset motion trajectory, and calibration of the camera fixed on the fixed turntable is implemented on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire the calibration plate image;
determining a boundary line of a calibration plate based on a calibration plate image acquired by a camera, and segmenting different calibration plate images;
coding and decoding the divided calibration plate images to obtain control points of each calibration plate;
and calibrating the camera internal parameters based on the control point coordinates and the corresponding world coordinates.
7. The multi-sensor calibration method according to claim 5, wherein [ A1] if the calibration requirement is a camera and IMU combined calibration process, the fixed turntable is controlled according to a preset motion trajectory, and calibration of the camera fixed on the fixed turntable is implemented on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire a calibration plate image;
determining a boundary line of a calibration plate based on a calibration plate image acquired by a camera, and segmenting different calibration plate images;
coding and decoding the segmented calibration plate images to obtain control points of each calibration plate and calculate external parameters of the camera;
determining the camera pose according to the motion track and initializing the node of the B spline;
and establishing an observation model cost function based on the nodes of the B-spline, optimizing to obtain external parameters of the camera and the IMU, and realizing the combined calibration process of the camera and the IMU.
8. The multi-sensor calibration method according to claim 5, wherein if the calibration requirement is a camera and LIDAR combined calibration process, the fixed turntable is controlled according to a preset motion trajectory, and calibration of the camera fixed on the fixed turntable is implemented on a plurality of preset calibration plates, specifically:
controlling the fixed turntable to move according to a preset movement track so as to enable the camera to acquire a calibration plate image;
determining a calibration plate plane and a boundary line based on a calibration plate image acquired by a camera, and segmenting different calibration plate planes;
extracting a point cloud plane and an intersection line of the point cloud plane based on point cloud data acquired by a laser radar;
and realizing the joint optimization of the camera and the LIDAR point-to-line and the LIDAR point-to-surface based on the calibration plate plane, the boundary line, the point cloud plane and the intersection line of the planes, and realizing the joint calibration process of the camera and the LIDAR.
9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements a multi-sensor calibration method as claimed in any one of claims 5 to 8 when executing the computer program.
10. A computer program product, characterized in that it causes a terminal device to execute a method for multi-sensor calibration according to any one of claims 5 to 8 when the computer program product is run on the terminal device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310154047.2A CN115908589A (en) | 2023-02-23 | 2023-02-23 | Multi-sensor calibration system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310154047.2A CN115908589A (en) | 2023-02-23 | 2023-02-23 | Multi-sensor calibration system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115908589A true CN115908589A (en) | 2023-04-04 |
Family
ID=86486800
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310154047.2A Pending CN115908589A (en) | 2023-02-23 | 2023-02-23 | Multi-sensor calibration system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115908589A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH05282042A (en) * | 1992-03-31 | 1993-10-29 | Suzuki Motor Corp | Position measuring instrument |
| CN111735479A (en) * | 2020-08-28 | 2020-10-02 | 中国计量大学 | Multi-sensor combined calibration device and method |
| CN112444798A (en) * | 2020-11-27 | 2021-03-05 | 杭州易现先进科技有限公司 | Multi-sensor equipment space-time external parameter calibration method and device and computer equipment |
| CN113140040A (en) * | 2021-04-26 | 2021-07-20 | 北京天地玛珂电液控制系统有限公司 | Multi-sensor fusion coal mine underground space positioning and mapping method and device |
| CN114494446A (en) * | 2020-10-23 | 2022-05-13 | 浙江舜宇智能光学技术有限公司 | External parameter calibration method and system and electronic equipment |
| WO2022143283A1 (en) * | 2020-12-30 | 2022-07-07 | 深圳市杉川机器人有限公司 | Camera calibration method and apparatus, and computer device and storage medium |
| CN114862899A (en) * | 2022-04-22 | 2022-08-05 | 阿里巴巴（中国）有限公司 | Equipment calibration method, system, device, electronic equipment and program product |
| CN114926547A (en) * | 2022-06-01 | 2022-08-19 | 海信电子科技（深圳）有限公司 | Calibration method of camera and IMU, electronic device and system |
-
2023
- 2023-02-23 CN CN202310154047.2A patent/CN115908589A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH05282042A (en) * | 1992-03-31 | 1993-10-29 | Suzuki Motor Corp | Position measuring instrument |
| CN111735479A (en) * | 2020-08-28 | 2020-10-02 | 中国计量大学 | Multi-sensor combined calibration device and method |
| CN114494446A (en) * | 2020-10-23 | 2022-05-13 | 浙江舜宇智能光学技术有限公司 | External parameter calibration method and system and electronic equipment |
| CN112444798A (en) * | 2020-11-27 | 2021-03-05 | 杭州易现先进科技有限公司 | Multi-sensor equipment space-time external parameter calibration method and device and computer equipment |
| WO2022143283A1 (en) * | 2020-12-30 | 2022-07-07 | 深圳市杉川机器人有限公司 | Camera calibration method and apparatus, and computer device and storage medium |
| CN113140040A (en) * | 2021-04-26 | 2021-07-20 | 北京天地玛珂电液控制系统有限公司 | Multi-sensor fusion coal mine underground space positioning and mapping method and device |
| CN114862899A (en) * | 2022-04-22 | 2022-08-05 | 阿里巴巴（中国）有限公司 | Equipment calibration method, system, device, electronic equipment and program product |
| CN114926547A (en) * | 2022-06-01 | 2022-08-19 | 海信电子科技（深圳）有限公司 | Calibration method of camera and IMU, electronic device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109270534B (en) | Intelligent vehicle laser sensor and camera online calibration method | |
| KR102126513B1 (en) | Apparatus and method for determining the pose of the camera | |
| CN112734852B (en) | Robot mapping method and device and computing equipment | |
| CN112907676B (en) | Calibration method, device and system of sensor, vehicle, equipment and storage medium | |
| Alismail et al. | Automatic calibration of a range sensor and camera system | |
| CN110261870A (en) | It is a kind of to synchronize positioning for vision-inertia-laser fusion and build drawing method | |
| CN112505065B (en) | Method for detecting surface defects of large part by indoor unmanned aerial vehicle | |
| Xie et al. | Infrastructure based calibration of a multi-camera and multi-lidar system using apriltags | |
| CN111401251B (en) | Lane line extraction method, lane line extraction device, electronic equipment and computer readable storage medium | |
| CN112577517A (en) | Multi-element positioning sensor combined calibration method and system | |
| CN103886107A (en) | Robot locating and map building system based on ceiling image information | |
| CN110634138A (en) | Bridge deformation monitoring method, device and equipment based on visual perception | |
| CN111964680B (en) | Real-time positioning method of inspection robot | |
| Cvišić et al. | Recalibrating the KITTI dataset camera setup for improved odometry accuracy | |
| CN109978954A (en) | The method and apparatus of radar and camera combined calibrating based on cabinet | |
| CN111553956A (en) | Calibration method and device of shooting device, electronic equipment and storage medium | |
| CN113643380A (en) | Mechanical arm guiding method based on monocular camera vision target positioning | |
| CN110673607B (en) | Feature point extraction method and device under dynamic scene and terminal equipment | |
| CN115239822A (en) | Real-time visual identification and positioning method and system for multi-module space of split type flying vehicle | |
| Manivannan et al. | Vision based intelligent vehicle steering control using single camera for automated highway system | |
| Bellavia et al. | Image orientation with a hybrid pipeline robust to rotations and wide-baselines | |
| KR20230003803A (en) | Automatic calibration through vector matching of the LiDAR coordinate system and the camera coordinate system | |
| CN115908589A (en) | Multi-sensor calibration system and method | |
| US12008785B2 (en) | Detection, 3D reconstruction and tracking of multiple rigid objects moving in relation to one another | |
| CN113239948B (en) | Data fusion method and system for millimeter wave radar and video image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |