CN112070841A - Rapid combined calibration method for millimeter wave radar and camera - Google Patents
Rapid combined calibration method for millimeter wave radar and camera Download PDFInfo
- Publication number
- CN112070841A CN112070841A CN202010616507.5A CN202010616507A CN112070841A CN 112070841 A CN112070841 A CN 112070841A CN 202010616507 A CN202010616507 A CN 202010616507A CN 112070841 A CN112070841 A CN 112070841A
- Authority
- CN
- China
- Prior art keywords
- camera
- radar
- corner reflector
- millimeter wave
- projection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000003384 imaging method Methods 0.000 claims abstract description 22
- 239000011159 matrix material Substances 0.000 claims abstract description 11
- 238000001514 detection method Methods 0.000 claims abstract description 7
- 238000005259 measurement Methods 0.000 description 8
- 230000001788 irregular Effects 0.000 description 6
- 230000004927 fusion Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 239000002184 metal Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003930 cognitive ability Effects 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C25/00—Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
- G01S7/4052—Means for monitoring or calibrating by simulation of echoes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
- G01S7/4052—Means for monitoring or calibrating by simulation of echoes
- G01S7/4082—Means for monitoring or calibrating by simulation of echoes using externally generated reference signals, e.g. via remote reflector or transponder
- G01S7/4095—Means for monitoring or calibrating by simulation of echoes using externally generated reference signals, e.g. via remote reflector or transponder the external reference signals being modulated, e.g. rotating a dihedral reflector or modulating a transponder for simulation of a Doppler echo
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Manufacturing & Machinery (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention relates to a quick combined calibration method for a millimeter wave radar and a camera, which comprises the following steps: s1, mounting the camera and the radar group on the motion platform, and placing the motion platform in a clear and flat place; s2, arranging a corner reflector right in front of the motion platform, adjusting the height of the corner reflector to be the same as that of the radar group, and adjusting the distance between the corner reflector and the radar group until the corner reflector appears in the image range of the camera; s3, calibrating camera internal parameters by using the pinhole camera model and adjusting the radar angle; s4, determining initial projection parameters of the millimeter wave coordinate system Xr-Yr to the camera coordinate system Xc-Yc; s5, adjusting the numerical values of the yaw angle and the translation distance in the external parameters by calculating the deviation of the imaging position of the corner reflector in the imaging plane and the projection position of the radar detection point of the corner reflector on the imaging plane; and S6, outputting the camera internal parameter matrix and the external parameter, and completing calibration. The corner reflector has high reflection intensity and small size, and better feedback can be obtained.
Description
Technical Field
The invention relates to the technical field of multi-sensor combined calibration, in particular to a rapid combined calibration method for a millimeter wave radar and a camera.
Background
For an unmanned motion platform, whether an intelligent driving vehicle on a regular road, an unmanned agricultural machine on an unstructured road or a mobile robot in an indoor scene, accurate sensing and identification of a target are carried out by using multi-sensor fusion, and the environmental cognitive ability of the target is improved, which is very important. The use of multi-sensor fusion can effectively avoid accidents caused by single sensor failure, and the advantages of the multi-sensors are complemented to remarkably improve the perception precision of the perception system. In different sensor fusion schemes, cameras are low in price and can provide rich semantic information, so that the cameras become an indispensable component of an unmanned motion platform sensing system. However, the camera is very susceptible to external environments such as variable illumination, and it is difficult to obtain accurate position information of the target; the millimeter wave radar is also a sensor with low price, has good distance and speed measurement precision, is less influenced by weather factors such as illumination, smoke dust, rain, snow and the like, and forms natural complementation with the camera. Therefore, the sensing fusion is carried out by using the camera and the millimeter wave radar, and the sensing scheme is low in price and stable in performance.
However, since the sensing data of the two sensors are described in their respective coordinate systems, when it is desired to use the fused data of the camera and the millimeter wave radar as the target detection result, the two sensors are first calibrated jointly to determine the transformation relationship between their coordinate systems.
The joint calibration generally includes self-calibration and inter-sensor calibration of each sensor. The calibration of each sensor is mainly used for determining parameters of the sensor, such as camera internal parameters, so that data can be accurately acquired by the sensor; the calibration between sensors is to determine the projection relationship of data in a single sensor coordinate system to another sensor coordinate system, so as to ensure the accurate matching of the data of the two sensors.
Some researches exist at home and abroad on the research of the combined calibration method of the millimeter wave radar and the camera, for example, Qinghua university rocaiya assumes that a normal vector of a millimeter wave radar plane is parallel to a vehicle longitudinal symmetry plane through a strict vehicle longitudinal symmetry plane, calibrates the radar and the camera respectively, and obtains the projection relation of a coordinate system of the radar and the camera through calculation. In the research, a good calibration effect is obtained, however, in order to obtain a longitudinal symmetry plane of the vehicle, researchers need to measure a plurality of symmetry points on the surface of the vehicle, calculate the midpoint of the vehicle, and then scan a vertical laser plane through a laser level meter, so that the method is complex to implement; in the research, Tao Wang et al of the Western university of transportation uses a metal plate to calibrate a camera and a millimeter wave radar, and from the result, the metal plate has a large area, so that the calibrated transverse deviation is too large, and a good effect is difficult to obtain. In addition, calibration is performed by using vehicles, pedestrians and the like as reference objects, but due to the problems of large target of the reference object, irregular shape, poor reflection intensity and the like, the situations that more than one projection point of the millimeter wave radar measuring point in the image plane, the lateral position of the projection point is inaccurate and the like are easily caused, and the combined calibration precision of the millimeter wave radar and the vision is influenced.
In summary, some of the existing millimeter wave and camera joint calibration methods have strong preconditions, and complex operations are often required to meet the required preconditions; some reference objects with larger targets and irregular shapes are selected, so that the calibrated transverse position is inaccurate, and the calibration precision is reduced.
A joint calibration method is needed, which can realize rapid calibration and avoid strong preconditions in use.
Disclosure of Invention
The invention aims to solve the problems that some work of the existing millimeter wave radar and camera combined calibration method in the prior art has strong preconditions, and complex operation is often required to meet the required preconditions; some reference objects with larger targets and irregular shapes are selected, so that the problem of calibration precision reduction caused by inaccurate transverse position of calibration is solved.
The invention provides a rapid combined calibration method for a millimeter wave radar and a camera, which comprises the following steps:
s1, mounting the camera and the radar group on the motion platform, and placing the motion platform in a clear and flat place;
s2, arranging a corner reflector right in front of the motion platform, adjusting the height of the corner reflector to be the same as that of the radar group, and adjusting the distance between the corner reflector and the radar group until the corner reflector appears in the image range of the camera;
s3, calibrating camera internal parameters by using the pinhole camera model and adjusting the radar angle;
s4, determining initial projection parameters of the millimeter wave coordinate system Xr-Yr to the camera coordinate system Xc-Yc;
s5, adjusting the numerical values of the external reference yaw angle and the translation distance by calculating the deviation of the imaging position of the corner reflector in the imaging plane and the projection position of the radar detection point on the imaging plane;
and S6, outputting the camera internal parameter matrix and the external parameter, and completing calibration.
The radar group can be a plurality of millimeter wave radars, can jointly mark a plurality of millimeter wave radars and camera, if the number more than or equal to two of millimeter wave radar, only need place the corner reflector in the position that a plurality of millimeter wave radars can detect simultaneously, and need adjust the position of corner reflector and make its appear in the camera image. The corner reflector has high reflection intensity, can be accurately obtained by the millimeter wave radar, avoids the problems of larger reference object target, irregular shape, poor reflection intensity and the like, easily generates the conditions that the number of projection points of a millimeter wave radar measuring point in an image plane is more than one, the transverse position of the projection point is inaccurate and the like, and influences the combined calibration precision of the millimeter wave radar and the vision. The size of the corner reflector is small, and better feedback can be obtained by utilizing the pixel deviation of the projection point of the corner reflector and the imaging point of the corner reflector in the image plane, so that the projection parameters are adjusted, and the joint calibration speed is improved.
The invention relates to a rapid combined calibration method for a millimeter wave radar and a camera, which is an optimal mode, and the specific method in the step S3 comprises the following steps:
s31, calibrating the camera internal reference by using a checkerboard calibration board, wherein the camera internal reference matrix is as follows:
wherein f isxAnd fyIs the focal length of the camera, u0And v0Is the image principal point coordinate;
s32, adjusting the transmitting and receiving planes of the radar group to be vertical to the ground;
s33, measuring the distance between the projection position of the radar group on the ground and the corner reflector, comparing the distance with the distance of the corner reflector measured by the radar group, and if the deviation is smaller than a threshold value, performing S4; if the deviation is greater than or equal to the threshold value, the roll angle and the pitch angle of the radar are adjusted, and the step S33 is repeated.
The rapid combined calibration method for the millimeter wave radar and the camera is used as an optimal mode, and S31 is used for adjusting the transmitting and receiving planes of the radar set to be vertical to the ground;
s32, measuring the distance between the projection position of the radar group on the ground and the corner reflector, comparing the distance with the distance of the corner reflector measured by the radar group, and if the deviation is smaller than a threshold value, performing S4; if the deviation is larger than or equal to the threshold value, adjusting the roll angle and the pitch angle of the radar, and repeating the step S32;
s33, calibrating by using checkerboardThe board is markd the camera internal reference, and the internal reference matrix of camera is:
wherein f isxAnd fyIs the focal length of the camera, u0And v0Is the image principal point coordinate.
Because the millimeter wave radar is mainly applied to measuring peripheral obstacles on the unmanned tractor, the normal plane of the millimeter wave radar needs to be ensured to be vertical to the ground as much as possible.
Measuring the distance between the projection position of the millimeter wave radar on the ground and the corner reflector by using a measuring tape, comparing the distance with the distance of the corner reflector measured by the millimeter wave radar, and if the deviation is smaller than a threshold value, proving that the roll angle and the pitch angle of the millimeter wave radar are basically eliminated; otherwise, the roll angle and the pitch angle of the millimeter wave radar are adjusted according to the feedback of the measurement error until the deviation is reduced to the acceptance range.
And calibrating the camera internal parameters by adjusting the radar and pinhole camera models simultaneously.
The invention relates to a rapid combined calibration method for a millimeter wave radar and a camera, which is an optimal mode, and the specific method in the step S4 comprises the following steps: measuring the translation distances Tx and Ty of the millimeter wave radar and the camera in the transverse direction and the longitudinal direction, and measuring the yaw angle R of the millimeter wave radar relative to the cameraθThereby determining initial projection parameters of the millimeter wave coordinate system Xr-Yr to the camera coordinate system Xc-Yc.
The millimeter wave radar outputs the horizontal and vertical coordinates of the target, and does not output vertical parameters, so that the conversion of the millimeter wave radar coordinate system and the camera coordinate system into a two-dimensional coordinate system, namely two-dimensional translation and rotation, can be considered.
The invention relates to a rapid combined calibration method for a millimeter wave radar and a camera, and as a preferred mode, the step S5 specifically comprises the following steps:
s51, projecting the target point of the corner reflector measured by the radar group on the image plane through the initial projection external parameter and the camera internal parameter, and setting the coordinate of the radar measuring point as (x)r，yr) The projected point of the coordinate system of the camera is (x)c，yc) In the imageThe coordinates of the projection point in the plane are (u, v), and the projection relation of the projection point to the radar measuring point in the camera coordinate system is
S52, projecting the target point of the corner reflector in the camera coordinate system to the image plane, wherein the projection equation is
S53, calculating the deviation of the imaging position of the corner reflector in the imaging plane and the projection position of the radar detection point on the imaging plane, and if the deviation is smaller than a threshold value, performing the step S54; if the deviation is greater than the threshold, adjusting the external parameters Tx, Ty, RθThen, repeating the step S53;
and S54, moving the corner reflector to different positions, and repeating the operation of the step S53 to calculate the projection deviation until the projection deviations at different positions are all smaller than the threshold value.
From the two projection formulas, the measurement points in the radar coordinate system are projected to the image coordinate system, namely the image plane, through the radar and camera coordinate system conversion matrix and the camera internal reference matrix respectively. In both equations, Kc has been determined through the camera calibration step, so only the rotation angle R for the coordinate system is requiredθiAnd the value of the transverse-longitudinal translation TxiAnd TyiAnd (5) calibrating.
Relative rotation angle R to a coordinate systemθiAnd the value of the transverse-longitudinal translation TxiAnd TyiPerforming preliminary measurement, determining initial external parameters, and obtaining [ x ] by calculationr1，yr1]And [ x ]r2，yr2]Initial projection position in the image plane. If the initial external reference measurement is accurate, the projection point of the corner reflector is exactly located at the imaging position, but the angle cannot be accurately measured usually, and the distance measurement has partial deviation. In general, R is increased or decreasedθThe projection point of the corner reflector moves left and right regularly on the image plane, so that R can be conveniently and quickly carried outθFine tuning of (3).
Through the projection process of the step, the target point of the corner reflector measured by the millimeter wave radar can be displayed in the image plane.
According to the rapid combined calibration method for the millimeter wave radar and the camera, as an optimal mode, external parameters comprise Tx, Ty and RθTx is the translation distance between the millimeter wave radar and the camera in the transverse direction, Ty is the translation distance between the millimeter wave radar and the camera in the longitudinal direction, RθIs the yaw angle of the millimeter wave radar relative to the camera.
According to the rapid combined calibration method for the millimeter wave radar and the camera, as an optimal mode, a radar group comprises at least two radars, and the radars are symmetrically arranged on the motion platform.
The invention has the following beneficial effects:
(1) the corner reflector has high reflection intensity and can be accurately obtained by the millimeter wave radar, so that the problems of large reference object target, irregular shape, poor reflection intensity and the like are avoided, the conditions that more than one projection point of a millimeter wave radar measuring point in an image plane, inaccurate transverse position of the projection point and the like are easily generated, and the combined calibration precision of the millimeter wave radar and vision is influenced;
(2) the size of the corner reflector is small, and better feedback can be obtained by utilizing the pixel deviation of a projection point of the corner reflector and an imaging point of the corner reflector in an image plane, so that the projection parameters are adjusted, and the joint calibration speed is improved;
(3) the method can be used for simultaneously carrying out combined calibration on a plurality of millimeter wave radars and the camera, and can be better adapted to a motion platform perception scheme which needs a plurality of millimeter wave radars and vision to be fused.
Drawings
FIG. 1 is a flow chart of a method for rapid calibration of millimeter wave radar and camera;
FIG. 2 is a left side view of the installation of a millimeter wave radar and a camera in a rapid combined calibration method of the millimeter wave radar and the camera;
fig. 3 is a top view of the installation of the millimeter wave radar and the camera in the rapid combined calibration method of the millimeter wave radar and the camera.
Reference numerals:
1. a motion platform; 2. a camera device; 3. a radar group; 4. a corner reflector.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
Example 1
As shown in fig. 1, a method for quickly calibrating a millimeter wave radar and a camera 2 in a combined manner includes the following steps:
s1, mounting the millimeter wave radar and the camera 2 to a fixed position of the motion platform 1, namely the position of a vehicle head, and placing the motion platform 1 in an open flat place, which can be an outdoor square or an indoor flat hall;
s2, placing the corner reflector 4 right in front of the millimeter wave radar to enable the corner reflector to be right opposite to the vehicle head, adjusting the height of the corner reflector to enable the reflector to be approximately as high as the millimeter wave radar, and adjusting the distance of the reflector to enable the reflector to appear in the image of the camera 2;
s3, calibrating internal parameters of the binocular camera 2 by using the pinhole camera model, and selecting a left target coordinate system as a camera coordinate system in the embodiment because millimeter wave radar measurement data only need to be projected to one image plane in general application. The calibration adopts a Zhangyingyou calibration method, namely a hard non-deformation checkerboard calibration board is used as a calibration reference object, the position of the calibration board is continuously moved, and the camera model parameters are solved through the corresponding relation between the coordinates of characteristic points (checkerboard angular points) in an image and world coordinates, so that the internal reference matrix of the camera is obtained
Wherein f isxAnd fyAre the focal length u of the camera, respectively0And v0Coordinates of the optical center of the camera under an image coordinate system;
and S4, adjusting the millimeter wave radar to enable the bottom surface to be parallel to the ground and the side surface to be perpendicular to the ground, and in order to guarantee the verticality, assisting with a plumb bob. If the plumb bob does not exist, the adjustment can be carried out according to the actual measurement effect;
s5, measuring the distance between the projection position of the radar group 3 on the ground and the corner reflector 4, comparing the distance with the distance of the corner reflector 4 measured by the radar group 3, and if the deviation is smaller than a threshold value, performing step S6; if the deviation is larger than or equal to the threshold value, adjusting the roll angle and the pitch angle of the radar, and repeating the step S5;
s6, measuring the translation distances Tx and Ty of the millimeter wave radar and the camera in the transverse direction and the longitudinal direction, and measuring the yaw angle R of the millimeter wave radar relative to the cameraθDetermining initial projection external parameters of the millimeter wave coordinate system Xr-Yr to the camera coordinate system Xc-Yc;
s7, projecting the target point of the corner reflector measured by the radar group 3 on an image plane through initial projection external parameters and camera internal parameters, and setting the coordinates of the radar measuring point as (x)r，yr) The projected point of the coordinate system of the camera is (x)c，yc) If the coordinates of the projection point in the image plane are (u, v), the projection relation of the projection point to the radar measuring point in the camera coordinate system is
S8, projecting the target point of the corner reflector in the camera coordinate system to the image plane, wherein the projection equation is
S9, calculating the deviation of the imaging position of the corner reflector 4 in the imaging plane and the projection position of the radar detection point on the imaging plane, and if the deviation is smaller than a threshold value, performing the step S10; if the deviation is greater than the threshold, adjusting the external parameters Tx, Ty, RθThen, repeating the step S9;
s10, moving the corner reflector 4 to at least three different positions of far, middle and near, and taking the left, middle and right positions as each distance with three different distances of far, middle and near, wherein 9 positions are the best. Wherein, the long distance is preferably 30-50m, the middle distance is preferably 10-30m, the short distance is preferably 5-10m, the step S9 is repeated, if the projection point of the corner reflector 4 has a pixel deviation with fixed size and direction with the image display position when moving to the positions, Tx and Ty can be adjusted to eliminate;
s11, outputting camera internal parameter matrix Kc, external parameters Tx, Ty, Rθ。
Since the sizes of pixels occupied by the corner reflector 4 are different when the corner reflector is far from the camera 2, the size of the image collected by the camera is 1280 × 720 pixels, in this embodiment, 20 pixels are selected as a deviation threshold, that is, the corner reflector 4 moves to 9 different positions, and the average deviation between the projection point of the position of each radar-acquired corner reflector 4 on the image plane and the actual image-forming position is within 20 pixels, then the calibration is considered to be completed. Recording camera internal parameters, internal parameters converted by the two millimeter wave radars and the camera coordinate system, and completing calibration.
The camera 2 in this embodiment may be a monocular camera, a binocular camera, a multi-view camera, an infrared camera, or the like.
As shown in fig. 2 and 3, the camera 2 is installed in the center of the front end of the moving platform 1, the radar group 3 is composed of two radar groups 3, the two radars are symmetrically arranged on the moving platform 1 at the two sides of the camera 2, and the corner reflector 4 is arranged in the front of the moving platform 1.
The radar group 3 can be a plurality of millimeter wave radars, that is, can jointly mark a plurality of millimeter wave radars and the camera 2, if the number of millimeter wave radars is more than or equal to two, only need place the corner reflector 4 in the position that a plurality of millimeter wave radars can detect simultaneously, and need adjust the position of corner reflector 4 and make it appear in the camera 2 image.
In conclusion, the millimeter wave radar and camera 2 calibration needs less constraint and assumption, and calibration errors caused by irregular reference objects and reference objects with small reflection intensity can be avoided by introducing the corner reflector 4; meanwhile, a plurality of sensors can be calibrated, and a plurality of positions are selected for deviation elimination, so that the calibration accuracy is improved; by observing or measuring the deviation change of the projection point and the imaging point of the corner reflector 4 caused by the external parameter change of the projection, the external parameter adjustment is directly fed back, the complicated mathematical calculation is not needed, the steps are simplified, and the rapid calibration of the sensor can be realized.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.
Claims (7)
1. A quick combined calibration method for a millimeter wave radar and a camera is characterized by comprising the following steps: the method comprises the following steps:
s1, mounting the camera (2) and the radar group (3) to the moving platform (1), and placing the moving platform (1) in an open flat position;
s2, arranging a corner reflector (4) right in front of the motion platform (1), adjusting the height of the corner reflector (4) to be the same as that of the radar group (3), and adjusting the distance between the corner reflector (4) and the radar group (3) to enable the corner reflector (4) to appear in the image range of the camera (2);
s3, calibrating the internal parameters of the camera (2) by using the pinhole camera model and adjusting the radar angle;
s4, determining initial projection parameters of the coordinate system Xr-Yr of the radar group (3) to the camera coordinate system Xc-Yc;
s5, adjusting the values of the yaw angle and the translation distance in the external parameters by calculating the deviation of the imaging position of the corner reflector (4) in the imaging plane from the projection position of the radar detection point on the imaging plane;
and S6, outputting the internal reference matrix and the external reference of the camera (2), and completing calibration.
2. The millimeter wave radar and camera rapid combined calibration method according to claim 1, characterized in that: the specific method of the step S3 is as follows:
s31, calibrating the internal reference of the camera (2) by using a checkerboard calibration board, wherein the internal reference matrix of the camera (2) is as follows:
wherein f isxAnd fyIs the focal length, u, of the camera (2)0And v0Is the image principal point coordinate;
s32, adjusting the transmitting and receiving planes of the radar group (3) to be vertical to the ground;
s33, measuring the distance between the projection position of the radar group (3) on the ground and the corner reflector (4), comparing the distance with the distance of the corner reflector (4) measured by the radar group (3), and if the deviation is smaller than a threshold value, performing the step S4; if the deviation is greater than or equal to the threshold value, the roll angle and the pitch angle of the radar are adjusted, and the step S33 is repeated.
3. The millimeter wave radar and camera rapid combined calibration method according to claim 1, characterized in that: the specific method of the step S3 is as follows:
s31, adjusting the transmitting and receiving planes of the radar group (3) to be vertical to the ground;
s32, measuring the distance between the projection position of the radar group (3) on the ground and the corner reflector (4), comparing the distance with the distance of the corner reflector (4) measured by the radar group (3), and if the deviation is smaller than a threshold value, performing the step S4; if the deviation is larger than or equal to the threshold value, adjusting the roll angle and the pitch angle of the radar, and repeating the step S32;
s33, calibrating the internal reference of the camera (2) by using a checkerboard calibration board, wherein the internal reference matrix of the camera (2) is as follows:
wherein f isxAnd fyIs the focal length, u, of the camera (2)0And v0Is the image principal point coordinate.
4. The millimeter wave radar and camera rapid combined calibration method according to claim 1, characterized in that: the specific method of the step S4 is as follows: measuring the translation distances Tx and Ty of the radar group (3) and the camera (2) in the transverse direction and the longitudinal direction, and measuring the radar group (3)) Yaw angle R relative to camera (2)θAnd thus determining initial projection parameters of the millimeter wave coordinate system Xr-Yr to the camera coordinate system Xc-Yc.
5. The millimeter wave radar and camera rapid combined calibration method according to claim 1, characterized in that: the step S5 specifically includes:
s51, projecting the target point of the corner reflector measured by the radar group (3) to an image plane through initial projection external parameters and the internal parameters of the camera (2), and setting the coordinates of the radar measuring point as (x)r，yr) The projection point of the system on the camera coordinate system is (x)c，yc) If the coordinates of the projection point in the image plane are (u, v), the projection relation of the projection point to the radar measuring point in the camera coordinate system is (u, v)
S52, projecting the target point of the corner reflector under the camera coordinate system to an image plane, wherein the projection equation is
S53, calculating the deviation of the projection position of the imaging position of the corner reflector (4) in the imaging plane and the radar detection point thereof on the imaging plane, and if the deviation is less than a threshold value, performing the step S54; if the deviation is greater than the threshold, adjusting the external parameters Tx, Ty, RθThen, repeating the step S53;
s54, moving the corner reflector (4) to different positions, and repeating the operation of the step S53 to calculate the projection deviation until the projection deviations at the different positions are all smaller than the threshold value.
6. The millimeter wave radar and camera rapid combined calibration method according to claim 1, characterized in that: the external parameters include Tx, Ty, RθThe Tx is the horizontal translation distance between the radar group (3) and the camera (2) and the Ty is the longitudinal translation distance between the radar group (3) and the camera (2)Translation distance, said RθIs the yaw angle of the radar group (3) relative to the camera (2).
7. The millimeter wave radar and camera rapid combined calibration method according to claim 1, characterized in that: the radar group (3) comprises at least two radars which are symmetrically arranged on the moving platform (1).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010616507.5A CN112070841A (en) | 2020-07-01 | 2020-07-01 | Rapid combined calibration method for millimeter wave radar and camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010616507.5A CN112070841A (en) | 2020-07-01 | 2020-07-01 | Rapid combined calibration method for millimeter wave radar and camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112070841A true CN112070841A (en) | 2020-12-11 |
Family
ID=73656186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010616507.5A Pending CN112070841A (en) | 2020-07-01 | 2020-07-01 | Rapid combined calibration method for millimeter wave radar and camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112070841A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112882024A (en) * | 2021-03-25 | 2021-06-01 | 浙江大华技术股份有限公司 | Radar detection method and apparatus, storage medium, and electronic apparatus |
| CN113256734A (en) * | 2021-05-20 | 2021-08-13 | 东风汽车集团股份有限公司 | Vehicle-mounted sensing sensor calibration method and system and electronic equipment |
| CN113281723A (en) * | 2021-05-07 | 2021-08-20 | 北京航空航天大学 | Calibration method for structural parameters between 3D laser radar and camera based on AR tag |
| CN113359097A (en) * | 2021-06-21 | 2021-09-07 | 北京易航远智科技有限公司 | Millimeter wave radar and camera combined calibration method |
| CN113514803A (en) * | 2021-03-25 | 2021-10-19 | 武汉光庭信息技术股份有限公司 | Combined calibration method for monocular camera and millimeter wave radar |
| CN114279468A (en) * | 2021-12-31 | 2022-04-05 | 北京理工大学 | Dynamic calibration method for millimeter wave radar and visual camera based on statistical analysis |
| CN114460552A (en) * | 2022-01-21 | 2022-05-10 | 苏州皓宇云联科技有限公司 | Road-end multi-sensor combined calibration method based on high-precision map |
| CN115346368A (en) * | 2022-07-30 | 2022-11-15 | 东南大学 | Traffic roadside sensing system and method based on integration of far and near view multiple sensors |
| CN115661270A (en) * | 2022-12-22 | 2023-01-31 | 陕西欧卡电子智能科技有限公司 | Online calibration method for millimeter wave radar and fisheye camera |
| CN116416319A (en) * | 2022-11-17 | 2023-07-11 | 南京理工大学 | Intelligent driving multi-type sensor calibration-oriented one-time combined calibration method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1693922A (en) * | 2005-05-10 | 2005-11-09 | 中国科学院电子学研究所 | Combined measuring device using machine interference synthetic aperture radar base line |
| US20190120934A1 (en) * | 2017-10-19 | 2019-04-25 | GM Global Technology Operations LLC | Three-dimensional alignment of radar and camera sensors |
| CN109828250A (en) * | 2019-03-28 | 2019-05-31 | 森思泰克河北科技有限公司 | A kind of Radar Calibration method, caliberating device and terminal device |
| CN110390697A (en) * | 2019-07-11 | 2019-10-29 | 浙江大学 | A kind of millimetre-wave radar based on LM algorithm and camera combined calibrating method |
-
2020
- 2020-07-01 CN CN202010616507.5A patent/CN112070841A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1693922A (en) * | 2005-05-10 | 2005-11-09 | 中国科学院电子学研究所 | Combined measuring device using machine interference synthetic aperture radar base line |
| US20190120934A1 (en) * | 2017-10-19 | 2019-04-25 | GM Global Technology Operations LLC | Three-dimensional alignment of radar and camera sensors |
| CN109828250A (en) * | 2019-03-28 | 2019-05-31 | 森思泰克河北科技有限公司 | A kind of Radar Calibration method, caliberating device and terminal device |
| CN110390697A (en) * | 2019-07-11 | 2019-10-29 | 浙江大学 | A kind of millimetre-wave radar based on LM algorithm and camera combined calibrating method |
Non-Patent Citations (1)
| Title |
|---|
| 罗逍;姚远;张金换: "《一种毫米波雷达和摄像头联合标定方法》", 《清华大学学报》, pages 1 - 5 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113514803A (en) * | 2021-03-25 | 2021-10-19 | 武汉光庭信息技术股份有限公司 | Combined calibration method for monocular camera and millimeter wave radar |
| CN112882024A (en) * | 2021-03-25 | 2021-06-01 | 浙江大华技术股份有限公司 | Radar detection method and apparatus, storage medium, and electronic apparatus |
| CN113281723A (en) * | 2021-05-07 | 2021-08-20 | 北京航空航天大学 | Calibration method for structural parameters between 3D laser radar and camera based on AR tag |
| CN113256734A (en) * | 2021-05-20 | 2021-08-13 | 东风汽车集团股份有限公司 | Vehicle-mounted sensing sensor calibration method and system and electronic equipment |
| CN113359097B (en) * | 2021-06-21 | 2022-09-02 | 北京易航远智科技有限公司 | Millimeter wave radar and camera combined calibration method |
| CN113359097A (en) * | 2021-06-21 | 2021-09-07 | 北京易航远智科技有限公司 | Millimeter wave radar and camera combined calibration method |
| CN114279468A (en) * | 2021-12-31 | 2022-04-05 | 北京理工大学 | Dynamic calibration method for millimeter wave radar and visual camera based on statistical analysis |
| CN114279468B (en) * | 2021-12-31 | 2022-06-14 | 北京理工大学 | Dynamic calibration method for millimeter wave radar and visual camera based on statistical analysis |
| CN114460552A (en) * | 2022-01-21 | 2022-05-10 | 苏州皓宇云联科技有限公司 | Road-end multi-sensor combined calibration method based on high-precision map |
| CN115346368A (en) * | 2022-07-30 | 2022-11-15 | 东南大学 | Traffic roadside sensing system and method based on integration of far and near view multiple sensors |
| CN115346368B (en) * | 2022-07-30 | 2024-01-05 | 东南大学 | Traffic road side sensing system and method based on integrated fusion of far-view and near-view multiple sensors |
| CN116416319A (en) * | 2022-11-17 | 2023-07-11 | 南京理工大学 | Intelligent driving multi-type sensor calibration-oriented one-time combined calibration method |
| CN116416319B (en) * | 2022-11-17 | 2023-11-24 | 南京理工大学 | Intelligent driving multi-type sensor calibration-oriented one-time combined calibration method |
| CN115661270A (en) * | 2022-12-22 | 2023-01-31 | 陕西欧卡电子智能科技有限公司 | Online calibration method for millimeter wave radar and fisheye camera |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112070841A (en) | Rapid combined calibration method for millimeter wave radar and camera | |
| CN112669393B (en) | Laser radar and camera combined calibration method | |
| CN107389026B (en) | A kind of monocular vision distance measuring method based on fixed point projective transformation | |
| CN110349221A (en) | A kind of three-dimensional laser radar merges scaling method with binocular visible light sensor | |
| CN103487034B (en) | Method for measuring distance and height by vehicle-mounted monocular camera based on vertical type target | |
| CN103499337B (en) | Vehicle-mounted monocular camera distance and height measuring device based on vertical target | |
| US20100235129A1 (en) | Calibration of multi-sensor system | |
| CN112880642B (en) | Ranging system and ranging method | |
| CN109961468A (en) | Volume measuring method, device and storage medium based on binocular vision | |
| CN110837080A (en) | Rapid calibration method of laser radar mobile measurement system | |
| CN111243029B (en) | Calibration method and device of vision sensor | |
| CN110208771B (en) | Point cloud intensity correction method of mobile two-dimensional laser radar | |
| CN110031829A (en) | A kind of targeting accuracy distance measuring method based on monocular vision | |
| CN112581545A (en) | Multi-mode heat source recognition and three-dimensional space positioning system, method and storage medium | |
| CN113808219B (en) | Deep learning-based radar auxiliary camera calibration method | |
| CN108648242A (en) | Two camera scaling methods and device without public view field are assisted based on laser range finder | |
| CN111736170B (en) | Device and method for monitoring deformation degree of cutting side slope | |
| CN115079143B (en) | Multi-radar external parameter quick calibration method and device for double-bridge steering mine card | |
| CN114413958A (en) | Monocular vision distance and speed measurement method of unmanned logistics vehicle | |
| CN114442073A (en) | Laser radar calibration method and device, vehicle and storage medium | |
| CN109712198B (en) | Calibration method of advanced driving assistance system | |
| CN111123260B (en) | Method for identifying state of environmental object by using millimeter wave radar and visible light camera | |
| CN111830519B (en) | Multi-sensor fusion ranging method | |
| CN113465572A (en) | Monocular distance measurement method and system based on camera imaging geometric relation | |
| CN117496467A (en) | Special-shaped lane line detection method based on fusion of monocular camera and 3D LIDAR |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |