Disclosure of Invention
In order to solve the problems in the prior art, the embodiment of the invention provides a vehicle sensor calibration method, a vehicle sensor calibration device and a storage medium, and the technical scheme is as follows:
in one aspect, a vehicle sensor calibration method is provided, and the method includes:
acquiring position measurement data of a laser radar and a sensor to be calibrated aiming at a calibration tool;
converting the position measurement data of the laser radar according to the external parameter matrix to be calibrated to obtain converted position data;
determining a loss value according to the conversion position data and the position measurement data corresponding to the sensor to be calibrated;
and reversely adjusting the external parameter matrix to be calibrated according to the loss value until the loss value meets a preset condition.
In another aspect, a vehicle sensor calibration apparatus is provided, the apparatus comprising:
a position measurement data acquisition module: the device is used for acquiring position measurement data of the laser radar and the sensor to be calibrated aiming at the calibration tool;
a conversion location data determination module: the position measurement data of the laser radar are converted according to the external parameter matrix to be calibrated to obtain conversion position data;
a loss value determination module: the loss value is determined according to the conversion position data and the position measurement data corresponding to the sensor to be calibrated;
a reverse adjustment module: and the external parameter matrix to be calibrated is reversely adjusted according to the loss value until the loss value meets a preset condition.
Further, when the sensor to be calibrated is a millimeter wave radar, the external reference matrix to be calibrated is a first external reference matrix;
the loss value determination module includes:
a first sum determination module: for determining a sum of the converted position data and a first origin translation amount; the first origin translation amount represents the translation amount from the origin of the laser radar coordinate system to the origin of the millimeter wave radar coordinate system;
a first loss value determination module: and the first loss value is determined according to the square of the first difference value.
Further, when the sensor to be calibrated is a millimeter wave radar, a precision improving component is arranged on the calibration tool, and the precision improving component is used for improving the calibration precision of the millimeter wave radar.
Further, when the sensor to be calibrated is a camera, the external reference matrix to be calibrated is a second external reference matrix;
the loss value determination module further includes:
a second sum determination module: for determining a sum of the converted position data and a second origin translation amount; the second origin translation amount represents the translation amount from the origin of the laser radar coordinate system to the origin of the camera coordinate system;
a second loss value determination module: and the device is used for determining a second difference value between the position measurement data corresponding to the camera and the sum value, and determining a second loss value according to the square of the second difference value.
Further, the apparatus further comprises:
a first rotation matrix and translation matrix determination module: the first rotating matrix and the first translation matrix are determined according to the first external parameter matrix when the first loss value meets the preset condition;
a second rotation matrix and translation matrix determination module: the second rotation matrix and the second translation matrix are determined according to the second external parameter matrix when the second loss value meets the preset condition;
a third rotation matrix determination module: for determining a third rotation matrix from the first and second rotation matrices;
a third translation matrix determination module: the device comprises a first translation matrix, a second translation matrix and a third translation matrix, wherein the first translation matrix and the second translation matrix are used for determining the first translation matrix and the second translation matrix;
a third external parameter matrix determination module: the third external parameter matrix is determined according to the third rotation matrix and the third translation matrix; and the third external parameter matrix represents an external parameter matrix for calibrating the camera by the millimeter wave radar.
Further, the apparatus further comprises:
a calibration result determination module: the calibration method comprises the steps of determining a calibration result according to a first external parameter matrix when the first loss value meets a preset condition, a second external parameter matrix when the second loss value meets a preset condition, and a third external parameter matrix; and visualizing the calibration result;
if the calibration result is within a preset error range, ending calibration;
and if the calibration result is not within the preset error range, re-executing the step of obtaining the position measurement data of the laser radar and the sensor to be calibrated aiming at the calibration tool until the calibration result is within the preset error range.
Another aspect provides a storage medium having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by a processor to implement the vehicle sensor calibration method as described above.
The invention provides a vehicle sensor calibration method, a vehicle sensor calibration device and a storage medium, which have the following technical effects:
according to the embodiment of the invention, position measurement data of a laser radar and a sensor to be calibrated aiming at a calibration tool are obtained, wherein the sensor to be calibrated is a millimeter wave radar and a camera; secondly, the position measurement data of the laser radar is converted according to the external parameter matrix to be calibrated to obtain the conversion position data of the position coordinate of the laser radar projected to the millimeter wave radar and the camera, the position of the sensor to be calibrated is converted by the same sensor, the precision of the combined calibration of the sensor is improved, then, according to the obtained conversion position data and the position measurement data of the sensor to be calibrated, determining a loss value, iterative calculation is carried out on the loss value to obtain an optimal loss value, and then the external parameter matrix to be calibrated is reversely adjusted according to the optimal loss value, calibrating the vehicle sensor according to the adjusted external parameter matrix to be calibrated, visualizing the calibration result, the calibration result can be rapidly and visually displayed, the observation is convenient, the accurate calibration of the vehicle sensor is realized through the technical scheme, and a good cushion is made for the fusion of the sensor.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
It is noted that the present specification provides the method steps as described in the examples or flowcharts, but may include more or less steps based on routine or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. In practice, the system or server product may be implemented in a sequential or parallel manner (e.g., parallel processor or multi-threaded environment) according to the embodiments or methods shown in the figures.
Example (b):
referring to fig. 1, a schematic flow chart of a vehicle sensor calibration method according to an embodiment of the present invention is shown, where the method specifically includes the following steps:
s101: acquiring position measurement data of a laser radar and a sensor to be calibrated aiming at a calibration tool;
in the embodiment of the invention, the position measurement data of a calibration tool are respectively obtained according to a laser radar and a sensor to be calibrated, wherein the sensor to be calibrated is a millimeter wave radar and a camera, the position measurement data of the laser radar is converted according to an external parameter matrix to be calibrated to obtain the conversion position data in the sensor to be calibrated, a loss value is determined according to the conversion position data and the position measurement data of the sensor to be calibrated, iterative calculation is carried out on the loss value to obtain the loss value meeting a preset condition, the external parameter matrix to be calibrated is reversely adjusted according to the loss value meeting the preset condition, and the vehicle sensor is calibrated according to the adjusted external parameter matrix to be calibrated.
S102: converting the position measurement data of the laser radar according to the external parameter matrix to be calibrated to obtain converted position data;
in the embodiment of the invention, when the sensor to be calibrated is a millimeter wave radar, the calibration tool is provided with the precision improving component, the precision improving component is arranged for improving the calibration precision of the millimeter wave radar, at the moment, the corresponding external parameter matrix to be calibrated is a first external parameter matrix, and the position measurement data of the laser radar is converted according to the first external parameter matrix, so that the converted position data of the position measurement data of the laser radar converted into the millimeter wave radar is obtained.
And when the sensor to be calibrated is a camera, the corresponding external parameter matrix to be calibrated is a second external parameter matrix, and the position measurement data of the laser radar is converted according to the second external parameter matrix to obtain the conversion position data of the position measurement data of the laser radar converted into the camera.
S103: determining a loss value according to the conversion position data and the position measurement data corresponding to the sensor to be calibrated;
in an optional implementation manner, when the sensor to be calibrated is a millimeter wave radar, it is determined that position measurement data of the laser radar is converted into conversion position data in the millimeter wave radar, a position coordinate value of the position measurement data of the laser radar converted into the millimeter wave radar is determined according to a sum of the conversion position data converted into the millimeter wave radar and a translation amount of a first origin, wherein the translation amount of the first origin represents a translation amount from an origin of a coordinate system of the laser radar to the origin of the coordinate system of the millimeter wave radar, the position coordinate value is subtracted from the position measurement data corresponding to the millimeter wave radar to obtain a first difference value, and the first difference value is squared to obtain a first loss value. It should be noted that, the first loss value is determined by summing the first difference values of the plurality of positions, which are brought in the process of calculating the first loss value, and the range, which can be measured by the laser radar, the millimeter wave radar and the camera together, should be covered as much as possible in the selection of the plurality of positions, which avoids the situation that the calculation of the first loss value is inaccurate due to too dense distance of the selected positions to a certain extent.
Specifically, the first loss value may be expressed as the following equation:
wherein L is
1Is a first loss value, x
radarAnd y
radarPosition measurement, x, for a millimeter-wave radar for a calibration tool
lidarAnd y
lidarFor position measurements of the lidar to a calibration tool,
a first rotation matrix corresponding to the conversion position data for converting the lidar to the millimeter wave radar, theta is a rotation angle of the lidar projected to the millimeter wave radar,
for the first origin translation amount, it should be noted that the first origin translation amount is equal to the first translation matrix in value.
By performing iterative calculation on the first loss value and stopping the iterative calculation until the first loss value meets the preset condition, and then reversely adjusting the first external reference matrix, specifically, obtaining an initial first loss value through the sum value of the position measurement data corresponding to the millimeter wave radar and the converted position data converted from the laser radar to the millimeter wave radar obtained by calculation and the first origin translation amount, judging whether the initial first loss value meets a preset condition or not, if not, then the rotation angle projected by the laser radar to the millimeter wave radar and the translation amount of the first origin are readjusted, the first loss value is repeatedly calculated until the preset condition is met, the iterative calculation is ended, and determining a first external parameter matrix through the rotation angle projected to the millimeter-wave radar by the corresponding laser radar when the iteration is finished and the translation amount of the first origin, and calibrating the millimeter-wave radar by the laser radar according to the adjusted first external parameter matrix.
In an optional embodiment, when the sensor to be calibrated is a camera, determining that position measurement data of the laser radar is converted into conversion position data in the camera, determining that the position measurement data of the laser radar is converted into a position coordinate value in the camera according to a sum of the conversion position data converted into the camera by the position measurement data of the laser radar and a second origin translation amount, wherein the second origin translation amount represents a translation amount from an origin of a coordinate system of the laser radar to the origin of the coordinate system of the camera, subtracting the position coordinate value from the position measurement data corresponding to the camera to obtain a second difference value, and squaring the second difference value to obtain a second loss value. It should be noted that, the second loss value is determined by summing the second differences at the plurality of positions, which are brought in during the second loss value calculation.
Specifically, the second loss value may be expressed as the following equation:
wherein L is2For the second loss value, Z and K are adjustable parameters, u and v, l are position measurements of the camera for the calibration tool, xlidarAnd ylidar、zlidarThe position measurement value of the laser radar for the calibration tool is obtained, R is a second rotation matrix corresponding to conversion position data converted from the laser radar to the camera, and T is a second origin point translation amount.
The iterative calculation is stopped until the second loss value meets the preset condition, and then adjusting the second external parameter matrix reversely, specifically, obtaining an initial second loss value through the position measurement data corresponding to the camera and the sum value of the converted position data converted from the laser radar to the camera and the second origin translation amount, judging whether the initial second loss value meets the preset condition or not, if not, readjusting the rotation angle projected to the camera by the laser radar and the translation amount of the second origin, repeatedly calculating a second loss value until a preset condition is met, ending iterative calculation, and determining a second appearance parameter matrix through the rotation angle projected to the camera by the corresponding laser radar when the iteration is finished and the second origin point translation amount, and calibrating the camera by the laser radar according to the adjusted second appearance parameter matrix.
S104: and reversely adjusting the external parameter matrix to be calibrated according to the loss value until the loss value meets a preset condition.
In the embodiment of the invention, the first external parameter matrix is reversely adjusted according to the first loss value to obtain the first loss value meeting the preset condition, the reverse adjustment method comprises the steps of carrying out iterative calculation on the first loss value by using a gradient descent method until the first loss value meets the preset condition, further determining the first external parameter matrix when the preset condition is met, and calibrating the millimeter wave radar by the laser radar according to the adjusted first external parameter matrix.
Similar to the calibration of the millimeter wave radar by the laser radar, the calibration of the camera by the laser radar is to reverse the second appearance parameter matrix according to the second loss value to obtain a second loss value meeting the preset condition, the reverse adjustment method is to perform iterative calculation on the second loss value by using a gradient descent method until the second loss value meets the preset condition, further determine the second appearance parameter matrix when the preset condition is met, and realize the calibration of the camera by the laser radar according to the adjusted second appearance parameter matrix.
In an optional implementation manner, after the millimeter wave radar and the camera are calibrated by the laser radar, a third external parameter matrix is determined according to the adjusted first external parameter matrix and the adjusted second external parameter matrix, wherein the third external parameter matrix represents the external parameter matrix for the millimeter wave radar to calibrate the camera. Specifically, a corresponding first rotation matrix and a corresponding first translation matrix are determined according to the adjusted first external parameter matrix, a corresponding second rotation matrix and a corresponding second translation matrix are determined according to the adjusted second external parameter matrix, a third rotation matrix is determined according to the first rotation matrix and the second rotation matrix, a third translation matrix is further determined according to the first translation matrix and the second translation matrix, and a third external parameter matrix is determined according to the third rotation matrix and the third translation matrix, so that calibration of the millimeter wave radar on the camera is realized, and further, combined calibration of the laser radar, the millimeter wave radar and the camera is realized.
Specifically, the adjusted first rotation matrix is denoted as R1And the adjusted second rotation matrix is denoted as R2And the adjusted second translation matrix is recorded as T1And the adjusted second translation matrix is recorded as T2And the third rotation matrix is denoted as R3And the third translation matrix is denoted as T3Then R is3=R2*R1 -1
T3=T2-R1 -1*T1
And further calibrating the camera by the millimeter wave radar according to the third external parameter matrix obtained by calculation.
Specifically, after the laser radar, the millimeter wave radar and the camera are calibrated to obtain calibration, the calibration result is verified, whether the calibration process is finished or not is judged according to the verification result, the verification process is to visualize the calibration result through a visualization tool to obtain a macroscopic calibration result, and whether the calibration process is finished or not is judged according to the result displayed on the visualization tool. Specifically, whether the calibration result is within a preset error range or not is judged, if the calibration result is within the preset error range, the calibration is finished, if the calibration result is not within the preset error range, the step of acquiring the position measurement data of the laser radar and the sensor to be calibrated aiming at the calibration tool is executed again, until the calibration result is within the preset error range, the calibration time of the vehicle sensor is reduced due to the implementation of the visual calibration result, the calibration process is simplified, meanwhile, an observer can conveniently and quickly acquire the calibration result, the efficiency and the accuracy of the calibration of the vehicle sensor are improved, and a good cushion is made for the fusion of the sensor.
According to the technical scheme of the embodiment of the invention, the position measurement data of the laser radar and the sensor to be calibrated aiming at the calibration tool are obtained, wherein the sensor to be calibrated is a millimeter wave radar and a camera; secondly, the position measurement data of the laser radar is converted according to the external parameter matrix to be calibrated to obtain the conversion position data of the position coordinate of the laser radar projected to the millimeter wave radar and the camera, the position of the sensor to be calibrated is converted by the same sensor, the precision of the combined calibration of the sensor is improved, then, according to the obtained conversion position data and the position measurement data of the sensor to be calibrated, determining a loss value, iterative calculation is carried out on the loss value to obtain an optimal loss value, and then the external parameter matrix to be calibrated is reversely adjusted according to the optimal loss value, calibrating the vehicle sensor according to the adjusted external parameter matrix to be calibrated, visualizing the calibration result, the calibration result can be rapidly and visually displayed, the observation is convenient, the accurate calibration of the vehicle sensor is realized through the technical scheme, and a good cushion is made for the fusion of the sensor.
The embodiment of the present invention further provides a vehicle sensor calibration apparatus, as shown in fig. 2, which is a structural block diagram of the vehicle sensor calibration apparatus provided in the embodiment of the present invention, and the apparatus includes:
position measurement data acquisition module 10: the device is used for acquiring position measurement data of the laser radar and the sensor to be calibrated aiming at the calibration tool;
the conversion position data determination module 20: the position measurement data of the laser radar are converted according to the external parameter matrix to be calibrated to obtain conversion position data;
loss value determination module 30: the loss value is determined according to the conversion position data and the position measurement data corresponding to the sensor to be calibrated;
the reverse adjustment module 40: and the external parameter matrix to be calibrated is reversely adjusted according to the loss value until the loss value meets a preset condition.
Further, when the sensor to be calibrated is a millimeter wave radar, the external reference matrix to be calibrated is a first external reference matrix;
the loss value determination module 30 includes:
a first sum determination module: for determining a sum of the converted position data and a first origin translation amount; the first origin translation amount represents the translation amount from the origin of the laser radar coordinate system to the origin of the millimeter wave radar coordinate system;
a first loss value determination module: and the first loss value is determined according to the square of the first difference value.
Further, when the sensor to be calibrated is a millimeter wave radar, a precision improving component is arranged on the calibration tool, and the precision improving component is used for improving the calibration precision of the millimeter wave radar.
Further, when the sensor to be calibrated is a camera, the external reference matrix to be calibrated is a second external reference matrix;
the loss value determination module 30 further includes:
a second sum determination module: for determining a sum of the converted position data and a second origin translation amount; the second origin translation amount represents the translation amount from the origin of the laser radar coordinate system to the origin of the camera coordinate system;
a second loss value determination module: and the device is used for determining a second difference value between the position measurement data corresponding to the camera and the sum value, and determining a second loss value according to the square of the second difference value.
Further, the apparatus further comprises:
a first rotation matrix and translation matrix determination module: the first rotating matrix and the first translation matrix are determined according to the first external parameter matrix when the first loss value meets the preset condition;
a second rotation matrix and translation matrix determination module: the second rotation matrix and the second translation matrix are determined according to the second external parameter matrix when the second loss value meets the preset condition;
a third rotation matrix determination module: for determining a third rotation matrix from the first and second rotation matrices;
a third translation matrix determination module: the device comprises a first translation matrix, a second translation matrix and a third translation matrix, wherein the first translation matrix and the second translation matrix are used for determining the first translation matrix and the second translation matrix;
a third external parameter matrix determination module: the third external parameter matrix is determined according to the third rotation matrix and the third translation matrix; and the third external parameter matrix represents an external parameter matrix for calibrating the camera by the millimeter wave radar.
Further, the apparatus further comprises:
a calibration result determination module: the calibration method comprises the steps of determining a calibration result according to a first external parameter matrix when the first loss value meets a preset condition, a second external parameter matrix when the second loss value meets a preset condition, and a third external parameter matrix; and visualizing the calibration result;
if the calibration result is within a preset error range, ending calibration;
and if the calibration result is not within the preset error range, re-executing the step of obtaining the position measurement data of the laser radar and the sensor to be calibrated aiming at the calibration tool until the calibration result is within the preset error range.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Embodiments of the present invention also provide a storage medium for implementing at least one instruction, at least one program, a code set, or a set of instructions related to a vehicle sensor calibration method in the method embodiments, where the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the vehicle sensor calibration method provided in the above method embodiments.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.