WO2022127532A1

WO2022127532A1 - Method and apparatus for calibrating external parameter of laser radar and imu, and device

Info

Publication number: WO2022127532A1
Application number: PCT/CN2021/132432
Authority: WO
Inventors: 张国龙; 梁宝华
Original assignee: 华为技术有限公司
Priority date: 2020-12-16
Filing date: 2021-11-23
Publication date: 2022-06-23
Also published as: CN114636993A

Abstract

A method for calibrating an external parameter of a laser radar and an IMU, a calibration apparatus (1000, 1100), and a device for solving the problem in the prior art that an inter-frame matching algorithm is not suitable for external parameter calibration of a laser radar installed obliquely. The method comprises: the calibration apparatus (1000, 1100) converting point cloud data collected by the laser radar to an IMU coordinate system according to an external parameter value to be optimized (S502); converting the point cloud data from the IMU coordinate system to a reference coordinate system according to the relative conversion relationship between the IMU coordinate system and the reference coordinate system (S503); performing iterative adjustment on said external parameter value according to the point cloud data in the reference coordinate system, to obtain a target external parameter value (S504), avoiding the influence of the external parameter calibration result from the installation angle of the laser radar, achieving automatic calibration, and improving the external parameter calibration efficiency.

Description

A method, device and equipment for external parameter calibration of lidar and IMU

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application filed on December 16, 2020, with the application number of 202011492455.1 and the application title of "A method, device and equipment for external parameter calibration of lidar and IMU", all of which are The contents are incorporated herein by reference.

technical field

The embodiments of the present application relate to the technical field of automatic driving, and in particular, to a method, device, and device for calibrating external parameters of a lidar and an IMU.

Background technique

High-precision maps are the basis and necessary condition for realizing geographic information data for lane-level navigation and monitoring of unmanned vehicles. High-precision maps mainly rely on inertial measurement unit (IMU), global navigation satellite system (GNSS), lidar and other sensors for acquisition and production. In order to represent the data of different sensors in the same coordinate system, it is necessary to calibrate the external parameter values between different sensors.

At present, the lidar on the acquisition vehicle is usually installed in parallel with the roof. During calibration, the point cloud data of the common features of two adjacent frames scanned by the lidar are used, and an inter-frame matching algorithm, such as iterative nearest point (iterative closest point), is used. The closest point, ICP) algorithm calculates and collects the pose changes of the vehicle to obtain the trajectory of the lidar, and then combines with the integrated navigation system to give the trajectory of the IMU. Then, the least squares method can be used to complete the external parameter calibration of the lidar relative to the IMU.

However, in order to improve the accuracy of ground feature recognition, when collecting high-precision maps, the radar in the collecting vehicle can be installed obliquely. It is mainly the features on the ground, on the sides of the vehicle and on the obliquely above the roof, and the features in front of the collected vehicle cannot be scanned, which leads to fewer features scanned between adjacent frames, resulting in a large matching error between frames, which affects the External parameter calibration accuracy.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method, device, and device for calibrating external parameters of a lidar and an IMU, which are used to solve the problem that the inter-frame matching algorithm in the prior art is not suitable for calibrating external parameters of an obliquely installed lidar.

In the first aspect, an embodiment of the present application provides an external parameter calibration method for a lidar and an IMU, which is applied to a calibration device. Both the lidar and the IMU are fixedly installed on a vehicle to be calibrated, and the method includes:

Obtain the first point cloud data collected by the lidar, where the first point cloud data is used to represent the position in the lidar coordinate system of the features around the to-be-calibrated vehicle collected by the to-be-calibrated vehicle traveling on the target path ;

In the current adjustment, according to the external parameter value to be optimized, the first point cloud data is converted from the lidar coordinate system to the IMU coordinate system to obtain the second point cloud data, and the external parameter value to be optimized uses for indicating a first relative transformation relationship between the lidar coordinate system and the IMU coordinate system;

According to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted into the reference coordinate system to obtain third point cloud data; the second relative transformation relationship is based on Obtained from the position and attitude of the vehicle to be calibrated collected by the IMU when the vehicle to be calibrated travels on the target path;

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies When the preset conditions are used, the current external parameter value to be optimized is used as the target external parameter value; otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the target external parameter value in the next adjustment. Optimized extrinsic parameter values.

Through the above design, the calibration device converts the point cloud data collected by the lidar from the lidar coordinate system to the reference coordinate system, and iteratively optimizes the external parameter values to be optimized according to the point cloud data in the reference coordinate system until the current adjustment and use When the external parameter value to be optimized makes the same feature collected by the lidar traveling to different positions in the same position in the reference coordinate system or the position difference satisfies the preset condition, the current external parameter value to be optimized is taken as The target external parameter value avoids the influence of the installation angle of the lidar, which leads to the low accuracy and efficiency of the external parameter calibration result, realizes automatic external parameter calibration, and improves the external parameter calibration efficiency and accuracy.

In a possible design, the first point cloud data is collected by the lidar at N first collection moments, and obtaining the second relative transformation relationship between the IMU coordinate system and the reference coordinate system includes: Acquiring measurement data collected by the IMU at M second collection moments, the measurement data including the linear acceleration and angular velocity of the vehicle to be calibrated collected by the IMU while the vehicle to be calibrated is traveling on the target path ; According to the measurement data, the second relative conversion relationships at M second collection moments are obtained respectively, and the second relative conversion relationships at the second collection moments are used to characterize the IMU coordinates at the second collection moment. The relative transformation relationship between the system and the reference coordinate system.

Through the above design, the calibration device obtains the relative transformation relationship between the IMU coordinate system and the reference coordinate system at M second acquisition moments respectively according to the linear acceleration and angular velocity of the calibrated vehicle collected by the IMU, which improves the accuracy of external parameter calibration.

In a possible design, according to the second relative conversion relationship between the IMU coordinate system and the reference coordinate system, converting the second point cloud data to the reference coordinate system to obtain third point cloud data, including: According to the second relative conversion relationships at the M second collection moments, respectively, third relative conversion relationships at the N first collection moments are obtained, and the third relative conversion relationships at the first collection moments are used to represent the The first acquisition moment, the relative transformation relationship between the IMU coordinate system and the reference coordinate system; according to the third relative transformation relationship of the i-th first acquisition moment, respectively The second point cloud data is converted to the reference coordinate system to obtain the i-th point cloud data at the first collection moment, and i is taken as a positive integer less than or equal to N to obtain N point cloud data at the first collection moment forming the third point cloud data.

Through the above design, the calibration device can obtain the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the N first acquisition moments, respectively, according to the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the M second acquisition moments. , to convert the second point cloud data of N first acquisition moments from the IMU coordinate system to the reference coordinate system, improving the efficiency and accuracy of the external parameter calibration of lidar and IMU.

In a possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies The preset conditions include: if the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, then determine the current adjustment to be used. The optimized external parameter value makes the same feature collected by the lidar traveling to different positions in the same position in the reference coordinate system or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is the first feature. The sum of variances corresponding to the coordinates of the feature in the three dimensions of the reference coordinate system respectively, and the first feature is any one of the X features.

Through the above design, according to the sum of the error parameters of the X features, it is judged whether the external parameter value to be optimized can make the same feature collected by the lidar travel to different positions in the same position in the reference coordinate system or the position difference meets the predetermined Setting the condition can also be understood as the external parameter value to be optimized so that the same feature collected by the lidar traveling to different positions overlaps or basically overlaps in the reference coordinate system, which improves the external parameter calibration efficiency. The external parameter value (ie the initial external parameter value) is insensitive and has good convergence.

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies The preset conditions include: if the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment are all smaller than the second threshold, then determine the external parameters to be optimized used in the current adjustment. The parameter value enables the same feature collected by the lidar to travel to different positions in the same position in the reference coordinate system or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is that the first feature is in The sum of variances corresponding to the coordinates of the three dimensions of the reference coordinate system respectively, and the first feature is any one of the X features.

Through the above design, according to the error parameters of the X features, it is judged whether the external parameter values to be optimized can make the same feature collected by the lidar travel to different positions in the same position in the reference coordinate system or the position difference satisfies the preset conditions , improve the accuracy of external parameter calibration, and it is not sensitive to the initial external parameter value, the convergence is good, and the efficiency of external parameter calibration is improved.

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies The preset conditions include: if the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, and the current adjustment uses the to-be-optimized If the difference between the external parameter value and the external parameter value to be optimized used in the last adjustment is less than the third threshold, the external parameter value to be optimized used in the current adjustment is determined so that the lidar travels to a different location to collect The position of the same feature in the reference coordinate system is the same or the position difference satisfies the preset condition; wherein, the error parameter of the first feature is that the coordinates of the first feature in the three dimensions of the reference coordinate system correspond to The sum of the variances of , the first feature is any one of the X features.

Through the above design, according to the sum of the error parameters of the X features, and according to the external parameter value to be optimized in two adjacent adjustments, the external parameter value to be optimized used in the current adjustment is determined so that the lidar travels to The position of the same feature collected at different positions in the reference coordinate system is the same or the position difference satisfies a preset condition, while ensuring the calibration accuracy and calibration efficiency of external parameters.

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies The preset conditions include: if the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is less than the second threshold, and the external parameters to be optimized used in the current adjustment The difference between the value of the external parameter to be optimized and the value of the external parameter to be optimized used in the last adjustment is smaller than the third threshold, then the value of the external parameter to be optimized used for the current adjustment is determined to make the lidar travel to different locations to collect the same feature. The position of the object in the reference coordinate system is the same or the position difference satisfies the preset condition; wherein, the error parameter of the first feature is the difference of the corresponding variances of the coordinates of the first feature in the three dimensions of the reference coordinate system. And, the first feature is any one of the X features.

Through the above design, according to the error parameters of the X features, and according to the external parameter values to be optimized in two adjacent adjustments, the external parameter values to be optimized used in the current adjustment are determined, so that the lidar travels to different locations to collect data The position of the same feature in the reference coordinate system is the same or the position difference satisfies the preset condition, while ensuring the calibration accuracy and calibration efficiency of external parameters.

In a possible design, according to the second relative conversion relationship between the IMU coordinate system and the reference coordinate system, converting the second point cloud data to the reference coordinate system to obtain third point cloud data, including: According to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted from the IMU coordinate system to the reference coordinate system to obtain fourth point cloud data; The compensated fourth point cloud data is used as the third point cloud data, and the fourth point cloud data after motion compensation is based on all data collected by the IMU when the vehicle to be calibrated travels on the target path. The motion compensation for the position, attitude and speed of the vehicle to be calibrated is described.

Through the above design, according to the pose and speed of the vehicle to be calibrated, motion compensation is performed on the point cloud data to eliminate the motion error caused by the motion of the vehicle body and improve the accuracy of external parameter calibration.

In a second aspect, an embodiment of the present application provides a method for calibrating external parameters of a laser radar and an IMU, which is applied to a calibration device. Both the laser radar and the IMU are fixedly installed on a vehicle to be calibrated. The method includes:

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies When the preset conditions are used, the current external parameter value to be optimized is used as the target external parameter value; otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the target external parameter value in the next adjustment. Optimized extrinsic parameter values.

Through the above design, the calibration device can convert the point cloud data from the lidar coordinate system to the IMU coordinate system after acquiring the point cloud data collected by the lidar, and directly according to the point cloud data in the IMU coordinate system, the external parameters to be optimized Iteratively adjust the value until the external parameter value to be optimized used in the current adjustment can make the same feature collected by the lidar traveling to different positions overlap or partially overlap in the IMU coordinate system, so as to avoid the external parameter calibration result being affected by the lidar. The influence of the installation angle improves the efficiency and accuracy of the external parameter calibration of the lidar.

In a possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies The preset conditions include: if the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, then determine the current adjustment to be used. The optimized external parameter value enables the same feature collected by the lidar to travel to different positions in the same position in the IMU coordinate system or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is the first The sum of variances corresponding to the coordinates of the feature in the three dimensions of the IMU coordinate system respectively, and the first feature is any one of the X features.

Through the above design, according to the sum of the error parameters of the X features, it is judged whether the external parameter value to be optimized can make the same feature collected by the lidar travel to different positions in the same position in the reference coordinate system or the position difference meets the predetermined Set conditions to improve the efficiency of external parameter calibration.

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies The preset conditions include: if the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment are all smaller than the second threshold, then determine the external parameters to be optimized used in the current adjustment. The parameter value enables the same feature collected by the lidar to travel to different positions in the same position in the IMU coordinate system or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is that the first feature is at The sum of variances corresponding to the coordinates of the three dimensions of the IMU coordinate system respectively, and the first feature is any one of the X features.

Through the above design, according to the error parameters of the X features, it is judged whether the external parameter values to be optimized can make the same feature collected by the lidar travel to different positions in the same position in the reference coordinate system or the position difference satisfies the preset conditions , to improve the accuracy of external parameter calibration.

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies The preset conditions include: if the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, and the current adjustment uses the to-be-optimized If the difference between the external parameter value and the external parameter value to be optimized used in the last adjustment is less than the third threshold, the external parameter value to be optimized used in the current adjustment is determined so that the lidar travels to a different location to collect The position of the same feature in the IMU coordinate system is the same or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is that the coordinates of the first feature in the three dimensions of the IMU coordinate system correspond to The sum of the variances of , the first feature is any one of the X features.

Through the above design, according to the sum of the error parameters of the X features, and according to the external parameter values to be optimized in two adjacent adjustments, the external parameter values to be optimized used in the current adjustment are determined, so that the lidar travels to different The position of the same feature collected from the position is the same in the reference coordinate system or the position difference satisfies a preset condition, while ensuring the calibration accuracy and calibration efficiency of external parameters.

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies The preset conditions include: if the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is less than the second threshold, and the external parameters to be optimized used in the current adjustment The difference between the value of the external parameter to be optimized and the value of the external parameter to be optimized used in the last adjustment is smaller than the third threshold, then the value of the external parameter to be optimized used for the current adjustment is determined to make the lidar travel to different locations to collect the same feature. The position of the object in the IMU coordinate system is the same or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is the difference between the corresponding variances of the coordinates of the first feature in the three dimensions of the IMU coordinate system. And, the first feature is any one of the X features.

In a third aspect, an embodiment of the present application provides a method for calibrating external parameters of a laser radar and an IMU, which is applied to a calibration device. Both the laser radar and the IMU are fixedly installed on a vehicle to be calibrated. The method includes: acquiring first point cloud data collected by a lidar, where the first point cloud data is used to represent the presence of features around the vehicle to be calibrated collected by the vehicle to be calibrated while driving on a target path in the lidar. The position in the coordinate system; in the current adjustment, according to the external parameter value to be optimized, and according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the fourth relative transformation between the lidar coordinate system and the reference coordinate system is obtained. The external parameter value to be optimized is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is Obtained according to the position and attitude of the to-be-calibrated vehicle collected by the IMU when the to-be-calibrated vehicle is traveling on the target path; based on the fourth relative transformation between the lidar coordinate system and the reference coordinate system relationship, convert the first point cloud data from the lidar coordinate system to the reference coordinate system to obtain second point cloud data; according to the second point cloud data, determine the external adjustment to be optimized for the current adjustment. When the parameter value makes the same feature collected by the lidar traveling to different positions in the same position in the IMU coordinate system or the position difference meets the preset condition, the current external parameter value to be optimized is used as the target external parameter. value, otherwise, adjust the external parameter value to be optimized, and use the adjusted external parameter value as the external parameter value to be optimized in the next adjustment.

Through the above design, the calibration device can directly convert the point cloud data from the lidar coordinate system to the reference coordinate system after acquiring the point cloud data collected by the lidar, and according to the point cloud data in the reference coordinate system, the external parameters to be optimized Iteratively adjust the value until the external parameter value to be optimized used in the current adjustment can make the same feature collected by the lidar traveling to different positions overlap or partially overlap in the IMU coordinate system, so as to avoid the external parameter calibration result being affected by the lidar. The influence of the installation angle improves the efficiency and accuracy of the external parameter calibration of the lidar.

In a fourth aspect, an embodiment of the present application provides an external parameter calibration method for a lidar and an IMU, and the method can be performed by a calibration device or a chip or a chip system in the calibration device, so as to realize the first aspect, the second aspect or the The method in any possible implementation manner performed by the calibration apparatus in the third aspect.

In a fifth aspect, the present application provides an external parameter calibration device for a lidar and an IMU, the calibration device including a module/unit for performing the method in any of the possible implementations of the first aspect, the second aspect or the third aspect . These modules/units can be implemented by hardware or by executing corresponding software by hardware.

In a sixth aspect, the present application provides a calibration device, comprising a processor and a memory, wherein the memory is used to store one or more computer programs; when the one or more computer programs stored in the memory are executed by the processor, the calibration is performed. The apparatus can implement the method in any possible implementation manner of the first aspect, the second aspect or the third aspect.

In a seventh aspect, the present application provides a computer program that, when the computer program runs on a computer, causes the computer to execute the method in any of the possible implementations of the first aspect, the second aspect or the third aspect. .

In an eighth aspect, the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a computer, the computer is made to execute the above-mentioned first aspect, second A method in any of the possible implementations of the aspect or the third aspect.

In a ninth aspect, the present application provides a chip, which is used to read a computer program stored in a memory and execute the method in any of the possible implementation manners of the first aspect, the second aspect or the third aspect.

In a tenth aspect, an embodiment of the present application further provides a chip system, where the chip system includes a processor for supporting a computer device to implement the method in any of the possible implementation manners of the first aspect, the second aspect, or the third aspect. . In a possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. The chip system can be composed of chips, and can also include chips and other discrete devices.

For the technical effects that can be achieved by any of the possible technical solutions in the fourth aspect to the tenth aspect, please refer to the description of the technical effects that can be achieved by the method in any of the possible implementations of the first aspect, the second aspect or the third aspect. , which will not be repeated here.

Description of drawings

1 is a schematic diagram of a possible calibration scenario provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of a possible position between a lidar and a collection vehicle provided in an embodiment of the present application;

3A is a schematic diagram of a possible calibration site provided in an embodiment of the present application;

3B is a schematic diagram of another possible calibration site provided in the embodiment of the present application;

FIG. 4 is a schematic diagram of a possible collection route provided in the embodiment of the present application;

FIG. 5 is a schematic flowchart of a first possible external parameter calibration method for a lidar and an IMU provided in an embodiment of the application;

6 is a schematic flowchart of a possible external parameter calibration method for a lidar and an IMU provided in an embodiment of the present application;

FIG. 7 is a schematic flowchart of another possible external parameter calibration method for a lidar and an IMU provided in an embodiment of the present application;

FIG. 8 is a schematic flowchart of a second possible external parameter calibration method for a lidar and an IMU provided in an embodiment of the present application;

FIG. 9 is a schematic flowchart of a third possible external parameter calibration method for a lidar and an IMU provided in an embodiment of the present application;

10 is a schematic structural diagram of a possible calibration device provided in an embodiment of the application;

FIG. 11 is a schematic structural diagram of another possible calibration device provided in the embodiment of the present application.

Detailed ways

In order to make the present application easier to understand, some basic concepts involved in the embodiments of the present application are first explained below. It should be noted that these explanations are for the purpose of making the embodiments of the present application easier to understand, and should not be regarded as limitations on the protection scope claimed by the present application.

1. Common coordinate system.

The public coordinate system, also known as the world coordinate system or the global coordinate system, whose coordinate origin is a fixed point in space. The common coordinate system is an absolute coordinate system, and all objects in the space can use the common coordinate system as a reference to determine the position of the object. Exemplarily, the common coordinate system may be a world coordinate system with east, north, and sky as the X-axis, Y-axis, and Z-axis.

2. External reference.

Generally, the parameters that affect the performance of lidar are divided into two types: internal parameters and external parameters. The internal parameters are determined when the lidar is manufactured, and the internal parameters may include, but are not limited to, the horizontal and vertical angles of each laser beam and the distance correction value. The external parameters refer to the offset distance and offset angle of the lidar relative to the IMU. For example, the offset distance of the lidar relative to the IMU means that the lidar is regarded as a particle, and the coordinates (x, y, z) of the particle in the IMU coordinate system O-x1y1z1 can indicate the relative distance of the lidar to the IMU. Offset distance; for another example, the lidar coordinate system O-x2y2z2 is established with the center of mass of the lidar as the origin. It is assumed that the lidar coordinate system O-x2y2z2 is rotated around the x2 axis, y2 axis and z2 axis respectively. After three angles, the x1 axis and the x2 axis are in the same direction in the three-dimensional space, the y1 axis and the y2 axis are in the same direction in the three-dimensional space, and the z1 axis and the z2 axis are in the same direction in the three-dimensional space, then, θ1, θ2 and θ3 can be regarded as the offset angle of the lidar relative to the IMU.

1 is a schematic diagram of a possible calibration scenario provided in an embodiment of the application, including a collection vehicle (the collection vehicle may also be referred to as a vehicle to be calibrated), a lidar, and an IMU. Both the lidar and the IMU are fixedly installed on the collection vehicle. In the process of collecting vehicles traveling according to the collecting route, the lidar can obtain the point cloud data of the features around the collecting route, and the IMU can obtain the linear acceleration and angular velocity of the collecting vehicle. Exemplarily, the acquisition vehicle further includes a global navigation satellite system (GNSS), and the GNSS is used to realize clock synchronization between the lidar and the IMU, and to provide the three-dimensional position of the acquisition vehicle in a common coordinate system.

FIG. 2 is a schematic diagram of a possible position between the lidar and the collection vehicle provided in the embodiment of the application. The inclination angle between the lidar and the collection vehicle is a set angle. As an example, the lidar is installed obliquely in the The vehicle is collected, that is, the inclination angle is the first set angle, for example, the first set angle may be 32°. In the process of collecting high-precision maps, when the collecting vehicle travels according to the collection route, the reflection intensity of the laser radar installed at an angle is higher, and the obtained road point cloud is dense, which improves the collection efficiency and accuracy of the high-precision road map. As another example, the lidar can also be installed in parallel with the acquisition vehicle, that is, the inclination angle is the second set angle, for example, the second set angle can be 1°.

The data collection requirements in the embodiments of the present application will be described below with reference to the accompanying drawings.

1) Calibration site requirements: refer to Figure 3A and Figure 3B, take the satellite navigation antenna set on the top of the acquisition vehicle as the center, and use the radius R meters, the elevation angle is more than α degrees, and the site without obstructions is used as the calibration site. For example, taking the satellite navigation antenna set on the top of the vehicle as the center, a site with a radius of 20 meters, an elevation angle of more than 45 degrees, and no obstructions is used as the calibration site. Exemplarily, the number of satellites received by the satellite navigation receiving device is greater than 30, and the positioning accuracy attenuation factor (PDOP) value is less than 1.5, so as to ensure that the acquired position of the collecting vehicle in the public coordinate system is more accurate and the measurement accuracy is improved. Among them, the satellite navigation antenna and the satellite navigation receiving device are both devices in the GNSS.

2) Feature requirements: features include plane features and high-altitude features. The plane feature may be a flat ground with a set area. For example, the plane feature is 1 square of ground, and the 1 square of ground is relatively flat and free of upslopes and potholes. The high-altitude feature is an object whose height from the ground reaches the set height. For example, the high-altitude feature can be the lamp head of a street lamp that is 3 meters above the ground. For another example, the high-altitude feature can be a sign that is 3 meters above the ground.

3) Collection route requirements: For example, the collection vehicle can pass the same feature in the forward and reverse directions during the driving process. For another example, the laser radar can scan the feature multiple times during the driving process of the collection vehicle. The acquisition route may also be referred to as the target route. Referring to Figure 4, Figure 4 includes two features, the ground and the lamp cap of the street lamp. L1, L2, L3, and L4 are different driving routes. The collection route of the collection vehicle can be A→L1→B→L2→C→ A→L3→B→L4→C→L4→B→L3→A→C→L2→B→L1→A. Further, in order to improve the precision and accuracy of external parameter calibration, when the collecting vehicle collects data along the collecting route, the traveling speed of the collecting vehicle is less than the set threshold, for example, the traveling speed of the collecting vehicle is less than 10km/h.

When the collection vehicle travels in the calibration site according to the collection route, the lidar collects the point cloud data of the characteristic objects, and the IMU obtains the linear acceleration and angular velocity of the collection vehicle. When performing external parameter calibration, the calibration device can convert the point cloud data collected by the lidar to the IMU coordinate system according to the external parameter values to be optimized; according to the linear acceleration and angular velocity of the collected vehicle collected by the IMU, the IMU coordinate system and reference The relative transformation relationship between the coordinate systems; according to the relative transformation relationship between the IMU coordinate system and the reference coordinate system, the point cloud data is converted from the IMU coordinate system to the reference coordinate system; according to the point cloud data of the reference coordinate system, it is optimized The extrinsic parameter value is adjusted to obtain the target extrinsic parameter value. By converting the point cloud data collected by the lidar to the reference coordinate system, the external parameter values to be optimized are optimized and adjusted based on the point cloud data in the reference coordinate system, so as to realize automatic calibration and improve the efficiency of external parameter calibration.

The technical solutions provided by the embodiments of the present application are described below with reference to the accompanying drawings.

Referring to FIG. 5 , FIG. 5 is a possible external parameter calibration method for a lidar and an IMU provided in an embodiment of the present application. The method can be performed by a calibration device or by a chip or a chip system in the calibration device. In the following, the execution subject of S501-S504 is taken as an example of the calibration device.

S501. The calibration device acquires first point cloud data collected by the lidar, where the first point cloud data is used to represent the position in the lidar coordinate system of the feature collected by the collecting vehicle traveling on the target path. Hereinafter, the lidar coordinate system is simply referred to as a radar coordinate system.

The first point cloud data includes N frames of point cloud data collected by lidar at N collection moments.

The first point cloud data may be obtained according to the lidar data collected by the lidar in the data collection stage, where the lidar data includes any one or more of the reflection intensity, scanning angle, and scanning direction of the lidar. In the process of collecting the data from the vehicle, the lidar collects the lidar data. When calibrating the external parameters of the lidar and the IMU, the calibration device can analyze the lidar data according to the internal parameters, and obtain the first point cloud data collected by the lidar. Exemplarily, the first point cloud data may be stored in a point cloud data (point cloud data, PCD) format. Exemplarily, during the data collection process, after the lidar data is collected by the lidar, the collected lidar data may be stored in the storage area. Further, when performing external parameter calibration, the calibration device can acquire lidar data from the storage area.

S502. In the current adjustment, the calibration device converts the first point cloud data from the lidar coordinate system to the IMU coordinate system according to the external parameter value to be optimized, and obtains the second point cloud data, and the external parameter value to be optimized is used for A first relative transformation relationship between the lidar coordinate system and the IMU coordinate system is indicated.

The external parameter value to be optimized in the first adjustment can also be called the initial external parameter value. The initial external parameter value includes the offset distance between the radar coordinate system and the IMU coordinate system, and\or, the radar coordinate system and the IMU coordinate Offset angle between systems. The initial extrinsic parameter values can be estimated according to the installation positions of the IMU and lidar in the acquisition vehicle.

The calibration device can perform rectangular coordinate transformation on the first point cloud data according to the initial external parameter value, and convert the first point cloud data from the radar coordinate system to the IMU coordinate system. Exemplarily, the calibration device performs displacement transformation on the first point cloud data according to the offset distance included in the initial external parameter value, and performs rotation axis transformation on the first point cloud data according to the offset angle included in the initial external parameter value. , to obtain the second point cloud data in the IMU coordinate system.

S503, the calibration device converts the second point cloud data to the reference coordinate system to obtain third point cloud data according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, and the relative transformation relationship between the IMU coordinate system and the reference coordinate system is based on It is obtained by collecting the position and attitude of the collecting vehicle collected by the IMU during the process of collecting the vehicle traveling on the target path.

Wherein, the second relative transformation relationship between the IMU coordinate system and the reference coordinate system may be obtained by the calibration device in the following manner: the calibration device obtains the measurement data collected by the IMU at M second collection moments, and the measurement data includes the collection of the vehicle traveling on the target path. The linear acceleration and angular velocity of the collection vehicle collected by the IMU during the process of collecting the ; The relative transformation relationship between the IMU coordinate system and the reference coordinate system at the time of acquisition.

The calibration device obtains third relative conversion relationships at N first collection moments respectively according to the second relative conversion relationships at the M second collection moments, respectively, and the third relative conversion relationships at the first collection moments are used to represent the first The relative transformation relationship between the IMU coordinate system and the reference coordinate system at the acquisition moment; according to the third relative transformation relationship at the i-th first acquisition moment, the second point cloud data at the i-th first acquisition moment are respectively converted to Referring to the coordinate system, the point cloud data of the i-th first collection moment is obtained, and i is taken as a positive integer less than or equal to N, so as to obtain N point cloud data of the first collection moment to form the third point cloud data.

Exemplarily, the calibration device obtains the third relative conversion relationship at the N first collection moments according to the second relative conversion relationships at the M second collection moments, respectively. Taking a single frame of point cloud data as an example, The single frame of point cloud data is any frame of point cloud data in the second point cloud data. As a possible situation, there are acquisition moments of single frame of point cloud data in the M second acquisition moments. According to the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the M second acquisition moments, it can be directly obtained. The third relative conversion relationship at the time of collection of the single frame of point cloud data is to obtain the relative conversion relationship between the IMU coordinate system and the reference coordinate system at the time of collection of the single frame of point cloud data.

As another possible situation, there is no single frame of point cloud data collection time in the M second collection moments, and the calibration device can use an interpolation algorithm to obtain The third relative conversion relationship at the acquisition moment of the single frame of point cloud data. The interpolation algorithm may adopt any one of linear interpolation algorithm, parabolic interpolation algorithm, Lagrangian interpolation algorithm, and Newton interpolation algorithm. The interpolation algorithm can be selected according to one or more of the severity of changes in the motion of the collected vehicle, the set interpolation accuracy, and the set real-time calculation requirements.

As a possible implementation, the reference coordinate system may refer to a common coordinate system. When the reference coordinate system is the common coordinate system, for the method of determining the second relative transformation relationship between the IMU coordinate system and the common coordinate system, refer to Embodiment 1 for details.

As another possible implementation manner, the reference coordinate system may also refer to a local coordinate system, and the local coordinate system includes a radar coordinate system at a fixed time, an IMU coordinate system at a fixed time, or a custom coordinate system, for example, 0 seconds (s) The radar coordinate system at the moment, for another example, the IMU coordinate system at the 0s moment. When the reference coordinate system is a local coordinate system, for the method of determining the second relative transformation relationship between the IMU coordinate system and the local coordinate system, refer to Embodiment 2 for details.

Exemplarily, the acquisition moment may be represented by GNSS time. Before collecting data from the vehicle, the IMU and LiDAR can be clocked through GNSS. Exemplarily, the GNSS can input the GNSS second pulse signal and the global positioning system (GPS) information (GPRMC) data frame with the recommended minimum data amount to the corresponding data interface of the lidar, and the lidar receives the GNSS second pulse signal. Then, calibrate the lidar clock according to the standard time contained in the GPRMC data frame. After the time synchronization between the IMU and the lidar is completed, both the IMU and the lidar can convert the time of data collection into GNSS time.

S504. The calibration device determines, according to the third point cloud data, the external parameter values to be optimized for the current adjustment, so that the same feature collected by the lidar travels to different positions in the same position in the reference coordinate system or the position difference satisfies the preset condition, take the current external parameter value to be optimized as the target external parameter value, otherwise, adjust the external parameter value to be optimized, and use the adjusted external parameter value as the external parameter value to be optimized in the next adjustment .

As a possible implementation manner, the third point cloud data is used to represent the three-dimensional coordinates of the X features in the reference coordinate system, where X is a positive integer. The calibration device directly determines the external parameter value to be optimized for the current adjustment according to the third point cloud data, so that the same feature collected by the lidar travels to different positions in the same position in the reference coordinate system or the position difference satisfies the preset condition When , take the current external parameter value to be optimized as the target external parameter value, otherwise, adjust the external parameter value to be optimized, and use the adjusted external parameter value as the external parameter value to be optimized in the next adjustment.

As another possible implementation manner, in order to improve the calibration efficiency and the calibration accuracy, after the calibration device acquires the third point cloud data, the third point cloud data can also be extracted with features. Further, the calibration device determines the external parameter value to be optimized for the current adjustment according to the extracted feature point cloud set of each feature, so that the lidar travels to different positions to collect the position of the same feature in the reference coordinate system. When the same or the position difference meets the preset conditions, the current external parameter value to be optimized is used as the target external parameter value, otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the next adjustment. The external parameter value to be optimized.

Exemplarily, if there are X feature objects in the calibration site, the calibration device extracts X feature point cloud sets from the third point cloud data according to the X reference coordinate ranges, and the X reference coordinate ranges are X feature objects. The reference coordinate range in the reference coordinate system. As an example, the calibration device first performs point cloud splicing on the third point cloud data in the reference coordinate system, and then performs feature extraction. Exemplarily, the calibration device performs point cloud splicing on the N frames of point cloud data included in the third point cloud data under the reference coordinate system to obtain the spliced third point cloud data; Feature extraction to obtain feature point cloud sets of X features. Among them, point cloud stitching refers to the process of unifying the data collected at different angles and different time points into the same coordinate system. For example, the third point cloud data includes point cloud data 1, point cloud data 2 and point cloud data 3. After the calibration device performs point cloud splicing on point cloud data 1, point cloud data 2 and point cloud data 3, the spliced point cloud data is obtained. The third point cloud data of . The calibration site includes feature A, and according to the reference coordinate range of feature A, the feature point cloud set of feature A is extracted from the third point cloud data after splicing.

As another example, the calibration device first performs feature extraction on the third point cloud data in the reference coordinate system, and then performs point cloud splicing. Exemplarily, the calibration device performs feature extraction on N frames of point cloud data contained in the third point cloud data under the reference coordinate system, respectively, to obtain each frame of feature data of X features; The corresponding N frames of feature data are point cloud spliced to obtain the third point cloud data after splicing. Taking the point cloud data 1 included in the third point cloud data as an example, the calibration device can perform feature extraction for a single frame of point cloud data according to the reference coordinate range of each feature, and the measurement accuracy of the reference coordinate range can be meter level. The calibration site contains feature A, and the calibration device extracts the point cloud data within the reference coordinate range of feature A from point cloud data 1 in the reference coordinate system according to the reference coordinate range of feature A, as the feature object A frame of feature point cloud of A. In the reference coordinate system, the feature extraction method in the single frame of point cloud data is used to extract the features of the N frames of point cloud data contained in the third point cloud data, and the feature point cloud of N frames of feature A can be obtained. , and splicing the N frames of feature point clouds of feature A, the feature point cloud set of feature A can be obtained.

Exemplarily, the calibration device can determine the external parameter value to be optimized used in the current adjustment through, but not limited to, the following two possible implementation manners, so that the same feature collected by the lidar travels to different positions in the reference coordinate system. The position is the same or the position difference meets the preset conditions:

The first possible implementation: when the calibration device determines that the first iteration stop condition is satisfied, it determines the external parameter value to be optimized used in the current adjustment, so that the same feature collected by the lidar travels to different positions in the reference coordinate system. The position is the same or the position difference meets the preset condition.

Exemplarily, the calibration device adopts an iterative optimization algorithm according to the extracted feature point cloud sets of each feature to adjust the external parameter values to be optimized until the first iteration stop condition is satisfied, and determines the to-be-optimized value used for the current adjustment. The extrinsic parameter value of , makes the position of the same feature collected by the lidar traveling to different positions in the reference coordinate system is the same or the position difference satisfies the preset condition. The iterative optimization algorithm may adopt, but is not limited to, any of the Gauss-Newton method, the conjugate gradient method, the gradient descent method, and the like.

When the calibration device adopts an iterative optimization algorithm, the error parameter is used as the objective function, and the external parameter value to be optimized is used as the optimization variable. The calibration device can calculate and obtain the error parameters of each feature according to the feature point cloud set of each feature. Among them, the error parameter of a feature is used to represent the sum of variances corresponding to the coordinates of a feature in the three dimensions of the reference coordinate system.

The first iteration stop condition may be, but not limited to, any one of the fixed number of iterations method, the fixed time method, and the pre- and post-difference method. The fixed cycle number method means that the iteration stops when the number of iterations reaches the threshold of the number of iterations. The fixed time method means that the iteration stops when the iteration duration reaches the duration threshold.

When the difference method is adopted, as an example, the stopping condition of the first iteration may be that the difference between the error parameters of the X features in the current iteration and the error parameters of the X features in the previous iteration is smaller than the second threshold.

Taking two features, feature A and feature B as an example, the calibration device adopts an iterative optimization algorithm. In the process of iterative optimization of the external parameter value to be optimized, in the reference coordinate system O-xyz, the statistical value of feature A is calculated. The variance of the feature point cloud set on the x-axis is 1, the variance on the y-axis is 2, and the variance on the z-axis is 3. According to the variance 1, variance 2 and variance 3 obtained by statistics, the feature point cloud set of feature A is obtained. The sum of the variances on each axis of the reference coordinate system, that is, the error parameter of the feature A is obtained. The calibration device is in the reference coordinate system O-xyz, and the variance of the feature point cloud set of the feature B on the x-axis is calculated 4 , variance 5 on the y-axis, variance 6 on the z-axis, according to the variance 4, variance 5 and variance 6 obtained by statistics, get the variance of the feature point cloud set of feature B on each axis of the reference coordinate system The sum, that is, the error parameter of the feature B is obtained. When the error parameter of feature A is less than the second threshold, and the error parameter of feature B is less than the second threshold, the iteration is stopped, and the adjusted external parameter value is output.

As another example, the first iteration stop condition may also be that the difference between the sum of the error parameters of the X features in the current iteration and the sum of the error parameters of the X features in the previous iteration is less than the first threshold.

Still taking feature A and feature B as an example, the calibration device adopts an iterative optimization algorithm. In the process of iterative optimization of the external parameter values to be optimized, in the reference coordinate system O-xyz, the feature point cloud set of feature A is counted. The variance on the x-axis is 1, the variance on the y-axis is 2, and the variance on the z-axis is 3. The sum of the variances of the feature point cloud set of the feature A on each axis of the reference coordinate system is obtained, that is, the feature is obtained. The error parameter of object A, in the reference coordinate system O-xyz, count the variance 4 on the x-axis, the variance 5 on the y-axis, and the variance 6 on the z-axis of the feature point cloud set of feature B to obtain the feature. The sum of the variances of the feature point cloud set of object B in each axis of the reference coordinate system, that is, the error parameter of feature B is obtained. Then, the calibration device adds the error parameter of feature A and the error parameter of feature B. And, the sum of the error parameters of feature A and feature B is obtained. When the sum of the error parameters of feature A and feature B is less than the first threshold, the iteration is stopped, and the adjusted extrinsic parameter value is output.

Using the iterative optimization algorithm, in the iterative optimization process of the external parameter value to be optimized, if the first iteration stop condition is not satisfied, the external parameter value to be optimized is adjusted according to the set step size. The set step size includes the step size of the offset distance and the step size of the offset angle. For example, the step size of the offset distance can be 0.1cm, and for example, the step size of the offset angle can be 0.01°.

As a second possible implementation manner, when the calibration device determines that the first iterative stop condition and the second iterative stop condition are satisfied, it determines the external parameter value to be optimized used in the current adjustment so that the lidar travels to different locations to collect the same feature The position of the object in the reference coordinate system is the same or the position difference meets the preset condition.

The second iterative stop condition may also adopt, but is not limited to, any one of the fixed number of iterations method, the fixed time method, and the pre- and post-difference method.

Taking the second iterative stop condition as an example, the difference method is used. third threshold.

Exemplarily, the third threshold includes a distance threshold and an angle threshold, and the calibration device judges whether the difference in the offset distance between the external parameter value used in the current adjustment and the external parameter value to be optimized used in the previous adjustment is less than the distance threshold. , and determine whether the difference in the offset angle between the external parameter value to be optimized used in the current adjustment and the external parameter value to be optimized used in the previous adjustment is smaller than the angle threshold.

If the difference between the offset distance between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the distance threshold, and the extrinsic parameter value to be optimized used in the current adjustment is equal to If the offset angle between the extrinsic parameter values to be optimized used in the last adjustment is smaller than the angle threshold, the extrinsic parameter value to be optimized used in the current adjustment is used as the target extrinsic parameter value.

Further, in order to eliminate the ghosting error of the point cloud caused by the motion of the collected vehicle, before the calibration device performs feature extraction on the third point cloud data, the calibration device can also use the position, attitude and speed of the collected vehicle collected by the IMU to determine the first point cloud data. Motion compensation is performed on point cloud data, second point cloud data or third point cloud data. Motion compensation is a method of describing the difference between adjacent frames to eliminate the influence of motion caused by any of the velocity, linear acceleration, or angular velocity of the acquisition vehicle.

Embodiment 1: Take the reference coordinate system as the common coordinate system as an example.

FIG. 6 is a possible external parameter calibration method for a lidar and an IMU provided in an embodiment of the present application. The method may be performed by a calibration device, or may be performed by a chip or a chip system in the calibration device.

S601. The calibration device acquires the measurement data collected by the IMU, and obtains a set of pose information of the IMU in the public coordinate system according to the measurement data collected by the IMU, where the pose information includes one or more of three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle Item, hereinafter the three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle are simply referred to as position, velocity, and attitude angle. The pose information of the IMU in the common coordinate system is used to describe the relative transformation relationship between the IMU coordinate system and the common coordinate system.

The pose information set includes pose information collected by the IMU at M collection moments.

The measurement information collected by the IMU includes linear acceleration and angular velocity. In the process of collecting the vehicle traveling on the collected route, the IMU can collect the linear acceleration and angular velocity of the collecting vehicle at N collection times. Further, after the IMU collects the measurement information, the measurement information may be stored in a storage area, for example, the storage area may be a hard disk, a portable notebook, etc. configured in the collection vehicle. When performing the external parameter calibration of the lidar and the IMU, the calibration device can obtain the measurement information collected by the stored IMU from the storage area.

Exemplarily, an inertial navigation system (INS) may be configured in the acquisition vehicle, and the INS includes an IMU. After the calibration device obtains the measurement information of the IMU, it performs inertial navigation calculation on the measurement information collected by the IMU through the inertial navigation system (INS), and obtains the INS navigation information. The INS navigation information includes the acquisition vehicle in the IMU coordinate system. One or more of the three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle of . The acquisition vehicle is also equipped with GNSS. During the process of the acquisition vehicle driving on the acquisition route, GNSS can collect GNSS measurement information. The GNSS measurement information includes the 3D position, 3D velocity, and 3D attitude angle of the collected vehicle in the public coordinate system. one or more. The calibration device adopts inertial navigation algorithm and fusion filtering algorithm to fuse INS navigation information and GNSS measurement information, and can obtain the pose information set of IMU in the public coordinate system.

S602, the calibration device acquires point cloud data collected by the lidar. The point cloud data collected by the lidar includes N frames of point cloud data collected by the lidar at N collection moments.

S603. The calibration device converts the point cloud data from the radar coordinate system to the IMU coordinate system according to the Mth group of external parameter values, and obtains point cloud data in the IMU coordinate system. M is a natural number.

When the value of M is 0, the 0th group of extrinsic parameter values can also be called initial extrinsic parameter values, and the initial extrinsic parameter values include the offset distance and offset angle between the radar coordinate system and the IMU coordinate system.

S604, the calibration device projects the point cloud data in the IMU coordinate system to the public coordinate system according to the pose information corresponding to the point cloud data collection time, and obtains the point cloud data in the public coordinate system.

Taking a single frame of point cloud data as an example, the single frame of point cloud data is any frame of point cloud data in the point cloud data in the IMU coordinate system.

As a possible situation, the collection moment of a single frame of point cloud data is the same as the collection moment of a piece of pose information in the pose information set. The calibration device can determine that the pose information corresponding to the collection moment of the single frame of point cloud data is the above one pose information.

As another possible situation, the collection moment of a single frame of point cloud data is different from the collection moment of the pose information in the pose information set. The calibration device can use an interpolation algorithm to calculate the corresponding pose information when the collection time is the collection time of a single frame of point cloud data, as the target pose information. pose information. Further, the calibration device projects the single frame of point cloud data in the IMU coordinate system to the common coordinate system according to the target pose information. For example, the GNSS time of a single frame of point cloud data is the 15th second, the GNSS time of the pose information 1 is the 14th second, and the GNSS time of the pose information 2 is the 16th second. Using the linear interpolation algorithm, the GNSS time of the 15th second is calculated. The pose information is the average value of the pose information 1 and the pose information 2, which is used as the target pose information.

Further, in order to eliminate the ghosting error of the point cloud caused by the movement of the collected vehicle, after projecting the point cloud data in the IMU coordinate system to the public coordinate system, the calibration device will correct the point cloud data according to the pose information and/or the measurement data of the IMU. Motion compensation is performed on point cloud data in a common coordinate system.

S605 , the calibration device extracts feature objects from the point cloud data in the public coordinate system, and obtains a feature point cloud set of each feature object.

S606, the calibration device adopts an iterative optimization algorithm to iteratively optimize the Mth group of external parameter values until the first iteration stop condition is satisfied, and obtains the M+1th group of external parameter values, see S504 for details.

When the calibration device adopts the iterative optimization algorithm, the error parameter is used as the objective function, and the Mth group of external parameter values is used as the optimization variable. The calibration device can calculate and obtain the error parameters of each feature according to the feature point cloud set of each feature. Taking the first iterative stop condition as an example, the first iteration stop condition may be that the difference between the error parameter of each feature in the current iteration and the error parameter of each feature in the previous iteration is smaller than the first error. threshold.

S607. The calibration device judges whether the second iteration stop condition is satisfied, if yes, executes S609, otherwise, executes S608.

Taking the second iterative stop condition using the front-to-back difference method as an example, the calibration device judges whether the difference between the offset distances between the M+1 group of extrinsic parameter values and the M-th group of extrinsic parameter values is smaller than the distance threshold, and determines whether the M-th Whether the difference in the offset angle between the +1 group of extrinsic parameter values and the Mth group of extrinsic parameter values is less than the angle threshold.

When the difference of the offset distance between the M+1 group of external parameter values and the Mth group of external parameter values is less than the distance threshold, and the relative deviation between the M+1th group of external parameter values and the M-th group of external parameter values When the offset angle of the angle is smaller than the angle threshold, S609 is performed, that is, the M+1 th group of extrinsic parameter values is taken as the target extrinsic parameter value, otherwise, S608 is performed.

S608, the calibration device replaces the Mth group of external parameter values with the M+1th group of external parameter values, and executes S603.

S609: The calibration device uses the Mth group of external parameter values as the target external parameter values.

Embodiment 2: Take the reference coordinate system as the local coordinate system as an example. FIG. 7 is a possible external parameter calibration method for a lidar and an IMU provided in an embodiment of the present application. The method may be performed by a calibration device, or may be performed by a chip or a chip system in the calibration device. In the following, the execution subject of S701-S709 is taken as an example of the calibration device.

S701. The calibration device obtains the measurement data collected by the IMU, and obtains the relative transformation relationship between the IMU coordinate system and the local coordinate system at the M collection moments according to the measurement data collected by the IMU.

As an example, the local coordinate system is the IMU coordinate system at a fixed time, for example, the IMU coordinate system at the fixed time is the IMU coordinate system at the 0s time. Taking the IMU coordinate system at a fixed time as the IMU coordinate system at time 0s as an example, the measurement information collected by the IMU includes the linear acceleration and angular velocity of the collection vehicle collected by the IMU at M collection times. After the calibration device obtains the measurement information collected by the IMU, according to the linear acceleration and angular velocity collected at the M collection times, the relative conversion relationship between the IMU coordinate system at the M collection times and the IMU coordinate system at the 0s time can be obtained through integral calculation. For example, the measurement data collected by the IMU includes the linear acceleration and angular velocity collected by the IMU at the time of 0s, and the linear acceleration and angular velocity collected by the IMU at the time of 1s. Integrate calculation to obtain the relative conversion relationship between the IMU coordinate system at 1s and the IMU coordinate system at 0s.

As another example, the local coordinate system is the radar coordinate system at a fixed time, for example, the IMU coordinate system at the fixed time is the radar coordinate system at the 0s time. Taking the IMU coordinate system at a fixed time as the radar coordinate system at time 0s as an example, after the calibration device obtains the measurement information collected by the IMU, according to the linear acceleration and angular velocity collected at the M collection times, through integral calculation, the M collection times can be obtained. The relative transformation relationship between the IMU coordinate system and the IMU coordinate system at time 0s. The calibration device can obtain the relative conversion relationship between the IMU coordinate system at the M acquisition moments and the radar coordinate system at the 0s time according to the initial external parameter value.

S702, the calibration device acquires the point cloud data collected by the lidar, for details, refer to S501.

S703, the calibration device converts the point cloud data from the radar coordinate system to the IMU coordinate system according to the Mth group of external parameter values, and obtains point cloud data in the IMU coordinate system, see S502 for details.

S704, the calibration device converts the point cloud data in the IMU coordinate system to the local coordinate system according to the relative transformation relationship between the IMU coordinate system and the local coordinate system, and obtains point cloud data in the local coordinate system, see S503 for details.

S705, the calibration device performs feature extraction on the point cloud data in the local coordinate system, and obtains a feature point cloud set of each feature. For details, refer to S504.

S706 , the calibration device uses an iterative optimization algorithm to iteratively optimize the Mth group of external parameter values until the first iterative stop condition is satisfied, and obtains the M+1th group of external parameter values, see S504 for details.

S707: The calibration device judges whether the second iteration stop condition is satisfied, and if so, executes S709; otherwise, executes S708.

S708, the calibration device replaces the Mth group of external parameter values with the M+1th group of external parameter values, and executes S703.

S709, the calibration device uses the Mth group of external parameter values as the target external parameter values.

Through the above method, the point cloud data collected by the lidar is converted into the local coordinate system through the external parameter values to be optimized and the relative transformation relationship between the IMU and the local coordinate system, and the point cloud data in the local coordinate system are characterized. Extract the feature points to obtain the feature point cloud data of each feature. Then iteratively optimizes the external parameter values to be optimized according to the characteristic point cloud data of each feature, realizes automatic calibration between the lidar and IMU coordinate systems, improves the calibration efficiency of the lidar and IMU coordinate systems, and avoids the installation of lidar. The effect of angle on calibration accuracy.

Based on the same concept, the embodiments of the present application provide a second possible external parameter calibration method for lidar and IMU. In the embodiment of the present application, the calibration device acquires the first point cloud data collected by the lidar; in the current adjustment, the calibration device is based on the external parameter value to be optimized and the second relative conversion relationship between the IMU coordinate system and the reference coordinate system. , the fourth relative transformation relationship between the lidar coordinate system and the reference coordinate system is obtained, and the external parameter value to be optimized is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system; The method for determining the second relative conversion relationship of the coordinate system is shown in Embodiment 1 and Embodiment 2; the calibration device converts the first point cloud data from the lidar coordinate system from the lidar coordinate system Convert the system to the reference coordinate system to obtain the second point cloud data; adjust the external parameter values to be optimized according to the second point cloud data; the calibration device determines the external parameter values to be optimized for the current adjustment according to the second point cloud data such that When the same feature collected by the lidar travels to different positions in the same position in the IMU coordinate system or the position difference satisfies the preset conditions, the current external parameter value to be optimized is used as the target external parameter value, otherwise, the external parameter value to be optimized is used. Adjust the external parameter value, and use the adjusted external parameter value as the external parameter value to be optimized in the next adjustment.

Taking the reference coordinate system as the local coordinate system as an example, FIG. 8 is the second possible external parameter calibration method of the laser radar and the IMU provided in the embodiment of the application. A chip or system-on-a-chip implementation. In the following, the execution subject of S801-S808 is taken as an example of the calibration device.

S801. The calibration device acquires the measurement data collected by the IMU, and obtains the relative transformation relationship between the IMU coordinate system and the local coordinate system at the M collection moments according to the measurement data collected by the IMU. The method for determining the relative transformation relationship between the IMU coordinate system and the local coordinate system at the M collection moments is S701.

S802, the calibration device acquires the point cloud data collected by the lidar, for details, refer to S501.

S803, the calibration device obtains the relative transformation relationship between the radar coordinate system and the local coordinate system at the N collection moments according to the Mth group of external parameter values and the relative transformation relationship between the IMU coordinate system and the local coordinate system at the M collection moments respectively .

Exemplarily, according to the relative transformation relationship between the IMU coordinate system and the local coordinate system at the M collection moments respectively, an interpolation algorithm is used to obtain the relative transformation relationship between the IMU coordinate system and the local coordinate system at the N collection moments respectively; M sets of external parameter values, and the relative transformation relationship between the IMU coordinate system and the local coordinate system at N acquisition moments, respectively, can obtain the relative transformation relationship between the radar coordinate system and the local coordinate system at N acquisition moments.

S804, the calibration device converts the point cloud data in the radar coordinate system to the local coordinate system according to the relative transformation relationship between the radar coordinate system and the local coordinate system at the N collection moments respectively, and obtains the point cloud data in the local coordinate system.

S805, the calibration device performs feature extraction on the point cloud data in the local coordinate system, and obtains a feature point cloud set of each feature. For details, refer to S504.

S806, the calibration device adopts an iterative optimization algorithm to adjust the Mth group of external parameter values until the first iteration stop condition is satisfied, and obtains the M+1th group of external parameter values, see S504 for details.

S807. The calibration device judges whether the second iterative convergence condition is satisfied, and if so, executes S809, otherwise, executes S808.

S808, the calibration device replaces the Mth group of external parameter values with the M+1th group of external parameter values, and executes S803.

S809, the calibration device takes the M+1 group of external parameter values as the target external parameter value.

Based on the same concept, a third possible external parameter calibration method for lidar and IMU is provided in the embodiments of the present application. In the embodiment of the present application, the calibration device obtains the first point cloud data collected by the lidar, and the first point cloud data is used to represent that the features around the vehicle to be calibrated collected by the vehicle to be calibrated driving on the target path are in the lidar coordinate system In the current adjustment, the calibration device converts the first point cloud data from the lidar coordinate system to the IMU coordinate system according to the external parameter values to be optimized to obtain the second point cloud data, and the external parameter values to be optimized are used for Indicate the first relative conversion relationship between the lidar coordinate system and the IMU coordinate system; the calibration device determines the external parameter value to be optimized for the current adjustment and uses according to the second point cloud data, so that the lidar travels to different locations to collect the same image When the positions of the features in the IMU coordinate system are the same or the position difference satisfies the preset conditions, the current external parameter value to be optimized is used as the target external parameter value, otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is adjusted. The extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

Referring to FIG. 9 , FIG. 9 is a third possible external parameter calibration method for lidar and IMU provided in the embodiment of the present application, and the method can be executed by a calibration device or by a chip or a chip system in the calibration device. .

S901, the calibration device acquires the point cloud data collected by the lidar. For details, refer to S501.

S902, the calibration device converts the point cloud data from the radar coordinate system to the IMU coordinate system according to the Mth group of external parameter values, and obtains point cloud data in the IMU coordinate system, see S502 for details.

S903, the calibration device performs feature extraction on the point cloud data in the IMU coordinate system to obtain a feature point cloud set of each feature. The process that the calibration device performs feature extraction on the point cloud data in the IMU coordinate system is similar to the process of performing feature extraction on the third point cloud data in S504, and will not be repeated here.

S904 , the calibration device uses an iterative optimization algorithm to iteratively optimize the Mth group of external parameter values until the first iterative stop condition is satisfied, and obtains the M+1th group of external parameter values.

S905, the calibration device judges whether the second iterative convergence condition is satisfied, if yes, executes S907, otherwise, executes S906.

S906, the calibration device replaces the Mth group of external parameter values with the M+1th group of external parameter values, and executes S902.

S907, the calibration device uses the Mth group of external parameter values as the target external parameter values.

Through the above method, after the point cloud data collected by the lidar is converted from the radar coordinate system to the IMU coordinate system according to the external parameter values to be optimized, the external parameter values to be optimized can be optimized directly according to the point cloud data in the IMU coordinate system. Improve the calibration efficiency of external parameters, and avoid the influence of the installation angle of the lidar on the calibration accuracy.

Based on the above-mentioned embodiment, the embodiment of the present application also provides a method for calibrating external parameters between multiple laser radars. When performing external parameter calibration between multiple laser radars, the method can A possible method is to obtain the extrinsic parameter values between each lidar and the IMU respectively, and obtain the extrinsic parameter values between multiple lidars according to the extrinsic parameter values between each lidar and the IMU.

Taking the external parameter calibration process between the first laser radar and the second laser radar as an example, see S501-S504, the first external parameter value between the first laser radar and the IMU can be obtained, and the second laser radar and the IMU can be obtained. The second extrinsic parameter value between . According to the relative relationship between the first extrinsic parameter value and the second extrinsic parameter value, the extrinsic parameter value between the first laser radar and the second laser radar is obtained.

Taking the external parameter calibration process between the first laser radar, the second laser radar, and the third laser radar as an example, see S501-S504, the first external parameter value between the first laser radar and the IMU, the second laser radar can be obtained. The second extrinsic parameter value between the radar and the IMU, and the third extrinsic parameter value between the third lidar and the IMU. According to the relative relationship between the first extrinsic parameter value and the second extrinsic parameter value, the extrinsic parameter value between the first laser radar and the second laser radar can be obtained. According to the relative relationship between the first extrinsic parameter value and the third extrinsic parameter value, the extrinsic parameter value between the first laser radar and the third laser radar can be obtained. According to the relative relationship between the second external parameter value and the third external parameter value, the external parameter value between the second laser radar and the third laser radar can be obtained.

Based on the same technical concept, the present application also provides an external parameter calibration device for lidar and IMU. The structure of the calibration device is shown in FIG. 10 , including a communication unit 1001 and a processing unit 1002 . The calibration device 1000 can be applied to the calibration device in the calibration method shown in FIGS. 5-9 . The functions of each unit in the calibration device 1000 will be introduced below.

The communication unit 1001 is used for receiving and sending data.

When the calibration device 1000 is applied to a calibration device, the communication unit 601 may also be referred to as a physical interface, a communication module, a communication interface, and an input/output interface.

In a possible application scenario, the calibration apparatus 1000 includes a communication unit 1001 and a processing unit 1002,

The communication unit 1001 is used to obtain the first point cloud data collected by the lidar, where the first point cloud data is used to represent the characteristic objects around the vehicle to be calibrated collected by the vehicle to be calibrated while driving on the target path. position in the radar coordinate system;

The processing unit 1002 is configured to convert the first point cloud data from the lidar coordinate system to the IMU coordinate system to obtain second point cloud data according to the external parameter value to be optimized in the current adjustment, the to-be-optimized external parameter value. The optimized external parameter value is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system; according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the The second point cloud data is converted to the reference coordinate system to obtain third point cloud data; the second relative conversion relationship is based on the to-be-calibrated vehicle collected by the IMU during the process of the to-be-calibrated vehicle traveling on the target path. The position and attitude of the vehicle are obtained; and, according to the third point cloud data, the external parameter value to be optimized used for the current adjustment is determined so that the same feature collected by the lidar traveling to different positions is in the reference When the positions in the coordinate system are the same or the position difference satisfies the preset conditions, the current external parameter value to be optimized is used as the target external parameter value, otherwise, the external parameter value to be optimized is adjusted, and the adjusted value is used. The external parameter value is used as the external parameter value to be optimized in the next adjustment.

In a possible design, the first point cloud data is collected by the lidar at N first collection moments, and the communication unit 1001 is further configured to acquire the IMU collected at M second collection moments The measured data includes the linear acceleration and angular velocity of the to-be-calibrated vehicle collected by the IMU when the to-be-calibrated vehicle is traveling on the target path;

When acquiring the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the processing unit 1002 is configured to: obtain, according to the measurement data, second relative transformation relationships at M second acquisition moments, respectively, The second relative transformation relationship at the second acquisition moment is used to represent the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the second acquisition moment.

In a possible design, according to the second relative conversion relationship between the IMU coordinate system and the reference coordinate system, when converting the second point cloud data to the reference coordinate system to obtain third point cloud data, The processing unit 1002 is used for:

According to the second relative conversion relationships at the M second collection moments, respectively, third relative conversion relationships at the N first collection moments are obtained, and the third relative conversion relationships at the first collection moments are used to represent the at the first acquisition moment, the relative transformation relationship between the IMU coordinate system and the reference coordinate system;

According to the third relative conversion relationship at the ith first collection moment, the second point cloud data at the ith first collection moment are respectively converted to the reference coordinate system to obtain the point cloud at the ith first collection moment For data, i is taken as a positive integer less than or equal to N, so as to obtain N point cloud data at the first collection moment to form the third point cloud data.

In a possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Under preset conditions, the processing unit 1002 is used for:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, the external parameter value to be optimized used in the current adjustment is determined such that The same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference meets a preset condition; wherein, the error parameter of the first feature is the first feature in the reference coordinate system. The sum of variances corresponding to the coordinates of the three dimensions of the coordinate system respectively, and the first feature is any one of the X features.

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is smaller than the second threshold, then determine the external parameter value to be optimized used in the current adjustment so that the laser The position of the same feature collected by the radar traveling to different positions in the reference coordinate system is the same or the position difference meets a preset condition; wherein, the error parameter of the first feature is the first feature in the reference coordinate system. The sum of variances corresponding to the coordinates of the three dimensions respectively, and the first feature is any one of the X features.

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, and the external parameter value to be optimized used in the current adjustment is the same as the above The difference between the external parameter values to be optimized used in one adjustment is smaller than the third threshold, and the external parameter value to be optimized used in the current adjustment is determined so that the same feature collected by the lidar traveling to different positions is at the location. The positions in the reference coordinate system are the same or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is the sum of the variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, so The first feature is any one of the X features.

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is less than the second threshold, and the value of the external parameter to be optimized used in the current adjustment is the same as the one used in the previous adjustment If the difference between the external parameter values to be optimized is smaller than the third threshold, the external parameter value to be optimized used for the current adjustment is determined so that the same feature collected by the lidar travels to different positions at the reference coordinates The positions in the reference coordinate system are the same or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is the sum of the variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature The feature is any of the X features.

According to the second relative conversion relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted from the IMU coordinate system to the reference coordinate system to obtain fourth point cloud data;

The fourth point cloud data after motion compensation is used as the third point cloud data, and the fourth point cloud data after motion compensation is collected by the IMU according to the process of the vehicle to be calibrated driving on the target path motion compensation for the position, attitude and speed of the vehicle to be calibrated.

In another possible application scenario, the calibration apparatus 1000 includes a communication unit 1001 and a processing unit 1002,

The processing unit 1002 is configured to convert the first point cloud data from the lidar coordinate system to the IMU coordinate system to obtain second point cloud data according to the external parameter value to be optimized in the current adjustment, the to-be-optimized external parameter value. The optimized extrinsic parameter value is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system; and, according to the second point cloud data, determine the extrinsic parameter to be optimized used for the current adjustment. When the parameter value makes the same feature collected by the lidar traveling to different positions in the same position in the IMU coordinate system or the position difference meets the preset condition, the current external parameter value to be optimized is used as the target external parameter. value, otherwise, adjust the external parameter value to be optimized, and use the adjusted external parameter value as the external parameter value to be optimized in the next adjustment.

In a possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Under preset conditions, the processing unit 1002 is used for:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, the external parameter value to be optimized used in the current adjustment is determined such that The position of the same feature collected by the lidar traveling to different positions in the IMU coordinate system is the same or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is the first feature in the IMU. The sum of variances corresponding to the coordinates of the three dimensions of the coordinate system respectively, and the first feature is any one of the X features.

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is smaller than the second threshold, then determine the external parameter value to be optimized used in the current adjustment so that the laser The same feature collected by the radar traveling to different positions has the same position in the IMU coordinate system or the position difference meets a preset condition; wherein, the error parameter of the first feature is the error parameter of the first feature in the IMU coordinate system. The sum of variances corresponding to the coordinates of the three dimensions respectively, and the first feature is any one of the X features.

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, and the external parameter value to be optimized used in the current adjustment is the same as the above The difference between the external parameter values to be optimized used in one adjustment is smaller than the third threshold, and the external parameter value to be optimized used in the current adjustment is determined so that the same feature collected by the lidar traveling to different positions is at the location. The positions in the IMU coordinate system are the same or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is the sum of the variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, so The first feature is any one of the X features.

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is less than the second threshold, and the value of the external parameter to be optimized used in the current adjustment is the same as the one used in the previous adjustment If the difference between the values of the external parameters to be optimized is smaller than the third threshold, the external parameter values to be optimized used in the current adjustment are determined so that the same feature collected by the lidar travels to different positions at the coordinates of the IMU The positions in the system are the same or the position difference satisfies a preset condition; wherein, the error parameter of the first feature is the sum of the variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature The feature is any of the X features.

The processing unit 1002 is used for obtaining the fourth relative transformation between the lidar coordinate system and the reference coordinate system according to the external parameter value to be optimized and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system in the current adjustment The external parameter value to be optimized is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is Obtained according to the position and attitude of the to-be-calibrated vehicle collected by the IMU when the to-be-calibrated vehicle is traveling on the target path; based on the fourth relative transformation between the lidar coordinate system and the reference coordinate system relationship, converting the first point cloud data from the lidar coordinate system to the reference coordinate system to obtain second point cloud data; and, according to the second point cloud data, determining the current adjustment to be optimized When the external parameter value of the laser radar travels to different positions and collects the same feature in the IMU coordinate system in the same position or the position difference meets the preset condition, the current external parameter value to be optimized is used as the target. The external parameter value, otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the external parameter value to be optimized in the next adjustment.

Based on the same technical concept, the present application also provides an external parameter calibration device for lidar and IMU. The calibration device 1100 can implement the functions of the calibration device in the calibration methods shown in FIGS. 5-9 . Referring to FIG. 11 , the calibration apparatus 1100 includes: a transceiver 1101 , a processor 1102 and a memory 1103 . The transceiver 1101 , the processor 1102 and the memory 1103 are connected to each other. Exemplarily, the processor 1102 may be configured to perform all the operations performed by the calibration apparatus in any of the embodiments shown in FIG. 5 to FIG. 9 except for the acquisition operation.

Exemplarily, the transceiver 1101 , the processor 1102 and the memory 1103 are connected to each other through a bus 1104 . The bus 1104 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 11, but it does not mean that there is only one bus or one type of bus.

The transceiver 1101 is used to receive and transmit data, and implement communication interaction with other devices.

It will be appreciated that the memory in Figure 11 of the present application may be either volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Wherein, the non-volatile memory may be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically programmable read-only memory (Erasable PROM, EPROM). Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. Volatile memory may be Random Access Memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synchlink DRAM, SLDRAM) ) and direct memory bus random access memory (Direct Rambus RAM, DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but not be limited to, these and any other suitable types of memory.

Based on the above embodiments, the embodiments of the present application further provide a computer program, when the computer program runs on a computer, the computer causes the computer to execute the calibration methods provided by the embodiments shown in FIGS. 5-9 .

Based on the above embodiments, the embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a computer, the computer executes the programs shown in FIGS. 5-9 . The calibration method provided by the illustrated embodiment. The storage medium may be any available medium that the computer can access. By way of example and not limitation, computer readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or be capable of carrying or storing instructions or data structures in the form of desired program code and any other medium that can be accessed by a computer.

Based on the above embodiments, an embodiment of the present application further provides a chip, which is used to read a computer program stored in a memory, and implement the calibration method provided by the embodiments shown in FIG. 5 to FIG. 9 .

Based on the above embodiments, the embodiments of the present application provide a chip system, where the chip system includes a processor for supporting a computer device to implement the functions involved in the calibration device in the embodiments shown in FIGS. 5-9 . In a possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. The chip system may be composed of chips, or may include chips and other discrete devices.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the protection scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

A method for calibrating external parameters of a laser radar and an inertial navigation unit IMU, which is applied to a calibration device, wherein the laser radar and the IMU are both fixedly installed on the vehicle to be calibrated, comprising:

Obtain the first point cloud data collected by the lidar, where the first point cloud data is used to represent the position in the lidar coordinate system of the features around the to-be-calibrated vehicle collected by the to-be-calibrated vehicle traveling on the target path ;

In the current adjustment, according to the external parameter value to be optimized, the first point cloud data is converted from the lidar coordinate system to the IMU coordinate system to obtain the second point cloud data, and the external parameter value to be optimized uses for indicating a first relative transformation relationship between the lidar coordinate system and the IMU coordinate system;

According to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted into the reference coordinate system to obtain third point cloud data; the second relative transformation relationship is based on Obtained from the position and attitude of the vehicle to be calibrated collected by the IMU when the vehicle to be calibrated travels on the target path;

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies When the preset conditions are used, the current external parameter value to be optimized is used as the target external parameter value; otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the target external parameter value in the next adjustment. Optimized extrinsic parameter values.
The method of claim 1, wherein the first point cloud data is collected by the lidar at N first collection moments, and a second difference between the IMU coordinate system and the reference coordinate system is obtained. Relative conversion relationships, including:

Acquiring measurement data collected by the IMU at M second collection moments, the measurement data including the linear acceleration and angular velocity of the vehicle to be calibrated collected by the IMU while the vehicle to be calibrated is traveling on the target path ;

According to the measurement data, second relative conversion relationships at M second collection moments are obtained respectively, and the second relative conversion relationships at the second collection moments are used to represent the IMU coordinate system at the second collection moment The relative transformation relationship with the reference coordinate system.
The method according to claim 2, wherein, according to a second relative conversion relationship between the IMU coordinate system and the reference coordinate system, converting the second point cloud data to the reference coordinate system to obtain a third Point cloud data, including:

According to the second relative conversion relationships at the M second collection moments, respectively, third relative conversion relationships at the N first collection moments are obtained, and the third relative conversion relationships at the first collection moments are used to represent the at the first acquisition moment, the relative transformation relationship between the IMU coordinate system and the reference coordinate system;

According to the third relative conversion relationship at the ith first collection moment, the second point cloud data at the ith first collection moment are respectively converted to the reference coordinate system to obtain the point cloud at the ith first collection moment For data, i is taken as a positive integer less than or equal to N, so as to obtain N point cloud data at the first collection moment to form the third point cloud data.
The method according to any one of claims 1-3, wherein the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, the external parameter values to be optimized for the current adjustment are determined so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Preset conditions, including:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, the external parameter value to be optimized used in the current adjustment is determined such that The same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies a preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature is any one of the X features .
The method according to any one of claims 1-3, wherein the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, the external parameter values to be optimized for the current adjustment are determined so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Preset conditions, including:

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is smaller than the second threshold, then determine the external parameter value to be optimized used in the current adjustment so that the laser The same feature collected by the radar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies the preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature is any one of the X features .
The method according to any one of claims 1-3, wherein the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, the external parameter values to be optimized for the current adjustment are determined so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Preset conditions, including:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, and the external parameter value to be optimized used in the current adjustment is the same as the above The difference between the external parameter values to be optimized used in one adjustment is smaller than the third threshold, and the external parameter value to be optimized used in the current adjustment is determined so that the same feature collected by the lidar traveling to different positions is at the location. The position in the reference coordinate system is the same or the position difference satisfies the preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature is any one of the X features .
The method according to any one of claims 1-3, wherein the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, the external parameter values to be optimized for the current adjustment are determined so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Preset conditions, including:

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is less than the second threshold, and the value of the external parameter to be optimized used in the current adjustment is the same as the one used in the previous adjustment If the difference between the external parameter values to be optimized is smaller than the third threshold, the external parameter value to be optimized used for the current adjustment is determined so that the same feature collected by the lidar travels to different positions at the reference coordinates The positions in the system are the same or the position difference meets the preset conditions;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature is any one of the X features .
The method according to any one of claims 1-7, wherein the second point cloud data is converted to the reference according to a second relative conversion relationship between the IMU coordinate system and the reference coordinate system The coordinate system gets the third point cloud data, including:

According to the second relative conversion relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted from the IMU coordinate system to the reference coordinate system to obtain the fourth point cloud data;

The fourth point cloud data after motion compensation is used as the third point cloud data, and the fourth point cloud data after motion compensation is collected by the IMU according to the process of the vehicle to be calibrated driving on the target path motion compensation for the position, attitude and speed of the vehicle to be calibrated.
A method for calibrating external parameters of a laser radar and an IMU, which is applied to a calibration device, wherein the laser radar and the IMU are both fixedly installed on the vehicle to be calibrated, including:

Obtain the first point cloud data collected by the lidar, where the first point cloud data is used to represent the position in the lidar coordinate system of the features around the to-be-calibrated vehicle collected by the to-be-calibrated vehicle traveling on the target path ;

In the current adjustment, according to the external parameter value to be optimized, the first point cloud data is converted from the lidar coordinate system to the IMU coordinate system to obtain the second point cloud data, and the external parameter value to be optimized uses for indicating a first relative transformation relationship between the lidar coordinate system and the IMU coordinate system;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies When the preset conditions are used, the current external parameter value to be optimized is used as the target external parameter value; otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the target external parameter value in the next adjustment. Optimized extrinsic parameter values.
The method according to claim 9, wherein the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Preset conditions, including:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, the external parameter value to be optimized used in the current adjustment is determined such that The same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies a preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature is any one of the X features .
The method according to claim 9, wherein the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Preset conditions, including:

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is smaller than the second threshold, then determine the external parameter value to be optimized used in the current adjustment so that the laser The same feature collected by the radar traveling to different positions has the same position in the IMU coordinate system or the position difference meets the preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature is any one of the X features .
The method according to claim 9, wherein the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Preset conditions, including:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, and the external parameter value to be optimized used in the current adjustment is the same as the above The difference between the external parameter values to be optimized used in one adjustment is smaller than the third threshold, and the external parameter value to be optimized used in the current adjustment is determined so that the same feature collected by the lidar traveling to different positions is at the location. The position in the IMU coordinate system is the same or the position difference meets the preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature is any one of the X features .
The method according to claim 9, wherein the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Preset conditions, including:

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is less than the second threshold, and the value of the external parameter to be optimized used in the current adjustment is the same as the one used in the previous adjustment If the difference between the values of the external parameters to be optimized is smaller than the third threshold, the external parameter values to be optimized used in the current adjustment are determined so that the same feature collected by the lidar travels to different positions at the coordinates of the IMU The positions in the system are the same or the position difference meets the preset conditions;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature is any one of the X features .
A method for calibrating external parameters of a laser radar and an IMU, which is applied to a calibration device, characterized in that both the laser radar and the IMU are fixedly installed on a vehicle to be calibrated, comprising:

Obtain the first point cloud data collected by the lidar, where the first point cloud data is used to represent the position in the lidar coordinate system of the features around the to-be-calibrated vehicle collected by the to-be-calibrated vehicle traveling on the target path ;

In the current adjustment, according to the external parameter value to be optimized, and according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the fourth relative transformation relationship between the lidar coordinate system and the reference coordinate system is obtained. The external parameter value of is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is based on the vehicle to be calibrated Obtained from the position and attitude of the vehicle to be calibrated collected by the IMU while driving on the target path;

converting the first point cloud data from the lidar coordinate system to the reference coordinate system to obtain second point cloud data according to the fourth relative conversion relationship between the lidar coordinate system and the reference coordinate system;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies When the preset conditions are used, the current external parameter value to be optimized is used as the target external parameter value; otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the target external parameter value in the next adjustment. Optimized extrinsic parameter values.
An external parameter calibration device for lidar and inertial navigation unit IMU, characterized in that both the lidar and the IMU are fixedly installed on the vehicle to be calibrated, comprising:

A communication unit, configured to acquire the first point cloud data collected by the lidar, where the first point cloud data is used to represent that the features around the to-be-calibrated vehicle collected by the to-be-calibrated vehicle traveling on the target path are in the lidar position in the coordinate system;

a processing unit, configured to convert the first point cloud data from the lidar coordinate system to the IMU coordinate system according to the external parameter values to be optimized in the current adjustment to obtain second point cloud data, the to-be-optimized The external parameter value of is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system; according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the The second point cloud data is converted to the reference coordinate system to obtain third point cloud data; the second relative conversion relationship is based on the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated driving on the target path and, according to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment so that the same feature collected by the lidar traveling to different positions is at the reference coordinate When the positions in the system are the same or the position difference meets the preset condition, the current external parameter value to be optimized is used as the target external parameter value, otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used. The parameter value is used as the external parameter value to be optimized in the next adjustment.
The apparatus according to claim 15, wherein the first point cloud data is collected by the lidar at N first collection moments, and the communication unit is further configured to obtain the IMU at M first collection moments. 2. Measurement data collected at the time of collection, where the measurement data includes the linear acceleration and angular velocity of the vehicle to be calibrated collected by the IMU while the vehicle to be calibrated is traveling on the target path;

When acquiring the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the processing unit is configured to:

According to the measurement data, second relative conversion relationships at M second collection moments are obtained respectively, and the second relative conversion relationships at the second collection moments are used to represent the IMU coordinate system at the second collection moment The relative transformation relationship with the reference coordinate system.
The device according to claim 16, wherein, according to a second relative transformation relationship between the IMU coordinate system and the reference coordinate system, converting the second point cloud data to the reference coordinate system to obtain a third When processing point cloud data, the processing unit is used to:

According to the second relative conversion relationships at the M second collection moments, respectively, third relative conversion relationships at the N first collection moments are obtained, and the third relative conversion relationships at the first collection moments are used to represent the at the first acquisition moment, the relative transformation relationship between the IMU coordinate system and the reference coordinate system;

According to the third relative conversion relationship at the ith first collection moment, the second point cloud data at the ith first collection moment are respectively converted to the reference coordinate system to obtain the point cloud at the ith first collection moment For data, i is taken as a positive integer less than or equal to N, so as to obtain N point cloud data at the first collection moment to form the third point cloud data.
The device according to any one of claims 15-17, wherein the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, the external parameter values to be optimized for the current adjustment are determined so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Under preset conditions, the processing unit is used to:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, the external parameter value to be optimized used in the current adjustment is determined such that The same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies a preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature is any one of the X features .
The device according to any one of claims 15-17, wherein the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Under preset conditions, the processing unit is used to:

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is smaller than the second threshold, then determine the external parameter value to be optimized used in the current adjustment so that the laser The same feature collected by the radar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies the preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature is any one of the X features .
The device according to any one of claims 15-17, wherein the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, the external parameter values to be optimized for the current adjustment are determined so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Under preset conditions, the processing unit is used to:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, and the external parameter value to be optimized used in the current adjustment is the same as the above The difference between the external parameter values to be optimized used in one adjustment is smaller than the third threshold, and the external parameter value to be optimized used in the current adjustment is determined so that the same feature collected by the lidar traveling to different positions is at the location. The position in the reference coordinate system is the same or the position difference satisfies the preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature is any one of the X features .
The device according to any one of claims 15-17, wherein the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, and X is a positive integer;

According to the third point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the reference coordinate system or the position difference satisfies Under preset conditions, the processing unit is used to:

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the last adjustment is less than the second threshold, and the value of the external parameter to be optimized used in the current adjustment is the same as the one used in the last adjustment If the difference between the values of the external parameters to be optimized is smaller than the third threshold, the external parameter values to be optimized used in the current adjustment are determined so that the same feature collected by the lidar travels to different positions at the reference coordinates The positions in the system are the same or the position difference meets the preset conditions;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the reference coordinate system, and the first feature is any one of the X features .
The apparatus according to any one of claims 15-21, wherein the second point cloud data is converted to the reference according to a second relative conversion relationship between the IMU coordinate system and the reference coordinate system When the coordinate system obtains the third point cloud data, the processing unit is used for:

According to the second relative conversion relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted from the IMU coordinate system to the reference coordinate system to obtain fourth point cloud data;

The fourth point cloud data after motion compensation is used as the third point cloud data, and the fourth point cloud data after motion compensation is collected by the IMU according to the process of the vehicle to be calibrated driving on the target path motion compensation for the position, attitude and speed of the vehicle to be calibrated.
An external parameter calibration device for lidar and IMU, characterized in that both the lidar and the IMU are fixedly installed on the vehicle to be calibrated, including:

A communication unit, configured to acquire the first point cloud data collected by the lidar, where the first point cloud data is used to represent that the features around the to-be-calibrated vehicle collected by the to-be-calibrated vehicle traveling on the target path are in the lidar position in the coordinate system;

a processing unit, configured to convert the first point cloud data from the lidar coordinate system to the IMU coordinate system according to the external parameter values to be optimized in the current adjustment to obtain second point cloud data, the to-be-optimized The extrinsic parameter value is used to indicate the first relative conversion relationship between the lidar coordinate system and the IMU coordinate system; and, according to the second point cloud data, determine the extrinsic parameter to be optimized used for the current adjustment When the position of the same feature collected by the lidar at different positions is the same in the IMU coordinate system or the position difference meets the preset condition, the current external parameter value to be optimized is used as the target external parameter value. , otherwise, adjust the external parameter value to be optimized, and use the adjusted external parameter value as the external parameter value to be optimized in the next adjustment.
The device according to claim 23, wherein the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Under preset conditions, the processing unit is used to:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, the external parameter value to be optimized used in the current adjustment is determined such that The same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies a preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature is any one of the X features .
The apparatus according to claim 24, wherein the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Under preset conditions, the processing unit is used to:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, the external parameter value to be optimized used in the current adjustment is determined such that The same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies a preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature is any one of the X features .
The apparatus according to claim 24, wherein the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Under preset conditions, the processing unit is used to:

If the difference between the sum of the error parameters of the X features in the current adjustment and the sum of the error parameters of the X features in the previous adjustment is less than the first threshold, and the external parameter value to be optimized used in the current adjustment is the same as the above The difference between the external parameter values to be optimized used in one adjustment is smaller than the third threshold, and the external parameter value to be optimized used in the current adjustment is determined so that the same feature collected by the lidar traveling to different positions is at the location. The position in the IMU coordinate system is the same or the position difference meets the preset condition;

The error parameter of the first feature is the sum of variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature is any one of the X features .
The apparatus according to claim 24, wherein the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, and X is a positive integer;

According to the second point cloud data, determine the external parameter value to be optimized used for the current adjustment, so that the same feature collected by the lidar traveling to different positions has the same position in the IMU coordinate system or the position difference satisfies Under preset conditions, the processing unit is used to:

If the difference between the error parameters of the X features in the current adjustment and the error parameters of the X features in the previous adjustment is less than the second threshold, and the value of the external parameter to be optimized used in the current adjustment is the same as the one used in the previous adjustment If the difference between the values of the external parameters to be optimized is smaller than the third threshold, the external parameter values to be optimized used in the current adjustment are determined so that the same feature collected by the lidar travels to different positions at the coordinates of the IMU The positions in the system are the same or the position difference meets the preset conditions;

The error parameter of the first feature is the sum of the variances corresponding to the coordinates of the first feature in the three dimensions of the IMU coordinate system, and the first feature is any one of the X features .
An external parameter calibration device for lidar and IMU, characterized in that both the lidar and the IMU are fixedly installed on the vehicle to be calibrated, including:

A communication unit, configured to acquire the first point cloud data collected by the lidar, where the first point cloud data is used to represent that the features around the to-be-calibrated vehicle collected by the to-be-calibrated vehicle traveling on the target path are in the lidar position in the coordinate system;

The processing unit is used for obtaining the fourth relative transformation relationship between the lidar coordinate system and the reference coordinate system according to the external parameter value to be optimized and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system in the current adjustment , the external parameter value to be optimized is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is based on Obtained from the position and attitude of the vehicle to be calibrated collected by the IMU when the vehicle to be calibrated travels on the target path; according to the fourth relative transformation relationship between the lidar coordinate system and the reference coordinate system , converting the first point cloud data from the lidar coordinate system to the reference coordinate system to obtain second point cloud data; and, according to the second point cloud data, determine the to-be-optimized data used for the current adjustment When the external parameter value makes the same feature collected by the lidar traveling to different positions in the same position in the IMU coordinate system or the position difference meets the preset condition, the current external parameter value to be optimized is used as the target external parameter. parameter value, otherwise, adjust the external parameter value to be optimized, and use the adjusted external parameter value as the external parameter value to be optimized in the next adjustment.
A calibration device, characterized in that it includes a memory and one or more processors; wherein, the memory is used to store computer program codes, and the computer program codes include computer instructions; when the computer instructions are executed by the processor When executing, the calibration device is made to execute the method according to any one of claims 1-8, or execute the method according to any one of claims 9-13, or execute the method according to claim 14. method.
A computer-readable storage medium, characterized by comprising computer instructions, when the computer instructions are executed in a calibration device, the calibration device is made to execute the method according to any one of claims 1-8, or to execute A method as claimed in any of claims 9-13, or performing a method as claimed in claim 14.
A computer program product, characterized in that, when the computer program product is run on a processor, the processor is caused to execute the method according to any one of claims 1-8, or to execute the method according to any one of claims 9-13 The method of any one of, or perform the method of claim 14 .