CN115902843A

CN115902843A - Multi-laser-radar calibration method and device and electronic equipment

Info

Publication number: CN115902843A
Application number: CN202211467175.4A
Authority: CN
Inventors: 王弘尧; 张振林
Original assignee: China Automotive Innovation Corp
Current assignee: China Automotive Innovation Corp
Priority date: 2022-11-22
Filing date: 2022-11-22
Publication date: 2023-04-04

Abstract

The application discloses a multi-laser radar calibration method, a multi-laser radar calibration device and electronic equipment, wherein the multi-laser radar calibration method can be applied to the field of cooperative automatic driving of a vehicle and a road, and comprises the following steps: acquiring a point cloud map and a point cloud image of each road end laser radar to be calibrated; marking a feature point set based on the point cloud map; determining a target point set corresponding to the characteristic point set in each point cloud image to obtain a target point set corresponding to each road end laser radar to be calibrated in the initial pose; for each road end laser radar to be calibrated, determining the optimized pose of the road end laser radar relative to the point cloud map based on the target point set under the corresponding initial pose; and determining external parameter parameters of the multiple road end laser radars to be calibrated based on the optimized pose of each road end laser radar to be calibrated relative to the point cloud map. The method and the device can well control the precision of the external parameters of the multiple laser radars, and can quickly meet the calibration requirements of a large number of road-end laser radars in intelligent traffic road-end application.

Description

Multi-laser-radar calibration method and device and electronic equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for calibrating multiple laser radars, and an electronic device.

Background

The lidar is a sensor commonly used in the field of V2X (Vehicle to observing), can provide semantic information, and is particularly important for the sensing function of a V2X system in calibration. The traditional calibration method usually uses a calibration plate to calibrate the laser radar, and the method requires that a plurality of laser radars can simultaneously observe the calibration plate and has larger requirement on a common visual area. However, the road-end laser radar is generally placed higher and cannot move, the wiring harness distribution is sparser, and a large common-view area does not exist, so that the external reference of the road-end laser radar is difficult to calibrate by using a traditional method.

Disclosure of Invention

In order to solve the problems in the prior art, embodiments of the present application provide a multi-lidar calibration method, apparatus, electronic device, and storage medium. The technical scheme is as follows:

on one hand, a multi-laser radar calibration method is provided, and the method comprises the following steps:

acquiring a point cloud map and a point cloud image of each to-be-calibrated road end laser radar in a plurality of to-be-calibrated road end laser radars; the point cloud map is obtained based on first point cloud data in a preset range acquired by a laser radar on an acquisition vehicle, the preset range comprises scanning ranges of the plurality of road end laser radars to be calibrated, and a point cloud image of each road end laser radar to be calibrated is an image generated based on second point cloud data acquired by the corresponding road end laser radar to be calibrated;

labeling a feature point set based on the point cloud map;

determining a target point set corresponding to the feature point set in each point cloud image to obtain a target point set corresponding to each road end laser radar to be calibrated in an initial pose;

for each road end laser radar to be calibrated, determining the optimized pose of the road end laser radar to be calibrated relative to the point cloud map based on the target point set under the corresponding initial pose;

and determining external parameter of the road end laser radars to be calibrated based on the optimized pose of each road end laser radar to be calibrated relative to the point cloud map.

In another aspect, a multi-lidar calibration apparatus is provided, the apparatus comprising:

the system comprises an information acquisition module, a road end laser radar calibration module and a road end image calibration module, wherein the information acquisition module is used for acquiring a point cloud map and a point cloud image of each road end laser radar to be calibrated in a plurality of road end laser radars to be calibrated; the point cloud map is obtained based on first point cloud data in a preset range acquired by a laser radar on an acquisition vehicle, the preset range comprises scanning ranges of the plurality of road end laser radars to be calibrated, and a point cloud image of each road end laser radar to be calibrated is an image generated based on second point cloud data acquired by the corresponding road end laser radar to be calibrated;

the characteristic marking module is used for marking a characteristic point set based on the point cloud map;

the point set matching module is used for determining a target point set corresponding to the characteristic point set in each point cloud image to obtain a target point set corresponding to each road end laser radar to be calibrated in an initial pose;

the optimization processing module is used for determining the optimized pose of each to-be-calibrated road end laser radar relative to the point cloud map based on the target point set under the corresponding initial pose;

and the external parameter determining module is used for determining external parameter among the road end laser radars to be calibrated based on the optimized pose of each road end laser radar to be calibrated relative to the point cloud map.

In an exemplary embodiment, the feature labeling module includes:

the area determining module is used for determining a map area which is overlapped with the scanning ranges of the multiple road end laser radars to be calibrated on the point cloud map;

and the facility marking module is used for determining a first preset number of road facilities in the map area and marking the road facilities as the feature point set.

In an exemplary embodiment, the point set matching module includes:

the point cloud moving module is used for moving the point cloud images aiming at each point cloud image so that the point cloud images are superposed with the point cloud map, and the initial pose of the to-be-calibrated road end laser radar is obtained;

and the point set marking module is used for determining a point set which is coincident with the characteristic point set on each point cloud image and marking the point set as a target point set corresponding to the road end laser radar to be calibrated corresponding to each point cloud image under the initial pose.

In an exemplary embodiment, the optimization processing module includes:

the adjacent point marking module is used for marking a second preset number of nearest adjacent points corresponding to the feature points on the point cloud map aiming at each feature point in the feature point set; the nearest neighbor point is a point on the point cloud map, the distance between the nearest neighbor point and the corresponding feature point is within a second preset number in ascending order;

the distance determining module is used for determining the distance between the target point and a second preset number of nearest neighbor points corresponding to the target feature point aiming at each target point in a target point set corresponding to each road-end laser radar to be calibrated under the initial pose, so as to obtain the target distance corresponding to the target point; the target characteristic points refer to characteristic points in the characteristic point set corresponding to the target points;

and the distance optimization module is used for optimizing the target distance of each target point in the target point set corresponding to each road end laser radar to be calibrated under the initial pose to obtain the optimized pose of each road end laser radar to be calibrated relative to the point cloud map.

In an exemplary embodiment, the distance determining module includes:

the adjacent point fitting module is used for fitting a second preset number of nearest adjacent points corresponding to the target characteristic points according to preset geometric elements to obtain a fitting result corresponding to the target point; the preset geometric elements comprise straight lines or planes;

and the first determining module is used for determining the distance between the target point and the fitting result to obtain the target distance corresponding to the target point.

In an exemplary embodiment, the distance optimization module includes:

the distance summing module is used for determining the sum of the target distances of all target points in a corresponding target point set under the corresponding initial pose aiming at each to-be-calibrated road end laser radar to obtain the target optimization distance corresponding to the to-be-calibrated road end laser radar;

the position and pose adjusting module is used for adjusting the initial position and pose of the to-be-calibrated road end laser radar, updating the target optimization distance corresponding to the to-be-calibrated road end laser radar based on the target point set corresponding to the to-be-calibrated road end laser radar under the adjusted initial position and finishing the adjustment of the initial position and pose of the to-be-calibrated road end laser radar until the updated target optimization distance is smaller than a preset distance threshold;

and the pose determining module is used for determining the initial pose of the to-be-calibrated road end laser radar after the adjustment is finished as the optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map.

In an exemplary embodiment, the external parameter determining module includes:

the radar determination module is used for determining a first road end laser radar and a second road end laser radar which are to be calibrated mutually in a plurality of road end laser radars to be calibrated;

the second determination module is used for determining a first optimized pose of the first path laser radar relative to the point cloud map and a second optimized pose of the second path laser radar relative to the point cloud map;

the first coordinate transformation module is used for determining a coordinate transformation matrix of the first road end laser radar relative to the second road end laser radar based on the first optimization pose and the second optimization pose, and the coordinate transformation matrix is used as an external parameter of the first road end laser radar;

and the second coordinate transformation module is used for determining a coordinate transformation matrix of the second road end laser radar relative to the first road end laser radar based on the first optimized pose and the second optimized pose, and the coordinate transformation matrix is used as an external parameter of the second road end laser radar.

In an exemplary embodiment, the apparatus further comprises a mapping module for making a point cloud map, the mapping module comprising:

the point cloud acquisition module is used for acquiring first point cloud data acquired by a laser radar on a vehicle at multiple moments in the moving process; the plurality of moments are moments separated by preset time periods, and the first point cloud data are point cloud data in the preset range acquired by a laser radar on the acquisition vehicle;

the map creation module is used for selecting a first moment from the moments and determining a current point cloud map based on first point cloud data acquired at the first moment;

the map updating module is used for selecting a second moment from the rest moments and updating the current point cloud map based on first point cloud data acquired at the second moment; the remaining time refers to the unselected time in the plurality of times;

the map optimization module is used for repeating the steps of selecting a second moment from the rest moments and updating the current point cloud map based on first point cloud data acquired at the second moment under the condition that the coverage degree of the updated current point cloud map and the preset range does not reach the preset coverage degree;

and the map determining module is used for taking the updated current point cloud map reaching the preset coverage degree as the point cloud map.

In another aspect, an electronic device is provided, which includes a processor and a memory, where at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded by the processor and executed to implement the multi-lidar calibration method according to any of the above aspects.

In another aspect, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the multi-lidar calibration method according to any of the above aspects.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the multi-lidar calibration method of any one of the aspects.

According to the method and the device for calibrating the road end laser radar point cloud map, the point cloud map is created for the area through the laser radar on the vehicle, road facilities are marked on the point cloud map, then the point cloud of the road end laser radar to be calibrated is matched to the point cloud map, and external reference calibration of multiple radars is achieved. The external reference calibration can be completed even if no common visual area exists between the two radars; meanwhile, the precision of the external parameter can be well controlled and almost only depends on the precision of the distance measurement of the radar; the calibration requirements of a large number of road-end laser radars in intelligent traffic road-end application can be quickly realized.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flowchart of a multi-lidar calibration method provided in an embodiment of the present application;

FIG. 2 is a schematic flow chart of a labeled feature point set provided in the embodiment of the present application;

FIG. 3 is a schematic flowchart of a labeling target point set provided in an embodiment of the present application;

FIG. 4 is a schematic flow chart of an optimization process provided by an embodiment of the present application;

FIG. 5 is a schematic flowchart of a multi-lidar external reference calibration provided by an embodiment of the present application;

fig. 6 is a schematic flow chart of manufacturing a point cloud map according to the embodiment of the present application;

fig. 7 is a block diagram of a multi-lidar calibration apparatus provided in an embodiment of the present application;

fig. 8 is a block diagram of a hardware structure of an electronic device according to an embodiment of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It is understood that in the specific implementation of the present application, related data such as user information is involved, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with relevant laws and regulations and standards in relevant countries and regions.

Please refer to fig. 1, which is a schematic flow chart illustrating a multi-lidar calibration method according to an embodiment of the present disclosure. It is noted that the present specification provides the method steps as described in the examples or flowcharts, but may include more or less steps based on routine or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. In actual system or product execution, sequential execution or parallel execution (e.g., parallel processor or multi-threaded environment) may be possible according to the embodiments or methods shown in the figures. Specifically, as shown in fig. 1, the method may include:

s101, a point cloud map and a point cloud image of each road end laser radar to be calibrated in a plurality of road end laser radars to be calibrated are obtained.

The point cloud map is obtained based on first point cloud data in a preset range acquired by a laser radar on the acquisition vehicle. Wherein the first point cloud data is point cloud data of objects in a laser radar scanning range on the collected vehicle

In specific implementation, the collection vehicle can move around a plurality of road end laser radars to be calibrated so as to collect point cloud data in a preset range of the road end laser radars through the laser radars on the collection vehicle, the point cloud data is used for manufacturing a point cloud map, a plurality of road end laser radars to be calibrated can be used for manufacturing a point cloud map, and each road end laser radar to be calibrated can be used for manufacturing a point cloud map.

The preset range comprises a scanning range of a plurality of road-end laser radars to be calibrated.

And the point cloud image of each road end laser radar to be calibrated is an image generated based on second point cloud data acquired by the corresponding road end laser radar to be calibrated. And the second point cloud data is the point cloud data of the object in the scanning range of the laser radar of the road end to be calibrated.

And S103, marking a characteristic point set based on the point cloud map.

The feature point set can comprise a ground plane point set, a road edge point set and a lamp post edge point set. In particular implementation, if the road edge point sets are labeled but not the light pole edge point sets, each feature point set should include multiple sets of road edges, and not all parallel to each other, in order to ensure that two independent coordinates in the horizontal plane direction can be uniquely solved.

In an exemplary embodiment, as shown in fig. 2, the step S103 may include:

s201, determining a map area which is overlapped with the scanning ranges of the road end laser radars to be calibrated on the point cloud map.

The map area is determined to ensure that the overlapping area of the point cloud map and the point cloud image of each road end laser radar to be calibrated is selected, namely, the marked feature point set is located in the common-view area of the laser radar on the collection vehicle and the road end laser radar to be calibrated.

S203, determining a first preset number of road facilities in the map area, and marking the road facilities as a feature point set.

The road facilities comprise the ground, a road edge and a lamp post, and the road facilities are located in a common-view area for collecting laser radars on vehicles and road-end laser radars to be calibrated. In specific implementation, the marking of the feature point set is completed by manually marking road facilities.

According to the technical scheme, the road facilities are marked in the common-view area by determining the common-view area of the laser radar on the collection vehicle and the road end laser radar to be calibrated, so that preliminary external reference calibration of the road end laser radar to be calibrated can be guaranteed based on the point cloud map.

And S105, determining a target point set corresponding to the characteristic point set in each point cloud image to obtain a target point set corresponding to each road-end laser radar to be calibrated in the initial pose.

In specific implementation, the labeling of the target point set needs to be performed on the basis of rough registration of the point cloud map and the point cloud image, and in the process, a coordinate transformation matrix of coordinates in the point cloud image and coordinates in the point cloud map of each road end laser radar to be calibrated can be determined, namely the initial pose of each road end laser radar to be calibrated relative to the point cloud map.

In an exemplary embodiment, as shown in fig. 3, the step S105 may include:

s301, aiming at each point cloud image, moving the point cloud image to enable the point cloud image to be overlapped with the point cloud map, and obtaining the initial pose of the road end laser radar to be calibrated.

In the step, a coordinate transformation matrix determined based on the moving process is determined as the initial pose of the road-end laser radar to be calibrated in the process of coarse registration of each point cloud image.

And S303, determining a point set which is coincident with the characteristic point set on each point cloud image, and labeling the point set as a target point set corresponding to each point cloud image and corresponding to the road-end laser radar to be calibrated under the initial pose.

And the target point set is a point set corresponding to the road facility which is superposed with the characteristic point set on the point cloud image.

In specific implementation, the feature point sets are respectively marked with numbers, after a target point set which is coincident with the feature point set on the point cloud image is determined, the target point set is also marked with corresponding numbers, and the feature point sets with different numbers and the target point set cannot be matched with each other.

According to the technical scheme, the point cloud image and the point cloud map of each to-be-calibrated road end laser radar are roughly registered, and the target point set corresponding to the feature point set on the point cloud map one by one is determined for each to-be-calibrated road end laser radar in the initial pose, so that the optimal pose of the to-be-calibrated road end laser radar relative to the point cloud map can be determined by gradually optimizing the position of the target point set relative to the point cloud map.

And S107, for each road end laser radar to be calibrated, determining the optimized position and pose of the road end laser radar to be calibrated relative to the point cloud map based on the target point set under the corresponding initial position and pose.

And optimizing the position of the target point set relative to the point cloud map by adjusting the initial pose, and determining the adjusted initial pose as the optimized pose when an expected optimization result is achieved.

In an exemplary embodiment, as shown in fig. 4, the step S107 may include:

s401, for each feature point in the feature point set, marking a second preset number of nearest neighbor points corresponding to the feature point on the point cloud map.

And the nearest neighbor point is a point on the point cloud map, wherein the distance between the nearest neighbor point and the corresponding feature point is within a second preset number in the ascending order.

And traversing each feature point, sending a traversal point cloud map from the current feature point, and marking a second preset number of nearest neighbor points corresponding to each feature point.

And S403, determining the distance between the target point and a second preset number of nearest neighbor points corresponding to the target feature point for each target point in a target point set corresponding to each road-end laser radar to be calibrated in the initial pose, and obtaining the target distance corresponding to the target point.

The target feature point refers to a feature point corresponding to the target point in the feature point set.

Here, the nearest neighbor point corresponding to the feature point corresponding to the target point is defaulted to be the point on the point cloud map closest to the target point.

The step S403 may include the following steps:

fitting a second preset number of nearest neighbor points corresponding to the target feature points according to preset geometric elements to obtain a fitting result corresponding to the target feature points; the preset geometric elements comprise straight lines or planes;

and determining the distance between the target point and the fitting result to obtain the target distance corresponding to the target point.

The preset geometric elements are different according to different types of feature point sets to which the target feature points belong. For the ground plane point set, presetting a geometric element as a plane; and for the road edge point set and the lamp post edge point set, the preset geometric elements are straight lines.

The target distance also corresponds to the two situations, and is different according to different types of target point sets to which the target points belong. For the ground plane point set, the target distance is the distance of a plane fitted by the target point and the nearest neighbor point corresponding to the target feature point; for the road edge point set and the lamp post edge point set, the target distance is the distance of a straight line which is fitted with the nearest adjacent point corresponding to the target point and the target characteristic point.

According to the technical scheme, the method and the device for optimizing the current pose provide a cut-in for subsequently optimizing the current pose through the error corresponding to the initial pose corresponding to the distance of the geometric elements, which are fitted to the nearest neighbor points corresponding to the target point and the target feature points.

S405, optimizing the target distance of each target point in the target point set corresponding to each road end laser radar to be calibrated under the initial pose to obtain the optimized pose of each road end laser radar to be calibrated relative to the point cloud map.

The target distance can reflect the error of the initial pose of the to-be-calibrated road end laser radar relative to the point cloud map compared with the actual pose of the to-be-calibrated road end laser radar relative to the world coordinate system to a certain extent, and the initial pose is optimized based on the error, so that the optimized pose of the to-be-calibrated road end laser radar is obtained.

And the optimization processing is to obtain a target distance based on the initial pose, slightly change the initial pose, reduce the target distance until the target distance reaches the minimum, and determine the adjusted initial pose which enables the target distance to be the minimum as the optimized pose of the road end laser radar to be calibrated.

According to the technical scheme of the embodiment of the application, the distance between the target point in the target point set corresponding to the road end laser radar to be calibrated under the initial pose and the nearest neighbor points corresponding to the target feature points in the second preset number is used for calibrating the error of the initial pose, and the distance is continuously reduced, so that the initial pose is continuously optimized.

The step S405 may include the steps of:

determining the sum of target distances of target points in a corresponding target point set under a corresponding initial pose aiming at each to-be-calibrated road end laser radar to obtain a target optimization distance corresponding to the to-be-calibrated road end laser radar;

adjusting the initial pose of the to-be-calibrated road-end laser radar, updating the target optimization distance corresponding to the to-be-calibrated road-end laser radar based on the target point set corresponding to the to-be-calibrated road-end laser radar under the adjusted initial pose, and ending the adjustment of the initial pose of the to-be-calibrated road-end laser radar until the updated target optimization distance is smaller than a preset distance threshold;

and determining the initial pose of the to-be-calibrated road end laser radar after the adjustment is finished as the optimized pose of the to-be-calibrated road end laser radar relative to the point cloud map.

The target optimization distance is an overall reflection of the error of the position of the target point relative to the point cloud map compared with the actual coordinate of the target point in the world coordinate system, and the error reflects the error of the initial pose of the road end laser radar to be calibrated relative to the point cloud map compared with the actual pose of the road end laser radar to be calibrated relative to the world coordinate system. And judging whether the error of the position of the target point set relative to the point cloud map of each road end laser radar to be calibrated under the initial pose of the road end laser radar to be calibrated reaches the minimum or not based on the target optimization distance.

The preset distance threshold is set according to the acceptable error of the initial pose of the road end laser radar to be calibrated relative to the point cloud map compared with the actual pose of the road end laser radar to be calibrated relative to the world coordinate system, and the smaller the preset distance threshold is, the more accurate the optimized pose is and the more accurate the calibration parameters are.

The initial pose of the road-end laser radar to be calibrated after the adjustment is finished means that the target optimization distance is the minimum under the initial pose, namely the error of the position of the target point relative to the point cloud map is the minimum compared with the error of the actual coordinate of the target point under the world coordinate system, and the initial pose is optimized at the moment.

In specific implementation, the initial pose of each road end laser radar to be calibrated is adjusted, the point cloud image is moved according to the initial pose, the target point set on the point cloud image is moved accordingly, and the distance between the target point in the moved target point set and the nearest neighbor points of the second preset number corresponding to the target feature points, namely the target optimization distance, is determined. Judging whether the target optimization distance reaches the minimum, and if so, determining the initial pose adjusted in the step as the optimization pose; and if the target optimization distance does not reach the minimum, continuing to adjust the initial pose, repeating the steps until the target optimization distance reaches the minimum, finishing the adjustment, and determining the initial pose after finishing the adjustment as the optimization pose. The method is a continuous heuristic process, and the initial pose is continuously adjusted to explore the minimum value of the error of the position of a target point relative to a point cloud map compared with the actual coordinate of the target point in a world coordinate system, so that the optimization of the initial pose is completed.

According to the technical scheme of the embodiment of the application, whether the initial pose is optimal or not is judged by judging whether the distance between the target point and the second preset number of nearest adjacent points corresponding to the target feature points is minimum or not, whether the initial pose is continuously adjusted or not is determined according to the optimal initial pose, the target optimization distance is shortened, and therefore the minimum error of the optimization pose is ensured.

S109, determining external parameter of the road end laser radars to be calibrated based on the optimized pose of each road end laser radar to be calibrated relative to the point cloud map.

And determining a coordinate transformation matrix among the plurality of road end laser radars to be calibrated through matrix operation based on the coordinate transformation matrix of each road end laser radar to be calibrated relative to the point cloud map in the plurality of road end laser radars to be calibrated.

According to the technical scheme, the point cloud map is created for the area by collecting the laser radar on the vehicle, the feature point set is marked on the point cloud map, and then the point cloud of the road end laser radar to be calibrated is matched to the point cloud map, so that the external reference calibration of multiple radars is realized. The external reference calibration can be completed even if no common visual area exists between the two radars. Meanwhile, the precision of the external parameter can be well controlled and almost only depends on the precision of the distance measurement of the radar. The calibration requirements of a large number of road-end laser radars in intelligent traffic road-end application can be quickly realized.

In an exemplary embodiment, as shown in fig. 5, the step S109 may include:

s501, a first road end laser radar and a second road end laser radar to be calibrated with each other are determined in a plurality of road end laser radars to be calibrated.

The coordinate transformation matrix of a plurality of road end laser radars to be calibrated relative to a point cloud map is known, the coordinate transformation matrix of the plurality of road end laser radars to be calibrated relative to each other is determined, the coordinate transformation matrix between every two road end laser radars needs to be determined, and therefore two road end laser radars to be calibrated relative to each other need to be determined.

S503, determining a first optimized pose of the first end laser radar relative to the point cloud map and a second optimized pose of the second end laser radar relative to the point cloud map.

The first optimization pose is an optimized coordinate transformation matrix of the first road end laser radar relative to the point cloud map, and the second optimization pose is an optimized coordinate transformation matrix of the second road end laser radar relative to the point cloud map.

And S505, determining a coordinate transformation matrix of the first path laser radar relative to the second path laser radar based on the first optimization pose and the second optimization pose as external parameters of the first path laser radar.

And obtaining a coordinate transformation matrix of the first path laser radar relative to the second path laser radar through matrix operation based on the coordinate transformation matrix of the first path laser radar relative to the point cloud map and the coordinate transformation matrix of the second path laser radar relative to the point cloud map.

And S507, determining a coordinate transformation matrix of the second path laser radar relative to the first path laser radar based on the first optimized pose and the second optimized pose, and taking the coordinate transformation matrix as an external parameter of the second path laser radar.

And obtaining a coordinate transformation matrix of the second road end laser radar relative to the first road end laser radar through matrix operation based on the coordinate transformation matrix of the first road end laser radar relative to the point cloud map and the coordinate transformation matrix of the second road end laser radar relative to the point cloud map.

According to the technical scheme, the external reference calibration of the multiple road-end laser radars to be calibrated can be completed by determining the optimized positions of the multiple road-end laser radars to be calibrated relative to the point cloud map so as to complete the external reference calibration of the multiple road-end laser radars to be calibrated.

The manufacturing of the point cloud map is explained below before the point cloud map and the point cloud image of each to-be-calibrated road end lidar in the plurality of to-be-calibrated road end radars are obtained, as shown in fig. 6, the method specifically includes the following steps:

s601, acquiring first point cloud data acquired by a laser radar on a vehicle at multiple moments in a moving process.

The plurality of times are times separated by a preset time period, and data is collected every 100 milliseconds in the embodiment.

And the first point cloud data is point cloud data in the preset range acquired by a laser radar on the acquisition vehicle.

S603, selecting a first moment from the moments, and determining a current point cloud map based on first point cloud data acquired at the first moment.

The method comprises the steps of randomly selecting a first moment from a plurality of moments, collecting first point cloud data of the first moment based on a laser radar on a collection vehicle, obtaining the current pose of a vehicle-mounted laser radar of the first moment based on a laser radar odometer on the collection vehicle, manufacturing a current point cloud map based on the first point cloud data and the current pose, and obtaining the current point cloud map based on the first point cloud data collected at the first moment, wherein the current point cloud map is a basic map of a subsequent update map.

S605, selecting a second moment from the rest moments, and updating the current point cloud map based on the first point cloud data acquired at the second moment.

Wherein the remaining time refers to a time that is not selected from the plurality of times.

And acquiring first point cloud data of a second moment based on the laser radar on the acquisition vehicle, obtaining the current pose of the vehicle-mounted laser radar of the second moment based on the laser radar odometer on the acquisition vehicle, determining a point cloud map of the second moment based on the first point cloud data and the current pose, and supplementing the point cloud map to the current point cloud map, namely finishing updating the current point cloud map.

And S607, under the condition that the coverage degree of the updated current point cloud map and the preset range does not reach the preset coverage degree, repeating the steps of selecting a second moment from the rest moments and updating the current point cloud map based on first point cloud data acquired at the second moment until the coverage degree of the updated current point cloud map and the preset range reaches the preset coverage degree.

The preset coverage degree is a specific value used for judging whether the coverage rate of the current point cloud map to the preset range reaches the standard or not.

The method is a cyclic updating process, and the coverage range of the current point cloud map is continuously expanded by collecting first point cloud data at different moments until the current point cloud map can cover the scanning ranges of a plurality of road-end laser radars to be calibrated.

And S609, taking the updated current point cloud map reaching the preset coverage degree as the point cloud map.

According to the technical scheme, the point cloud map is manufactured by collecting the point cloud data collected by the laser radar on the vehicle in the scanning range of the multiple road end laser radars to be calibrated, the point cloud map is used as a reference for optimizing the initial pose of the road end laser radar to be calibrated relative to the point cloud map, the point cloud data in different ranges at different moments are collected by the movement of the vehicle, the point cloud maps at different moments correspond to the point cloud maps in different ranges, the current point cloud map is updated by manufacturing the point cloud maps at different moments, and the point cloud maps in different ranges are spliced, so that the finally completed point cloud map covers the scanning range of the multiple road end laser radars to be calibrated.

Corresponding to the multiple lidar calibration methods provided in the foregoing embodiments, embodiments of the present application also provide a multiple lidar calibration apparatus, and because the multiple lidar calibration apparatus provided in embodiments of the present application corresponds to the multiple lidar calibration methods provided in the foregoing embodiments, the implementation of the foregoing multiple lidar calibration method is also applicable to the multiple lidar calibration apparatus provided in this embodiment, and is not described in detail in this embodiment.

Please refer to fig. 7, which is a schematic structural diagram of a multi-lidar calibration apparatus provided in an embodiment of the present application, where the apparatus has a function of implementing the multi-lidar calibration method in the foregoing method embodiment, and the function may be implemented by hardware or by hardware executing corresponding software. As shown in fig. 7, the apparatus may include:

the system comprises an information acquisition module, a road end laser radar calibration module and a road end image calibration module, wherein the information acquisition module is used for acquiring a point cloud map and a point cloud image of each road end laser radar to be calibrated in a plurality of road end laser radars to be calibrated; the point cloud map is obtained based on first point cloud data in a preset range acquired by a laser radar on the acquisition vehicle, the preset range comprises scanning ranges of the plurality of road end laser radars to be calibrated, and a point cloud image of each road end laser radar to be calibrated is an image generated based on second point cloud data acquired by the corresponding road end laser radar to be calibrated;

and the external parameter determining module is used for determining external parameter among the plurality of road end laser radars to be calibrated based on the optimized position and posture of each road end laser radar to be calibrated relative to the point cloud map.

In an exemplary embodiment, the feature labeling module includes:

and the facility marking module is used for determining a first preset number of road facilities in the map area, and marking the road facilities as the feature point set.

In an exemplary embodiment, the point set matching module includes:

and the point set marking module is used for determining a point set which is coincident with the characteristic point set on each point cloud image and marking the point set as a target point set which corresponds to each point cloud image and corresponds to the to-be-calibrated road-end laser radar in the initial pose.

In an exemplary embodiment, the optimization processing module includes:

the adjacent point marking module is used for marking a second preset number of nearest adjacent points corresponding to the feature points on the point cloud map aiming at each feature point in the feature point set; the nearest neighbor point is a point on the point cloud map, wherein the distance between the nearest neighbor point and the corresponding feature point is within a second preset number in ascending order;

the distance determining module is used for determining the distance between the target point and a second preset number of nearest neighbors corresponding to the target feature point aiming at each target point in a target point set corresponding to each road-end laser radar to be calibrated in the initial pose, so as to obtain the target distance corresponding to the target point; the target characteristic points refer to characteristic points in the characteristic point set corresponding to the target points;

and the distance optimization module is used for optimizing the target distance of each target point in the target point set corresponding to each road end laser radar to be calibrated under the initial pose so as to obtain the optimized pose of each road end laser radar to be calibrated relative to the point cloud map.

In an exemplary embodiment, the distance determining module includes:

In an exemplary embodiment, the distance optimization module includes:

a pose adjusting module, configured to adjust an initial pose of the to-be-calibrated road end lidar, update a target optimization distance corresponding to the to-be-calibrated road end lidar based on a target point set corresponding to the to-be-calibrated road end lidar in the adjusted initial pose, and terminate adjustment of the initial pose of the to-be-calibrated road end lidar until the updated target optimization distance is smaller than a preset distance threshold;

In an exemplary embodiment, the external parameter determining module includes:

the second determining module is used for determining a first optimized pose of the first path laser radar relative to the point cloud map and a second optimized pose of the second path laser radar relative to the point cloud map;

the point cloud acquisition module is used for acquiring first point cloud data acquired by a laser radar on a vehicle at multiple moments in the moving process; the multiple moments are moments separated by a preset time period, and the first point cloud data are point cloud data in the preset range acquired by a laser radar on the acquisition vehicle;

It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, the division of each functional module is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.

The embodiment of the present application provides an electronic device, which includes a processor and a memory, where at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded and executed by the processor to implement any one of the multiple lidar calibration methods provided in the above method embodiments.

The memory may be used to store software programs and modules, and the processor may execute various functional applications and data processing by operating the software programs and modules stored in the memory. The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs needed by functions and the like; the storage data area may store data created according to use of the apparatus, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory may also include a memory controller to provide the processor access to the memory.

The method embodiments provided in the embodiments of the present application may be executed in a computer terminal, a server, or a similar computing device, that is, the electronic device may include a computer terminal, a server, or a similar computing device. Fig. 8 is a block diagram of a hardware structure of an electronic device for operating a multi-lidar calibration method according to an embodiment of the present application, and as shown in fig. 8, an internal structure of the computer device may include, but is not limited to: a processor, a network interface, and a memory. The processor, the network interface, and the memory in the computer device may be connected by a bus or in other manners, and fig. 8 shown in the embodiment of the present specification is exemplified by being connected by a bus.

The processor (or CPU) is a computing core and a control core of the computer device. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI, mobile communication interface, etc.). Memory (Memory) is a Memory device in a computer device used to store programs and data. It is understood that the memory herein may be a high-speed RAM storage device, or may be a non-volatile storage device (non-volatile memory), such as at least one magnetic disk storage device; optionally, at least one memory device located remotely from the processor. The memory provides a storage space that stores an operating system of the electronic device, which may include, but is not limited to: a Windows system (an operating system), a Linux system (an operating system), an Android system, an IOS system, etc., which are not limited in the present invention; also, one or more instructions, which may be one or more computer programs (including program code), are stored in the memory space and are adapted to be loaded and executed by the processor. In an embodiment of the present specification, the processor loads and executes one or more instructions stored in the memory to implement the multi-lidar calibration method provided in the foregoing method embodiment.

Embodiments of the present application further provide a computer-readable storage medium, which may be disposed in an electronic device to store at least one instruction or at least one program for implementing a multi-lidar calibration method, where the at least one instruction or the at least one program is loaded and executed by the processor to implement any one of the multi-lidar calibration methods provided in the foregoing method embodiments.

Optionally, in this embodiment, the storage medium may include but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and various media capable of storing program codes.

It should be noted that: the sequence of the embodiments of the present application is only for description, and does not represent the advantages or disadvantages of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A multi-laser radar calibration method is characterized by comprising the following steps:

labeling a feature point set based on the point cloud map;

determining a target point set corresponding to the characteristic point set in each point cloud image to obtain a target point set corresponding to each road-end laser radar to be calibrated in an initial pose;

2. The method of claim 1, wherein the labeling a feature point set based on the point cloud map comprises:

determining a map area which is overlapped with the scanning ranges of the multiple road end laser radars to be calibrated on the point cloud map;

and determining a first preset number of road facilities in the map area, and marking the road facilities as a feature point set.

3. The method for calibrating multiple lidar according to claim 1, wherein the determining a target point set corresponding to the feature point set in each point cloud image to obtain a target point set corresponding to each to-be-calibrated road-end lidar in an initial pose comprises:

moving the point cloud image to enable the point cloud image to be overlapped with the point cloud map, so as to obtain an initial pose of the road end laser radar to be calibrated;

and determining a point set which is coincident with the characteristic point set on each point cloud image, and labeling the point set as a target point set corresponding to each point cloud image at the initial pose of the road end laser radar to be calibrated.

4. The method according to claim 1, wherein the determining, for each road-end lidar to be calibrated, an optimized pose of the road-end lidar to be calibrated with respect to the point cloud map based on a set of target points at a corresponding initial pose comprises:

for each feature point in the feature point set, marking a second preset number of nearest neighbor points corresponding to the feature point on the point cloud map; the nearest neighbor point is a point on the point cloud map, wherein the distance between the nearest neighbor point and the corresponding feature point is within a second preset number in ascending order;

determining the distance between the target point and a second preset number of nearest neighbor points corresponding to the target feature point aiming at each target point in a target point set corresponding to each road end laser radar to be calibrated under the initial pose, and obtaining the target distance corresponding to the target point; the target characteristic points refer to characteristic points in the characteristic point set corresponding to the target points;

and optimizing the target distance of each target point in the target point set corresponding to each road end laser radar to be calibrated under the initial pose to obtain the optimized pose of each road end laser radar to be calibrated relative to the point cloud map.

5. The method for calibrating multiple lidar according to claim 4, wherein the determining the distance between the target point and a second predetermined number of nearest neighboring points corresponding to the target feature point to obtain the target distance corresponding to the target point comprises:

6. The multi-lidar calibration method according to claim 4, wherein the optimizing the target distance of each target point in the target point set corresponding to each to-be-calibrated link laser radar in the initial pose to obtain the optimized pose of each to-be-calibrated link laser radar relative to the point cloud map comprises:

7. The method according to claim 1, wherein the determining external parameters of the road-end lidar to be calibrated relative to each other based on the optimized pose of each road-end lidar to be calibrated relative to the point cloud map comprises:

determining a first road end laser radar and a second road end laser radar to be calibrated mutually in a plurality of road end laser radars to be calibrated;

determining a first optimized pose of the first end laser radar relative to the point cloud map and a second optimized pose of the second end laser radar relative to the point cloud map;

determining a coordinate transformation matrix of the first road end laser radar relative to the second road end laser radar based on the first optimization pose and the second optimization pose as external parameters of the first road end laser radar;

and determining a coordinate transformation matrix of the second path laser radar relative to the first path laser radar based on the first optimized pose and the second optimized pose, and taking the coordinate transformation matrix as an external parameter of the second path laser radar.

8. The method of claim 1, wherein before the point cloud map and the point cloud image of each of the plurality of road-end lidar to be calibrated, the method further comprises:

acquiring first point cloud data acquired by a laser radar on a vehicle at multiple moments in a moving process; the multiple moments are moments separated by a preset time period, and the first point cloud data are point cloud data in the preset range acquired by a laser radar on the acquisition vehicle;

selecting a first moment from the plurality of moments, and determining a current point cloud map based on first point cloud data acquired at the first moment;

selecting a second moment from the rest moments, and updating the current point cloud map based on first point cloud data acquired at the second moment; the remaining time refers to the unselected time in the plurality of times;

under the condition that the coverage degree of the updated current point cloud map and the preset range does not reach the preset coverage degree, repeating the step of selecting a second moment from the rest moments and updating the current point cloud map based on first point cloud data acquired at the second moment until the coverage degree of the updated current point cloud map and the preset range reaches the preset coverage degree;

and taking the updated current point cloud map when the preset coverage degree is reached as the point cloud map.

9. A multi-laser radar calibration device is characterized in that the method comprises the following steps:

the information acquisition module is used for acquiring a point cloud map and a point cloud image of each to-be-calibrated road end laser radar in a plurality of to-be-calibrated road end laser radars; the point cloud map is obtained based on first point cloud data in a preset range acquired by a laser radar on the acquisition vehicle, the preset range comprises scanning ranges of the plurality of road end laser radars to be calibrated, and a point cloud image of each road end laser radar to be calibrated is an image generated based on second point cloud data acquired by the corresponding road end laser radar to be calibrated;

10. An electronic device, comprising a processor and a memory, wherein at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded by the processor and executed to implement the method according to any one of claims 1 to 8.