CN116165639A

CN116165639A - Multi-laser radar calibration method and device, terminal equipment and storage medium

Info

Publication number: CN116165639A
Application number: CN202211713329.3A
Authority: CN
Inventors: 郭雪梅; 毛巨洪; 胡攀攀
Original assignee: Wuhan Wanji Photoelectric Technology Co Ltd
Current assignee: Wuhan Wanji Photoelectric Technology Co Ltd
Priority date: 2022-12-29
Filing date: 2022-12-29
Publication date: 2023-05-26

Abstract

The application provides a multi-laser radar calibration method, device, terminal equipment and storage medium, firstly, according to point cloud data of an artificial feature board and reflectivity data of the artificial feature board under each laser radar coordinate system to be calibrated, calibrating feature points of each feature graph of the artificial feature board under each laser radar coordinate system, and then calibrating relative pose of each laser radar according to the feature points of each feature graph of the artificial feature board under each laser radar coordinate system.

Description

Multi-laser radar calibration method and device, terminal equipment and storage medium

Technical Field

The application belongs to the technical field of safety, and particularly relates to a multi-laser radar calibration method, a multi-laser radar calibration device, terminal equipment and a storage medium.

Background

Along with the continuous development of unmanned technology, the types and the number of sensors equipped for automatic driving vehicles are continuously increased, wherein the laser radar sensor has the characteristics of high distance sensing precision, high reflectivity of the object surface in a perceivable scene, high anti-interference capability and the like, and is widely applied to a plurality of automatic driving scenes. But a single laser cannot acquire complete information in an autopilot scenario. Therefore, it is necessary to combine a plurality of lidars. In this application scenario, it is important to obtain the positional relationship and coordinate transformation between the laser radars.

The calibration method between the multi-laser radars adopts black and white grids as the calibration plates, but the accurate positions of the calibration plates in a world coordinate system are required to be known, the positions of the calibration plates are required to be moved, the process of the method is complex, and the requirement on the calculation performance of equipment is high.

Disclosure of Invention

The embodiment of the application provides a multi-laser radar calibration method, a multi-laser radar calibration device, terminal equipment and a storage medium, which can solve the problems of complex external parameter calibration process and high equipment requirements among a plurality of laser radars in the prior art.

In one aspect of the present application, an embodiment provides a multi-lidar calibration method, including:

calibrating feature points of each feature graph of the artificial feature board under each laser radar coordinate system according to point cloud data of the artificial feature board under each laser radar coordinate system to be calibrated and reflectivity data of the artificial feature board;

and calibrating the relative pose of each laser radar according to the characteristic point positions of each characteristic graph of the artificial characteristic plate under each laser radar coordinate system.

In an alternative embodiment, the method further comprises:

acquiring initial scanning point cloud data under each laser radar coordinate system to be calibrated and initial reflectivity data of a scanning artificial feature plate; the initial scanning point cloud data comprise scanning point cloud data of an artificial feature plate;

And carrying out plane detection on the initial scanning point cloud data and the initial reflectivity data by using a preset plane constraint condition to generate point cloud data of an artificial characteristic plate and reflectivity data of the artificial characteristic plate under each laser radar coordinate system to be calibrated.

In an alternative embodiment, before performing plane detection on the initial scanning point cloud data and the initial reflectivity data by using a preset plane constraint condition, the method further includes:

and performing primary visual interface frame selection filtering on the initial scanning point cloud data.

In an alternative embodiment, before acquiring the initial scan point cloud data, the multi-lidar calibration method further includes:

aiming at each laser radar, arranging the artificial feature plate parallel to the Z axis of the laser radar and within a set distance from the laser radar;

and acquiring the initial scanning point cloud data containing the artificial feature plate under a laser radar coordinate system, wherein in the initial scanning point cloud data, the number of point clouds on all feature patterns of the artificial feature plate is higher than a first set point cloud threshold.

In an alternative embodiment, before calibrating the feature points of each feature pattern of the artificial feature board under each laser radar coordinate system, the multi-laser radar calibration method further includes:

And the artificial feature plate is manufactured by adopting two materials with different reflectivities.

In an alternative embodiment, calibrating feature points of each feature pattern of the artificial feature board under each laser radar coordinate system according to point cloud data of the artificial feature board under each laser radar coordinate system to be calibrated and reflectivity data of the artificial feature board, including:

determining edge points of each feature graph in the point cloud data according to the reflectivity data of the artificial feature plate for each laser radar;

determining whether edge points of the feature pattern on the artificial feature board are detected according to a second set point cloud threshold and the number of the edge points;

if yes, combining the first graph feature size and the second graph feature size, and carrying out random sampling feature graph segmentation detection on the edge points to obtain feature points of each feature graph;

if the number of the detected feature points is higher than the number of the feature patterns on the artificial feature plate, arranging and combining the feature points to form a plurality of groups of feature point combinations;

and calibrating the feature points of each feature pattern of the artificial feature plate under the corresponding laser radar coordinate system according to the combination of the plurality of groups of feature points.

In an optional embodiment, calibrating feature points of each feature pattern of the artificial feature board under each laser radar coordinate system according to the multiple sets of feature point combinations includes:

comparing the differences between each group of feature point combinations and the actual feature point combinations of the feature patterns of the artificial feature plate, and determining a feature point combination with the minimum difference by combining with setting a fault tolerance threshold;

and calibrating each feature point in the feature point combination with the minimum difference as the feature point of each feature pattern of the artificial feature plate under the corresponding laser radar coordinate system.

In an alternative embodiment, the method further comprises:

and if the number of the detected feature points is smaller than the number of the feature patterns on the artificial feature board, or the feature point combination does not meet the set fault tolerance threshold, re-executing the steps of acquiring initial scanning point cloud data under each laser radar coordinate system to be calibrated and scanning initial reflectivity data of the artificial feature board until the number of the detected feature points is higher than the number of the feature patterns on the artificial feature board.

In an alternative embodiment, the feature pattern is a circle and an arc, the circle is a solid structure, the arc is a hollowed-out structure, and the feature point is a circle center.

In an alternative embodiment, the first graphical feature size is smaller than the second graphical feature size, and the circular shape of the circular arc is outside the artificial feature board.

In an optional embodiment, the calibrating the relative pose of each laser radar according to the feature point of each feature pattern of the artificial feature board under each laser radar coordinate system includes:

and determining the relative pose among the plurality of laser radars by a nearest point searching method and a singular value decomposition algorithm.

Another aspect of the present application provides a multi-lidar calibration device, including:

the first calibration module is used for calibrating feature points of each feature graph of the artificial feature plate under each laser radar coordinate system according to the point cloud data of the artificial feature plate under each laser radar coordinate system to be calibrated and the reflectivity data of the artificial feature plate;

and the second calibration module is used for calibrating the relative pose of each laser radar according to the characteristic point positions of each characteristic graph of the artificial characteristic plate under each laser radar coordinate system.

In an alternative embodiment, the artificial feature plate comprises a solid circle and a hollow arc, the center of the hollow arc is located at the outer side of the artificial feature plate, and the arc edge of the arc forms the edge of the artificial feature plate.

In still another aspect, an embodiment of the present application provides a terminal device, including: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the multi-laser radar calibration method when executing the computer program.

In yet another aspect, embodiments of the present application provide a computer storage medium having a computer program stored thereon, which when executed by a processor implements a multi-lidar calibration method as described above.

Compared with the prior art, the embodiment of the application has the beneficial effects that: according to the point cloud data of the artificial feature plate and the reflectivity data of the artificial feature plate under each laser radar coordinate system to be calibrated, the feature point positions of the feature patterns of the artificial feature plate under each laser radar coordinate system are calibrated, and then the relative pose of each laser radar is calibrated according to the feature point positions of the feature patterns of the artificial feature plate under each laser radar coordinate system, so that the method has the advantages of being high in precision, high in operation speed, low in cost, simple in device, free of complex algorithm and the like, and is suitable for calibrating static multi-laser radars, such as calibrating multiple laser radars in the vehicle production process.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments or the description of the prior art, it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a system architecture of a multi-lidar calibration method according to an embodiment of the present application;

FIG. 2 is a flow chart of a multi-lidar calibration method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a specific flow chart of a method for calibrating multiple lidar according to an embodiment of the present application;

fig. 4 is a schematic flowchart of step S1 in the multi-lidar calibration method according to an embodiment of the present application;

FIG. 5 is a schematic structural view of an artificial feature board according to an embodiment of the present application;

FIG. 6 is a schematic block diagram of a multi-lidar calibration device according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

In addition, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

In the current calibration method for the multi-laser radar, black and white grids are adopted as calibration plates, but the accurate positions of the calibration plates in a world coordinate system are required to be known, the positions of the calibration plates are required to be moved, the process of the method is complex, and the requirement on the calculation performance of equipment is high, so that the method has the defect.

In view of this, the application provides a multi-laser radar calibration method, device, terminal equipment and storage medium, firstly calibrating feature points of each feature graph of an artificial feature board under each laser radar coordinate system according to point cloud data of the artificial feature board under each laser radar coordinate system to be calibrated and reflectivity data of the artificial feature board; and then, according to the characteristic point positions of each characteristic pattern of the artificial characteristic plate under each laser radar coordinate system, the relative pose of each laser radar is calibrated, so that the method has the advantages of high precision, high operation speed, low cost, simple device, no complicated algorithm and the like, and is suitable for calibrating static multi-laser radar, such as calibrating a plurality of laser radars in the vehicle production process.

The multi-laser radar calibration method, device, terminal equipment and storage medium provided by the application are described in detail below with reference to the accompanying drawings.

Fig. 1 shows a schematic architecture of a multi-lidar calibration device according to an embodiment of the present application, where, as shown in fig. 1, the multi-lidar calibration architecture includes: the system comprises a plurality of laser radars 2, an artificial characteristic plate 1 and a multi-laser radar calibration device 3, wherein the multi-laser radar calibration device 3 can calibrate characteristic points of each characteristic graph of the artificial characteristic plate under each laser radar coordinate system according to point cloud data of the artificial characteristic plate under each laser radar coordinate system to be calibrated and reflectivity data of the artificial characteristic plate; and calibrating the relative pose of each laser radar according to the characteristic point positions of each characteristic graph of the artificial characteristic plate under each laser radar coordinate system.

In this embodiment, the laser radar may be a laser radar, that is, a laser is used as a light source for transmitting, and a photoelectric detection technology is used for ranging. Each laser radar in the application comprises a transmitting system, a receiving system, an information processing part and the like. The emission system is composed of various lasers, such as a carbon dioxide laser, a neodymium-doped yttrium aluminum garnet laser, a semiconductor laser, a solid laser with tunable wavelength, an optical beam expanding unit and the like; the receiving system adopts a combination of a telescope and various photoelectric detectors, such as a photomultiplier tube, a semiconductor photodiode, an avalanche photodiode, infrared and visible light multi-component detection devices and the like.

In the specific embodiment of the application, the laser radar adopts 2 working modes of pulse or continuous wave, and the detection method can be divided into the laser radars such as Mie scattering, rayleigh scattering, raman scattering, brillouin scattering, fluorescence, doppler and the like according to different detection principles, and the application is not limited.

Fig. 2 shows a flow chart of a multi-lidar calibration method, as shown in fig. 2, including:

s1: calibrating feature points of each feature graph of the artificial feature board under each laser radar coordinate system according to point cloud data of the artificial feature board under each laser radar coordinate system to be calibrated and reflectivity data of the artificial feature board;

s2: and calibrating the relative pose of each laser radar according to the characteristic point positions of each characteristic graph of the artificial characteristic plate under each laser radar coordinate system.

According to the multi-laser radar calibration method, firstly, the characteristic points of each characteristic graph of the artificial characteristic board under each laser radar coordinate system are calibrated according to the point cloud data of the artificial characteristic board under each laser radar coordinate system to be calibrated and the reflectivity data of the artificial characteristic board, and then the relative pose of each laser radar is calibrated according to the characteristic points of each characteristic graph of the artificial characteristic board under each laser radar coordinate system, so that the multi-laser radar calibration method has the advantages of being high in precision, fast in operation speed, low in cost, simple in device, free of complex algorithm and the like, and is suitable for calibrating static multi-laser radars, such as calibrating multiple laser radars in the vehicle production process.

In embodiments of the present application, the artificial feature board may include one or more feature patterns, such as circles, squares, etc., which are not limited in this application.

Further, in this embodiment of the present application, the feature point may be a preset arbitrary point on the feature pattern, and a center, a center of gravity, an edge point, and the like may be generally selected.

In the prior art, the artificial feature board is generally in a black and white chessboard shape, namely, the artificial feature board is composed of black square grids and hollowed square grids, and then external parameter calibration is carried out.

The point cloud data of the artificial feature board under the coordinate system of each laser radar to be calibrated, specifically, the point cloud data detected by each laser radar through laser scanning, it can be understood that, as shown in fig. 1, the relative positions of the laser radar and the artificial feature board are different, so that the obtained point cloud data are also different.

Because there is the difference in the position of every laser radar relative to the artificial feature board, therefore when the laser radar scans the artificial feature board, the artificial feature board often is not parallel relative to the scanning plane of laser radar, consequently the reflectivity of artificial feature board is different, and when the laser radar is in the position to the artificial feature board, the reflectivity is highest, and this application is not repeated.

The following describes the above steps of the present application in detail, and in an alternative embodiment, as shown in fig. 3, further includes:

s01: acquiring initial scanning point cloud data under each laser radar coordinate system to be calibrated and initial reflectivity data of a scanning artificial feature plate; the initial scan point cloud data includes scan point cloud data of an artificial feature board.

In the step, the artificial feature plate is placed in a laser radar sensing range to obtain real reflectivity data of the object surface in the laser radar sensing range, namely, initial scanning point cloud data and initial reflectivity data of the next frame of a laser radar coordinate system to be calibrated are obtained.

In this embodiment, the process of acquiring one frame of point cloud data and reflectivity data under one laser radar coordinate system to be calibrated is as follows: firstly, an artificial characteristic plate manufactured by two different reflectivities is perpendicular to the ground, is placed in a sensing range of a laser radar to be calibrated, meets a distance threshold value 1, is parallel to a Z axis of the laser radar, and can display a complete artificial characteristic plate in a point cloud display module, and a laser data acquisition module acquires one frame of point cloud data and reflectivity data of laser radar scanning to be calibrated.

S02: and carrying out plane detection on the initial scanning point cloud data and the initial reflectivity data by using a preset plane constraint condition to generate point cloud data of an artificial characteristic plate and reflectivity data of the artificial characteristic plate under each laser radar coordinate system to be calibrated.

Specifically, in this embodiment, the initial scan point cloud data is subjected to a primary frame selection filtering operation, and the artificial feature plate point cloud data and the artificial feature plate reflectivity are subjected to data extraction processing by using a random sampling plane detection algorithm and a plane constraint condition 1, so as to obtain the artificial feature plate point cloud data and the reflectivity data.

Specifically, the method and the device can provide a point cloud display interface, manually input control and manually frame selection, and perform point cloud frame selection processing of the minimum range containing the artificial feature plate on the point cloud data to obtain primarily filtered point cloud data.

In an alternative embodiment, before the initial scan point cloud data is acquired, the multi-lidar calibration method further includes:

Specifically, the artificial characteristic calibration plate can be placed in the perception range of the laser radar and is placed perpendicular to the ground, and the Z axis of the laser radar is kept parallel to the calibration plate, so that the characteristic graph of the artificial characteristic plate has the number of point clouds meeting the point cloud threshold value 1.

It can be seen that in alternative embodiments, the method can be summarized as follows: before calibrating the feature points of each feature pattern of the artificial feature board under each laser radar coordinate system, the multi-laser radar calibration method further comprises the following steps: and the artificial feature plate is manufactured by adopting two materials with different reflectivities.

According to the application, in the preferred embodiment, the feature pattern is circular, the difference between each angle and each direction can be reduced, so that the fact that the artificial feature plate and the laser radar are inclined relatively in the vertical plane direction is not required to be considered, the black-white square grid is adopted for calibration in the prior art, and due to the fact that the technical principle is different, when the black-white square grid is used in the application, whether the square grid is in an upright state relative to the laser radar or not is also required to be considered, namely, two sides of the square grid are in a horizontal direction relative to laser, the other two sides are in a vertical state relative to the laser, or the square grid and the attitude of the laser radar are kept consistent, namely, in the point cloud image of the artificial feature plate obtained by shooting any one laser radar, the point cloud of the square grid is in the same attitude.

Further, in the preferred embodiment of the present application, the feature circle 1 and the feature circle 2 may be set, where the feature circle 2 may be a virtual feature circle, for example, only a part of an arc of the feature circle 2 is reserved on the artificial feature board, the arc may be hollowed, and the feature circle 1 may be solid, where the artificial feature board includes a solid circle and a hollowed arc, a center of the hollowed arc is located at an outer side of the artificial feature board, and an arc edge of the arc forms an edge of the artificial feature board.

In the preferred embodiment of the present application, the radii of the circles and the arcs may be different, and in other embodiments, the radii of each circle may be the same or different, and in general, for simplicity of calculation, only the radii of the circles and the arcs may be different.

The process of calibrating by using the artificial feature board in the present application will be described in detail.

In an alternative embodiment, step S1, namely, calibrating feature points of each feature pattern of the artificial feature board in each laser radar coordinate system according to point cloud data of the artificial feature board and reflectivity data of the artificial feature board in each laser radar coordinate system to be calibrated, as shown in fig. 4, includes:

S11: determining edge points of each feature graph in the point cloud data according to the reflectivity data of the artificial feature plate for each laser radar;

s12: determining whether edge points of the feature pattern on the artificial feature board are detected according to a second set point cloud threshold and the number of the edge points;

s13: if yes, combining the first graph feature size and the second graph feature size, and carrying out random sampling feature graph segmentation detection on the edge points to obtain feature points of each feature graph;

s14: if the number of the detected feature points is higher than the number of the feature patterns on the artificial feature plate, arranging and combining the feature points to form a plurality of groups of feature point combinations;

s15: and calibrating the feature points of each feature pattern of the artificial feature plate under the corresponding laser radar coordinate system according to the combination of the plurality of groups of feature points.

Specifically, the embodiment defines initial scan point cloud data as first point cloud data, point cloud data of the artificial feature board under each laser radar coordinate system after plane constraint processing as second point cloud data, initial reflectivity data as first reflectivity data, and reflectivity data of the artificial feature board under each laser radar coordinate system after plane constraint processing as second reflectivity data.

Further, a first set point cloud threshold is defined as a point cloud threshold 1, a second set point cloud threshold is defined as a point cloud threshold 2, and the first feature size is smaller than the second feature size.

Specifically, in step S11, firstly, an artificial feature board made of two different reflectivities is perpendicular to the ground, placed in a sensing range of a laser radar to be calibrated, and satisfies a distance threshold 1, and is parallel to a Z axis of the laser radar, so that a complete artificial feature board can be displayed in a point cloud display module, and one frame of point cloud data and reflectivity data scanned by the laser radar to be calibrated are obtained by a laser data obtaining module; and classifying the point cloud data and the reflectivity data on the artificial feature plate according to the scanning wire bundles of the laser radars to obtain all the point cloud data and the reflectivity data of the wire bundles of each laser radar.

And then distinguishing the characteristic dot cloud from the characteristic plate dot cloud by using the second reflectivity data of the artificial characteristic plate and the reflectivity threshold value 1 to obtain the edge dot cloud of the characteristic circle.

After the edge point cloud of the feature circle is obtained, calculating the reflectivity difference value between the point clouds of the wire harness of each laser radar, and screening the edge point of the feature circle through the reflectivity threshold value 1.

In step S12, it is determined whether the edge points of the feature circle satisfy the point cloud threshold 2, and if not, the calibration is performed again, that is, the operation is restarted from S01.

In step S13, as shown in fig. 5, assuming that the number of circles 1 on the artificial feature board is 3, the number of arcs is 4, and performing random sampling feature pattern segmentation detection on the edge points by combining the first pattern feature size and the second pattern feature size to obtain feature points of each feature pattern, specifically, performing random sampling circle segmentation detection on the edge points by using a circle radius threshold 1 to obtain a group of to-be-determined circle centers with the number of circle centers being greater than or equal to 3; if the number of the circle centers to be determined is smaller than 3, performing random sampling circular segmentation detection on the edge points by using a circle radius threshold 2 to obtain a circle center group to be determined, wherein the number of the circle centers is greater than or equal to 4; and if the number of the circle centers to be calibrated is smaller than 4, acquiring one frame of point cloud data and reflectivity data of laser radar scanning to be calibrated again.

Namely, by means of two different shapes, namely a circle and an arc, and configuring the circle center of the arc outside the artificial feature plate, the circle center can be detected through two circle radius thresholds, and when the number of the circle centers of the two circles is detected to be smaller than the real number, calibration is reconfigured.

Then in step S14, if the number of the detected feature points is higher than the number of the feature patterns on the artificial feature board, the feature points are arranged and combined to form a plurality of groups of feature point combinations, specifically, the to-be-determined circle center group is arranged and combined to obtain a plurality of groups of circle center groups with the number of 3 or 4.

Then in step S15, calibrating feature points of each feature pattern of the artificial feature board under the corresponding lidar coordinate system according to the combination of the feature points, wherein the fault tolerance threshold 1 is used for geometric judgment, and the circle center group which does not meet the condition is excluded until a circle center group with optimal geometric distribution is obtained; if the circle center group with the optimal geometric distribution cannot be obtained, a frame of point cloud data and reflectivity data of laser radar scanning to be calibrated are obtained again, and the calculation is carried out again until the circle center group with the optimal geometric distribution is obtained.

In this embodiment, after obtaining the optimal circle center group, a laser radar to be calibrated is manually selected as a main laser radar, and the circle center group under each laser radar coordinate system except the main laser radar and the circle center group of the main laser radar are registered by using ICP to perform pose transformation matrix estimation by using a pose transformation matrix estimation module, so as to complete external parameter calibration.

It can be understood that, when the computer implementation is performed, if the number of the detected feature points is smaller than the number of the feature patterns on the artificial feature board, or the feature point combination does not meet the set fault tolerance threshold, the steps of acquiring the initial scan point cloud data under each laser radar coordinate system to be calibrated and scanning the initial reflectivity data of the artificial feature board are executed again until the number of the detected feature points is higher than the number of the feature patterns on the artificial feature board, which is not repeated in the application.

In some embodiments, the ICP registration includes an SVD decomposition method, specifically,

in the embodiment of the application, ICP (iterative closest point) is utilized, so that the accuracy is high, and the feature points do not need to be extracted; and because the optimal combination is selected in the characteristic circle center combination, the rough registration is finished, and the local optimization cannot be caused.

The ICP algorithm is utilized to repeatedly select a corresponding relation point pair and calculate optimal rigid transformation until the convergence accuracy requirement of correct registration is met, specifically, by finding rotation and translation parameters, the point clouds under two different coordinate systems are used, one point cloud coordinate system (a main laser radar coordinate system) is used as a global coordinate system, the other point cloud (other laser radar coordinate systems) is subjected to rotation and translation, the overlapping parts of the two groups of point clouds are completely overlapped, the input of the algorithm is the coordinate of one circle center in the circle center group under the main laser radar coordinate system and the target point cloud, namely the optimal circle center group under the main laser radar coordinate system and the optimal circle center group under other laser radar coordinate systems, and the output of the algorithm is a rotation and translation transformation matrix, namely the pose transformation matrix.

The specific process for calculating the pose transformation matrix is as follows:

assuming that the point cloud { Q } is a target point cloud (coordinates of one of the circle centers in the circle center group under the main lidar coordinate system), { P } is a source point cloud (coordinates of the circle center in the circle center group under the other lidar coordinate systems to be registered), pi (i E1, 2..n), qi is the point closest to pi in { E }.

An RT transformation matrix from { P } to { Q } is then calculated, i.e. a rotation matrix R and a translation matrix T. If the transformation parameters are accurate, each point pi in the point cloud { P } should be completely coincident with a point qi in the point cloud { Q } after transformation, namely: qi=rpi+t, but due to the presence of noise, it is impossible for all points to coincide completely, defining an objective function, and minimizing the objective function, R, T is the transformation parameter sought.

Firstly, searching the nearest point qi (which can be realized by using a kd-tree nearest neighbor searching algorithm) in { P } for each point Pi in { P }, forming point pairs corresponding to each other one by one, calculating the mass centers of two groups of point clouds, respectively recording as Up and Uq, removing the mass centers of the two groups of point clouds, constructing a matrix H, carrying out SVD decomposition on the H matrix to obtain R and T, obtaining R, carrying out space transformation on a space to be registered by using the T matrix to obtain a new point set after obtaining the R, substituting the T matrix into an objective function, stopping iterative calculation if the average distance between the new transformation point set and the reference point set is smaller than a given threshold value, otherwise, continuing iteration by using the new transformation point set as a new { Pi } until the requirement of the objective function is met.

And then calculating the relative pose of each two laser radars by using a pose transformation matrix estimation module according to the characteristic circle center group, thereby completing calibration, namely, the relative pose of any two laser radars can be determined by the coordinates of the corresponding points because the coordinates of each point on the slave laser radars in the corresponding main laser radars can be known by the RT matrix.

Specific scenarios of the present application are described in detail below.

Step 1, placing an artificial feature board in a laser radar sensing range, and acquiring next frame point cloud data and reflectivity data of a laser radar coordinate system to be calibrated by utilizing a laser data acquisition module;

step 2, preprocessing the point cloud data and the reflectivity data by using an operable point cloud display module and an artificial feature board point cloud data and reflectivity data extraction module to obtain the point cloud data and the reflectivity data on the artificial feature board;

step 3, utilizing a characteristic circle edge point detection module to perform edge point detection processing on the point cloud data and the reflectivity data on the artificial characteristic plate, and determining edge point clouds of the characteristic circle

Step 4, detecting the circle centers of the edge points by using a characteristic circle center group detection module to obtain a characteristic circle center group;

Step 5, repeating the steps 1-4 to obtain a characteristic circle center group under each laser radar coordinate system to be calibrated, and respectively storing the characteristic circle center groups;

and 6, calculating the relative pose of each two laser radars by using a pose transformation matrix estimation module according to the characteristic circle center group, thereby completing calibration.

The step 1 is a process of acquiring one frame of point cloud data and reflectivity data under a laser radar coordinate system to be calibrated, wherein the process is as follows: firstly, arranging an artificial characteristic plate manufactured by two different reflectivities on a sensing range of a laser radar to be calibrated in a manner of being perpendicular to the ground, meeting a distance threshold value 1, and being parallel to a Z axis of the laser radar, so that the complete artificial characteristic plate can be displayed in a point cloud display module, and obtaining one frame of point cloud data and reflectivity data of laser radar scanning to be calibrated by a laser data acquisition module;

and 2, preprocessing the point cloud data and the reflectivity data by using a point cloud display module and an artificial characteristic board point cloud data and reflectivity data extraction module as follows:

step 21, performing manual frame selection on the point cloud data by an operable point cloud display module through mouse and keyboard control, and performing point cloud frame selection processing of a minimum range containing an artificial feature plate on the point cloud data to obtain primarily filtered point cloud data;

Step 22, for the point cloud data of the primary filtering, performing data extraction processing on the point cloud data of the artificial feature plate and the reflectivity of the artificial feature plate by using a random sampling plane detection algorithm and a plane constraint condition 1 in a data extraction module to obtain the point cloud data and the reflectivity data of the artificial feature plate;

and 3, using a characteristic circle edge point detection module to perform edge point detection processing on the point cloud data and the reflectivity data on the artificial characteristic plate, wherein the specific operation is as follows:

step 31, classifying the point cloud data and the reflectivity data on the artificial feature board according to the scanning wire bundles of the laser radars to obtain all the point cloud data and the reflectivity data of the wire bundles of each laser radar;

step 32, calculating a reflectivity difference value between point clouds of the wire harness of each laser radar;

step 33, screening and obtaining edge points of the feature circle through a reflectivity threshold value 1;

step 34, judging whether the edge points of the feature circle meet a point cloud threshold 2, and if not, entering step 1;

the specific process of detecting the circle center of the edge point by using the characteristic circle center group detection module in the step 4 is as follows:

step 41, performing random sampling circular segmentation detection on the edge points by using a circular radius threshold value 1, and entering step 42 when a to-be-determined circle center group with the circle center number being more than or equal to 3 is obtained; if the number of the circle centers to be determined is less than 3, performing random sampling circular segmentation detection on the edge points by using a circle radius threshold 2, and if the number of the circle centers to be determined is greater than or equal to 4, entering a step 42; if the number of the circle centers to be calibrated is smaller than 4, entering the step 1 to acquire one frame of point cloud data and reflectivity data of laser radar scanning to be calibrated again;

Step 42, for the to-be-determined circle center group, obtaining a plurality of groups of circle center groups with the number of 3 or 4 by using permutation and combination;

step 43, performing geometric judgment on the multiple groups of circle center groups by using the fault tolerance threshold 1, and removing the circle center groups which do not meet the condition until a group of circle centers with optimal geometric distribution is obtained; if the circle center group with the optimal geometric distribution cannot be obtained, entering the step 1 to re-obtain one frame of point cloud data and reflectivity data of laser radar scanning to be calibrated, and re-calculating until the circle center group with the optimal geometric distribution is obtained;

step 5, manually selecting one laser radar to be calibrated as a main laser radar, and performing pose transformation matrix estimation on a circle center group under each laser radar coordinate system except the main laser radar and the circle center group of the main laser radar by utilizing ICP registration by utilizing a pose transformation matrix estimation module so as to finish external parameter calibration; the ICP registration includes an SVD decomposition method.

It can be seen that the beneficial effects of the invention are: the device is simple, and a complex algorithm is avoided; searching for the edge points of the characteristic circle by utilizing the reflectivity, and solving the problem that the edge points cannot be found correctly due to the tailing points; various types of multi-laser radars can be calibrated; and calibrating the multi-laser radar rapidly and accurately.

Based on the same inventive concept, a multiple lidar calibration device 10, as shown in fig. 6, comprises: the first calibration module 1 is used for calibrating feature points of each feature graph of the artificial feature plate under each laser radar coordinate system according to point cloud data of the artificial feature plate under each laser radar coordinate system to be calibrated and reflectivity data of the artificial feature plate; and the second calibration module 2 is used for calibrating the relative pose of each laser radar according to the characteristic point positions of each characteristic graph of the artificial characteristic plate under each laser radar coordinate system.

According to the laser radar calibration device, through the arrangement of the first calibration module and the second calibration module, firstly, according to the point cloud data of the artificial feature plate and the reflectivity data of the artificial feature plate under each laser radar coordinate system to be calibrated, the feature point positions of each feature pattern of the artificial feature plate under each laser radar coordinate system are calibrated, and then, according to the feature point positions of each feature pattern of each artificial feature plate under each laser radar coordinate system, the relative pose of each laser radar is calibrated, so that the device has the advantages of being high in precision, high in operation speed, low in cost, simple in device, free of complex algorithm and the like, and is suitable for calibrating static multiple laser radars, such as calibrating multiple laser radars in the vehicle production process.

The multi-laser radar calibration method and device implemented by other execution bodies in the above embodiments are the same as or similar to the effects in the foregoing embodiments of the present application, and are not described herein in detail.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not limit the implementation process of the embodiment of the present application in any way.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

In order to implement the above embodiment, the present application further proposes a terminal device. Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 7, the terminal apparatus 600 includes:

a memory 610 and at least one processor 620, a bus 630 connecting the different components (including the memory 610 and the processor 620), the memory 610 storing a computer program which when executed by the processor 620 implements the multi-lidar calibration method described in the embodiments of the present application.

Bus 630 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Terminal device 600 typically includes a variety of electronic device readable media. Such media can be any available media that is accessible by terminal device 600 and includes both volatile and nonvolatile media, removable and non-removable media.

Memory 610 may also include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 640 and/or cache memory 650. The terminal device 600 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 660 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 7, commonly referred to as a "hard disk drive"). Although not shown in fig. 7, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 630 through one or more data medium interfaces. Memory 610 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of the embodiments of the present application.

A program/utility 680 having a set (at least one) of program modules 670 may be stored in, for example, memory 610, such program modules 670 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 670 generally perform the functions and/or methods in the embodiments described herein.

The terminal device 600 can also communicate with one or more external devices 690 (e.g., keyboard, pointing device, display 691, etc.), one or more devices that enable a user to interact with the terminal device 600, and/or any device (e.g., network card, modem, etc.) that enables the terminal device 600 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 692. Also, terminal device 600 can communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through network adapter 693. As shown, network adapter 693 communicates with other modules of terminal device 600 over bus 630. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with terminal device 600, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.

The processor 620 executes various functional applications and data processing by running programs stored in the memory 610.

It should be noted that, the implementation process and the technical principle of the terminal device in this embodiment refer to the foregoing explanation of the multi-lidar calibration method in this embodiment, and are not repeated herein.

Embodiments of the present application also provide a computer storage medium storing a computer program which, when executed by a processor, implements steps that may be implemented in the various method embodiments described above.

The present embodiments provide a computer program product which, when run on a terminal device, causes the terminal device to perform steps that enable the respective method embodiments described above to be implemented.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer storage medium, where the computer program may implement the steps of the method embodiments described above when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the apparatus/terminal device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims

1. The multi-laser radar calibration method is characterized by comprising the following steps of:

2. The method as recited in claim 1, further comprising:

3. The method of claim 2, wherein the step of determining the position of the substrate comprises,

before the initial scanning point cloud data is acquired, the multi-laser radar calibration method further comprises the following steps:

4. The method of claim 1, wherein calibrating feature points of feature patterns of the artificial feature board in each lidar coordinate system according to the point cloud data of the artificial feature board and the reflectivity data of the artificial feature board in each lidar coordinate system to be calibrated, comprises:

5. The method of claim 4, wherein calibrating feature points of feature patterns of the artificial feature board under the each lidar coordinate system according to the plurality of sets of feature point combinations comprises:

6. The method of claim 5, wherein determining edge points of each feature pattern in the point cloud data based on the reflectivity data of the artificial feature plate comprises:

classifying the point cloud data and the reflectivity data on the artificial feature plate according to the scanning wire bundles of the laser radars to obtain all the point cloud data and the reflectivity data of the wire bundles of each laser radar;

calculating the reflectivity difference between the point clouds of the wire harness of each laser radar;

and screening according to a preset reflectivity threshold to obtain edge points of the feature pattern.

7. The method of claim 5, wherein the feature pattern is a circle and an arc, the circle is a solid structure, the arc is a hollowed-out structure, and the feature point is a circle center.

8. The method according to claim 1, wherein calibrating the relative pose of each laser radar according to the feature points of each feature pattern of the artificial feature board under each laser radar coordinate system comprises:

9. A multi-lidar calibration device, comprising:

10. The multi-lidar calibration device of claim 9, wherein the artificial feature plate comprises a solid circle and a hollowed-out arc, the center of the hollowed-out arc is located on the outer side of the artificial feature plate, and the arc edge of the arc forms the edge of the artificial feature plate.