CN113496491A - Road surface segmentation method and device based on multi-line laser radar - Google Patents
Road surface segmentation method and device based on multi-line laser radar Download PDFInfo
- Publication number
- CN113496491A CN113496491A CN202010193980.7A CN202010193980A CN113496491A CN 113496491 A CN113496491 A CN 113496491A CN 202010193980 A CN202010193980 A CN 202010193980A CN 113496491 A CN113496491 A CN 113496491A
- Authority
- CN
- China
- Prior art keywords
- laser radar
- point cloud
- cloud data
- road surface
- radar point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 82
- 238000000034 method Methods 0.000 title claims abstract description 34
- 239000011159 matrix material Substances 0.000 claims description 14
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000001186 cumulative effect Effects 0.000 claims description 4
- 238000013519 translation Methods 0.000 claims description 4
- 230000004927 fusion Effects 0.000 abstract description 4
- 238000012545 processing Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 102100026278 Cysteine sulfinic acid decarboxylase Human genes 0.000 description 3
- 230000004888 barrier function Effects 0.000 description 3
- 108010064775 protein C activator peptide Proteins 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 239000002904 solvent Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/06—Topological mapping of higher dimensional structures onto lower dimensional surfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20128—Atlas-based segmentation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Traffic Control Systems (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The invention discloses a road surface segmentation method and a road surface segmentation device based on a multi-line laser radar, wherein the road surface segmentation method comprises the following steps: step S1, scanning the surrounding environment of the vehicle by using a multi-line laser radar, acquiring laser radar point cloud data under a laser radar coordinate system, and converting the laser radar point cloud data into laser radar point cloud data under the vehicle coordinate system; step S2, projecting the laser radar point cloud data under the vehicle coordinate system to a two-dimensional grid map; step S3, according to the two-dimensional grid map, road surface segmentation is carried out based on the local accumulated height difference and the regional laser beam distribution threshold value respectively; and step S4, overlapping the results of road surface segmentation based on the local accumulated height difference and the regional laser beam distribution threshold to obtain a complete road surface segmentation result. The method adopts a method of local feature fusion, effectively solves the problem of road surface segmentation of the automatic driving automobile under the working condition of complex environment, has simple steps, is easy to realize, and has strong robustness and real-time performance.
Description
Technical Field
The invention belongs to the technical field of automatic driving of automobiles, and particularly relates to a road surface segmentation method and device based on a multi-line laser radar.
Background
With the continuous development of automobile technology, the intellectualization of vehicles has become a trend. The automatic driving technology is an important expression of automobile intellectualization, and the environment perception technology is an important link in the automatic driving technology. By collecting and processing information of various sensors, lane lines, vehicles, pedestrians, obstacles, passable areas and the like are identified, a local digital environment map is constructed, and a basis is provided for decision-making of automatic driving behaviors.
The multi-line laser radar is used for dividing the road surface where the intelligent automobile runs, so that the accessible area of the road surface can be extracted, and the safe running of the automatic driving automobile is guaranteed. For dividing a road surface by applying the multi-line laser radar, the difference characteristics shown by scanning the laser radar to an obstacle and the ground are generally adopted for extraction. Common features include maximum and minimum height difference features in a grid, tangential neighborhood radius ratio features, and the like. The maximum and minimum height difference in the grids can be obtained by calculating the height difference between the highest point and the lowest point in the barrier grids, if the difference value exceeds a set threshold value, the position is considered as a barrier point, all the barrier points in the grids are extracted by the method, and the road surface segmentation is realized. The tangential neighborhood radius ratio is obtained by comparing the ratio of a certain laser point cloud and the distance between the tangential neighborhood point and the origin to judge whether the point is an obstacle point, if the ratio is smaller than a certain threshold, the point is an obstacle point, and the road surface segmentation is finally realized by traversing all the laser point clouds.
The method for dividing the road surface by utilizing the maximum and minimum height difference in the grids only has a good effect on the nearby obstacles, but fails on the short obstacles at the far distance. The tangential neighborhood radius ratio based approach still fails for short and long obstacles at distance and can cause false detections when the vehicle jolts and vibrates. In addition, the driving environment of an autonomous automobile has a complicated variety, mainly including a driving road, a weather environment, a vehicle state, and the like. The driving road mainly comprises a structured urban road, an unstructured rural road and a field road, wherein high-order noise is introduced due to the fact that the unstructured road is not smooth; weather environments mainly include environments such as sunny days and rainy days, and compared with the sunny days, the rainy days change the laser radar wiring harness from diffuse reflection to mirror reflection, so that data loss is caused; self-propelled vehicles sometimes jolt during travel, causing local coordinate system oscillations. The technical method for road surface segmentation by adopting the multi-line laser radar has great influence on the factors.
Disclosure of Invention
The technical problem to be solved by the embodiments of the present invention is to provide a road surface segmentation method and device based on a multi-line laser radar, so as to improve the robustness and real-time performance of road surface segmentation under complex environment working conditions.
In order to solve the technical problem, the invention provides a road surface segmentation method based on a multi-line laser radar, which comprises the following steps:
step S1, scanning the surrounding environment of the vehicle by using a multi-line laser radar, acquiring laser radar point cloud data under a laser radar coordinate system, and converting the laser radar point cloud data into laser radar point cloud data under the vehicle coordinate system;
step S2, projecting the laser radar point cloud data under the vehicle coordinate system to a two-dimensional grid map;
step S3, according to the two-dimensional grid map, road surface segmentation is carried out based on the local accumulated height difference and the regional laser beam distribution threshold value respectively;
and step S4, overlapping the results of road surface segmentation based on the local accumulated height difference and the regional laser beam distribution threshold to obtain a complete road surface segmentation result.
Further, the step S1 converts the lidar point cloud data from the lidar coordinate system to the vehicle coordinate system as follows:
[Vx,Vy,Vz]＝C[Hx,Hy,Hz]
hx, Hy and Hz are respectively an abscissa, an ordinate and a height coordinate of laser radar point cloud under a laser radar coordinate system, Vx, Vy and Vz are respectively an abscissa, an ordinate and a height coordinate of the laser radar point cloud under a vehicle coordinate system, and C is a matrix calibrated between the vehicle coordinate system and the laser radar coordinate system in advance.
Further, the air conditioner is provided with a fan,
r is a 3 × 3 rotation matrix and T is a 1 × 3 translation matrix.
Further, the step S2 projects the three-dimensional lidar point cloud data based on the vehicle coordinate system into the two-dimensional grid map by:
I[i,j]＝MV[x,y,z]
i [ I, j ] represents the coordinates of the laser radar point cloud in the two-dimensional grid map, I is a vertical coordinate and represents the row of the two-dimensional grid map, and j is a horizontal coordinate and represents the column of the two-dimensional grid map; and V [ x, y, z ] is a three-dimensional coordinate of the laser radar point cloud under a vehicle coordinate system, x is an abscissa, y is an ordinate, z is a height coordinate, and M is a conversion matrix for projecting the laser radar point cloud data under the vehicle coordinate system to a two-dimensional grid map.
Further, in step S3, the road surface segmentation based on the local cumulative height difference specifically includes:
counting and recording the minimum height value of the laser radar point cloud data in each grid after a frame of laser radar point cloud data is projected to a two-dimensional grid map;
calculating the difference value between the height of certain laser radar point cloud data and the minimum height value of the laser radar point cloud data of the grid where the certain laser radar point cloud data is located;
if the difference value is larger than the height difference threshold value, respectively calculating the difference value between the height of the laser radar point cloud data and the minimum height value of the laser radar point cloud data of each grid in a region with a preset size by taking a two-dimensional grid map pixel corresponding to the laser radar point cloud data as a center;
if the difference value is larger than the height difference threshold value, counting once, and when the counting value exceeds a preset accumulated threshold value, marking the laser radar point cloud data as a non-road surface point;
traversing the current frame laser radar point cloud data, extracting all non-road surface points of the frame, and marking in the two-dimensional grid map.
Further, the one-frame lidar point cloud data used in step S3 is lidar point cloud data obtained by scanning the vehicle surroundings with the front P laser beams according to the angular distribution of the laser beams of the multi-line lidar mounted on the roof in the vertical direction, and processed in steps S1 and S2.
Further, the height difference threshold is set to 10cm to 20cm, the area of a predetermined size is m × n, m and n are set to 10 to 20, and the size of the count threshold is 0.25 × m × n.
Further, in step S3, the dividing the road surface based on the regional laser beam distribution threshold specifically includes:
counting the number of wire harnesses of the laser radar point cloud data at each grid position of the two-dimensional grid map after the one-frame laser radar point cloud data is projected to the two-dimensional grid map;
and traversing the two-dimensional grid map in a subarea manner, and marking the grid position as a non-road point when the quantity of the laser beams at a certain grid position exceeds the distribution threshold of the area.
Further, the division mode of the region is as follows: under a vehicle coordinate system, in the same abscissa range, the ordinate symmetrically divides a plurality of areas from the origin from near to far.
The invention also provides a road surface segmentation device based on the multi-line laser radar, which comprises:
the coordinate conversion unit is used for converting laser radar point cloud data under a laser radar coordinate system, which is obtained by scanning the surrounding environment of the vehicle by the multi-line laser radar, into laser radar point cloud data under the vehicle coordinate system;
the projection unit is used for projecting the laser radar point cloud data under the vehicle coordinate system into a two-dimensional grid map;
the road surface segmentation unit is used for segmenting the road surface based on the local accumulated height difference and the regional laser beam distribution threshold value respectively according to the two-dimensional grid map;
and the superposition unit is used for superposing the results of road surface segmentation based on the local accumulated height difference and the regional laser beam distribution threshold to obtain complete road surface segmentation results.
Further, the road surface segmentation unit specifically includes:
the statistical module is used for statistically recording the minimum height value of the laser point clouds in each grid after a frame of laser radar point cloud data is projected to the two-dimensional grid map;
the first calculation module is used for calculating the difference value between the height of certain laser radar point cloud data and the minimum height value of the laser radar point cloud data of the grid where the certain laser radar point cloud data is located;
the second calculation module is used for calculating the difference value between the height of the laser radar point cloud data and the minimum height value of the laser radar point cloud data of each grid in a preset size area by taking a two-dimensional grid map pixel corresponding to the laser radar point cloud data as a center when the difference value calculated by the first calculation module is larger than a height difference threshold value;
the counting module is used for counting once when the difference value calculated by the second calculating module is greater than the height difference threshold value;
and the marking module is used for marking the laser radar point cloud data as a non-road surface point when the counting value of the counting module exceeds a preset accumulative threshold value.
Further, the road surface segmentation unit specifically includes:
the counting module is used for counting the number of the wire harnesses of the laser radar point cloud data at each grid position of the two-dimensional grid map after the one-frame laser radar point cloud data is projected to the two-dimensional grid map;
and the marking module is used for traversing the two-dimensional grid map in different areas, and marking the grid position as a non-road point when the number of the laser beams at the certain grid position exceeds the distribution threshold of the area where the laser beams are located.
The embodiment of the invention has the following beneficial effects: the method adopts the method of local feature fusion, can adaptively segment the road surface to the maximum extent for different driving environment working conditions, effectively solves the problem of road surface segmentation of the automatic driving automobile under the complex environment working conditions of unstructured roads, rainy days, vehicle bump vibration and the like, has simple steps, is easy to realize, and has strong robustness and real-time performance.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a road surface segmentation method based on a multiline laser radar according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a two-dimensional grid map according to an embodiment of the present invention.
Fig. 3 is a schematic flow chart of the road surface segmentation based on the local cumulative height difference in the embodiment of the present invention.
Fig. 4 is a schematic view of a specific process of road surface segmentation based on a distribution threshold of laser beams in a partitioned area in an embodiment of the present invention.
Detailed Description
The following description of the embodiments refers to the accompanying drawings, which are included to illustrate specific embodiments in which the invention may be practiced.
Aiming at the difficulty of dividing the multi-line laser radar road surface: the invention discloses a road surface segmentation method and a road surface segmentation device based on a multi-line laser radar, which are innovatively developed by adopting the multi-line laser radar, and stably realize road surface segmentation under various environmental working conditions based on multi-feature fusion.
Referring to fig. 1, an embodiment of the present invention provides a road surface segmentation method based on a multiline laser radar, including:
step S1, scanning the surrounding environment of the vehicle by using a multi-line laser radar, acquiring laser radar point cloud data under a laser radar coordinate system, and converting the laser radar point cloud data into laser radar point cloud data under the vehicle coordinate system;
step S2, projecting the laser radar point cloud data based on the vehicle coordinate system to a two-dimensional grid map;
step S3, according to the two-dimensional grid map, road surface segmentation is carried out based on the local accumulated height difference and the regional laser beam distribution threshold value respectively;
and step S4, overlapping the results of road surface segmentation based on the local accumulated height difference and the regional laser beam distribution threshold to obtain a complete road surface segmentation result.
Specifically, step S1 first performs lidar point cloud data acquisition and resolution. The laser radar is installed on the top of the vehicle. The laser radar used in this embodiment is a multi-line laser radar, such as a laser radar with more than 16 lines (16 lines, 32 lines, 64 lines, 128 lines, etc.), and preferably a 64-line laser radar (HDL-64E S3) of Velodyne, which uses UDP communication to transmit data in a PCAP packet format and obtains laser radar point cloud data by parsing the PCAP packet.
After the PCAP package is analyzed to obtain the laser radar point cloud data, the laser radar point cloud data needs to be preprocessed, including internal parameter correction of the laser radar and construction of an undirected graph structure. And then, converting the laser radar point cloud data from the laser radar coordinate system to the vehicle coordinate system according to the following mode:
[Vx,Vy,Vz]＝CHx ,Hy,Hz]
hx, Hy and Hz are respectively an abscissa, an ordinate and a height coordinate of a certain laser radar point cloud under a laser radar coordinate system, Vx, Vy and Vz are respectively an abscissa, an ordinate and a height coordinate of the laser radar point cloud under a vehicle coordinate system, and C is a matrix calibrated between the vehicle coordinate system and the laser radar coordinate system in advance and comprises a rotation matrix R and a translation matrix T. In particular, the amount of the solvent to be used,
r is a 3 × 3 rotation matrix and T is a 1 × 3 translation matrix. The origin of the vehicle coordinate system is set as the center point of the head of the vehicle.
Step S2 projects the three-dimensional lidar point cloud data based on the vehicle coordinate system into the two-dimensional grid map by:
I[i,j]＝MV[x,y,z]
i [ I, j ] represents the coordinates of the laser radar point cloud data in a two-dimensional grid map, I is a vertical coordinate and represents the row of the grid map, and j is a horizontal coordinate and represents the column of the grid map; and V [ x, y and z ] is a three-dimensional coordinate of the laser radar point cloud under a vehicle coordinate system, x is an abscissa, y is an ordinate, z is a height coordinate, and M is a conversion matrix of the laser radar point cloud data projected to the two-dimensional grid map under the vehicle coordinate system.
The two-dimensional grid map itself of this embodiment is an image, and includes the coordinates of the pixels and the corresponding pixel values at the coordinates, I [ I, j ] obtained by the projection transformation in step S2 is the coordinates of the pixels of the two-dimensional grid map, and the pixel values corresponding to the coordinates are the heights Zi. Specifically, the two-dimensional grid map of the present embodiment is shown in fig. 2, and has a size of 512 × 512, and the origin is the upper left corner (0, 0). The coordinates of the two-dimensional grid map are defined according to the range of road surface segmentation by the laser radar, and a vehicle head central point which is taken as an origin in a vehicle coordinate system is arranged at a pixel point (256,411) of the two-dimensional grid map. Setting the actual geometric distance represented by each pixel in the transverse direction of the two-dimensional grid map to be 20 cm; the actual geometric distance represented by each pixel in front of the vehicle is 20cm in the longitudinal direction, and the actual geometric distance represented by each pixel behind the vehicle is 50cm, so that the two-dimensional grid map of the present embodiment shows a range of about 80m in front of the vehicle, about 50m behind the vehicle, and about 100m in the lateral direction (50 m each in the left and right). It will be appreciated that the actual geometric distance represented by each pixel in the two-dimensional grid map may be set according to the range of road surface segmentation by the lidar, and the foregoing setting is merely an example.
Step S3 will perform two types of road surface segmentation, respectively: the road surface segmentation based on the local cumulative height difference and the road surface segmentation based on the regional laser beam distribution threshold are described below.
A. Road surface segmentation based on local accumulated height difference
The road surface segmentation based on the local accumulated height difference is to select the front P laser beams (vehicle close-range positions) for processing according to the angle distribution of each laser beam of the 64-line laser radar installed on the roof in the vertical direction, and the processing flow is shown in FIG. 3.
Projecting a frame of laser radar point cloud data to a two-dimensional grid map, counting and recording the height minimum value Zmin [ i, j ] (i, j represents the position of a certain grid in the two-dimensional grid map, namely the row and column of the grid map in which the certain grid is located) of the laser radar point cloud data in each grid, traversing the current frame of laser radar point cloud data, and extracting non-road surface points:
for certain laser radar point cloud data, calculating a difference value (Zi-Zmin [ i, j ]) of the height Zi of the laser radar point cloud data and the height minimum value Zmin [ i, j ] of the laser radar point cloud data of a grid where the laser radar point cloud data is located, then judging whether the difference value is larger than a preset height difference Threshold value Threshold1, if Zi-Zmin [ i, j ] is larger than Threshold1, taking a two-dimensional grid map pixel of a corresponding position as a center, respectively calculating the height difference of the height Zi of the laser radar point cloud data and the height minimum value Zmin [ i, j ] of the laser radar point cloud data of each grid in an m multiplied by N area, if Zi-Zmin [ i, j ] is larger than Threshold1, counting once, and if the counting value exceeds a preset accumulative Threshold value N, marking the point cloud as a non-road surface point; and traversing the point cloud data of the laser radar of the current frame, extracting all non-road surface points of the frame, and marking in the two-dimensional grid map.
As an example, P is 50, the height Threshold1 may be set to 10cm to 20cm, the size of the m × N region may be set, m and N may be set to 10 to 20, and the count Threshold N is 0.25 × m × N.
The traditional road surface segmentation method based on the height difference in the grids is greatly influenced by a single noise point, and error segmentation is easily caused; according to the road surface segmentation method based on the local accumulated height difference, the interference of noise is greatly reduced by comparing the laser point clouds in the grids of the surrounding area for many times, the robustness is high, and the long-distance (sparse laser radar scanning line beams) road surface segmentation is improved to a certain extent.
B. Road surface segmentation based on regional laser beam distribution threshold
In different areas of the grid map, the density degree of the laser beam distribution can reflect the distribution of the obstacle points, so that the road surface segmentation processing can be performed, and the processing flow is shown in fig. 4.
And after one frame of laser radar point cloud data is projected to the two-dimensional grid map, counting the number of the wire harnesses of the laser radar point cloud data at each grid position of the two-dimensional grid map. Then, the two-dimensional grid map is traversed in a subarea mode, and when the number of laser beams at a certain grid position exceeds a distribution Threshold value Threshold2, the grid position is marked as a non-road surface point.
The distribution Threshold2 corresponding to different region positions of the grid is shown in the following table 1:
TABLE 1 distribution Threshold Threshold2 corresponding to different region positions of the grid
| Abscissa (m) | Ordinate (m) | Threshold of distribution 2 |
| [-50,50] | [-3,3] | 10 |
| [-50,50] | [-5,-3]&&[3,5] | 8 |
| [-50,50] | [-10,-5]&&[5,10] | 6 |
| [-50,50] | [-15,-10]&&[10,15] | 5 |
| [-50,50] | [-20,-15]&&[15,20] | 4 |
| [-50,50] | [-30,-20]&&[20,30] | 3 |
| [-50,50] | [-50,-30]&&[30,80] | 2 |
The coordinate system shown in table 1 is a vehicle coordinate system, the origin of coordinates is the center point of the head of the vehicle, the abscissa is a negative value and represents the left side of the vehicle, and the positive value represents the right side of the vehicle; negative values on the ordinate represent the rear of the vehicle and positive values represent the front. The division mode of the region is as follows: under a vehicle coordinate system, in the same abscissa range, the ordinate symmetrically divides a plurality of areas from the origin from near to far. It can also be seen that the closer the distance from the front to the rear of the vehicle, the larger the distribution threshold, and the more the number of laser beams (exceeding the distribution threshold) is required to be determined as a non-road point.
Step S4 superimposes the two road surface segmentation results of step S3 as a complete road surface segmentation result based on a 64-line laser radar. And the superposition is to integrate non-road points marked by the two road surface segmentation methods in the two-dimensional grid map and display the non-road points on the two-dimensional grid map. It will be appreciated that the non-road points marked by the two methods may coincide and the final superposition shows a union of the two road segmentation results without duplicate results.
Corresponding to the road surface segmentation method based on the multi-line laser radar in the first embodiment of the invention, the second embodiment of the invention provides a road surface segmentation device based on the multi-line laser radar, which comprises the following steps:
the coordinate conversion unit is used for converting laser radar point cloud data under a laser radar coordinate system, which is obtained by scanning the surrounding environment of the vehicle by the multi-line laser radar, into laser radar point cloud data under the vehicle coordinate system;
the projection unit is used for projecting the laser radar point cloud data under the vehicle coordinate system into a two-dimensional grid map;
the road surface segmentation unit is used for segmenting the road surface based on the local accumulated height difference and the regional laser beam distribution threshold value respectively according to the two-dimensional grid map;
and the superposition unit is used for superposing the results of road surface segmentation based on the local accumulated height difference and the regional laser beam distribution threshold to obtain complete road surface segmentation results.
Further, the road surface segmentation unit specifically includes:
the statistical module is used for statistically recording the minimum height value of the laser point clouds in each grid after a frame of laser radar point cloud data is projected to the two-dimensional grid map;
the first calculation module is used for calculating the difference value between the height of certain laser radar point cloud data and the minimum height value of the laser radar point cloud data of the grid where the certain laser radar point cloud data is located;
the second calculation module is used for calculating the difference value between the height of the laser radar point cloud data and the minimum height value of the laser radar point cloud data of each grid in a preset size area by taking a two-dimensional grid map pixel corresponding to the laser radar point cloud data as a center when the difference value calculated by the first calculation module is larger than a height difference threshold value;
the counting module is used for counting once when the difference value calculated by the second calculating module is greater than the height difference threshold value;
and the marking module is used for marking the laser radar point cloud data as a non-road surface point when the counting value of the counting module exceeds a preset accumulative threshold value.
Further, the road surface segmentation unit specifically includes:
the counting module is used for counting the number of the wire harnesses of the laser radar point cloud data at each grid position of the two-dimensional grid map after the one-frame laser radar point cloud data is projected to the two-dimensional grid map;
and the marking module is used for traversing the two-dimensional grid map in different areas, and marking the grid position as a non-road point when the number of the laser beams at the certain grid position exceeds the distribution threshold of the area where the laser beams are located.
For the working principle and process of the multi-line lidar based road surface segmentation device of the present embodiment, reference is made to the description of the first embodiment of the present invention, which is not repeated herein.
Compared with the prior art, the invention has the beneficial effects that: the invention adopts a method of local feature fusion, can adaptively segment the road surface to the maximum extent for different driving environment working conditions, effectively solves the problem of road surface segmentation of the automatic driving automobile under the complex environment working conditions of unstructured road, rainy days, vehicle bump vibration and the like, and has strong robustness and real-time performance.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Claims (12)
1. A road surface segmentation method based on a multiline laser radar is characterized by comprising the following steps:
step S1, scanning the surrounding environment of the vehicle by using a multi-line laser radar, acquiring laser radar point cloud data under a laser radar coordinate system, and converting the laser radar point cloud data into laser radar point cloud data under the vehicle coordinate system;
step S2, projecting the laser radar point cloud data under the vehicle coordinate system to a two-dimensional grid map;
step S3, according to the two-dimensional grid map, road surface segmentation is carried out based on the local accumulated height difference and the regional laser beam distribution threshold value respectively;
and step S4, overlapping the results of road surface segmentation based on the local accumulated height difference and the regional laser beam distribution threshold to obtain a complete road surface segmentation result.
2. The road surface segmentation method according to claim 1, wherein the step S1 is to convert the lidar point cloud data from the lidar coordinate system to the vehicle coordinate system in the following manner:
[Vx,Vy,Vz]＝C[Hx,Hy,Hz]
hx, Hy and Hz are respectively an abscissa, an ordinate and a height coordinate of laser radar point cloud under a laser radar coordinate system, Vx, Vy and Vz are respectively an abscissa, an ordinate and a height coordinate of the laser radar point cloud under a vehicle coordinate system, and C is a matrix calibrated between the vehicle coordinate system and the laser radar coordinate system in advance.
3. The road surface segmentation method according to claim 2,
r is a 3 × 3 rotation matrix and T is a 1 × 3 translation matrix.
4. The road surface segmentation method according to claim 1, wherein the step S2 projects three-dimensional lidar point cloud data based on a vehicle coordinate system into a two-dimensional grid map by:
I[i,j]＝MV[x,y,z]
i [ I, j ] represents the coordinates of the laser radar point cloud in the two-dimensional grid map, I is a vertical coordinate and represents the row of the two-dimensional grid map, and j is a horizontal coordinate and represents the column of the two-dimensional grid map; and V [ x, y, z ] is a three-dimensional coordinate of the laser radar point cloud under a vehicle coordinate system, x is an abscissa, y is an ordinate, z is a height coordinate, and M is a conversion matrix for projecting the laser radar point cloud data under the vehicle coordinate system to a two-dimensional grid map.
5. The road surface segmentation method according to claim 1, wherein the step S3 of performing road surface segmentation based on the local cumulative height difference specifically includes:
counting and recording the minimum height value of the laser radar point cloud data in each grid after a frame of laser radar point cloud data is projected to a two-dimensional grid map;
calculating the difference value between the height of certain laser radar point cloud data and the minimum height value of the laser radar point cloud data of the grid where the certain laser radar point cloud data is located;
if the difference value is larger than the height difference threshold value, respectively calculating the difference value between the height of the laser radar point cloud data and the minimum height value of the laser radar point cloud data of each grid in a region with a preset size by taking a two-dimensional grid map pixel corresponding to the laser radar point cloud data as a center;
if the difference value is larger than the height difference threshold value, counting once, and when the counting value exceeds a preset accumulated threshold value, marking the laser radar point cloud data as a non-road surface point;
traversing the current frame laser radar point cloud data, extracting all non-road surface points of the frame, and marking in the two-dimensional grid map.
6. The road surface segmentation method according to claim 5, wherein the one-frame lidar point cloud data used in step S3 is lidar point cloud data obtained by scanning the vehicle surroundings with P laser beams before the selection according to the angle distribution of each laser beam of the multi-line lidar mounted on the roof in the vertical direction and processed in steps S1 and S2.
7. The road surface division method according to claim 5, wherein the height difference threshold value is set to 10cm to 20cm, the predetermined size of the region is m x n, m and n are set to 10 to 20, and the count threshold value is 0.25 x m x n.
8. The method for dividing a road surface according to claim 1, wherein the step S3 of dividing a road surface based on the threshold of the laser beam distribution in the divided regions specifically includes:
counting the number of wire harnesses of the laser radar point cloud data at each grid position of the two-dimensional grid map after the one-frame laser radar point cloud data is projected to the two-dimensional grid map;
and traversing the two-dimensional grid map in a subarea manner, and marking the grid position as a non-road point when the quantity of the laser beams at a certain grid position exceeds the distribution threshold of the area.
9. The road surface segmentation method according to claim 8, wherein the regions are divided in a manner that: under a vehicle coordinate system, in the same abscissa range, the ordinate symmetrically divides a plurality of areas from the origin from near to far.
10. A road surface segmentation device based on multi-line laser radar is characterized by comprising:
the coordinate conversion unit is used for converting laser radar point cloud data under a laser radar coordinate system, which is obtained by scanning the surrounding environment of the vehicle by the multi-line laser radar, into laser radar point cloud data under the vehicle coordinate system;
the projection unit is used for projecting the laser radar point cloud data under the vehicle coordinate system into a two-dimensional grid map;
the road surface segmentation unit is used for segmenting the road surface based on the local accumulated height difference and the regional laser beam distribution threshold value respectively according to the two-dimensional grid map;
and the superposition unit is used for superposing the results of road surface segmentation based on the local accumulated height difference and the regional laser beam distribution threshold to obtain complete road surface segmentation results.
11. A road surface segmentation device according to claim 10, characterized in that the road surface segmentation unit comprises in particular:
the statistical module is used for statistically recording the minimum height value of the laser point clouds in each grid after a frame of laser radar point cloud data is projected to the two-dimensional grid map;
the first calculation module is used for calculating the difference value between the height of certain laser radar point cloud data and the minimum height value of the laser radar point cloud data of the grid where the certain laser radar point cloud data is located;
the second calculation module is used for calculating the difference value between the height of the laser radar point cloud data and the minimum height value of the laser radar point cloud data of each grid in a preset size area by taking a two-dimensional grid map pixel corresponding to the laser radar point cloud data as a center when the difference value calculated by the first calculation module is larger than a height difference threshold value;
the counting module is used for counting once when the difference value calculated by the second calculating module is greater than the height difference threshold value;
and the marking module is used for marking the laser radar point cloud data as a non-road surface point when the counting value of the counting module exceeds a preset accumulative threshold value.
12. A road surface segmentation device according to claim 10, characterized in that the road surface segmentation unit comprises in particular:
the counting module is used for counting the number of the wire harnesses of the laser radar point cloud data at each grid position of the two-dimensional grid map after the one-frame laser radar point cloud data is projected to the two-dimensional grid map;
and the marking module is used for traversing the two-dimensional grid map in different areas, and marking the grid position as a non-road point when the number of the laser beams at the certain grid position exceeds the distribution threshold of the area where the laser beams are located.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010193980.7A CN113496491B (en) | 2020-03-19 | 2020-03-19 | Road surface segmentation method and device based on multi-line laser radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010193980.7A CN113496491B (en) | 2020-03-19 | 2020-03-19 | Road surface segmentation method and device based on multi-line laser radar |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113496491A true CN113496491A (en) | 2021-10-12 |
| CN113496491B CN113496491B (en) | 2023-12-15 |
Family
ID=77993289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010193980.7A Active CN113496491B (en) | 2020-03-19 | 2020-03-19 | Road surface segmentation method and device based on multi-line laser radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113496491B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114322856A (en) * | 2021-12-16 | 2022-04-12 | 青岛慧拓智能机器有限公司 | Mining area pavement flatness detection method and device, storage medium and equipment |
| CN114387585A (en) * | 2022-03-22 | 2022-04-22 | 新石器慧通（北京）科技有限公司 | Obstacle detection method, detection device, and travel device |
| CN114415661A (en) * | 2021-12-15 | 2022-04-29 | 中国农业大学 | Planar laser SLAM and navigation method based on compressed three-dimensional space point cloud |
| CN114627073A (en) * | 2022-03-14 | 2022-06-14 | 一汽解放汽车有限公司 | Terrain recognition method, terrain recognition device, computer equipment and storage medium |
| CN114659513A (en) * | 2022-03-11 | 2022-06-24 | 北京航空航天大学 | Point cloud map construction and maintenance method for unstructured road |
| CN114755695A (en) * | 2022-06-15 | 2022-07-15 | 北京海天瑞声科技股份有限公司 | Method, device and medium for detecting road surface of laser radar point cloud data |
| CN114814796A (en) * | 2022-07-01 | 2022-07-29 | 陕西欧卡电子智能科技有限公司 | Method, device and equipment for extracting water surface travelable area based on high-precision map |
| WO2024040954A1 (en) * | 2022-08-24 | 2024-02-29 | 北京京东乾石科技有限公司 | Point cloud semantic segmentation network training method, and point cloud semantic segmentation method and apparatus |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1308752A2 (en) * | 2001-11-02 | 2003-05-07 | Fuji Jukogyo Kabushiki Kaisha | Monitoring system of the outside of a vehicle and method therefore |
| US20130282208A1 (en) * | 2012-04-24 | 2013-10-24 | Exelis, Inc. | Point cloud visualization of acceptable helicopter landing zones based on 4d lidar |
| US20140118716A1 (en) * | 2012-10-31 | 2014-05-01 | Raytheon Company | Video and lidar target detection and tracking system and method for segmenting moving targets |
| KR101421128B1 (en) * | 2014-04-30 | 2014-07-22 | 국방과학연구소 | Extraction method of building regions using segmented 3d raw datd based on laser radar |
| CN104005325A (en) * | 2014-06-17 | 2014-08-27 | 武汉武大卓越科技有限责任公司 | Pavement crack detecting device and method based on depth and gray level images |
| CN105404898A (en) * | 2015-11-26 | 2016-03-16 | 福州华鹰重工机械有限公司 | Loose-type point cloud data segmentation method and device |
| CN106204705A (en) * | 2016-07-05 | 2016-12-07 | 长安大学 | A kind of 3D point cloud segmentation method based on multi-line laser radar |
| WO2018205119A1 (en) * | 2017-05-09 | 2018-11-15 | 深圳市速腾聚创科技有限公司 | Roadside detection method and system based on laser radar scanning |
| CN108983248A (en) * | 2018-06-26 | 2018-12-11 | 长安大学 | It is a kind of that vehicle localization method is joined based on the net of 3D laser radar and V2X |
| CN110320504A (en) * | 2019-07-29 | 2019-10-11 | 浙江大学 | A kind of unstructured road detection method based on laser radar point cloud statistics geometrical model |
| WO2019242174A1 (en) * | 2018-06-21 | 2019-12-26 | 华南理工大学 | Method for automatically detecting building structure and generating 3d model based on laser radar |
| WO2020020146A1 (en) * | 2018-07-25 | 2020-01-30 | 深圳市商汤科技有限公司 | Method and apparatus for processing laser radar sparse depth map, device, and medium |
| WO2020043041A1 (en) * | 2018-08-27 | 2020-03-05 | 腾讯科技（深圳）有限公司 | Method and device for point cloud data partitioning, storage medium, and electronic device |
-
2020
- 2020-03-19 CN CN202010193980.7A patent/CN113496491B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1308752A2 (en) * | 2001-11-02 | 2003-05-07 | Fuji Jukogyo Kabushiki Kaisha | Monitoring system of the outside of a vehicle and method therefore |
| US20130282208A1 (en) * | 2012-04-24 | 2013-10-24 | Exelis, Inc. | Point cloud visualization of acceptable helicopter landing zones based on 4d lidar |
| US20140118716A1 (en) * | 2012-10-31 | 2014-05-01 | Raytheon Company | Video and lidar target detection and tracking system and method for segmenting moving targets |
| KR101421128B1 (en) * | 2014-04-30 | 2014-07-22 | 국방과학연구소 | Extraction method of building regions using segmented 3d raw datd based on laser radar |
| CN104005325A (en) * | 2014-06-17 | 2014-08-27 | 武汉武大卓越科技有限责任公司 | Pavement crack detecting device and method based on depth and gray level images |
| CN105404898A (en) * | 2015-11-26 | 2016-03-16 | 福州华鹰重工机械有限公司 | Loose-type point cloud data segmentation method and device |
| CN106204705A (en) * | 2016-07-05 | 2016-12-07 | 长安大学 | A kind of 3D point cloud segmentation method based on multi-line laser radar |
| WO2018205119A1 (en) * | 2017-05-09 | 2018-11-15 | 深圳市速腾聚创科技有限公司 | Roadside detection method and system based on laser radar scanning |
| WO2019242174A1 (en) * | 2018-06-21 | 2019-12-26 | 华南理工大学 | Method for automatically detecting building structure and generating 3d model based on laser radar |
| CN108983248A (en) * | 2018-06-26 | 2018-12-11 | 长安大学 | It is a kind of that vehicle localization method is joined based on the net of 3D laser radar and V2X |
| WO2020020146A1 (en) * | 2018-07-25 | 2020-01-30 | 深圳市商汤科技有限公司 | Method and apparatus for processing laser radar sparse depth map, device, and medium |
| WO2020043041A1 (en) * | 2018-08-27 | 2020-03-05 | 腾讯科技（深圳）有限公司 | Method and device for point cloud data partitioning, storage medium, and electronic device |
| CN110320504A (en) * | 2019-07-29 | 2019-10-11 | 浙江大学 | A kind of unstructured road detection method based on laser radar point cloud statistics geometrical model |
Non-Patent Citations (3)
| Title |
|---|
| M. NIKOLOVA; A. HERO: "Segmentation of a road from a vehicle-mounted radar and accuracy of the estimation", 《PROCEEDINGS OF THE IEEE INTELLIGENT VEHICLES SYMPOSIUM 2000 (CAT. NO.00TH8511)》 * |
| 张达;李霖;李游;: "基于车载激光扫描的城市道路提取方法", 测绘通报, no. 07 * |
| 程子阳;任国全;张银: "扫描线段特征用于三维点云地面分割", 《光电工程》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114415661A (en) * | 2021-12-15 | 2022-04-29 | 中国农业大学 | Planar laser SLAM and navigation method based on compressed three-dimensional space point cloud |
| CN114415661B (en) * | 2021-12-15 | 2023-09-22 | 中国农业大学 | Planar laser SLAM and navigation method based on compressed three-dimensional space point cloud |
| CN114322856B (en) * | 2021-12-16 | 2023-09-15 | 青岛慧拓智能机器有限公司 | Mining area pavement evenness detection method, device, storage medium and equipment |
| CN114322856A (en) * | 2021-12-16 | 2022-04-12 | 青岛慧拓智能机器有限公司 | Mining area pavement flatness detection method and device, storage medium and equipment |
| CN114659513B (en) * | 2022-03-11 | 2024-04-09 | 北京航空航天大学 | Unstructured road-oriented point cloud map construction and maintenance method |
| CN114659513A (en) * | 2022-03-11 | 2022-06-24 | 北京航空航天大学 | Point cloud map construction and maintenance method for unstructured road |
| CN114627073A (en) * | 2022-03-14 | 2022-06-14 | 一汽解放汽车有限公司 | Terrain recognition method, terrain recognition device, computer equipment and storage medium |
| CN114627073B (en) * | 2022-03-14 | 2024-06-04 | 一汽解放汽车有限公司 | Terrain recognition method, apparatus, computer device and storage medium |
| CN114387585B (en) * | 2022-03-22 | 2022-07-05 | 新石器慧通（北京）科技有限公司 | Obstacle detection method, detection device, and travel device |
| CN114387585A (en) * | 2022-03-22 | 2022-04-22 | 新石器慧通（北京）科技有限公司 | Obstacle detection method, detection device, and travel device |
| CN114755695A (en) * | 2022-06-15 | 2022-07-15 | 北京海天瑞声科技股份有限公司 | Method, device and medium for detecting road surface of laser radar point cloud data |
| CN114814796A (en) * | 2022-07-01 | 2022-07-29 | 陕西欧卡电子智能科技有限公司 | Method, device and equipment for extracting water surface travelable area based on high-precision map |
| CN114814796B (en) * | 2022-07-01 | 2022-09-30 | 陕西欧卡电子智能科技有限公司 | Method, device and equipment for extracting water surface travelable area based on high-precision map |
| WO2024040954A1 (en) * | 2022-08-24 | 2024-02-29 | 北京京东乾石科技有限公司 | Point cloud semantic segmentation network training method, and point cloud semantic segmentation method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113496491B (en) | 2023-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113496491B (en) | Road surface segmentation method and device based on multi-line laser radar | |
| US10860870B2 (en) | Object detecting apparatus, object detecting method, and computer program product | |
| EP3418943B1 (en) | Object detecting apparatus, object detecting method, and computer-readable medium | |
| CN110210389B (en) | Multi-target identification tracking method for road traffic scene | |
| WO2021259344A1 (en) | Vehicle detection method and device, vehicle, and storage medium | |
| US8036424B2 (en) | Field recognition apparatus, method for field recognition and program for the same | |
| US11373532B2 (en) | Pothole detection system | |
| CN110705458B (en) | Boundary detection method and device | |
| CN107064955A (en) | barrier clustering method and device | |
| CN110979313B (en) | Automatic parking positioning method and system based on space map | |
| CN111461048B (en) | Vision-based parking lot drivable area detection and local map construction method | |
| CN113110451A (en) | Mobile robot obstacle avoidance method with depth camera and single line laser radar fused | |
| CN112666535A (en) | Environment sensing method and system based on multi-radar data fusion | |
| CN112487907B (en) | Dangerous scene identification method and system based on graph classification | |
| CN112213735A (en) | Laser point cloud noise reduction method for rainy and snowy weather | |
| CN109241855B (en) | Intelligent vehicle travelable area detection method based on stereoscopic vision | |
| CN105512641A (en) | Method for using laser radar scanning method to calibrate dynamic pedestrians and vehicles in video in snowing or raining state | |
| CN115775378A (en) | Vehicle-road cooperative target detection method based on multi-sensor fusion | |
| CN114660568B (en) | Laser radar obstacle detection method and device | |
| CN113734176A (en) | Environment sensing system and method for intelligent driving vehicle, vehicle and storage medium | |
| CN113838060A (en) | Perception system for autonomous vehicle | |
| US11373389B2 (en) | Partitioning images obtained from an autonomous vehicle camera | |
| CN114743179A (en) | Panoramic visible driving area detection method based on semantic segmentation | |
| CN114594767A (en) | Unmanned vehicle path re-planning method based on air-ground cooperation | |
| CN114495059A (en) | Road surface identification method and device for sanitation sweeper |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |