CN117830977A - Multi-mode travelable region detection algorithm - Google Patents
Multi-mode travelable region detection algorithm Download PDFInfo
- Publication number
- CN117830977A CN117830977A CN202311446232.5A CN202311446232A CN117830977A CN 117830977 A CN117830977 A CN 117830977A CN 202311446232 A CN202311446232 A CN 202311446232A CN 117830977 A CN117830977 A CN 117830977A
- Authority
- CN
- China
- Prior art keywords
- point
- drivable
- algorithm
- points
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 26
- 238000000034 method Methods 0.000 claims abstract description 7
- 238000013135 deep learning Methods 0.000 claims abstract description 4
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims description 3
- 240000004050 Pentaglottis sempervirens Species 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention discloses a multi-mode drivable area detection algorithm, which specifically comprises the following steps: the method specifically comprises the following steps: step S1, detecting and outputting a drivable area in front of a vehicle through a laser radar algorithm; step S2, visually detecting a drivable region in front of the vehicle by using a YOLOP deep learning algorithm to obtain a probability Pi that each pixel point in the image belongs to the drivable region (u,v) ∈[0,1]The method comprises the steps of carrying out a first treatment on the surface of the The YOLOP algorithm framework is shown in fig. 2: only the backbones and Drivable area segment head of fig. 2 are reserved for the detection of the travelable region, and the detection speed is increased. The multi-mode model can simultaneously utilize a plurality of different data inputs, so that information can be more comprehensively understood and processed; multimodal models can utilize a variety of different data inputs to better understand the context and context of information; the multimodal model can utilize a variety of different modelsTo better cope with data noise and input variations.
Description
Technical Field
The invention relates to the field of automatic driving, in particular to a multi-mode drivable area detection algorithm.
Background
Multimodal perception refers to the ability to obtain environmental information and user intent through multiple perception means (e.g., visual, auditory, tactile, etc.); the method can combine data of different sensing modes, and further improve understanding and analysis capability of environment and users.
The following problems exist in the detection algorithm of the travelable area of the laser radar in the prior art: the detection resolution in the lateral direction is low; is easy to be influenced by rain and snow weather conditions; the high-harness laser radar has high manufacturing cost; the following problems are also presented with the runable region detection algorithm using images alone: the detection resolution in the longitudinal direction is low; is easily affected by weather conditions such as illumination.
Disclosure of Invention
The invention provides a multi-mode travelable region detection algorithm, which adopts a multi-mode fusion method to comprehensively consider the travelable region detection results of vision and laser radar, so that the algorithm is more robust and has higher accuracy.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: the multi-mode drivable area detection algorithm specifically comprises the following steps:
step S1, detecting and outputting a drivable area in front of a vehicle through a laser radar algorithm;
step S2, visually detecting a drivable region in front of the vehicle by using a YOLOP deep learning algorithm to obtain a probability Pi that each pixel point in the image belongs to the drivable region (u,v) ∈[0,1]The method comprises the steps of carrying out a first treatment on the surface of the The YOLOP algorithm framework is shown in fig. 2:
only the backbones and Drivable area segment head of fig. 2 are reserved for detecting the driving area, so that the detection speed is improved
Step S3, converting the drivable area obtained in the first step into an image coordinate system through calibration information to obtain that each pixel point in the image of the laser radar belongs to the drivable areaProbability Pl of (2) (u,v) ∈{0,1}；
S4, fusing detection results of the laser radar and the camera;
and S5, converting the result of the drivable area into a vehicle coordinate system according to the calibration information for use by a downstream module in the automatic driving system.
The specific scheme flow of the step S1 is as follows:
s1.1, preprocessing original point cloud to remove miscellaneous points;
s1.2, eliminating ground points from the input point cloud by using a ground segmentation algorithm;
step S1.3, setting a height threshold h, and removing points with higher heights;
s1.4, reducing the dimension of the point cloud with the ground points and higher points removed to a bird' S eye view two-dimensional view angle;
and S1.5, rasterizing the aerial two-dimensional point cloud after the dimension reduction to obtain a drivable region, marking the grid with the point cloud as an undrivable grid after rasterization, and marking the grid without the point cloud as a drivable grid.
The specific steps of the step S1.2 are as follows:
step S1.21, dividing a sensing area into pilar (columns) on two scales with different sizes;
s1.22, finding the lowest point of each villar in the Z direction to obtain a minuz;
step S1.23, for any point, certain large and small villars belong to at the same time, calculating the difference between the point and the minimum point minuz of the two villars where the point is located, and respectively recording as d1 and d2. If the difference value between d1 and d2 is smaller than the threshold value, the point is marked as a ground point, otherwise, the point is an obstacle point;
in step S1.24, the ground points are removed, and the remaining points are regarded as obstacle points.
And in the step S4, fusing detection results of the laser radar and the camera: final probability that each pixel point in the image belongs to a travelable region
Said arrangementSetting a threshold value theta, if P (u,v) >θ, the pixel is a travelable region, otherwise it is a non-travelable region.
The beneficial effects of adopting above technical scheme are:
1. the information richness of the invention: the multimodal model can utilize a variety of different data inputs simultaneously to more fully understand and process information. For example, by using both image and speech inputs, features of objects and environmental context can be more accurately recognized and understood, thereby improving the accuracy of image recognition and speech recognition.
2. Context awareness capability of the present invention: multimodal models can utilize a variety of different data inputs to better understand the context and context of information. For example, in the field of natural language processing, multimodal models can utilize image or video input to better understand implicit information and emotional tendency in a language.
3. Robustness and stability of the present invention: the multimodal model can utilize a variety of different data inputs to better address data noise and input variations. For example, in the face recognition field, the multimodal model can use both image and voice input, so as to better cope with problems of face shielding, light variation and the like.
Drawings
FIG. 1 is a grid mark diagram of a point cloud free system of the present invention;
FIG. 2 is a frame diagram of the YOLOP algorithm of the present invention.
Detailed Description
The following detailed description of the embodiments of the invention, given by way of example only, is presented in the accompanying drawings to aid in a more complete, accurate and thorough understanding of the concepts and aspects of the invention, and to aid in its practice, by those skilled in the art.
As shown in fig. 1 to 2, the invention relates to a multi-mode drivable region detection algorithm, which considers interaction between an automatic driving vehicle and a scene target and combines high-precision map lane constraint, thereby improving the safety and accuracy of track prediction in a complex scene.
Specifically, as shown in fig. 1 to 2, the method specifically includes the following steps:
step S1, detecting and outputting a drivable area in front of a vehicle through a laser radar algorithm;
step S2, visually detecting a drivable region in front of the vehicle by using a YOLOP deep learning algorithm to obtain a probability Pi that each pixel point in the image belongs to the drivable region (u,v) ∈[0,1]；
Step S3, converting the travelable region obtained in the first step into an image coordinate system through calibration information to obtain the probability Pl that each pixel point belongs to the travelable region in the image by the laser radar (u,v) ∈{0,1}；
S4, fusing detection results of the laser radar and the camera;
and S5, converting the result of the drivable area into a vehicle coordinate system according to the calibration information for use by a downstream module in the automatic driving system.
The specific scheme flow of the step S1 is as follows:
s1.1, preprocessing original point cloud to remove miscellaneous points;
s1.2, eliminating ground points from the input point cloud by using a ground segmentation algorithm;
step S1.3, setting a height threshold h, and removing points with higher heights;
s1.4, reducing the dimension of the point cloud with the ground points and higher points removed to a bird' S eye view two-dimensional view angle;
and S1.5, rasterizing the aerial two-dimensional point cloud after the dimension reduction to obtain a drivable region, marking the grid with the point cloud as an undrivable grid after rasterization, and marking the grid without the point cloud as a drivable grid.
The specific steps of the step S1.2 are as follows:
step S1.21, dividing a sensing area into pilar (columns) on two scales with different sizes;
s1.22, finding the lowest point of each villar in the Z direction to obtain a minuz;
step S1.23, for any point, certain large villar and small villar belong to at the same time, calculating the difference value of the point and the minimum point minuz of the two villars where the point is positioned, and respectively marking as d1 and d2; if the difference value between d1 and d2 is smaller than the threshold value, the point is marked as a ground point, otherwise, the point is an obstacle point;
in step S1.24, the ground points are removed, and the remaining points are regarded as obstacle points.
The detection results of the laser radar and the camera are fused in the step S4, and the final probability that each pixel point in the image belongs to the drivable areaSetting a threshold value theta, if P (u,v) >θ, the pixel is a travelable region, otherwise it is a non-travelable region.
While the invention has been described above by way of example with reference to the accompanying drawings, it is to be understood that the invention is not limited to the particular embodiments described, but is capable of numerous insubstantial modifications of the inventive concept and solution; or the invention is not improved, and the conception and the technical scheme are directly applied to other occasions and are all within the protection scope of the invention.
Claims (5)
1. A multi-modal travelable region detection algorithm characterized by: the method specifically comprises the following steps:
step S1, detecting and outputting a drivable area in front of a vehicle through a laser radar algorithm;
step S2, visually detecting a drivable region in front of the vehicle by using a YOLOP deep learning algorithm to obtain a probability Pi that each pixel point in the image belongs to the drivable region (u,v) ∈[0,1]；
Step S3, converting the travelable region obtained in the first step into an image coordinate system through calibration information to obtain the probability Pl that each pixel point belongs to the travelable region in the image by the laser radar (u,v) ∈{0,1}；
S4, fusing detection results of the laser radar and the camera;
and S5, converting the result of the drivable area into a vehicle coordinate system according to the calibration information for use by a downstream module in the automatic driving system.
2. A multi-modal drivable area detection algorithm as claimed in claim 1, in which: the specific scheme flow of the step S1 is as follows:
s1.1, preprocessing original point cloud to remove miscellaneous points;
s1.2, eliminating ground points from the input point cloud by using a ground segmentation algorithm;
step S1.3, setting a height threshold h, and removing points with higher heights;
s1.4, reducing the dimension of the point cloud with the ground points and higher points removed to a bird' S eye view two-dimensional view angle;
and S1.5, rasterizing the aerial two-dimensional point cloud after the dimension reduction to obtain a drivable region, marking the grid with the point cloud as an undrivable grid after rasterization, and marking the grid without the point cloud as a drivable grid.
3. A multi-modal drivable area detection algorithm as claimed in claim 2, in which: the specific steps of the step S1.2 are as follows:
step S1.21, dividing a sensing area into pilar (columns) on two scales with different sizes;
s1.22, finding the lowest point of each villar in the Z direction to obtain a minuz;
step S1.23, for any point, certain large and small villars belong to at the same time, calculating the difference between the point and the minimum point minuz of the two villars where the point is located, and respectively recording as d1 and d2. If the difference value between d1 and d2 is smaller than the threshold value, the point is marked as a ground point, otherwise, the point is an obstacle point;
in step S1.24, the ground points are removed, and the remaining points are regarded as obstacle points.
4. A multi-modal drivable area detection algorithm as claimed in claim 1, in which: and in the step S4, fusing detection results of the laser radar and the camera:
each pixel point in the image belongs to a drivable areaFinal probability
5. The multi-modal drivable area detection algorithm as set forth in claim 4, wherein: the setting threshold value theta, if P (u,v) >θ, the pixel is a travelable region, otherwise it is a non-travelable region.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311446232.5A CN117830977A (en) | 2023-11-02 | 2023-11-02 | Multi-mode travelable region detection algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311446232.5A CN117830977A (en) | 2023-11-02 | 2023-11-02 | Multi-mode travelable region detection algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117830977A true CN117830977A (en) | 2024-04-05 |
Family
ID=90504620
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311446232.5A Pending CN117830977A (en) | 2023-11-02 | 2023-11-02 | Multi-mode travelable region detection algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117830977A (en) |
-
2023
- 2023-11-02 CN CN202311446232.5A patent/CN117830977A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110175576B (en) | Driving vehicle visual detection method combining laser point cloud data | |
| US20200041284A1 (en) | Map road marking and road quality collecting apparatus and method based on adas system | |
| US11532151B2 (en) | Vision-LiDAR fusion method and system based on deep canonical correlation analysis | |
| Greenhalgh et al. | Recognizing text-based traffic signs | |
| CN103324930B (en) | A kind of registration number character dividing method based on grey level histogram binaryzation | |
| Gomez et al. | Traffic lights detection and state estimation using hidden markov models | |
| Kheyrollahi et al. | Automatic real-time road marking recognition using a feature driven approach | |
| CN108388641B (en) | Traffic facility map generation method and system based on deep learning | |
| CN111179152B (en) | Road identification recognition method and device, medium and terminal | |
| Haque et al. | A computer vision based lane detection approach | |
| CN110738121A (en) | front vehicle detection method and detection system | |
| CN111563412B (en) | Rapid lane line detection method based on parameter space voting and Bessel fitting | |
| AU2020102039A4 (en) | A high-precision multi-targets visual detection method in automatic driving scene | |
| CN105373135A (en) | Method and system for guiding airplane docking and identifying airplane type based on machine vision | |
| CN105787524A (en) | License plate identification method based on OpenCV and license plate identification system based on OpenCV | |
| CN112825192B (en) | Object identification system and method based on machine learning | |
| KR102363719B1 (en) | Lane extraction method using projection transformation of 3D point cloud map | |
| CN109635737A (en) | Automobile navigation localization method is assisted based on pavement marker line visual identity | |
| CN104881661A (en) | Vehicle detection method based on structure similarity | |
| CN104915642A (en) | Method and apparatus for measurement of distance to vehicle ahead | |
| Rasib et al. | Pixel level segmentation based drivable road region detection and steering angle estimation method for autonomous driving on unstructured roads | |
| CN107220632B (en) | Road surface image segmentation method based on normal characteristic | |
| CN105447431A (en) | Docking airplane tracking and positioning method and system based on machine vision | |
| CN111310651B (en) | Water surface detection method based on polarization camera and RGB-D sensor | |
| CN112733678A (en) | Ranging method, ranging device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |