CN110992731B - Laser radar-based 3D vehicle detection method and device and storage medium - Google Patents
Laser radar-based 3D vehicle detection method and device and storage medium Download PDFInfo
- Publication number
- CN110992731B CN110992731B CN201911273416.XA CN201911273416A CN110992731B CN 110992731 B CN110992731 B CN 110992731B CN 201911273416 A CN201911273416 A CN 201911273416A CN 110992731 B CN110992731 B CN 110992731B
- Authority
- CN
- China
- Prior art keywords
- data
- vehicle
- laser radar
- detection
- vehicle detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/164—Centralised systems, e.g. external to vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Optical Radar Systems And Details Thereof (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to a laser radar-based 3D vehicle detection method, a laser radar-based 3D vehicle detection device and a storage medium, wherein 3D data of the surrounding environment of an automatic driving vehicle are collected based on a laser radar; manually labeling the target vehicle in each frame of 3D data aiming at the 3D data; for the artificially labeled 3D data, encoding Z-axis data of the 3D data by utilizing convolution and pooling operations, and compressing the artificially labeled 3D data into a 2D top view; and aiming at the compressed 2D top view, training by using an image detection network to obtain a laser radar-based 3D vehicle detection model. According to the invention, 3D data is directly acquired by the 3D laser radar for detection, so that the steps of 2D image calibration and projection in the existing detection method are omitted, the detection steps are effectively reduced, and the detection efficiency is further improved.
Description
Technical Field
The invention relates to a laser radar-based 3D vehicle detection method and device and a storage medium.
Background
The automatic driving truck needs to sense the surrounding environment in the driving process, the information of surrounding vehicles is particularly important, and the information of the surrounding vehicles needs to be known in time and detected to track and predict the vehicles, so that the normal operation of automatic driving can be ensured.
When vehicle detection is carried out, the traditional method depends on 2D image information, vehicle detection is carried out on a 2D image, then a related calibration method is utilized, a 2D image detection structure is projected onto a 3D laser radar, and then the 3D laser radar data is utilized for analysis, so that the 3D information of the vehicle is obtained. The 3D information of the vehicle includes a 3D position of the vehicle, a vehicle size, and an orientation of the vehicle.
Therefore, the conventional method for obtaining the 3D information of the vehicle has more flows, complicated steps and relatively large calculated amount.
Disclosure of Invention
The invention aims to provide a laser radar-based 3D vehicle detection method and system, which effectively reduce detection steps and further improve detection efficiency.
Based on the same inventive concept, the invention has three independent technical schemes:
1. A3D vehicle detection method based on laser radar is characterized by comprising the following steps:
step 1: acquiring 3D data of the surrounding environment of the autonomous vehicle based on the laser radar;
step 2: manually labeling the target vehicle in each frame of 3D data aiming at the 3D data;
and step 3: for the artificially labeled 3D data, encoding Z-axis data of the 3D data by utilizing convolution and pooling operations, and compressing the artificially labeled 3D data into a 2D top view;
and 4, step 4: and aiming at the compressed 2D top view, training by using an image detection network to obtain a laser radar-based 3D vehicle detection model.
Further, in the step 2, fusion is carried out by utilizing multi-frame 3D data according to a known position, and then manual marking is carried out; after manual labeling, single frame 3D data is re-projected.
Further, in step 4, the image detection network adopts CenterNet or fast-RCNN.
Further, the method comprises a step 5 of evaluating the effect of the 3D vehicle detection model by utilizing the manual labeling data.
Further, in step 5, when the evaluation effect does not meet the requirement, increasing the 3D data acquisition amount of the surrounding environment of the automatic driving vehicle, and repeating the steps 1 to 4 until the detection effect of the 3D vehicle detection model meets the requirement.
2. A3D vehicle detection device based on laser radar, characterized by comprising:
the laser radar is used for acquiring 3D data of the surrounding environment of the automatic driving vehicle;
and the detection module is used for realizing the method and obtaining the 3D vehicle detection model based on the laser radar.
3. A computer-readable storage medium having a computer program stored thereon, characterized in that: which when executed by a processor implements the method described above.
The invention has the following beneficial effects:
the method is based on the laser radar, and 3D data of the surrounding environment of the automatic driving vehicle are collected; manually labeling the target vehicle in each frame of 3D data aiming at the 3D data; for the artificially labeled 3D data, encoding Z-axis data of the 3D data by utilizing convolution and pooling operations, and compressing the artificially labeled 3D data into a 2D top view; and aiming at the compressed 2D top view, training by using an image detection network to obtain a laser radar-based 3D vehicle detection model. According to the invention, 3D data is directly acquired by the 3D laser radar for detection, so that the steps of 2D image calibration and projection in the existing detection method are omitted, the detection steps are effectively reduced, and the detection efficiency is further improved. According to the invention, the Z-axis data of the 3D data is encoded by using the convolution and pooling operation, and then the 3D data is compressed into the 2D top view for detection, so that the calculated amount is effectively reduced. Compared with 2D image detection of the existing detection method, the method provided by the invention can be used for directly detecting on the laser radar, can obtain a sensing result without a dead angle of 360 degrees, and has a larger sensing range.
According to the method, fusion is carried out by utilizing multi-frame 3D data according to a known position, and then manual marking is carried out; after manual labeling, single frame 3D data is re-projected. Because the point cloud after the fusion is dense, compared with the data of a single frame, the target vehicle is more clearly visible in the surrounding environment, the manual marking is more convenient, and the manual marking efficiency is effectively improved.
The method utilizes the manual marking data to carry out effect evaluation on the 3D vehicle detection model. And when the evaluation effect does not meet the requirement, increasing the 3D data acquisition amount of the surrounding environment of the automatic driving vehicle until the detection effect of the 3D vehicle detection model meets the requirement. The method further ensures the accuracy of vehicle detection.
Drawings
FIG. 1 is a general flow diagram of a lidar-based 3D vehicle detection method of the present invention.
Detailed Description
The present invention is described in detail with reference to the embodiments shown in the drawings, but it should be understood that these embodiments are not intended to limit the present invention, and those skilled in the art should understand that functional, methodological, or structural equivalents or substitutions made by these embodiments are within the scope of the present invention.
The first embodiment is as follows:
3D vehicle detection method based on laser radar
The method comprises the following steps:
step 1: based on the lidar, 3D data of the environment surrounding the autonomous vehicle is collected.
The truck carrying the laser radar runs in different scenes and cities, the laser radar is used for collecting data and recording the data in the mobile hard disk, and the data collection needs to cover different scenes as much as possible, so that the diversity of the data is ensured.
Step 2: and aiming at the 3D data, manually marking the target vehicle in each frame of 3D data, namely marking a 3D detection frame of the vehicle in each frame of 3D data.
Fusing multi-frame 3D data according to a known position, and then manually marking; after manual labeling, single frame 3D data is re-projected.
Because the laser radar is sparse, the 3D vehicles are not distinguished well sometimes, and multi-frame data are fused according to known positions to form dense point cloud data, and then manual marking is carried out. Because the point cloud after fusion is dense, compared with single-frame data, objects such as target vehicles and the like are more clearly visible, and the marking is more facilitated.
And step 3: and for the artificially labeled 3D data, encoding the Z-axis data of the 3D data by utilizing convolution and pooling operations, and compressing the artificially labeled 3D data into a 2D top view.
Unlike conventional 2D deep learning detection networks on images, the data of lidar is in 3D form. Although 3D convolution operation can be carried out on 3D data at present, the 3D calculation amount is large and the consumed memory is large, so the method adopts a mode of firstly encoding the Z-axis data of the 3D data by using the convolution and pooling operation and then compressing the 3D data into a 2D top view, and the calculation amount is effectively reduced.
In specific implementation, the space is firstly gridded by using the x and y axes, and a grid is formed by multiplying 10 centimeters by 10 centimeters. Each grid contains all points on the Z-axis in 10 x 10 cm space. And performing convolution and pooling operations on all point clouds in each grid for multiple times to obtain a representation vector of each grid, namely completing the encoding of the Z axis in the 3D data. Each network is represented by a vector, and 3D data is successfully compressed into a 2D top view structure.
And 4, step 4: and aiming at the compressed 2D top view, training by using an image detection network to obtain a laser radar-based 3D vehicle detection model.
In specific implementation, a CenterNet image detection technology is adopted. The centret is an anchor-free object detection technology, and directly regresses the central position and the detection frame of an object by utilizing multiple convolution and pooling operations of images.
Example two:
3D vehicle detection method based on laser radar
In the second embodiment, further comprising
And 5, evaluating the effect of the 3D vehicle detection model by utilizing the manual marking data.
And when the evaluation effect does not meet the requirement, increasing the 3D data acquisition amount of the surrounding environment of the automatic driving vehicle, and repeating the steps 1 to 4 until the detection effect of the 3D vehicle detection model meets the requirement.
After the training of the model is finished, the effect of the model can be tested by using some manually marked data, and the network model can be evaluated by using the accuracy and the recall rate.
And 6, releasing the model. When the model evaluation meets the requirements, the model can be released, and the vehicle detection is carried out on the automatic driving vehicle in real time for sensing and positioning the environmental information.
Example three: 3D vehicle detection method based on laser radar
In step 4, the image detection network adopts multiplexing fast-RCNN and CenterNet. The fast RCNN is an object detection technology based on anchor points, firstly, a plurality of anchor points are defined for each pixel in an image in advance, then the anchor points are associated with training truth values to obtain positive and negative training samples, meanwhile, the feature representation of the image is obtained by utilizing multiple convolution and pooling operations, and classification and regression of the anchor points are completed by combining the positive and negative samples. The centret is an anchor-free object detection technology, and directly regresses the central position and the detection frame of an object by utilizing multiple convolution and pooling operations of images.
The rest is the same as the second embodiment.
Example four:
3D vehicle detection device based on laser radar
The method comprises the following steps:
the laser radar is used for acquiring 3D data of the surrounding environment of the automatic driving vehicle;
and the detection module is used for realizing the method of the first embodiment or the second embodiment and obtaining the laser radar-based 3D vehicle detection model.
Example five:
computer readable storage medium
The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of one or both embodiments.
The above-listed detailed description is only a specific description of a possible embodiment of the present invention, and they are not intended to limit the scope of the present invention, and equivalent embodiments or modifications made without departing from the technical spirit of the present invention should be included in the scope of the present invention.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Claims (4)
1. A3D vehicle detection method based on laser radar is characterized by comprising the following steps:
step 1: acquiring 3D data of the surrounding environment of the autonomous vehicle based on the laser radar;
step 2: manually labeling the target vehicle in each frame of 3D data aiming at the 3D data;
and step 3: for the artificially labeled 3D data, encoding Z-axis data of the 3D data by utilizing convolution and pooling operations, and compressing the artificially labeled 3D data into a 2D top view;
and 4, step 4: aiming at the compressed 2D top view, training by using an image detection network to obtain a laser radar-based 3D vehicle detection model;
in the step 2, fusion is carried out by utilizing multi-frame 3D data according to a known position, and then manual marking is carried out; after manual labeling, projecting the single-frame 3D data again;
the method comprises the following steps of 5, utilizing manual labeling data to evaluate the effect of a 3D vehicle detection model, and evaluating the model by utilizing accuracy and recall rate;
in the step 5, when the evaluation effect does not meet the requirement, increasing the 3D data acquisition amount of the surrounding environment of the automatic driving vehicle, and repeating the steps 1 to 4 until the detection effect of the 3D vehicle detection model meets the requirement;
step 6, model release; and after the model evaluation meets the requirements, releasing the model, and detecting the vehicle on the automatic driving vehicle in real time for sensing and positioning the environmental information.
2. The lidar-based 3D vehicle detection method of claim 1, wherein: in step 4, the image detection network adopts CenterNet or fast-RCNN, or multiplexing CenterNet and fast-RCNN.
3. A3D vehicle detection device based on laser radar, characterized by comprising:
the laser radar is used for acquiring 3D data of the surrounding environment of the automatic driving vehicle;
a detection module for implementing the method of claim 1 or 2, obtaining a lidar based 3D vehicle detection model.
4. A computer-readable storage medium having a computer program stored thereon, characterized in that: the computer program, when executed by a processor, implements the method of claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911273416.XA CN110992731B (en) | 2019-12-12 | 2019-12-12 | Laser radar-based 3D vehicle detection method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911273416.XA CN110992731B (en) | 2019-12-12 | 2019-12-12 | Laser radar-based 3D vehicle detection method and device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110992731A CN110992731A (en) | 2020-04-10 |
| CN110992731B true CN110992731B (en) | 2021-11-05 |
Family
ID=70092780
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911273416.XA Active CN110992731B (en) | 2019-12-12 | 2019-12-12 | Laser radar-based 3D vehicle detection method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110992731B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113643226B (en) * | 2020-04-27 | 2024-01-19 | 成都术通科技有限公司 | Labeling method, labeling device, labeling equipment and labeling medium |
| CN111476902B (en) * | 2020-04-27 | 2023-10-24 | 北京小马慧行科技有限公司 | Labeling method and device for objects in 3D point cloud, storage medium and processor |
| CN111582213A (en) * | 2020-05-15 | 2020-08-25 | 北京铁科时代科技有限公司 | Automobile identification method based on Centernet |
| CN111860379A (en) * | 2020-07-27 | 2020-10-30 | 联通智网科技有限公司 | Method and device for establishing surrounding vehicle model, storage medium and computer equipment |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9699450B2 (en) * | 2012-10-04 | 2017-07-04 | Qualcomm Incorporated | Inter-view predicted motion vector for 3D video |
| CN108345822B (en) * | 2017-01-22 | 2022-02-01 | 腾讯科技（深圳）有限公司 | Point cloud data processing method and device |
| EP3613208A4 (en) * | 2017-06-02 | 2020-06-24 | SZ DJI Technology Co., Ltd. | Systems and methods for multi-target tracking and autofocusing based on deep machine learning and laser radar |
| CN107578427B (en) * | 2017-07-31 | 2021-05-18 | 深圳市易成自动驾驶技术有限公司 | Method and device for detecting dynamic obstacle and computer readable storage medium |
| CN109543601A (en) * | 2018-11-21 | 2019-03-29 | 电子科技大学 | A kind of unmanned vehicle object detection method based on multi-modal deep learning |
| CN110097047B (en) * | 2019-03-19 | 2021-10-08 | 同济大学 | Vehicle detection method based on deep learning and adopting single line laser radar |
| CN110070511B (en) * | 2019-04-30 | 2022-01-28 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic device and storage medium |
| CN110210398A (en) * | 2019-06-03 | 2019-09-06 | 宁波智能装备研究院有限公司 | A kind of three-dimensional point cloud semantic segmentation mask method |
| CN110531376B (en) * | 2019-08-23 | 2022-04-22 | 畅加风行(苏州)智能科技有限公司 | Obstacle detection and tracking method for port unmanned vehicle |
-
2019
- 2019-12-12 CN CN201911273416.XA patent/CN110992731B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110992731A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110992731B (en) | Laser radar-based 3D vehicle detection method and device and storage medium | |
| KR101829556B1 (en) | Lidar-based classification of object movement | |
| CN104778720B (en) | A kind of fast volume measuring method based on space invariance characteristic | |
| CN112418245B (en) | Electromagnetic emission point positioning method based on urban environment physical model | |
| CN111179300A (en) | Method, apparatus, system, device and storage medium for obstacle detection | |
| CN111141264B (en) | Unmanned aerial vehicle-based urban three-dimensional mapping method and system | |
| CN108280843A (en) | A kind of video object detecting and tracking method and apparatus | |
| CN104424630A (en) | Three-dimension reconstruction method and device, and mobile terminal | |
| CN110073362A (en) | System and method for lane markings detection | |
| CN112257605B (en) | Three-dimensional target detection method, system and device based on self-labeling training sample | |
| US20240017747A1 (en) | Method and system for augmenting lidar data | |
| CN103729846B (en) | LiDAR point cloud data edge detection method based on triangular irregular network | |
| CN112419512B (en) | Air three-dimensional model repairing system and method based on semantic information | |
| CN109564687A (en) | Learning method and program | |
| CN112082553A (en) | Indoor positioning method and positioning device based on WIFI and laser radar and robot | |
| CN113706633B (en) | Three-dimensional information determination method and device for target object | |
| CN104463240A (en) | Method and device for controlling list interface | |
| CN114494618A (en) | Map generation method and device, electronic equipment and storage medium | |
| US20230401837A1 (en) | Method for training neural network model and method for generating image | |
| CN111145203A (en) | Lane line extraction method and device | |
| Zhu et al. | Potentials of RGB-D cameras in as-built indoor environment modeling | |
| US20230080540A1 (en) | Lidar simulation system | |
| CN114299161A (en) | Data processing method, device, equipment and computer storage medium | |
| WO2022126380A1 (en) | Three-dimensional point cloud segmentation method and apparatus, and movable platform | |
| CN110148205A (en) | A kind of method and apparatus of the three-dimensional reconstruction based on crowdsourcing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |