TWI673190B - Vehicle detection method based on optical radar - Google Patents

Abstract

An optical radar-based vehicle detection method, the steps include: (A) obtaining a three-dimensional point cloud data by an optical radar; (B) subjecting the three-dimensional point cloud data to a segmented ground processing to remove ground information; (C) An object extraction process is performed on the three-dimensional point cloud data to obtain a three-dimensional point cloud image including candidate point cloud groups of the car; (D) performing a two-dimensional mapping process on the three-dimensional point cloud image to obtain a two-dimensional map; (E ) Perform a three-dimensional and a two-dimensional feature extraction on the three-dimensional point cloud image and the two-dimensional image, and use the three-dimensional and two-dimensional features to determine the position of the car, thereby enhancing the processing of the image data obtained by the optical radar, and more Can distinguish distant cars and environmental blocks, improve the ability to identify cars at night or in bad weather.

Description

Optical radar-based vehicle detection method

The present invention relates to an image processing method, and more particularly to a vehicle detection method for optical radar.

Optical radar (LIDAR) is a very important component in autonomous driving systems. From the perspective of many traditional suppliers and start-up companies, the obstacle detection of autonomous vehicles in the future will still rely on solid-state optical radar, because this is the most cost-effective and In a reliable way, optical radar is usually used to detect vehicles and pedestrians. Compared with millimeter wave (MMW) radar, optical radar has higher spatial resolution and higher ranging accuracy, but also has the following disadvantages: it cannot be used in harsh environments. In the event of rain or fog, the detection distance will be shorter than that of MMW radar in this situation. However, the recent improvements in the sensitivity of optical radar can have better performance in harsh environments, and it has now become an important sensor and is widely used in autonomous driving.

At present, there are many excellent papers discussing objects detected by optical radar. For example: B.Li proposed a vehicle detection system based on full convolutional network and Lidar data (B.Li, T. Zhang, and T.Xia.Vehicle detection from 3d lidar using fully convolutional network. In Robotics: Science and Systems, 2016), Navarro-Serment proposed a method for identifying three-dimensional data based on geometric and motion characteristics (LENavarro-Serment, C. Mertz, and M. Hebert, "Pedestrian detection and tracking using three-dimensional LADAR data," Int. J. Robot. Res., Vol. 29, no. 12, pp. 1516-1528, May 2010.), proposed by Kidono The characteristics of the slice represent the contour of the object at the same height level, which improves the recognition performance of distant objects (K.Kidono, T. Miyasaka, A. Watanabe, T. Naito, and J. Miura, "Pedestrian recognition using high-definition LIDAR, "In Proc. IEEE IV Symp., 2011, pp.405-410.).

It is known to use optical radar (Light detection and ranging system, LIDAR) to detect the distance of obstacles by transmitting electromagnetic radiation. However, although optical radar has a range detection of up to 50 meters, as the detection distance becomes longer, the point Cloud data will also become sparse, making obstacles harder to identify.

In the current literature, there have been quite a few researches on vehicle detection. Visual image-based methods have been developed for many years. Although these methods have had good results under clear image conditions, they may be bad at night. In the weather environment, the low-definition images obtained are often indistinguishable, so it is known that the range of optical radar can be more than 50 meters, but at about 40 meters, the point cloud data has become quite sparse. It is not possible to clearly identify distant objects; as cars can travel at speeds of up to 100 km / h, for autonomous driving, distant car detection is also very important, so failure to identify distant cars may cause serious as a result of.

Therefore, the industry currently needs to develop an optical radar-based automotive detection method that can enhance texture features through image processing to avoid the impact of external harsh environments and the problem of measuring distances too far. It can have both accuracy and cost at the same time, without the need to modify other hardware equipment. At the expense of manufacturing costs, and then strengthen the ability to identify cars, to increase the purpose of car safety.

In view of the shortcomings of the above-mentioned conventional technologies, the main object of the present invention is to provide an optical radar-based vehicle detection method that integrates an optical radar, a three-dimensional point cloud data, an object extraction process, and a three-dimensional and two-dimensional feature extraction. In order to complete the automotive image enhancement processing in optical radar 3D point cloud data.

In order to achieve the above object, according to a solution proposed by the present invention, an optical radar-based vehicle detection method is provided, including: (A) obtaining a three-dimensional point cloud data by an optical radar; (B) obtaining the three-dimensional point cloud The data is subjected to a segmented ground processing to remove the ground information; (C) the 3D point cloud data is subjected to an object extraction process to obtain a 3D point cloud image containing the candidate point cloud group of the car; (D) the 3D point cloud image is processed A two-dimensional mapping process is performed to obtain a two-dimensional map; (E) performing a three-dimensional and a two-dimensional feature extraction on the three-dimensional point cloud map and the two-dimensional map, and using the three-dimensional and two-dimensional features to determine a car position.

In the three-dimensional point cloud data obtained by the optical radar in step (A) above, the X, Y, and Z axes have been subjected to normalization processing to remove repetitive and inconsistent dependencies, protect the data, and let the three-dimensional point cloud data The application is more flexible; the segmentation ground processing in step (B) can use a Random Sample Consensus to remove the ground. The so-called random sample consensus is different from the traditional method, and it is not as much data as possible to obtain an initial solution. Only then is it used to eliminate invalid data points, but like the three-dimensional points in the present invention For cloud data, use random and feasible sample data sets to find the consistency to expand this (ground) set, and then eliminate this set to eliminate the ground information in the 3D point cloud data.

The object extraction process in step (C) can use the distance between the points in the 3D point cloud data for grouping. For example, when the distance between two points is less than 0.2m (but not limited to this), the two points can be marked. For the same group, the candidate point cloud group of pedestrians is screened by the ratio of the length, width, and height of each group.

The two-dimensional mapping process in step (D) may include the following steps: (a) mapping the three-dimensional point cloud image to the two-dimensional image; (b) expanding the two-dimensional image using a binary image to eliminate noise; (c) ) Use the morphological calculation to calculate the outline of each object in the two-dimensional map; (d) Fill the area of each object in the two-dimensional map. The above steps can effectively remove noise and make step (C) screen out The candidate point cloud group of the car is strengthened in the two-dimensional map to make it more obvious.

In step (E), the extracted three-dimensional and two-dimensional features are simultaneously used for classification by machine learning. After considering the three-dimensional and two-dimensional features, the position of the car is further determined.

In step (E), the three-dimensional and two-dimensional features are subjected to deep convolution processing and then subjected to point-wise convolution processing to reduce the parameter amount of the three-dimensional and two-dimensional features.

In step (E), the two-dimensional feature is subjected to deep convolution processing, and then subjected to point-by-point convolution processing to obtain a two-dimensional depth feature. The three-dimensional feature is processed through a three-dimensional convolution network, and then the three-dimensional feature is taken. Convolutional network features become three-dimensional For the depth feature, the three-dimensional depth feature and the two-dimensional feature depth feature can be used to determine the position of the car.

The above summary and the following detailed description and drawings are to further explain the methods, means and effects adopted by this creation to achieve the intended purpose. The other purposes and advantages of this creation will be explained in the subsequent description and drawings.

S101‧‧‧Step (A)

S102‧‧‧Step (B)

S103‧‧‧Step (C)

S104‧‧‧Step (D)

S105‧‧‧Step (E)

The first diagram is a flowchart of an optical radar-based vehicle detection method of the present invention; the second diagram is an embodiment of a two-dimensional mapping process of the present invention; the third diagram is an embodiment of the present invention, a general automobile detection method Result (a) and result (b) of the vehicle detection method of the present invention.

The following is a specific example to illustrate the implementation of this creation. Those who are familiar with this technique can easily understand the advantages and effects of this creation from the content disclosed in this manual.

The present invention uses an optical radar (Light detection and ranging system, LIDAR) to detect the distance of an object by transmitting electromagnetic radiation. Although the optical radar has a range detection of 50 meters, as the detection distance becomes longer, the point cloud data will also The more sparse it is, the more difficult it is to identify obstacles. Because it is at night or in bad weather, the low-resolution images obtained are often not clear. According to experience, the optical radar sensing distance can be as long as 50 meters. Above but big At about 40 meters, the three-dimensional point cloud has become quite sparse, and distant objects cannot be clearly identified. Therefore, one of the main objects of the present invention is to improve the detection of distant places by using only optical radar equipment. Detecting the car's position early in the correct position of the car can give autopilot a sufficient reaction distance, so that the optical radar is not affected by light, and can instantly sense information about the shape and distance of obstacles around the clock, day or night.

Please refer to the first figure, which is a flowchart of an optical radar-based vehicle detection method according to the present invention. As shown in the first figure, the present invention provides an optical radar-based vehicle detection method. The steps include: (A) borrowing Obtain a three-dimensional point cloud data S101 from an optical radar, where the three-dimensional point cloud data includes normalized XYZ axes; step (B) perform a segmented ground processing on the three-dimensional point cloud data to remove ground information S102, where The segmented ground processing uses a Random Sample Consensus to remove the ground; step (C) performs an object extraction process on the 3D point cloud data to obtain a 3D point cloud image including a candidate point cloud group of the car S103 Among them, the object extraction processing uses the distance between points in the 3D point cloud data to perform grouping; step (D) performs a 2D mapping process on the 3D point cloud image to obtain a 2D map S104; step (E) Perform one-dimensional and one-dimensional feature extraction on the three-dimensional point cloud image and the two-dimensional image, and use the three-dimensional and two-dimensional features to determine a car position S105.

Examples

Step 1: Optical radar (LIDAR) data acquisition. In this embodiment, LIDAR is used to obtain three-dimensional point cloud data. After pre-processing, the X, Y, and Z axes are normalized. The normalization process is used to delete repetitive and inconsistent dependencies to protect the data and make the use of 3D point cloud data more flexible. Step 2: Segmentation of the ground. This example uses Random Sample Consensus (RANSAC for short) ) Remove the ground. The so-called random sample consensus is different from the traditional method. It is not to use as much data as possible to obtain an initial solution, and then use it to eliminate invalid data points. Instead, use the three-dimensional point cloud data as in this embodiment. Randomly feasible sample data set, find the consistency to expand the ground set, and then eliminate this ground set, then the ground information in the 3D point cloud data can be eliminated; Step 3: Object extraction processing, this embodiment is based on the point The distance between them is divided into groups, and the KD tree is used as the search method. When the distance between two points is less than 0.2m, the two points are marked as the same group, and then the pedestrian candidate point cloud group is preliminarily filtered by the length, width, and height of each group; Four: classification processing, this embodiment uses the three-dimensional and mapped two-dimensional features extracted feature vector calculation and evaluation, using machine learning to classify Whether it is a car, its implementation can use the well-known two-dimensional features to increase accuracy, group the measured three-dimensional point clouds, use the size of the car to extract candidate point cloud groups, and then extract features from the three-dimensional point clouds. To solve the problem of long distance and sparse points, we map the 3D point cloud to a 2D plane. This process needs to calculate the distance and angle of the point cloud group, and then map the 2D plane to the 2D plane after rotating the point cloud, and then pass the contour. Judging and filling to form a two-dimensional image, and then extracting the corresponding two-dimensional contour and its related two-dimensional features, such as: direction gradient histogram (HOG), Image Feature Extraction (LBP), Haar-like, and finally use machine learning for classification based on traditional 3D features and proposed 2D features; Please refer to the second figure, which is a two-dimensional mapping process implementation history of the present invention. In the above embodiment, the three-dimensional point cloud is mapped to a two-dimensional plane to generate a two-dimensional mapping process. The two-dimensional mapping process includes the following steps. : (A) Map the three-dimensional point cloud map into a two-dimensional map. This process needs to calculate the distance and angle to the point cloud group, and then map the two-dimensional plane after rotating the point cloud. (B) Use binarization. The image expands the two-dimensional image to eliminate noise; (c) calculates the outline of each object in the two-dimensional image using morphological calculations; (d) fills the area (gap) of each object in the two-dimensional image.

As shown in the second figure, the original point cloud is at the top of the figure, and the two-dimensional image information after contour determination and filling is below the figure, and then the corresponding two-dimensional contour and its related two-dimensional features are extracted (for example: HOG, LBP, Haar-like), in order to further improve the accuracy, the present invention proposes to convert the 3D point cloud into volume pixels, first find the bounding boxes of all candidate cars in the image, and then divide them into N * N * N Block, N is 30 in this embodiment, and then calculate the number of points in each block and normalize the number in all blocks to between 0 and 1, thereby turning all blocks into complex stereo pixels, After conversion into volume pixels, the 3D convolutional neural network is used to create this 3D convolutional neural network. The input layer is 30 * 30 * 30 stereo pixels. The number of kernels in each convolutional layer is 30 * 30 and the kernel The size is 5 * 5 * 5, this embodiment extracts 500-dimensional features from the last layer of this 3D convolutional network as deep learning features, and then The two-dimensional map is extracted through a two-dimensional convolutional neural network to extract two-dimensional depth features. The neural network decomposes the standard convolution process into two processes: deep convolution and point-by-point convolution. This conversion process can be Significantly reduce the amount of parameters and calculations. The depth convolution refers to the first convolution of only the images of each channel of the image (for example, the three channels of RGB images). After the depth convolution, the number of channels does not change. Will change (that is, three channels before convolution, or three channels after convolution), and pointwise convolution refers to convolution for each point, and the final convolution is changed by the number of convolution kernels set The number of completed channels does not change the image size. Compared to the standard type of convolution, because the image size is reduced by deep convolution before point-by-point convolution, so all convolution operations can be reduced. The amount of parameters and the amount of calculations are finally combined with traditional three-dimensional, two-dimensional features and deep features using machine learning for classification.

Please refer to the third figure, which is an embodiment of the present invention. The results of the general vehicle detection method (a) and the results of the vehicle detection method (b) of the present invention, as shown in the figure, use only three-dimensional features to determine the car. The point cloud of cars far away will be very sparse. Only using three-dimensional features may not be able to identify distant cars, so that distant cars cannot be detected, as shown in Figure 3 (a); Figure 3 (b) uses the traditional 3D The two-dimensional features and the deep features are classified by the machine detection results generated by machine learning. As shown in the figure, the distant cars can be detected by the detection method of the present invention.

The present invention uses optical radar to develop a "optical radar-based vehicle detection method". This system has great advantages for optical radar to detect distant objects. Improvement can avoid related problems such as inaccuracy when making car judgments solely based on 3D features. In actual smart vehicles, accurate car distance is needed to help remind the driver, and it must work well regardless of day or night. In contrast, the optical radar has a range detection of up to 50 meters, which is enough to provide the processing time for vehicle dynamic judgment when the vehicle speed is increased. The 360-degree layered scan (64 layers) can depict the outline of the objects around the vehicle, which is beneficial. For object recognition and classification, it is quite suitable for outdoor and vehicle detection. The method of the invention can improve the accuracy of optical radar to identify cars. It can be further integrated with RGB-Camera to optimize the system, and it can improve the accuracy. In the future, the application of optical radars in automatic driver assistance systems or industrial related industries can be increased. For example, optical radars can also be used indoors. In large automated factories, the method of the present invention is used to detect working machines. When there are automated machines near multiple machines, When moving the area, the machine can stop automatically immediately to avoid collision.

The above-mentioned embodiments are only for illustrative purposes to explain the features and effects of this creation, and are not intended to limit the scope of the substantial technical content of this creation. Anyone who is familiar with this technique can modify and change the above embodiments without departing from the spirit and scope of the creation. Therefore, the scope of protection of the rights of this creation should be listed in the scope of patent applications described below.

Claims (7)

An optical radar-based vehicle detection method, the steps include: (A) obtaining a three-dimensional point cloud data by an optical radar; (B) subjecting the three-dimensional point cloud data to a segmented ground processing to remove ground information; (C) An object extraction process is performed on the three-dimensional point cloud data to obtain a three-dimensional point cloud image including candidate point cloud groups of the car; (D) performing a two-dimensional mapping process on the three-dimensional point cloud image to obtain a two-dimensional map; (E ) Perform one-dimensional and one-dimensional feature extraction on the three-dimensional point cloud image and the two-dimensional image, and use the three-dimensional and two-dimensional features to determine the position of the car. According to the optical radar-based vehicle detection method described in item 1 of the patent application scope, the two-dimensional mapping process includes the following steps: (a) mapping the three-dimensional point cloud image to the two-dimensional image; (b) using two The valued image expands the two-dimensional map to eliminate noise; (c) calculates the outline of each object in the two-dimensional map using morphological calculations; (d) fills the area of each object in the two-dimensional map. According to the optical radar-based automobile detection method described in the second patent application scope, wherein the two-dimensional feature is selected from one of HOG, LBP, Haar-like, or a mixture thereof. According to the optical radar-based vehicle detection method described in item 3 of the scope of the patent application, in step (E), the three-dimensional and two-dimensional features are used for classification by machine learning to determine the position of the vehicle. According to the optical radar-based vehicle detection method described in item 1 of the scope of patent application, wherein step (E) further includes the two-dimensional feature is subjected to deep convolution processing, and then subjected to point-by-point convolution processing to obtain a 2D depth features. According to the optical radar-based automobile detection method described in item 5 of the scope of patent application, wherein step (E) further includes the three-dimensional feature after being processed by a three-dimensional convolution network, and the characteristics of the three-dimensional convolution network are taken. Become a three-dimensional depth feature. According to the optical radar-based automobile detection method described in the sixth item of the patent application scope, in step (E), the three-dimensional depth feature and the two-dimensional feature depth feature are further used to determine the car position.