CN110097047A - A vehicle detection method using single-line lidar based on deep learning - Google Patents

Abstract

The invention relates to a vehicle detection method based on deep learning using a single-line laser radar, comprising the following steps: 1) collecting feature point cloud data of the vehicle through a single-line laser radar, and performing preprocessing to obtain a binary image; 2) performing a binary image on the binary image Marking, obtaining the location of the vehicle, generating a training data set, and using a clustering method to generate a region of interest; 3) Constructing a deep convolutional neural network and its loss function suitable for target vehicle detection; 4) Carrying out the region of interest After augmentation, it is input into the deep convolutional neural network, and the parameters in the convolutional neural network are trained and updated according to the difference between the output value and the training true value to obtain the optimal network parameters, and according to the trained deep convolutional neural network Carry out vehicle position detection and obtain vehicle position. Compared with the prior art, the invention has the advantages of high robustness, low detection cost and the like.

Description

A vehicle detection method using single-line lidar based on deep learning

technical field

The invention relates to the technical field of intelligent driving, in particular to a vehicle detection method based on deep learning and using single-line laser radar.

Background technique

In the field of intelligent driving, vehicle detection is one of the key tasks to ensure the safe driving of unmanned vehicles. For vehicle detection, 3D lidar or camera is mostly used as the sensor at present, limited by the less data information of single-line lidar, it is rare to use single-line lidar as a vehicle detection sensor. However, the cost of 3D lidar is high, and the point cloud information obtained from the collected data is too much, which requires huge computing resources. Using a camera as a detection sensor can achieve higher detection accuracy, but the camera is greatly affected by light, especially at night, heavy fog, sandstorms, etc. It is difficult to complete the detection of surrounding vehicles due to limited image acquisition.

Contents of the invention

The purpose of the present invention is to provide a vehicle detection method based on deep learning and using single-line laser radar in order to overcome the above-mentioned defects in the prior art.

The purpose of the present invention can be achieved through the following technical solutions:

A vehicle detection method using single-line laser radar based on deep learning, comprising the following steps:

1) Collect the characteristic point cloud data of the vehicle through single-line lidar, and perform preprocessing to obtain a binary image;

2) Mark the binary image, obtain the position of the vehicle in it, generate a training data set, and use a clustering method to generate a region of interest;

3) Construct a deep convolutional neural network and its loss function suitable for target vehicle detection;

4) The region of interest is augmented and then input into the deep convolutional neural network, and the parameters in the convolutional neural network are trained and updated according to the difference between the output value and the training true value to obtain the optimal network parameters, and according to the training A good deep convolutional neural network performs vehicle position detection and obtains the vehicle position.

Described step 1) specifically comprises the following steps:

11) Convert the collected feature point cloud data into a globally unified Cartesian coordinate system with the single-line laser radar as the coordinate origin;

12) Grid the point cloud data after the coordinate transformation, and convert it into a binary image.

In described step 11), conversion expression is:

(x _j0 ,y _j0 )=(x _j1 ,y _j1 )R+t

Among them, (r _j , φ _j ) is the position coordinate of point j in the original point cloud data, (x _j1 , y _j1 ) is the position coordinate of point j transformed into the Cartesian coordinate system with the lidar as the coordinate origin, ( x _j0 , y _j0 ) are the coordinates in the global unified Cartesian coordinate system, R is the transformation rotation matrix, and t is the transformation translation vector.

In the step 2), the annotation content includes the pixel-level annotation of the image and the bounding box annotation of the target vehicle.

The deep convolutional neural network is a Faster R-CNN convolutional neural network, which takes a binary image of a set size as input, and outputs the position and confidence of the target vehicle corresponding to the input binary image.

In described step 3), the expression of the loss function of deep convolutional neural network is:

Loss＝L _cls (p,u)+λ[u＝1]L _loc (t ^u ,v)

L _cls (p,u)=-log(p)

x＝(t ^u -v)

Among them, L _cls (p, u) is the target classification detection loss sub-function, L _loc (t ^u , v) is the distance loss sub-function, p is the predictor for the target category, u is the actual factor of the corresponding category, and λ is The weighted weight of the loss function, when u=1, it means that the target vehicle is in the region of interest, when u=0, it means that the region of interest is the background, t ^u is the predicted position factor, v is the real position factor in the training sample, x is the deviation between the predicted value and the true value.

In the step 2), the clustering methods include density clustering, mean value clustering and Mean-Shift clustering.

In the described step 4), the region of interest is augmented specifically as follows:

The image of the region of interest is randomly flipped horizontally, cropped, and uniformly scaled to a fixed size, and the annotation data is also flipped, cropped, and scaled accordingly.

In the described step 4), the training depth convolutional neural network is specifically:

According to the loss function, the parameters of the deep convolutional neural network are iteratively updated using the gradient descent backpropagation method, and the network parameters obtained after iterating to the maximum set number of times are used as the optimal network parameters to complete the training.

Compared with the prior art, the present invention has the following advantages:

1. High robustness: Since the single-line laser radar can collect accurate vehicle point cloud data under various complex working conditions, supplemented by a vehicle detection algorithm with a high robustness deep learning neural network, this paper The invented vehicle detection method has high robustness, and can guarantee the relative accuracy of detection results even under complex working conditions.

2. Low detection cost: The vehicle detection algorithm proposed in the present invention uses a single-line laser radar as a sensor, and the detection cost is lower than that of a 3D laser radar.

Description of drawings

Fig. 1 is a flowchart of the detection method of the present invention.

Fig. 2 is a schematic diagram of the vehicle deep convolutional network structure in the embodiment of the present invention.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

Example

The present invention provides a method for vehicle detection based on deep learning using single-line laser radar, which uses single-line laser radar as a detection sensor to obtain point cloud data of the vehicle, performs preprocessing and then inputs it to a deep convolutional neural network, and finally obtains the vehicle location and confidence. As shown in Figure 1, the method includes the following steps:

(1) Use single-line lidar to collect vehicle characteristic point cloud data, and preprocess the collected point cloud data;

(2) Manually mark the location of the vehicle in the collected data, and construct the data set used for training

(3) Construct a deep convolutional neural network and loss function suitable for vehicle detection

(4) Use the clustering method to generate the region of interest for the training data obtained in step (2), and simultaneously input the training data and region of interest augmentation into the deep convolutional neural network constructed in step (3). According to the difference between the output value and the training true value, the parameters in the convolutional neural network are trained and updated, and finally the ideal network parameters are obtained.

In the present embodiment, the preprocessing to point cloud data in step (1) includes two following steps such as coordinate conversion and image conversion to point cloud data:

(1-1) The single-line laser radar in this embodiment is located in front of the vehicle in the embodiment, and the single-line laser radar collects surrounding point cloud data at a certain frame rate, and the collection results take the laser radar as the coordinate origin and store them in polar coordinates . Convert the collected point cloud data into Cartesian coordinate system.

The conversion expression is as follows:

In the above formula, (r _j , φ _j ) represents a certain point in the collected original point cloud data, and (x _j1 , y _j1 ) is a certain point in the collected point cloud data converted into a flute with the lidar as the coordinate origin. Representation in the Karl coordinate system.

(1-2) Image processing of point cloud data. In this embodiment, the upper limit of the distance collected by the laser radar is set to 25m, and the collection angle of the single-line laser radar is 180°. The point cloud data after coordinate transformation is gridded, and then the binary image with a size of 250×250 is adjusted. If If there is a data point in the corresponding grid, it is set to 1, otherwise it is set to 0.

In this embodiment, step (2) is to construct a data set required for deep learning training. After processing the collected lidar point cloud data, it is necessary to manually label the training data to form the data set required for training. The labeling methods include but are not limited to pixel-level labeling of images and bounding box labeling of target vehicles. When labeling, at least the position of the target vehicle must be included, but the attitude information of the target vehicle can be expanded and added.

In this embodiment, step (3) is to construct a deep convolutional neural network and a loss function suitable for target vehicle detection. The construction of the deep convolutional neural network is directly related to the training data set prepared in step (2). In this embodiment, step (2) uses the bounding box label of the target vehicle, so in this embodiment, the deep convolutional neural network The structure is similar to that of Fast R-CNN, and the construction of the main structure refers to Fast R-CNN. The convolutional neural network structure is shown in Figure 2.

In this embodiment, in step (3), the loss function of the deep convolutional neural network is composed of two weighted parts:

Loss＝L _cls (p,u)+λ[u＝1]L _loc (t ^u ,v)

(3-1) Construct the target classification loss function L _cls (p,u), where p is the predictor for the target category, and u represents the actual factor of the corresponding category. It is usually constructed using a log loss function, where p represents the predicted probability of a certain category, and the closer p is to 1, the higher the confidence and the smaller the loss.

L _cls (p,u)=-log(p)

(3-2) Construct the target detection distance L _loc (t ^u ,v). Among them, λ represents the weighted weight of the loss function, and λ=1 can usually be taken. [u=1] means 1 is taken when the ROI is the target vehicle, and 0 is taken when the ROI is the background, that is, if the current ROI is something irrelevant to the environment, its distance loss is not considered. The t ^u in the formula represents the predicted position factor, and v represents the real position factor in the training sample. Usually, the distance loss sub-function is constructed with the smooth Manhattan distance formula smooth _L1 (x), and its expression is: where x=(t ^u -v), represents the deviation between the predicted value and the real value.

In this embodiment, the method of generating the region of interest by clustering in step (4) adopts a density clustering algorithm. The specific implementation is to input the point cloud data pair before conversion into the density clustering function, classify the point cloud data pair through the density clustering function, and then convert the point cloud data pair to the corresponding point in the binary image, The binary image points of the same category are surrounded by the smallest rectangular frame, and the obtained image within the rectangular frame is the region of interest. When implementing density clustering, it is necessary to specify the minimum logarithm of data points required by the clusters generated by the clustering, and the domain distance threshold in each type of cluster. In this embodiment, the minimum logarithm of data points is set to 5, and the domain distance The distance threshold is set to 0.5m.

In this embodiment, when the region of interest is input to the deep convolutional neural network, since the size of the region of interest depends on the size of the clustering result, in order to ensure the normal operation of the deep convolutional network, the image data is first convoluted Feature extraction, and then convert the feature image of the region of interest into a uniform size through ROI space pooling.

In this embodiment, the augmentation of the training data in step (4) mainly includes random horizontal flipping, cropping, and uniform scaling of the image to a fixed size, and corresponding flipping, cropping, and scaling of the labeled data. Here Basically, the obtained images are normalized by channel, and the fixed size used in this embodiment is 250×250.

In this embodiment, when initializing the training network model in step (4), first use the SoftMax loss function on ImageNet or other image classification data sets to pre-train the object feature extraction network, and the obtained parameter values are used as the initial parameters of the network. parameter.

In this embodiment, when training the network in step (4), the weighted loss function is used to calculate the comprehensive loss value, and then the gradient is calculated by backpropagation, and the network parameters are updated using an optimizer such as Adam, and a certain number of iterations is used to obtain the final the result of. And the final parameter results are set as the network model parameters of the target vehicle detector for use in detecting the target vehicle.

In a word, the present invention provides a method of target vehicle detection based on deep learning using single-line lidar, which uses single-line lidar as a detection sensor to obtain the point cloud data of the target vehicle, and then input it to the deep convolutional neural network after preprocessing , and finally get the position and confidence of the target vehicle. The detection performance is excellent, and at the same time, it has high robustness, low implementation cost, and is easy to deploy to the existing intelligent parking robot for target vehicle detection.

It is obvious that those skilled in the art can easily make various modifications to these embodiments, and apply the general principles described here to other embodiments without creative efforts. Therefore, the present invention is not limited to the embodiments herein. Improvements and modifications made by those skilled in the art according to the disclosure of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims (9)

1. a kind of vehicle checking method for using single line laser radar based on deep learning, which comprises the following steps:

1) the feature point cloud data of vehicle is acquired by single line laser radar, and is pre-processed to obtain binary picture；

2) to being labeled in binary picture, the wherein position where vehicle is obtained, generates training dataset, and using cluster Method generates area-of-interest；

3) building is suitable for the depth convolutional neural networks and its loss function of target vehicle detection；

4) it is inputted in depth convolutional neural networks after carrying out augmentation to area-of-interest, and according to the difference of output valve and training true value The different parameter in convolutional neural networks is trained update, obtains optimal network parameter, and roll up according to trained depth Product neural network carries out vehicle location detection, obtains vehicle location.

2. a kind of vehicle checking method for using single line laser radar based on deep learning according to claim 1, special Sign is, the step 1) stated specifically includes the following steps:

11) collected feature point cloud data is converted to using single line laser radar as the polar coordinate system of coordinate origin global unified Cartesian coordinate system；

12) to the point cloud data grid gridding after coordinate conversion, binary picture is converted to.

3. a kind of vehicle checking method for using single line laser radar based on deep learning according to claim 1, special Sign is, in the step 11), transformed representation are as follows:

(x_j0,y_j0)=(x_j1,y_j1)R+t

Wherein, (r_j,φ_j) be original point cloud data midpoint j position coordinates, (x_j1,y_j1) it is that point j is converted to and is with laser radar Position coordinates in the cartesian coordinate system of coordinate origin, (x_j0,y_j0) it is the global coordinate unified in cartesian coordinate system, R is Spin matrix is converted, t is translation vector.

4. a kind of vehicle checking method for using single line laser radar based on deep learning according to claim 1, special Sign is, in the step 2), marked content includes that the Pixel-level mark of image and the bounding box of target vehicle mark.

5. a kind of vehicle checking method for using single line laser radar based on deep learning according to claim 1, special Sign is that the depth convolutional neural networks are Faster R-CNN convolutional neural networks, with the binary system being sized Image is as input, to export with position and the confidence level of target vehicle is corresponded on input binary picture.

6. a kind of target vehicle detection method based on deep learning according to claim 1, which is characterized in that described In step 3), the expression formula of the loss function of depth convolutional neural networks are as follows:

Loss=L_cls(p, u)+λ [u=1] L_loc(t^u,v)

L_cls(p, u)=- log (p)

X=(t^u-v)

Wherein, L_cls(p, u) is target classification Detectability loss subfunction, L_loc(t^u, v) and it is range loss subfunction, p is for mesh The predictive factor of classification is marked, u is the practical factor of corresponding classification, and λ is the weighting weight of loss function, indicates feeling as u=1 Interest region is target vehicle, indicates that area-of-interest is background, t as u=0^uFor the location factor of prediction, v is training sample True location factor in this, x are the deviation of predicted value and true value.

7. a kind of target vehicle detection method based on deep learning according to claim 1, which is characterized in that described In step 2), the clustering method includes Density Clustering, mean cluster and Mean-Shift cluster.

8. a kind of target vehicle detection method based on deep learning according to claim 1, which is characterized in that described In step 4), augmentation is carried out to area-of-interest specifically:

Region of interest area image is carried out Random Level overturning, cuts and uniformly zooms to fixed size, and labeled data Also it overturn, cut and is scaled accordingly.

9. a kind of target vehicle detection method based on deep learning according to claim 1, which is characterized in that described In step 4), training depth convolutional neural networks specifically:

According to loss function, declines back-propagation method using gradient, the parameter of depth convolutional neural networks is iterated more Newly, the network parameter obtained after iteration to maximum being set number completes training as optimal network parameter.