CN111598069B - A deep learning-based method for analyzing the lane-changing area of expressway vehicles - Google Patents

Abstract

The invention discloses a highway vehicle lane change area analysis method based on deep learning, which comprises the steps of firstly carrying out structural modeling on lanes and lane lines of a road; simultaneously detecting the outer frame of the vehicle in the high-definition monitoring video of the highway; tracking the vehicle track in the video according to the detection result of the vehicle in each frame of image; combining the vehicle track with the road structured data, identifying the vehicle lane change according to the lane area where the vehicle passes, and detecting the vehicle lane change position according to the intersection position of the vehicle frame and the lane line; and finally, carrying out clustering analysis on the lane changing positions of the vehicles passing through in different time periods to obtain the lane changing hot spot areas of the vehicles on the expressway in different time periods. The method has the advantages of simple steps and accurate result, detects the lane changing behavior of the vehicle and analyzes the lane changing area by using the camera data in the expressway, and provides powerful support for the fine traffic management and lane design of the expressway.

Description

A deep learning-based method for analyzing the lane-changing area of expressway vehicles

technical field

The invention relates to the field of driving behavior detection, in particular to a deep learning-based lane-changing area analysis method for expressway vehicles.

Background technique

With the rapid economic and social development, people's demand for transportation is increasing, especially for long-distance travel. According to statistics, at the end of 2019, the number of civilian vehicles in the country was 261.5 million, an increase of 21.22 million over the end of the previous year. The contradiction between the growth of traffic demand and the existing road conditions is becoming more and more prominent. As the main traffic artery between cities, the traffic pressure is also increasing day by day. Reasonable traffic management can effectively reduce the occurrence of traffic congestion. In order to improve the level of traffic management, it is necessary to accurately analyze the driving behavior of vehicles on the highway.

Lane changing of highway vehicles is the most common driving behavior. Lane changing refers to the driving behavior of vehicles on the road changing lanes due to needs. Reasonable lane changing on the expressway can increase the speed of the entire traffic flow, relieve traffic pressure and improve road capacity, but too frequent lane changing and unreasonable lane changing will increase the instability of traffic flow and increase traffic flow. danger. In recent years, vehicle lane changing has gradually become an important cause of traffic accidents, and its safety issues have to attract people's attention. Therefore, in order to ensure the driving safety and make full and reasonable use of the road resources of the expressway, it is necessary to detect and analyze the lane-changing behavior of vehicles on the expressway.

According to the analysis of the existing research, the data collected by the expressway mainly include traffic flow, vehicle speed, lane occupancy rate, etc. These data are the macro-state data of traffic, and more micro-traffic data is needed in the fine traffic management. . Among them, vehicle lane-changing information is very important micro-traffic data. In traditional vehicle lane change detection, vehicle lane change parameters are usually obtained by using on-board GPS positioning data. Such methods are limited by the positioning accuracy of GPS, and not all vehicles are equipped with GPS, which makes the detection of lane-changing behavior of expressway vehicles inaccurate and incomplete. Therefore, the present invention provides a deep learning-based expressway vehicle. Lane change area analysis method.

SUMMARY OF THE INVENTION

In order to solve the above problems, the present invention provides a method for analyzing the lane-changing area of expressway vehicles based on deep learning. At the same time, it analyzes based on the detection results to provide support for refined traffic management and lane design. To achieve this purpose, the present invention provides a deep learning-based method for analyzing the lane-changing area of expressway vehicles, which includes the following steps:

(1) According to the surveillance video of the expressway, extract the road background, divide the lane area according to the lanes and lane lines in the road background, and obtain the road structured data;

(2) Use the target detection model based on deep learning to detect the vehicle in each frame of the surveillance video, and obtain the outer frame of the vehicle;

(3) Track the vehicle trajectory in the surveillance video according to the detection result of the vehicle in each frame of image;

(4) Combine the vehicle trajectory with the road structured data, identify the vehicle lane change according to the lane area the vehicle passes through, and detect the vehicle lane change position according to the intersection of the outer frame of the vehicle and the lane line;

(5) Cluster analysis is performed on the lane-changing positions of vehicles passing by in different time periods, and the lane-changing hot spots of expressway vehicles in different time periods are obtained.

As a further improvement of the present invention, the step (1) includes the following sub-steps:

(1.1) For the surveillance video on the highway, select consecutive T frames of video images, superimpose and sum the collected images, and then obtain the average value as the road background. The calculation formula is:

In the formula, P is the road background picture, I _i is the ith frame picture, and T is the number of consecutive frames;

(1.2) Load the road background, draw road lane lines and lane areas on the road background, calculate the minimum circumscribed rectangle of all lane areas as the lane area, and save all the result data in json format to obtain road structured data;

As a further improvement of the present invention, the step (2) includes the following steps:

The step (2) includes the following substeps:

(2.1) According to the lane area obtained in step (1), crop the lane area image from the video frame image;

(2.2) Input the image of the lane area into the trained target detection model based on deep learning, and output the outer frame of the vehicle in the picture after calculation. The target detection model is EfficientDet, which uses the manually marked vehicle detection frame data for training.

As a further improvement of the present invention, in the step (3), the SORT method is used to track the vehicle trajectory in the surveillance video.

As a further improvement of the present invention, the step (4) includes the following substeps:

(4.1) Create a new variable D={key, value} to record the lane number where the vehicle is located, where the key is the vehicle number and the value is the lane number;

(4.2) For the trajectory results of each frame, traverse all vehicles, calculate the lane number where the vehicle is located, and then analyze according to the lane number and vehicle number:

If the lane number does not exist, analyze the next vehicle directly;

If the vehicle number is not in the key of variable D, add the vehicle number and the lane number of the current vehicle to variable D;

If the vehicle number is already in the key of variable D, it is further judged whether the lane number has changed: if the lane number has not changed, the next vehicle is directly analyzed; if the lane number has changed, it is considered that a lane change has occurred;

(4.3) For the vehicle that has changed lanes, use the edge of the lower edge of the outer frame of the vehicle and the lane line to calculate the intersection to obtain the coordinates of the intersection point on the lane line, and use this point as the lane change position point.

As a further improvement of the present invention, in the step (4.2), the lane number where the vehicle is located is calculated, and the specific method is:

Use the trisecting points A and B of the lower edge of the outer frame of the vehicle as the judgment points of the lane where the vehicle is located, and use the ray method to judge whether points A and B are within the polygon of the lane area; if both points A and B are in the same lane In the lane area, the car is considered to be in this lane.

As a further improvement of the present invention, the step (5) includes the following substeps:

(5.1) Divide each lane line into N equal parts, select the vehicle lane changing position data between the time period t1-t2, and count the number of lane changes in each lane and other segments;

(5.2) Use Gaussian filtering to smooth the number of lane changes in each lane and other segments;

(5.3) The three-dimensional analysis space of the lane-changing area is constructed with three elements of time, lane-changing position, and lane-changing times, and the lane-changing area is visually analyzed.

Compared with the prior art, a deep learning-based expressway vehicle lane changing area analysis method of the present invention has the following technical effects:

(1) The present invention uses an efficient and accurate target detection method in the field of deep learning for the detection of highway vehicles, and uses a lightweight front-end interactive method to model road lane lines and lane areas. The combination of the two greatly improves vehicle lane changing. The convenience and accuracy of detection reduces the use of other sensors such as GPS, and belongs to the interdisciplinary application of deep learning algorithms in the field of intelligent transportation.

(2) The present invention can obtain the hotspot areas of the expressway vehicle lane changing in different time periods by performing cluster analysis on the lane changing position, solve the problem that the vehicle lane changing is difficult to analyze in the traditional method, and provide data support for refined traffic management .

Description of drawings

Fig. 1 is the overall flow chart of the method of the present invention;

Fig. 2 is the picture display diagram of 9 consecutive frames in the video;

Figure 3 is a road background map;

Figure 4 shows the result of drawing lane lines and lane areas;

FIG. 5 is a graph of vehicle detection results.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments:

The present invention provides a method for analyzing the lane-changing area of expressway vehicles based on deep learning. The purpose of the present invention is to detect the lane-changing behavior of vehicles based on the deep learning method by using cameras widely deployed on the expressway, and at the same time based on the detection results. Perform analysis to provide support for refined traffic management and lane design.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless defined as herein, are not to be taken in an idealized or overly formal sense. explain.

As an embodiment, a surveillance video on a certain expressway is selected as a data source, and FIG. 2 is an image of 9 consecutive frames in the video.

As shown in Figure 1, a deep learning-based method for analyzing the lane-changing area of expressway vehicles includes the following steps:

(1) For the high-definition surveillance video on the highway, extract the road background, and use the front-end interaction to construct a structured modeling of the road lanes and lane lines, that is, to divide the lane area; it includes the following sub-steps:

(1.1) For the monitored video images, select continuous 100 frames of video images, preferably, there are no vehicles in the continuous 100 frames of video images. The collected pictures are superimposed and summed, and then the average value is taken as the road background. The result is shown in Figure 3. The calculation formula is:

In the formula, P is the road background picture, I _i is the ith frame picture, T is the number of consecutive frames, which is 100 in this embodiment.

(1.2) Load the road background image on the front-end page, draw the road lane line and lane area on the road background image, and the drawing result is shown in Figure 4. At the same time, the minimum circumscribed rectangle of all lane areas is calculated as the lane area, and all the result data are calculated as the lane area. Save in json format.

(2) Use the target detection model based on deep learning to detect the vehicle in each frame of the high-definition surveillance video of the highway, and obtain the outer frame of the vehicle (Bounding box), including the following sub-steps:

(2.1) According to the lane area obtained in step (1.2), the image of the lane area is cropped from the video frame image, so as to reduce the calculation amount of target detection and the data amount of data transmission;

(2.2) Input the image of the lane area into the vehicle detection model based on EfficientDet, and output the outer frame of the vehicle in the picture after calculation. The vehicle detection model uses the manually marked vehicle detection frame data for training in advance, and the model can be provided in the form of API. Call the interface, and the detection results are shown in Figure 5; the vehicle detection model is not limited to EfficientDet, but can also use models such as SSD, Yolo, and Faster RCNN.

(3) According to the detection result of the vehicle in each frame of image, the SORT method is used to track the vehicle trajectory in the video. The SORT method mainly includes the steps of obtaining the adjacency matrix, simple matching, Hungarian matching, and state update. In addition, methods such as Deep SORT can also be used.

(4) Combine the vehicle trajectory with the road structured data, identify the vehicle lane change according to the lane area the vehicle passes through, and detect the vehicle lane change position according to the intersection of the outer frame of the vehicle and the lane line, including the following sub-steps :

(4.1) Create a dictionary variable D={key, value} to record the lane number where the vehicle is located. The key is the vehicle number, which is obtained from the number when the vehicle is detected in step (2), and the value is the lane number;

(4.2) For the trajectory results of each frame, traverse all vehicles, calculate the lane number where the vehicle is located, and then analyze according to the lane number and vehicle number. If the lane number does not exist, analyze the next vehicle directly. If the vehicle number does not exist In the key of variable D, the vehicle number and the lane number where the current car is located are added to variable D. If the vehicle number is already in the key of variable D, it is further judged whether the lane number has changed: if the lane number has not changed, then The next vehicle is directly analyzed, and a lane change is considered to have occurred if the lane number has changed. Among them, for the calculation of the lane number where the vehicle is located, the following method can be used: use the trisecting points A and B of the lower edge of the outer frame of the vehicle as the judgment points of the lane where the vehicle is located, and use the ray method to judge whether points A and B are in the lane. In the polygon of the area, that is, the horizontal ray is made from this point. If there is an odd number of intersections with a polygon of a lane area, it is in the lane area, otherwise it is not in the lane area. If the two points A and B are in the same lane area, then The vehicle is considered to be in this lane.

(4.3) For the vehicle that has changed lanes, use the edge of the lower edge of the outer frame of the vehicle and the lane line to calculate the intersection to obtain the coordinates of the intersection point on the lane line, and use this point as the lane change position point.

(5) Perform a cluster analysis on the lane-changing positions of vehicles passing by in different time periods, and obtain the lane-changing hot spots of expressway vehicles in different time periods, including the following sub-steps:

(5.1) Divide each lane line into N equal parts, select the vehicle lane change position data between time period t1-t2, and count the number of lane changes in each lane and other segments. For example, divide each lane line into 10 equal parts, select the vehicle lane change position data between 8:00-10:00 in the time period, and count the number of lane changes per 10 minutes in each lane and other segments, lane 1 The statistical results of lane change between lanes and 2 are shown in Table 1;

Table 1. Original table of statistics on the number of lane changes

(5.2) Use Gaussian filtering to smooth the number of lane changes in each lane and other segments, and the formula is:

where x is a random variable (in this embodiment, the position index of the Gaussian kernel), μ is the expected value of the Gaussian distribution, and σ is the standard deviation of the Gaussian distribution.

In the smoothing, a 1*3 Gaussian kernel is selected, where μ is 0, σ is 0.8, and the resulting Gaussian kernel is [0.239, 0.522, 0.239]. The results of lane change before lane 1 and 2 after statistical smoothing are shown in Table 2. ;

Table 2 Statistical Gaussian smoothing table for the number of lane changes

(5.3) The three-dimensional analysis space of the lane-changing area is constructed with the three elements of time, lane-changing lane line position, and lane-changing times, and the visual analysis of the lane-changing area is more intuitive.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention in any other form, and any modifications or equivalent changes made according to the technical essence of the present invention still fall within the scope of protection of the present invention. .

Claims (5)

1. A highway vehicle lane change area analysis method based on deep learning is characterized by comprising the following specific steps:

(1) extracting a road background according to a monitoring video of the expressway, and dividing lane areas according to lanes and lane lines in the road background to obtain road structured data;

(2) detecting the vehicle in each frame of image of the monitoring video by adopting a target detection model based on deep learning to obtain the outer frame of the vehicle;

(3) tracking the vehicle track in the monitoring video according to the detection result of the vehicle in each frame of image;

(4) combining the vehicle track with the road structured data, identifying the vehicle lane change according to the lane area where the vehicle passes, and detecting the vehicle lane change position according to the intersection position of the outer frame of the vehicle and the lane line; the method comprises the following substeps:

(4.1) newly establishing a variable D = { key, value }, and recording the lane number of the vehicle, wherein key is the vehicle number, and value is the lane number;

(4.2) traversing all vehicles according to the track result of each frame of picture, calculating the lane number of the vehicle, and analyzing according to the lane number and the vehicle number:

if the lane number does not exist, directly analyzing the next vehicle;

if the vehicle number is not in the key of the variable D, adding the vehicle number and the lane number where the current vehicle is located into the variable D;

and if the vehicle number is already in the key of the variable D, further judging whether the lane number is changed: if the lane number is not changed, directly analyzing the next vehicle, and if the lane number is changed, considering that lane change occurs;

the lane number of the vehicle is calculated, and the specific method comprises the following steps:

using trisection points A and B of the lower edge of an outer frame of the vehicle as judgment points of a lane where the vehicle is located, and judging whether the point A and the point B are in a polygon of a lane area by adopting a ray method; if the two points A and B are in the same lane area, the vehicle is considered to be in the lane;

(4.3) for the vehicles with lane change, performing intersection calculation by using the edge of the lower edge of the outer frame of the vehicle and a lane line to obtain intersection point coordinates on the lane line, and taking the point as a lane change position point;

(5) and carrying out clustering analysis on the lane changing positions of the vehicles passing through in different time periods to obtain the lane changing hot spot areas of the vehicles on the expressway in different time periods.

2. The method for analyzing the lane change area of the highway vehicle based on the deep learning of claim 1, wherein the method comprises the following steps: the step (1) comprises the following substeps:

(1.1) for the monitoring video on the expressway, selecting continuous T frame video pictures, superposing and summing the collected pictures, and then solving an average value as a road background, wherein the calculation formula is as follows:

in the formula (I), the compound is shown in the specification,Pthe picture is a background picture of the road,I _i is as followsiThe number of frames of a picture is,Tis a continuous frame number;

and (1.2) loading a road background, drawing a road lane line and lane areas on the road background, and calculating the minimum circumscribed rectangle of all the lane areas as the lane areas.

3. The method for analyzing the lane change area of the highway vehicle based on the deep learning of claim 1, wherein the method comprises the following steps: the step (2) comprises the following substeps:

(2.1) cutting out a lane area image from the video frame image according to the lane area obtained in the step (1);

and (2.2) inputting the lane area image into a trained target detection model based on deep learning, calculating and outputting the outer frame of the vehicle in the picture, wherein the target detection model is EfficientDet, and training by using the manually marked vehicle detection frame data.

4. The method for analyzing the lane change area of the highway vehicle based on the deep learning of claim 1, wherein the method comprises the following steps: in the step (3), the vehicle track in the monitoring video is tracked by adopting an SORT method.

5. The method for analyzing the lane change area of the highway vehicle based on the deep learning of claim 1, wherein the method comprises the following steps: the step (5) comprises the following substeps:

(5.1) dividing each lane line into N equal parts, selecting lane change position data of vehicles in a time period of t1-t2, and counting lane change times in equal segments of each lane;

(5.2) smoothing the number of conversion passes in each lane equal segment by using Gaussian filtering;

and (5.3) constructing a three-dimensional analysis space of the lane change region by using three elements of time, lane change position and lane change number, and performing visual analysis on the lane change region.

Images