CN114299461A - Two-stage-based vehicle illegal lane change identification method - Google Patents

Abstract

The invention discloses a two-stage vehicle violation lane change recognition method, comprising the steps of: starting the vehicle and the camera, and reading the camera image in real time as an input; processing the input image to detect lane lines; and comparing the changes of the base points of the lane lines , to determine whether the vehicle changes lanes and the direction of the lane change, if not, re-read the image for identification, otherwise intercept the image of the lane change lane line and proceed to the next step; apply the lane line type prediction model to the equipment, and carry out Recognition, if the recognition result is one of solid line, solid virtual or double solid line, it is illegal lane change, and an alarm prompt is issued. The invention can identify lane lines, vehicle lane changes and lane line types of lane changes; improves the residual module, improves the recognition accuracy of the algorithm, can quickly complete the detection and identification of lane lines, and facilitates embedded transplant applications.

Description

A two-stage vehicle illegal lane change recognition method

technical field

The invention belongs to the technical field of intelligent transportation, and in particular relates to a two-stage vehicle violation lane change identification method.

Background technique

In recent years, with the increase in the popularity of motor vehicles, the contradiction between the rapid growth of vehicle sales and the lack of road construction infrastructure has appeared in many countries, posing a huge threat to people's lives and property safety. According to statistics, about 1.3 million people die in traffic accidents every year in the world, and 80% of traffic accidents are directly or indirectly caused by driver errors, such as fatigue driving, inattention, bad driving habits, etc. Among them, about 50% of car accidents are caused by vehicles changing lanes illegally.

With the improvement of computer computing power and the acceleration of industrial applications, the field of intelligent transportation has developed rapidly, and the Advanced Driver Assistance System (ADAS) has attracted the attention and research of a large number of workers, and has become an important innovative technology to improve road safety. , In this technology, since lane line detection can not only be used for lane departure warning and lane keeping assist, but also help to identify illegal lane changes and maintain road safety and driver safety, it has become the most basic and most important. One of the challenging jobs.

A large number of methods have been applied to lane line detection, which can be mainly divided into traditional machine learning algorithms and deep learning. Some workers process the input image by modifying the morphological operation to obtain a good effect of lane line detection. Some workers obtain lane outlines by segmenting images by training deep learning models. Some workers use detection algorithms to directly determine the location of lane lines. Due to the lack of data sets for lane type recognition and the lack of lane type recognition algorithms, the existing methods have achieved good results in lane line position detection, but there are fewer lane line type recognition algorithms for lane changes, whether it is illegal The false recognition rate of lane change is high, and it is difficult to apply to embedded.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a two-stage vehicle illegal lane change recognition method, which can improve the illegal lane change recognition rate and effectively maintain road safety. In order to solve the problem of the lack of data sets for lane line type recognition, by arranging public data sets and using existing conditions to record images in actual scenes, a lane line type recognition data set is produced for the training of recognition algorithms. In order to solve the problem of low accuracy of the lane line type recognition algorithm and difficult embedded application, by analyzing the characteristics of the lane line type and the requirements of embedded transplantation, first use the morphological operation to detect the lane line, and judge whether there is a lane change. If there is, extract the front image and enter the lane line recognition; through the performance comparison and the embedded transplant speed requirements, the recognition algorithm selects resnet50 as the basic model, and at the same time, according to the characteristics of the lane line type and the characteristics of the resnet50, the resnet50 is improved to obtain Higher recognition accuracy.

A two-stage illegal lane change identification method proposed by the present invention, the specific execution includes the following implementation steps:

The vehicle and ADAS cameras are started, and the camera images are read in real time as input;

Process the input image to detect lane lines;

Compare the changes of the base point of the lane line to determine whether the vehicle changes lanes and the direction of the lane change, if not, re-read the image for identification, otherwise, intercept the image of the lane change lane line for the next step;

Apply the lane line type prediction model to the equipment to identify the lane change lane line. If the recognition result is one of solid line, solid dotted line or double solid line, it is a violation of lane change and an alarm will be issued.

Further, the processing of the input image to detect lane lines includes:

Perform inverse perspective transformation on the input image to generate a bird's-eye view;

Perform grayscale processing and Gaussian filtering on the bird's-eye view, eliminate noise and interference information in the image to obtain a binary image, and determine the two base points of the lane line by calculating the number of pixel points in each column;

Combined with the sliding window and the base point of the lane line, the pixel coordinates belonging to the left and right lane lines are searched, and then fitted and mapped to the original image.

Further, performing inverse perspective transformation on the input image to generate a bird's-eye view, using the following transformation matrix:

Further, performing grayscale processing and Gaussian filtering on the bird's-eye view, eliminating noise and interference information in the image to obtain a binary image, and determining the calculation of the two base points of the lane line by calculating the number of pixel points in each column. The process is:

Gray=R×0.299+G×0.587+B×0.114

为图像从左到右每列像素为255的个数，l_lbase为左车道线基点，l_rbase为右车道线基点。In the formula, R, G and B are the three channels of the bird's-eye view image respectively, Gray is the processed grayscale image, G(x,y) is the grayscale value after Gaussian filtering, w is the width of the input image,

The number of pixels in each column of the image from left to right is 255, l _lbase is the base point of the left lane line, and l _rbase is the base point of the right lane line.

Further, the method for judging whether the vehicle changes lanes and the direction of the lane change are:

为第二帧图像的左车道线基点，

为第一帧图像的左车道线基点，若第一帧和第二帧基点差值大于阈值β则为右变道，小于-β则为左变道。In the formula,

is the base point of the left lane line of the second frame image,

is the base point of the left lane line of the first frame image. If the difference between the base point of the first frame and the second frame is greater than the threshold β, it is a right lane change, and if it is less than -β, it is a left lane change.

Further, the lane line type prediction model is established through the following steps:

Define the identified lane line type, and obtain the lane line type data set used to train the illegal lane change identification algorithm;

Preprocess the image channels and dimensions to satisfy the model input;

Build training models using convolutional layers, pooling layers, new residual blocks, atrous convolutions, and full connections;

Set the loss function and constraint parameters for calculating the recognition rate and adjusting the weight of the model, where the loss function uses the cross entropy loss function;

The model is trained using the lane line type recognition dataset, and a converged recognition model is obtained after multiple iterations.

Further, the new residual block is formed by adding hole convolution to the residual block, which is used to solve the loss of feature information caused by downsampling and retain rich feature information; the input image is first convolved and maximum pooled to obtain X, and then Input the new residual block, obtain O ^c (X) after convolution processing and O ^k (X) after hole convolution processing, and through skip connection and addition operation, retain richer image information, Complete this module and loop through this module; finally pass through the fully connected layer to output the image category.

The beneficial effects of the present invention are as follows:

The two-stage vehicle illegal lane change recognition method proposed by the present invention comprehensively considers the lane line problem, not only considers the lane line detection and vehicle lane change, but also identifies the lane line type of the lane change, which is helpful to determine whether the vehicle is Change lanes illegally and better maintain road traffic safety.

The two-stage vehicle violation lane change recognition method proposed by the present invention takes into account the characteristics of the lane lines and the characteristics of the existing algorithm comprehensively when recognizing the type of lane lines, improves the residual module of resnet50, and makes full use of the image information, which improves the recognition accuracy of the algorithm.

The first stage of the present invention uses morphological operation and filtering technology, and the second stage uses the resnet50 framework. In general, the detection and identification of lane lines can be quickly completed, and the embedded transplantation application is convenient.

Description of drawings

Fig. 1 is the schematic flow chart of the present invention;

Figure 2 is a schematic diagram of the lane line types contained in the identification data set;

Figure 3 is a schematic diagram of the improved resnet50 residual module;

Figure 4 is a schematic diagram of the lane line detection stage;

FIG. 5 is a schematic diagram of the lane line change of the vehicle changing lanes and the extraction of the lane line recognition area.

Detailed ways

The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way, and any transformation or replacement based on the teachings of the present invention belongs to the protection scope of the present invention.

Step 1: Define the identified lane line type, and obtain the lane line type data set used to train the illegal lane change identification algorithm, as follows:

Create a lane line recognition data set, where the lane line recognition data set contains multiple categories of lane line detection data, each lane line recognition data corresponds to a lane line type, and the number of lane lines is known and determined, refer to Figure 2.

According to the type of lane lines and the performance of the recognition algorithm, the types of lane lines are divided into six categories, that is, the number of recognition types is 6, which are solid line (yellow solid line and white solid line), dashed line (yellow dashed line and white dashed line), double Solid line (double white solid line and double yellow solid line), double yellow dashed line, solid dashed line, dashed solid line. With this as the goal, a data set for illegal lane change recognition is produced by synthesizing the data set publicly available on the Internet and the images obtained by the actual in-vehicle equipment.

Step 2: Establish a lane line type prediction model based on recognition. The establishment process is as follows:

2a: Preprocess the image channels and dimensions to satisfy the model input;

2b: Build the convolution layer, pooling layer, new residual block, hole convolution and full connection used in the training model. After analyzing the lane line type information and the characteristics of the resnet50 residual block, add hole convolution to the classic residual block. Difference block, used to solve the loss of feature information caused by downsampling, and retain more abundant feature information;

As shown in Figure 3, 1×1 convolutional layer, 3×3, rate2 convolutional layer, 3×3, rate4 convolutional layer, 3×3, rate8 convolutional layer and pooling layer are connected in parallel. The 64 1×1 convolutions of the classic residual block in the residual block reduce the 256-dimensional channel to 64-dimensional, and then go through 64 3×3 convolutions, and finally restore to 256 channels through 64 1×1 convolutions , on the other hand, it is connected to the classical residual block by atrous convolution through the 1×1 convolution layer and the Dropout layer in the new residual block. Referring to Figure 3, the input image is first subjected to convolution and maximum pooling to obtain X, and then the residual block proposed by the present invention is input to obtain O ^c (X) after convolution processing and O ^k (X) after hole convolution processing. X), and through skip connection and addition operations, retain richer image information, complete this module and cycle through this module; finally, go through the fully connected layer (not marked in the figure) to output the image category. ResNet is the abbreviation of Residual Network, which is widely used in the field of target classification and a part of the backbone classical neural network for computer vision tasks. The classical residual block in this embodiment is the residual block in ResNet50.

2c: Set the loss function and constraint parameters for calculating the recognition rate and adjusting the weight of the model, where the loss function is to use the cross entropy loss. The mathematical formula of the cross-entropy loss function is as follows:

where y _i represents the label of sample i, positive class is 1, negative class is 0, p _i represents the probability that sample i is predicted to be positive class, and N is the number of samples.

2d: Use the produced lane line type recognition data set to train the model, and obtain a converged recognition model after multiple iterations;

Step 3: The vehicle and ADAS camera are started, and the camera image is read in real time as input;

Step 4: Process the input image to detect lane lines, specifically:

4a: Perform inverse perspective transformation on the input image to generate a bird's-eye view. The transformation matrix is:

4b: Perform grayscale processing and Gaussian filtering on the bird's-eye view, eliminate noise and interference information in the image to obtain a binary image, and determine the two base points of the lane line by calculating the number of pixels in each column. The calculation process is as follows:

Gray=R×0.299+G×0.587+B×0.114

为图像从左到右每列像素为255的个数，l_lbase为左车道线基点，l_rbase为右车道线基点。In the formula, R, G and B are the three channels of the bird's-eye view image respectively, Gray is the processed grayscale image, G(x,y) is the grayscale value after Gaussian filtering, w is the width of the input image,

The number of pixels in each column of the image from left to right is 255, l _lbase is the base point of the left lane line, and l _rbase is the base point of the right lane line.

4c: Combine the sliding window and the base point of the lane line to search for the pixel coordinates belonging to the left and right lane lines, then fit and map to the original image, and mark 1 and 2 respectively from the left to right lane lines.

Step 5: Compare the changes of the base point of the lane line to determine whether the vehicle has changed lanes and the direction of the lane change, if not, return to step 4, otherwise, intercept the image of the lane change lane line and go to step 6. The judgment method is:

为第二帧图像的左车道线基点，

为第一帧图像的左车道线基点，若第一帧和第二帧基点差值大于阈值β则为右变道，小于-β则为左变道。In the formula,

is the base point of the left lane line of the second frame image,

is the base point of the left lane line of the first frame image. If the difference between the base point of the first frame and the second frame is greater than the threshold β, it is a right lane change, and if it is less than -β, it is a left lane change.

Step 6: Apply the detection model to the equipment, identify the lane changing lane lines, and determine whether it is an illegal lane change. If the identification result is a solid line, a solid virtual line, or a double solid line, it is a violation of lane change, and an alarm prompt is issued.

Example

The specific implementation method of this embodiment is as described above, and the specific steps are not described in detail, but the accuracy of lane line type recognition is shown below. The test data of this implementation case comes from the actual scene, and the pictures used are affected by various environments such as lighting, shadows, and standing water.

In order to display the advantages of the present invention more intuitively, the second stage of the present invention is quantitatively analyzed, and the selection criteria are:

In the formula, p is the recognition accuracy, T _i is the number of correct identifications of the i-th type, I _i is the total number of the i-th type, and k is the number of categories.

In the same experimental environment, after taking the average of many experiments, the accuracy of resnet18 is 70%, the accuracy of resnet50 is 77%, and the accuracy of the present invention is 85%, which fully demonstrates the effectiveness and superiority of the present invention.

The beneficial effects of the present invention are as follows:

The two-stage vehicle illegal lane change recognition method proposed by the present invention comprehensively considers the lane line problem, not only considers the lane line detection and vehicle lane change, but also identifies the lane line type of the lane change, which is helpful to determine whether the vehicle is Change lanes illegally and better maintain road traffic safety.

The two-stage vehicle violation lane change recognition method proposed by the present invention takes into account the characteristics of the lane lines and the characteristics of the existing algorithm comprehensively when recognizing the type of lane lines, improves the residual module of resnet50, and makes full use of the image information, which improves the recognition accuracy of the algorithm.

The first stage of the present invention uses morphological operation and filtering technology, and the second stage uses the resnet50 framework. In general, the detection and identification of lane lines can be quickly completed, and the embedded transplantation application is convenient.

As used herein, the word "preferred" means serving as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as advantageous over other aspects or designs. Rather, use of the word "preferred" is intended to present concepts in a specific manner. The term "or" as used in this application is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless specified otherwise or clear from context, "X employs A or B" is meant to naturally include either of the permutations. That is, "X uses A or B" is satisfied in any of the preceding examples if X uses A; X uses B; or X uses both A and B.

Furthermore, although the present disclosure has been shown and described with respect to one implementation or implementation, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular with respect to the various functions performed by the above-described components (eg, elements, etc.), the terms used to describe such components are intended to correspond to any component that performs the specified function of the component (eg, which is functionally equivalent) (unless otherwise indicated), even if not structurally equivalent to the disclosed structures that perform the functions of the exemplary implementations of the present disclosure shown herein. Furthermore, although a particular feature of the present disclosure has been disclosed with respect to only one of several implementations, such feature may be combined with one or other features of other implementations as may be desirable and advantageous for a given or particular application combination. Also, to the extent that the terms "including," "having," "containing," or variations thereof, are used in the detailed description or the claims, such terms are intended to include in a manner similar to the term "comprising."

Each functional unit in this embodiment of the present invention may be integrated into one processing module, or each unit may exist physically alone, or multiple or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. The above-mentioned apparatuses or systems may execute the storage methods in the corresponding method embodiments.

To sum up, the above-mentioned embodiment is an embodiment of the present invention, but the embodiment of the present invention is not limited by the embodiment, and any other changes that deviate from the spirit and principle of the present invention, Modifications, substitutions, combinations, and simplifications should all be equivalent substitutions, which are all included within the protection scope of the present invention.

Claims (7)

1. A two-stage-based vehicle illegal lane change identification method is characterized by comprising the following steps:

starting the vehicle and the ADAS camera, and reading the camera image in real time as input;

processing an input image and detecting a lane line;

comparing the change condition of the lane line base point, judging whether the vehicle changes lane or not and the lane changing direction, if not, reading the image again for recognition, otherwise, intercepting the lane changing lane line image for the next step;

and applying the lane line type prediction model to equipment, identifying the lane change lane line, and if the identification result is one of a solid line, a virtual line or a double solid line, indicating that the lane change is illegal, and sending an alarm prompt.

2. The method for identifying a lane change violation of a vehicle according to claim 1, wherein the processing the input image and the detecting the lane line comprises:

carrying out inverse perspective transformation on the input image to generate a bird's-eye view;

carrying out gray processing and Gaussian filtering on the aerial view, eliminating noise and interference information in the image to obtain a binary image, and determining two base points of the lane line by calculating the number of pixel points in each row;

and searching pixel coordinates belonging to the left lane line and the right lane line by combining the sliding window and the lane line base point, and then fitting and mapping the pixel coordinates to the original image.

3. The two-stage-based lane change identification method for the violation of the vehicle according to claim 2, wherein the input image is subjected to inverse perspective transformation to generate an aerial view, and a transformation matrix is used as follows:

4. the two-stage-based vehicle illegal lane change identification method according to claim 2, wherein the calculation process of performing gray processing and gaussian filtering on the aerial view, eliminating noise and interference information in an image to obtain a binary image, and determining two base points of a lane line by calculating the number of pixels in each row is as follows:

Gray＝R×0.299+G×0.587+B×0.114

wherein R, G and B are the three channels of the bird's eye view image,gray is the processed Gray image, G (x, y) is the gaussian filtered Gray value, w is the width of the input image,

the number of pixels in each column is 255 from left to right, l_lbaseIs a base point of the left lane line,/_rbaseIs a base point of the right lane line.

5. The two-stage-based vehicle violation lane-changing identification method according to claim 2, wherein the method for judging whether the vehicle changes lane and the lane-changing direction is as follows:

in the formula (I), the compound is shown in the specification,

is the left lane line base point of the second frame image,

and if the difference value between the base points of the first frame and the second frame is larger than a threshold value beta, the lane change is performed on the right side, and if the difference value is smaller than-beta, the lane change is performed on the left side.

6. The two-stage-based vehicle violation lane-change identification method according to claim 1, wherein the lane line type prediction model is built by:

defining and identifying lane line types, and acquiring a lane line type data set for training an illegal lane change identification algorithm;

preprocessing the image channel and size to meet the model input;

building a training model by using the convolution layer, the pooling layer, the new residual block, the cavity convolution sum and the full connection;

setting a loss function and a constraint parameter for calculating the recognition rate and adjusting the model weight, wherein the loss function uses a cross entropy loss function;

and training the model by using the lane line type recognition data set, and obtaining a converged recognition model after iteration for multiple times.

7. The two-stage-based vehicle violation lane-changing identification method according to claim 6, wherein the new residual block is formed by adding a hole convolution to the residual block, so as to solve the problem of feature information loss caused by downsampling and retain rich feature information; the input image is firstly convolved and maximally pooled to obtain X, and then the X is input into the new residual block to obtain O after being convolved^c(X) and O subjected to hole convolution processing^k(X) and through skip join and add operations, retaining richer image information, completing the module and cycling through the module; and finally, outputting the image category through a full connection layer.

Images