CN117333845A - Real-time detection method for small target traffic sign based on improved YOLOv5s - Google Patents

Abstract

The invention discloses a real-time detection method for a small target traffic sign based on improved YOLOv5s, which specifically comprises the following steps: (1) creating a traffic sign image dataset; (2) adding annotation information to the image in the dataset; (3) Constructing a small target traffic sign real-time detection model based on improved YOLOv5 s; (4) employing a new loss function; (5) training the model using the training set and the verification set; (6) And testing the model by adopting a test set, and obtaining a final small target traffic sign real-time detection model. Compared with the prior art, the real-time detection method for the small target traffic sign based on the improved YOLOv5s can effectively improve the detection precision of the small target traffic sign and greatly improve the real-time performance of traffic sign detection.

Description

A real-time detection method of small target traffic signs based on improved YOLOv5s

Technical field

The invention relates to the field of artificial intelligence, and specifically relates to a real-time detection method of small target traffic signs.

Background technique

With the continuous development of autonomous driving technology, Traffic Sign Recognition (TSR) has become one of the key components of the Advanced Driver Assistance Systems (ADAS). TSR can efficiently and accurately identify images captured by cameras and provide drivers with instructions, warnings and other information in real time. In the actual road driving process, the proportion of the image occupied by traffic signs when they first enter the field of view is usually less than 1%, which belongs to small target detection. Small target traffic signs have problems such as blur and unclear details in the image. Therefore, it is difficult to detect small target traffic signs. The rapid and accurate recognition of target traffic signs is of great significance to the development of assisted driving systems.

Traditional traffic sign detection methods mainly focus on color segmentation, feature extraction of shapes, contours and other features, and then complete feature classification through classifiers to achieve detection and recognition of road traffic signs. Traditional traffic sign detection algorithms use manual methods to extract features, and then use these features as input to the algorithm. These detection algorithms have high dimensions, large amounts of calculation, and are time-consuming and labor-intensive. There are real-time, stability, and accuracy issues in the traffic sign detection process. Not good.

With the rapid development of deep learning, improving the target detection technology of convolutional neural models has become a current research hotspot. Convolutional Neural Network (CNN) is a mainstream deep learning algorithm, including R-CNN, Fast-R-CNN, Fast-R-CNN, Mask-R-CNN, Alex Net, YOLO, etc. Model. These models have achieved good results in target detection in different application scenarios. Target detection algorithms can be divided into two categories: two-stage detection algorithms that use region candidate frames and single-stage detection algorithms that do not use region candidate frames. The two-stage algorithm has high accuracy but slow detection speed. The improved algorithm improves the speed to a certain extent. However, due to the setting of the candidate area, the two-stage model cannot better meet the real-time detection requirements of small target traffic signs during high-speed driving. Therefore, a single-stage algorithm is usually used in traffic sign recognition. In terms of single-stage target detection algorithm, although the real-time performance is improved compared to the two-stage detection algorithm, the accuracy is lower. Especially in the recognition and detection of small target traffic signs, the detection accuracy is not high, which can easily lead to missed detections and false detections. Moreover, due to the large number of model parameters and high computational complexity, the real-time detection performance is not good. Migrating the model to mobile devices has the disadvantage of It must be difficult.

Contents of the invention

In view of the problems existing in the existing technology, the present invention provides a real-time detection method of small target traffic signs based on improved YOLOv5s. By improving the YOLOv5s target detection model, utilizing feature enhancement and adopting the NWD measurement method, the detection accuracy of small target traffic signs is solved. It is not high, has poor real-time performance, has a large number of model parameters, has high computational complexity, and is not suitable for deployment on mobile devices.

The technical solution provided by the invention includes the following steps:

Step 1: Obtain traffic sign images to form the first data set;

Step 2: Add annotation information to the images in the first data set to form a second data set, and divide the second data set into a training set, a verification set and a test set;

Step 3: Build a traffic sign detection model based on improved YOLOv5s;

Step 4: Use a new loss function;

Step 5: Use the training set and verification set to train the traffic sign detection model based on improved YOLOv5s;

Step 6: Use the traffic sign test set to test the optimal model obtained through training, and obtain the final small target traffic sign real-time detection model.

Further, in step 1, the traffic sign images in the first data set can be captured by a digital camera, collected from the Internet, or obtained from surveillance videos.

Preferably, in step 2, the training set, verification set and test set can be divided according to the ratio of 7:2:1.

Further, the step 3 specifically includes step 3.1 to step 3.3:

Step 3.1: Combine the C3 module and ConvMixer in the YOLOv5s model to form the CSPCM module, and use the CSPCM module to replace the C3 module at the last layer of the YOLOv5s backbone network and the C3 module at the last layer in the neck network respectively;

Step 3.2: Replace the remaining C3 modules in the backbone network and neck network of the YOLOv5s model with the lightweight convolution module C3_Faster, that is, use C3_Faster to replace the last layer of C3 in the backbone network and neck network except the ones that have been replaced in step 3.1. The remaining C3 modules outside the module;

Step 3.3: The output layer adds a small target detection head to the existing three detection heads.

Further, in step 4, the NWD metric method is introduced into the CIoU loss function, and the NWD metric is used to optimize the CIoU loss function of YOLOv5s. The optimized loss function formula is:

L＝(1-β)*(1-NWD(N _a , N _b ))+β*(1-CIoU) (1)

In formula (1), L is the optimized loss function, NWD is the normalized Wasserstein distance, N _a and N _b are given by and/> Modeled Gaussian distribution, a represents the real box, b represents the predicted box, cx _a , cy _a represents the center point coordinates of the real box, w _a , _ha represents the width and height of the real box; cx _b , cy _b represents prediction The center point coordinates of the box, w _b and h _b represent the width and height of the prediction box; β is the weight proportion coefficient, CIoU is the loss function in the original YOLOv5s, and its calculation formula is:

In formula (2), ρ ² (b _A ,b _B ) represents the Euclidean distance between the center point of the real box and the prediction box, c represents the diagonal distance of the minimum limiting rectangle between the prediction box and the real box, α is the weight factor, v is the aspect ratio consistency, and IoU is the overlap ratio between the real box and the predicted box.

Further, the step 5 specifically includes step 5.1 to step 5.4:

Step 5.1: Set the training parameters of the real-time detection model of small target traffic signs based on improved YOLOv5s. The model training parameters specifically include: learning rate, momentum, weight attenuation, optimizer, number of iteration rounds, and batch size;

Step 5.2: Input the training set and verification set images and corresponding labels into the improved YOLOv5s small target traffic sign real-time detection model, use the back propagation algorithm to calculate the gradient of the loss function on the model parameters, and adjust the model by minimizing the loss function parameters so that it gradually approaches the optimal solution;

Step 5.3: Use the optimizer to update the model parameters in the direction of gradient descent; until the loss functions of the training set and verification set no longer decrease, and the evaluation indicators such as accuracy P, recall R, and mAP no longer decrease. improve;

Step 5.4: Save the trained model parameters as the optimal model.

Further, the step 6 specifically includes step 6.1 to step 6.3:

Step 6.1: Input the test set into the improved optimal model described in step 5;

Step 6.2: Calculate model performance indicators: Performance indicators specifically include accuracy P (Precision), recall rate R, mAP, number of parameters, computational complexity (GFLOPs), and model size. The specific formula is as follows:

Among them, P is the precision rate, R is the recall rate, mAP is the average precision rate mean of all categories, AP is the average precision rate, m is the total number of traffic sign categories, TP represents the number of positive samples that are correctly recognized as positive samples, FP represents the number of negative samples that are incorrectly identified as positive samples, and FN represents the number of positive samples that are incorrectly identified as negative samples;

Step 6.3: When the performance indicators meet the accuracy requirements, obtain the final real-time detection model of small target traffic signs based on improved YOLOv5s.

Compared with the prior art, the beneficial effects of the present invention are:

The invention discloses a real-time detection method of small target traffic signs based on improved YOLOv5s. By introducing the CSPCM module and adding a small target detection head, the detection accuracy of small target traffic signs is improved; the C3_Faster network structure is used to reduce model parameters and calculation complexity. degree, which improves the real-time performance of traffic sign detection; the NWD measurement method is introduced in the CIoU loss function, and a new loss function is adopted, which effectively avoids large losses caused by small targets and accelerates the convergence of the model.

Description of drawings

Figure 1 is a flow chart of the real-time detection method of small target traffic signs based on improved YOLOv5s according to the present invention;

Figure 2 is a schematic structural diagram of the model structure of the real-time detection method of small target traffic signs based on improved YOLOv5s according to the present invention;

Figure 3 is a schematic diagram of the CSPCM structure;

Figure 4 is a schematic diagram of the C3_Faster structure;

Detailed ways

In order to make the technical solutions, structural features, achieved objectives and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with specific embodiments and accompanying drawings. It should be noted that the specific embodiments described here are only used to explain the present invention more clearly and are not used to limit the present invention.

Figure 1 is a flow chart of a real-time detection method for small target traffic signs based on improved YOLOv5s disclosed by the present invention. The implementation process is as follows:

Step 1: Obtain traffic sign images to form a first data set; the traffic sign images in the first data set can be photographed by a digital camera, collected from the Internet, or obtained from surveillance videos;

In this embodiment, in order to better evaluate the detection effect of a real-time detection method for small target traffic signs based on improved YOLOv5s disclosed in the present invention, the public data set TT100K (Tsinghua-Tencent 100K) traffic sign data set was used. Using the TT100K data set, the data set needs to be preprocessed: First, count the traffic sign categories that appear less than 200 times in the TT100K data set, delete the labels corresponding to the traffic sign categories that appear less than 200 times, and the labels that only contain this category. The pictures are finally made into a data set containing 36 types of traffic signs, which forms the first data set.

Step 2: Use labelme annotation tool to add annotation information to the images in the first data set. Since the public data set TT100K used in this embodiment already has annotation information, this step is skipped. Convert the json format label file in the first data set described in this embodiment into the txt format label file required by YOLOv5s to form a second data set, and divide the second data set according to 7:2:1 for training Set, verification set and test set; the divided training set includes 6598 pictures, the test set includes 1889 pictures, and the verification set includes 970 pictures. The image size in the second data set described in this embodiment is 640*640.

Step 3: Construct a traffic sign detection model based on improved YOLOv5s. The improved YOLOv5s model structure is shown in Figure 2. The model construction process specifically includes steps 3.1 to 3.3:

Step 3.1: Replace the Bottleneck module in the C3 module with the ConvMixer module to form a new convolution module CSPCM; replace the C3 module of the last layer of Backbone and the C3 module of the last layer of Neck with the new convolution CSPCM module;

Further, the structure of the ConvMixer module is shown in Figure 3. It operates in a mixed manner of separating space and channel dimensions, and maintains the same size and resolution throughout the model; in ConvMixer, the input is first passed through a Depthwise Conv( Dwconv), that is, a grouped convolution with the number of groups equal to the number of channels; and then through a PointwiseConv (PWCONV), which is a 1×1 convolution; after each convolution operation, it is followed by an activation function z′ ₁ and a z _{1+ 1} BatchNorm layer. This design can effectively reduce the amount of model parameters, improve the real-time performance of the model, and effectively improve the model's feature expression ability.

Step 3.2: Use the lightweight convolution module C3_Faster to replace the remaining C3 modules in the backbone network and neck network of the YOLOv5s model, that is, use C3_Faster to replace the C3 modules in the last layer of the backbone network and neck network that have been replaced in step 3.1. The remaining C3 modules;

The lightweight convolution module C3_Faster uses Pconv convolution; compared with depth convolution (DWConv) and group convolution (GConv), PConv can reduce the calculation amount of FLOPs, reduce computational redundancy and memory access, Figure 4 shows C3_Faster The structure consists of a PConv layer followed by 2 Conv layers, which are shown together as an inverted residual block, the intermediate layer has an extended number of channels, and after each intermediate Conv, there is a normalization and activation layer.

Step 3.3: The output layer adds a small target detection head to the existing three detection heads, which can process multi-scale feature information more efficiently, make the detector more sensitive to small targets, and improve the detection of small target traffic signs. Detection performance.

Step 4: Use a new loss function, introduce the NWD metric method into the CIoU loss function, combine CIoU with the NWD metric, and propose a new loss function; the new loss function can effectively avoid large losses caused by small targets. , to speed up the convergence of the model. The specific steps include steps 4.1 to 4.3:

Step 4.1: Model the bounding box as a two-dimensional Gaussian distribution, and use the Gaussian distribution corresponding to the predicted target and the actual target to calculate the Wasserstein distance between the predicted box and the real box. The calculation formula is:

In formula (1), is the Wasserstein distance, N _a and N _b are given by/> and Modeled Gaussian distribution, a represents the real box, b represents the predicted box, cx _a , cy _a represents the center point coordinates of the real box, w _a , _ha represents the width and height of the real box; cx _b , cy _b represents prediction The center point coordinates of the box, w _b , h _b represent the width and height of the prediction box;

Step 4.2: Calculate the normalized Wasserstein distance between them. The calculation formula is:

In formula (2), NWD (N _a , N _b ) is the normalized Wasserstein distance, and C is a constant closely related to the data set. Step 4.3: Based on the proportional relationship between CIoU and NWD, a new loss function is proposed, and its calculation formula is:

L＝(1-β)*(1-NWD(N _a , N _b ))+β*(1-CIoU) (3)

In formula (3), L is the optimized loss function, β is the weight proportion coefficient, CIoU is the loss function in the original YOLOv5s, and its calculation formula is:

In formula (4), ρ ² (b _A , b _B ) represents the Euclidean distance between the center point of the real frame and the prediction frame, c represents the diagonal distance of the minimum limiting rectangle between the prediction frame and the real frame, α is the weight factor, v is the aspect ratio consistency, and IntersectionoverUnion (IoU) is the object detection method used to measure the overlap between the predicted bounding box and the real bounding box. The higher the IoU value, the greater the overlap between the predicted frame and the real frame, and the better the detection effect. The IoU calculation formula is:

In formula (5): B is the prediction box; B ^GT is the real box.

Step 5: Input the training set and verification set into the small target traffic sign model based on improved YOLOv5s described in step 3 for training, specifically including steps 5.1 to 5.4:

Step 5.1: Set the training parameters of the small target traffic sign real-time detection model based on improved YOLOv5s. The model training parameters include: learning rate, momentum, weight attenuation, optimizer, number of iteration rounds, and batch size;

In this embodiment, the optimizer is SGD, the initial learning rate 1r0 is 0.01, the momentum is 0.937, the weight decay weight_decay is 0.0005, the batch size is 16, and the iteration round Epoch is 300.

Step 5.2: Input the training set and verification set images and corresponding labels into the improved YOLOv5s small target traffic sign real-time detection model, and use the back propagation algorithm to calculate the gradient of the loss function on the model parameters. The backpropagation algorithm is an efficient method for calculating gradients. It uses the chain rule to calculate the gradient of each parameter with respect to the loss function. Specifically, backpropagation calculates the gradient of each parameter layer by layer by propagating the loss function backward from the output layer. In this process, the gradient of each parameter represents the rate of change of the loss function for that parameter, that is, how the loss function changes as the parameter changes. Adjust the model parameters by minimizing the loss function to gradually approach the optimal solution.

Step 5.3: After calculating the gradients of the model parameters, use the optimizer to update these parameters. The optimizer updates parameters in the opposite direction of the gradient based on the gradient information of the parameters. Parameters with larger gradients are updated in larger steps, while parameters with smaller gradients are updated in smaller steps. By continuously updating the model parameters iteratively, the value of the loss function can be gradually reduced. By minimizing the loss function, the values of the model parameters are adjusted so that they gradually approach the optimal solution, that is, the parameter value at which the loss function reaches the minimum value, the difference between the model's prediction results and the true value is minimized, and at the same time, mAP, recall Evaluation indicators such as rate R and accuracy P also no longer improve.

Step 5.4: Save the trained model parameters as the optimal model.

Step 6: Use the test set to test the optimal model described in step 5, evaluate the test set test results, and meet the accuracy requirements, that is, obtain the final real-time detection model of small target traffic signs based on improved YOLOv5s. Specifically, The step 6 further includes steps 6.1 to 6.3:

Step 6.1: Input the test set into the optimal model described in step 5;

Step 6.2: Calculate model performance indicators: Performance indicators specifically include accuracy P, recall rate R, mAP, number of parameters, computational complexity GFLOPs, and model size. The specific calculation formula is as follows:

Among them, P is the precision rate, R is the recall rate, mAP is the average precision rate mean of all categories, AP is the average precision rate, m is the total number of traffic sign categories, TP represents the number of positive samples that are correctly recognized as positive samples, FP represents the number of negative samples that are incorrectly identified as positive samples, and FN represents the number of positive samples that are incorrectly identified as negative samples;

Step 6.3: When the performance indicators meet the accuracy requirements, obtain the final real-time detection model of small target traffic signs based on improved YOLOv5s.

In this embodiment, in order to verify the improved model effect disclosed in the present invention, the test set is input into the small target traffic sign real-time detection model based on improved YOLOv5s and the YOLOv5s model is tested. The evaluation indicators are calculated separately for the two models, and the evaluation indicator data are shown in Table 1. As can be seen from Table 1, the real-time detection model of small target traffic signs disclosed in the present invention has achieved higher detection accuracy than the original YOLOv5s model in terms of accuracy P, mAP@0.5 and other indicators. In terms of performance indicators such as model parameter quantity, model size, and computational complexity GFLOPs, the improved model disclosed in the present invention is lower than the original YOLOv5s model, so that the model has better lightweight indicators and is easier to deploy on mobile devices.

Table 1 Comparative experimental results

The above is only an embodiment of the present invention, and does not limit the patent scope of the present invention. For those skilled in the art, the present invention can have various modifications and changes, all within the spirit and principles of the present invention. , any modifications, equivalent substitutions, improvements, etc. shall be included in the protection scope of the present invention.

Claims (3)

1. The real-time detection method for the small target traffic sign based on the improved YOLOv5s is characterized by comprising the following steps of:

step 1: acquiring a traffic sign image to form a first data set; the traffic sign images in the first data set can be shot by a digital camera, collected and obtained from a network or obtained from a monitoring video;

step 2: adding annotation information to the images in the first data set to form a second data set, and dividing the second data set into a training set, a verification set and a test set;

step 3: constructing a small target traffic sign real-time detection model based on improved YOLOv5s, wherein the construction of the model further comprises the steps of 3.1 to 3.3:

step 3.1: combining a C3 module in the Yolov5s model with ConvMixer to form a CSPCM module, and respectively replacing the C3 module of the last layer of the Yolov5s backbone network and the C3 module of the last layer of the neck network with the CSPCM module;

step 3.2: replacing the rest of C3 modules in the Yolov5s model backbone network and the neck network by using a lightweight convolution module C3_Faster, namely replacing the rest of C3 modules except the last layer of C3 modules in the backbone network and the neck network which are replaced in the step 3.1 by using C3_Faster;

step 3.3: the output layer is provided with a small target detection head based on the existing 3 detection heads;

step 4: the new loss function is adopted, and the specific method is as follows:

introducing an NWD measurement method into the CIoU loss function, optimizing the CIoU loss function of Yolov5s by using the NWD measurement, wherein the optimized loss function formula is as follows:

L＝(1-β)*(1-NWD(N _a ,N _b ))+β*(1-CIoU) (1)

NWD is normalized Wasserstein distance, N _a ,N _b Is composed ofAnd-> The modeled gaussian distribution, a represents the true box, b represents the predicted box, cx _a ,,cy _a Representing the coordinates of the center point, w, of the real frame _a 、h _a Representing the width and height of the real frame; cx (cx) _b ,cy _b Representing the coordinates of the central point of the prediction frame, w _b ,h _b Representing the width and height of the prediction box; beta is a weight proportionality coefficient, CIoU is a loss function in original YOLOv5s, and the CIoU calculation formula is:

in the formula (2), ρ ² (b _A ,b _B ) Representing the center points of the real frames and the predicted framesEuclidean distance between, c represents the diagonal distance of the smallest defined rectangle of the predicted and real frames, α is the weight factor, v is the aspect ratio uniformity, ioU is the overlap ratio between the real and predicted frames;

step 5: training the small target traffic sign real-time detection model based on the improved YOLOv5s by using the training set and the verification set, and saving the trained model as an optimal model, and further comprising the steps of 5.1 to 5.4:

step 5.1: setting the small target traffic sign real-time detection model training parameters based on the improved YOLOv5s, wherein the model training parameters comprise: learning rate, momentum, weight decay, optimizer, iteration round number, batch size;

step 5.2: inputting the training set and verification set images and the corresponding labels into the small target traffic sign real-time detection model of the improved YOLOv5s, calculating the gradient of a loss function to model parameters by using a back propagation algorithm, and adjusting the model parameters to gradually approach an optimal solution by minimizing the loss function;

step 5.3: updating model parameters by using an optimizer SGD (generalized algorithm D), so that the model parameters are updated towards the gradient descending direction until the loss functions of the training set and the verification set are not reduced any more, and the evaluation index mAP, the recall rate R and the accuracy rate P are not improved any more;

step 5.4: saving the trained model parameters as an optimal model;

step 6: and testing the optimal model by adopting the test set, evaluating the test result of the test set, and meeting the precision requirement to obtain the final real-time detection model of the small target traffic sign based on the improved YOLOv5 s.

2. The real-time detection method of small target traffic sign based on improved YOLOv5s according to claim 1, wherein in the step 2, the training set, the verification set and the test set are divided according to a ratio of 7:2:1.

3. The real-time detection method of small target traffic sign based on improved YOLOv5s according to claim 1, wherein the step 6 further comprises steps 6.1 to 6.3:

step 6.1: inputting the test set into the optimal model in the step 5;

step 6.2: calculating a model performance index: the accuracy P, the recall R, mAP, the parameter quantity, the calculation complexity GFLOPs and the model size are calculated according to the following specific calculation formulas:

wherein P is the accuracy, R is the recall, mAP is the average accuracy average of all the categories, AP is the average accuracy, m is the total number of categories of traffic sign, TP represents the number of positive samples correctly identified as positive samples, FP represents the number of negative samples incorrectly identified as positive samples, and FN represents the number of positive samples incorrectly identified as negative samples;

step 6.3: and when the performance index meets the precision requirement, obtaining a small target traffic sign real-time detection model based on the improved YOLOv5 s.