CN116129327A - Infrared vehicle detection method based on improved YOLOv7 algorithm - Google Patents

Abstract

The invention discloses an infrared vehicle detection method based on an improved YOLOv7 algorithm, which comprises the following steps of; step 1: collecting vehicle videos on a traffic channel, and carrying out frame extraction and image preprocessing to obtain an infrared vehicle image data set; step 2: constructing a new trunk feature extraction network Conv31 containing 31 convolution blocks; step 3: connecting the new trunk feature extraction network with the original predicted network of the YOLOv7 to form a new network model Conv31-YOLOv7; step 4: the training data set obtained in the step 1 is sent to a new network model Conv31-YOLOv7 in the step 3, and a small batch of random gradient descent algorithm is adopted for training, so that a trained infrared vehicle detection model is obtained; step 5: and sending the infrared vehicle video on the traffic road, which is acquired by the infrared thermal imaging equipment in real time, into a trained infrared vehicle detection model according to frames to obtain real-time position information, scale information and confidence of the vehicle. The invention obviously improves the detection accuracy on the premise of ensuring higher detection speed.

Description

An infrared vehicle detection method based on improved YOLOv7 algorithm

Technical Field

The invention belongs to the technical field of vehicle detection, and in particular relates to an infrared vehicle detection method based on an improved YOLOv7 algorithm.

Background Art

Infrared target detection technology refers to the automatic extraction of target location information from infrared images. Given the advantages of infrared thermal imaging, infrared target detection technology can be applied to vehicle detection scenarios on traffic roads and can adapt to conditions such as darkness, strong light, and extreme weather. Therefore, the breakthrough of this technology has important theoretical significance and practical value in fields such as autonomous driving and intelligent transportation.

Traditional infrared vehicle detection methods usually first use methods such as gradient directional histograms to extract target features, and then use positive and negative samples to train classifiers such as support vector machines to classify target features. This method has a slow detection speed and cannot meet the requirements of timeliness. It also has problems such as limited application scenarios, poor robustness, and weak generalization ability.

In recent years, with the rapid development of artificial intelligence technology, infrared vehicle detection methods based on convolutional neural networks have been widely used. It can automatically abstract and extract features from images through convolutional neural networks, and has high detection accuracy and strong robustness.

At present, there are two main types of target detection algorithms based on deep learning. One type is a two-stage detection algorithm, which divides the detection process into two stages. The first stage generates candidate regions of the image to be detected, and the second stage classifies and regresses the generated candidate regions to obtain the final detection results. The first stage of this type of algorithm is relatively time-consuming, and the overall detection accuracy is high, but the detection speed is slow, and generally cannot meet the real-time requirements. Representative algorithms include R-CNN, Fast R-CNN, Faster R-CNN, etc. The other type is a single-stage detection algorithm, which unifies the above two-stage detection process into an end-to-end regression process, combining the two steps of region selection and detection judgment into one. The detection accuracy is low, but the detection speed is fast. Representative algorithms include YOLO and SSD, etc.

The target detection algorithm based on deep learning has good detection effect in visible light image target detection scenarios. However, in infrared target detection scenarios, since infrared images are single-channel images with unclear features, feature extraction of infrared vehicle targets is more difficult. As a result, the detection accuracy of current mainstream target detection algorithms is generally low and it is difficult to meet actual needs.

Summary of the invention

In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide an infrared vehicle detection method based on an improved YOLOv7 algorithm, which significantly improves the detection accuracy while ensuring a high detection speed.

In order to achieve the above object, the technical solution adopted by the present invention is:

An infrared vehicle detection method based on an improved YOLOv7 algorithm comprises the following steps:

Step 1: Collect vehicle videos on traffic roads, perform frame extraction and image preprocessing, and obtain infrared vehicle image datasets;

Step 2: Improve the backbone feature extraction network of the YOLOv7 algorithm, that is, discard the backbone feature extraction network in the YOLOv7 algorithm, build a new backbone feature extraction network Conv31 containing 31 convolutional blocks, and replace the backbone feature extraction network in the YOLOv7 algorithm;

Step 3: Connect the new backbone feature extraction network with the original YOLOv7 prediction network to form a new network model Conv31-YOLOv7;

Step 4: Send the training data set obtained in step 1 to the network model Conv31-YOLOv7 in step 3, and use the small batch stochastic gradient descent algorithm for training to obtain a trained infrared vehicle detection model;

Step 5: The infrared vehicle video on the traffic road collected in real time by the infrared thermal imaging device is sent frame by frame to the trained infrared vehicle detection model to obtain the real-time position information, scale information and confidence of the vehicle.

The steps of frame extraction and image preprocessing in step 1 are specifically as follows:

(1.1) Collect infrared vehicle videos on traffic roads, read the first 10,000 frames of the video, set the resolution of the output image to 640×640, output each frame in sequence in image format, obtain 10,000 infrared vehicle images, and annotate the location information of the vehicle targets in the obtained infrared vehicle images to create an infrared vehicle image dataset. The dataset contains 10,000 infrared vehicle images with a resolution of 640×640;

(1.2) The infrared vehicle image dataset is divided into a training dataset and a test dataset in a ratio of 9:1. That is, 9000 infrared images are randomly selected from the dataset to form the training set, and the remaining 1000 infrared images form the test set.

The step 2 is specifically as follows:

(2.1) Abandon the backbone feature extraction network in the YOLOv7 algorithm and construct a new backbone feature extraction network Conv31 to replace the backbone feature extraction network in the YOLOv7 algorithm. Construct a new backbone feature extraction network Conv31, which contains 31 convolution blocks, and each convolution block has the following structure:

The first convolutional block: contains a convolutional layer with an input channel of 1, an output channel of 32, a kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 32, and a LeakyReLU activation function layer with a complex slope of 0.1;

The second convolutional block: contains a convolutional layer with 32 input channels, 16 output channels, a kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 16 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The third convolutional block: contains a convolutional layer with 16 input channels, 32 output channels, a kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The fourth convolutional block: contains a convolutional layer with 32 input channels, 32 output channels, a kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The fifth and seventh convolution blocks: contain a convolutional layer with 32 input channels, 64 output channels, a kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 64 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The sixth convolutional block: contains a convolutional layer with 64 input channels, 32 output channels, a kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 8th convolutional block: contains a convolutional layer with 64 input channels, 64 output channels, a kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 64 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 9th and 11th convolutional blocks: contain a convolutional layer with 64 input channels, 128 output channels, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 128 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 10th convolutional block: contains a convolutional layer with 128 input channels, 64 output channels, a kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 64 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 12th convolutional block: contains a convolutional layer with 128 input channels, 128 output channels, a kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 128 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 13th and 15th convolutional blocks: contain a convolutional layer with 128 input channels, 256 output channels, a kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 256 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 14th convolutional block: contains a convolutional layer with 256 input channels, 128 output channels, a kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 128 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 16th convolution block, the 19th convolution block, the 21st convolution block, and the 23rd convolution block: contain a convolution layer with an input channel number of 256, an output channel number of 512, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 17th convolutional block, the 20th convolutional block, and the 22nd convolutional block: contain a convolutional layer with an input channel number of 512, an output channel number of 256, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with a channel number of 256, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 18th convolutional block: contains a convolutional layer with 256 input channels, 256 output channels, a kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 256 channels, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 24th convolution block, the 27th convolution block, the 29th convolution block, and the 31st convolution block: contain a convolution layer with an input channel number of 512, an output channel number of 1024, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 1024, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 25th, 28th, and 30th convolution blocks all contain a convolution layer with an input channel number of 1024, an output channel number of 512, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1;

The 26th convolutional block: contains a convolutional layer with an input channel number of 512, an output channel number of 512, a convolution kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1;

(2.2) Connect the above 31 convolution blocks in sequence to obtain a new backbone feature extraction network Conv31 with the following structure:

1st convolution block -> 2nd convolution block -> 3rd convolution block -> 4th convolution block -> 5th convolution block -> 6th convolution block -> 7th convolution block -> 8th convolution block -> 9th convolution block -> 10th convolution block -> 11th convolution block -> 12th convolution block -> 13th convolution block -> 14th convolution block -> 15th convolution block -> 16th convolution block -> 17th convolution block -> 18th convolution block -> 19th convolution block -> 20th convolution block -> 21st convolution block -> 22nd convolution block -> 23rd convolution block -> 24th convolution block -> 25th convolution block -> 26th convolution block -> 27th convolution block -> 28th convolution block -> 29th convolution block -> 30th convolution block -> 31st convolution block;

(2.3) Replace the backbone feature extraction network in the YOLOv7 algorithm with Conv31.

The step 3 is specifically as follows:

Connect the 16th convolutional block in the new backbone feature extraction network Conv31 obtained in step 2 to the first prediction branch of the YOLOv7 prediction network;

Connect the 24th convolutional block in the new backbone feature extraction network Conv31 to the second prediction branch of the YOLOv7 prediction network;

Connect the 31st convolutional block in the new backbone feature extraction network Conv31 to the 3rd prediction branch of the YOLOv7 prediction network.

The internal module connection relationship of the YOLOv7 prediction network is:

The connection relationship between the modules of the first prediction branch is as follows:

16th convolution block->branch convolution block 1->Multi_Concat_Block1->RepConv1->Detection head 1;

The connection relationship between the modules of the second prediction branch is as follows:

24th convolution block->branch convolution block 2->Multi_Concat_Block2->Multi_Concat_Block3->RepConv2->Detection head 2;

The connection relationship between the modules of the third prediction branch is as follows:

31st convolution block->Multi_Concat_Block4->RepConv3->Detection head 3;

The connection relationship between the prediction branch modules is as follows:

31st convolution block -> upsampling convolution block 1 -> upsampling layer 1 -> Multi_Concat_Block2 -> upsampling convolution block 2 -> upsampling layer 2 -> Multi_Concat_Block1;

Multi_Concat_Block1->TransitionBlock1->Multi_Concat_Block2->TransitionBlock2->Multi_Concat_Block4.

The step 4 is specifically as follows:

(4.1) Set the training parameters: the number of training rounds is 200, the number of infrared vehicle images selected for one training is set to 16, the learning rate is set to 0.001, and the confidence threshold and IOU ignore threshold are both set to 0.5;

(4.2) Input 16 images of 9000 infrared vehicle images in the training set into the Conv31-YOLOv7 model at a time. Each time, the output is the offset value ( _tx , _ty , _tw , _th ) of the target bounding box relative to the labeled box and the target confidence p, where _tx is the offset value of the target bounding box relative to the labeled box in the x direction, _ty is the offset value of the target bounding box relative to the labeled box in the y direction, _tw is the offset value of the target bounding box relative to the width of the labeled box, and _th is the offset value of the target bounding box relative to the height of the labeled box;

(4.3) The offset value ( _tx , _ty , _tw , _th ) is calculated using the following coordinate offset formula to obtain the position, width and height of the predicted box:

Among them, _{bx ,} by is the position of the prediction box, _cx , _cy is the position of the annotation box, _bw , _bh are the width and height of the prediction box, _pw , _ph are the width and height of the annotation box;

(4.4) Substitute the position, width, height and confidence of the predicted box (b _x , _by , b _w , b _h , p) and the position, width, height and confidence of the labeled box into the loss function to calculate the loss value, and use the mini-batch stochastic gradient descent algorithm to update its weights;

(4.5) Repeat (4.2)-(4.4) until the loss value stabilizes and no longer decreases, then stop training to obtain a trained infrared vehicle detection model.

The step 5 is specifically as follows:

Infrared thermal imaging equipment is used to collect infrared vehicle videos on traffic roads in real time, and the videos are fed frame by frame into the trained infrared vehicle detection model to obtain the real-time location information and confidence of the vehicle.

Beneficial effects of the present invention:

The present invention replaces the backbone feature extraction network of the YOLOv7 algorithm with a new backbone feature extraction network Conv31, Conv31 contains 31 convolution blocks, each of which contains a convolution layer, a total of 31 convolution layers, and increasing the number of convolution layers to increase the network depth can effectively improve the detection accuracy to a certain extent. The present method increases the number of convolution layers by stacking convolution blocks, which can deepen the network depth, strengthen the feature extraction ability, and effectively improve the detection accuracy; the 16th convolution block, the 24th convolution block and the 31st convolution block of Conv31 can respectively extract the shallow features, middle features and deep features of the infrared vehicle target. By connecting these three convolution blocks with three prediction branches, the fusion of multi-scale features can be realized, and the ability of the network model to detect infrared vehicle targets of different scales can be improved, thereby further improving the detection accuracy. The test results show that compared with other vehicle detection methods based on convolutional neural networks, the present invention can significantly improve the detection accuracy while ensuring a high detection speed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of the implementation of the present invention.

FIG. 2 is a diagram of the Conv31-YOLOv7 network structure constructed in the present invention.

FIG. 3 is a detection schematic diagram of the present invention in an actual scenario.

DETAILED DESCRIPTION

The present invention will be further described in detail below in conjunction with the accompanying drawings.

As shown in Figure 1:

Step 1: Build an infrared vehicle dataset.

(1.1) Collect infrared vehicle videos on traffic roads, read the first 10,000 frames of the video, set the resolution of the output image to 640×640, output each frame in sequence in image format, and obtain 10,000 infrared vehicle images. Annotate the location information of the vehicle targets in the obtained infrared vehicle images to create an infrared vehicle image dataset. The dataset contains 10,000 infrared images with a resolution of 640×640.

(1.2) The infrared vehicle image dataset is divided into a training dataset and a test dataset in a ratio of 9:1. That is, 9000 infrared images are randomly selected from the dataset to form the training set, and the remaining 1000 infrared images form the test set.

Step 2: Build a new backbone feature extraction network.

This step constructs a new backbone feature extraction network based on the improvement of the backbone feature extraction network of the existing YOLOv7 algorithm. The network model in the YOLOv7 algorithm includes a backbone feature extraction network and a prediction network. This step only improves its backbone feature extraction network, which is specifically implemented as follows:

(2.1) Abandon the backbone feature extraction network in the YOLOv7 algorithm and construct a new backbone feature extraction network Conv31 to replace the backbone feature extraction network in the YOLOv7 algorithm. Construct a new backbone feature extraction network Conv31, which contains 31 convolution blocks, each of which has the following structure:

The first convolution block: contains a convolution layer with an input channel of 1, an output channel of 32, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 32, and a LeakyReLU activation function layer with a complex slope of 0.1.

The second convolution block: contains a convolution layer with 32 input channels, 16 output channels, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 16 channels, and a LeakyReLU activation function layer with a complex slope of 0.1.

The third convolutional block: contains a convolutional layer with 16 input channels, 32 output channels, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1.

The fourth convolutional block: contains a convolutional layer with 32 input channels, 32 output channels, a convolution kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 5th and 7th convolutional blocks contain a convolutional layer with 32 input channels, 64 output channels, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 64 channels, and a LeakyReLU activation function layer with a complex slope of 0.1.

The sixth convolutional block: contains a convolutional layer with 64 input channels, 32 output channels, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 8th convolutional block: contains a convolutional layer with 64 input channels, 64 output channels, a convolution kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 64 channels, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 9th and 11th convolutional blocks contain a convolutional layer with 64 input channels, 128 output channels, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 128 channels, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 10th convolutional block: contains a convolutional layer with an input channel number of 128, an output channel number of 64, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with a channel number of 64, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 12th convolutional block: contains a convolutional layer with an input channel number of 128, an output channel number of 128, a convolution kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with a channel number of 128, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 13th and 15th convolutional blocks contain a convolutional layer with an input channel number of 128, an output channel number of 256, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 256, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 14th convolutional block: contains a convolutional layer with an input channel number of 256, an output channel number of 128, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with a channel number of 128, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 16th convolution block, the 19th convolution block, the 21st convolution block, and the 23rd convolution block: contain a convolution layer with an input channel number of 256, an output channel number of 512, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 17th convolution block, the 20th convolution block, and the 22nd convolution block: contain a convolution layer with an input channel number of 512, an output channel number of 256, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with a channel number of 256, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 18th convolutional block: contains a convolutional layer with an input channel number of 256, an output channel number of 256, a convolution kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with a channel number of 256, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 24th convolution block, the 27th convolution block, the 29th convolution block, and the 31st convolution block: contain a convolution layer with an input channel number of 512, an output channel number of 1024, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 1024, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 25th convolution block, the 28th convolution block, and the 30th convolution block all contain a convolution layer with an input channel number of 1024, an output channel number of 512, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1.

The 26th convolutional block: contains a convolutional layer with an input channel number of 512, an output channel number of 512, a convolution kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1.

(2.2) Connect the above 31 convolution blocks in sequence to obtain a new backbone feature extraction network Conv31 with the following structure:

1st convolution block -> 2nd convolution block -> 3rd convolution block -> 4th convolution block -> 5th convolution block -> 6th convolution block -> 7th convolution block -> 8th convolution block -> 9th convolution block -> 10th convolution block -> 11th convolution block -> 12th convolution block -> 13th convolution block -> 14th convolution block -> 15th convolution block -> 16th convolution block -> 17th convolution block -> 18th convolution block -> 19th convolution block -> 20th convolution block -> 21st convolution block -> 22nd convolution block -> 23rd convolution block -> 24th convolution block -> 25th convolution block -> 26th convolution block -> 27th convolution block -> 28th convolution block -> 29th convolution block -> 30th convolution block -> 31st convolution block.

(2.3) Replace the backbone feature extraction network in the YOLOv7 algorithm with Conv31.

Step 3: Build a new network model Conv31-YOLOv7.

Referring to Figure 2, the new backbone feature extraction network is connected to the prediction network of YOLOv7 according to the following structural relationship to form a new network model Conv31-YOLOv7:

Connect the 16th convolutional block in the new backbone feature extraction network Conv31 to the first prediction branch of the YOLOv7 prediction network.

Connect the 24th convolutional block in the new backbone feature extraction network Conv31 to the 2nd prediction branch of the YOLOv7 prediction network.

Connect the 31st convolutional block in the new backbone feature extraction network Conv31 to the 3rd prediction branch of the YOLOv7 prediction network.

Step 4: Train the new network model Conv31-YOLOv7.

(4.1) Set the training parameters: the number of training rounds is 200, the number of infrared vehicle images selected for one training is set to 16, the learning rate is set to 0.001, and the confidence threshold and IOU ignore threshold are both set to 0.5.

(4.2) The 9000 infrared vehicle images in the training set are input into the Conv31-YOLOv7 model 16 at a time. Each time the output is the offset value ( _tx , _ty , _tw , _th ) of the target bounding box relative to the labeled box and the target confidence p, where _tx is the offset value of the target bounding box relative to the labeled box in the x direction, _ty is the offset value of the target bounding box relative to the labeled box in the y direction, _tw is the offset value of the target bounding box relative to the width of the labeled box, and _th is the offset value of the target bounding box relative to the height of the labeled box.

(4.3) The offset value ( _tx , _ty , _tw , _th ) is calculated using the following coordinate offset formula to obtain the position, width and height of the predicted box:

Among them, _bx , _by are the positions of the predicted box, _cx , _cy are the positions of the labeled box, _bw , _bh are the width and height of the predicted box, and _pw , _ph are the width and height of the labeled box.

(4.4) Substitute the position, width, height and confidence of the predicted box ( _bx , _by , _bw , _bh , p) and the position, width, height and confidence of the labeled box into the loss function to calculate the loss value, and use the mini-batch stochastic gradient descent algorithm to update its weights.

(4.5) Repeat (4.2)-(4.4) until the loss value stabilizes and no longer decreases, then stop training to obtain a trained infrared vehicle detection model.

Step 5: Use the trained model to perform infrared vehicle detection.

Infrared thermal imaging equipment is used to collect infrared vehicle videos on traffic roads in real time, and the videos are fed frame by frame into the trained infrared vehicle detection model to obtain the real-time location information and confidence of the vehicle.

The effect of the present invention is further illustrated by the following simulation experiments and measured data:

1. Simulation and test environment

The simulation and actual measurement of the present invention use the Windows 10 operating system, an NVIDIA GeForce GTX2060 GPU for acceleration, and the deep learning framework used is pytorch 1.8.1.

2. Simulation Content

Simulation 1: Use the same training set and parameters as the present invention to train other target detection models based on convolutional neural networks to obtain respective trained infrared vehicle detection models.

1000 test set images are sent one at a time to the trained model of the present invention for testing, and the accuracy and speed of the infrared vehicle detection of the present invention with an IOU threshold of 0.5 are obtained.

Using the same test set as the present invention, the accuracy and speed of infrared vehicle detection of other methods are tested with an IOU threshold of 0.5.

The method of the present invention is compared with the infrared vehicle detection method based on YOLOv7 through simulation experiments. The comparison results are shown in Table 1:

Table 1

Comparing the present invention with YOLOv7, it is found that the present invention method can detect 31 images per second, which is slightly lower than the detection speed of 33 images per second of YOLOv7. The accuracy of the present invention method for infrared vehicle detection with an IOU threshold of 0.5 is 94.36%, and the accuracy of YOLOv7 for infrared vehicle target detection is 91.89%. The detection accuracy of the present invention method is significantly improved compared with YOLOv7, ensuring a higher detection accuracy.

3. Test content

Infrared thermal imaging equipment is used to collect infrared vehicle videos on traffic roads in real time, and the videos are sent frame by frame to the infrared vehicle detection model that has been trained by the present invention to obtain the real-time position information and confidence of the vehicle, as shown in FIG3 .

The large rectangular box in Figure 3 represents the predicted box surrounding the vehicle in the infrared image, and the small rectangular box above the large rectangular box shows the confidence of the vehicle target.

The specific implementation methods described above are only descriptions of the preferred methods of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims (6)

1. An infrared vehicle detection method based on an improved YOLOv7 algorithm, characterized in that it comprises the following steps; Step 1: Collect vehicle videos on traffic roads, perform frame extraction and image preprocessing, and obtain infrared vehicle image datasets; Step 2: Improve the backbone feature extraction network of the YOLOv7 algorithm, that is, discard the backbone feature extraction network in the YOLOv7 algorithm, build a new backbone feature extraction network Conv31 containing 31 convolutional blocks, and replace the backbone feature extraction network in the YOLOv7 algorithm; Step 3: Connect the new backbone feature extraction network with the original YOLOv7 prediction network to form a new network model Conv31-YOLOv7; Step 4: Send the training data set obtained in step 1 to the network model Conv31-YOLOv7 in step 3, and use the small batch stochastic gradient descent algorithm for training to obtain a trained infrared vehicle detection model; Step 5: The infrared vehicle video on the traffic road collected in real time by the infrared thermal imaging device is sent frame by frame to the trained infrared vehicle detection model to obtain the real-time position information, scale information and confidence of the vehicle. 2. An infrared vehicle detection method based on an improved YOLOv7 algorithm according to claim 1, characterized in that the steps of frame extraction and image preprocessing in step 1 are specifically: (1.1) Collect infrared vehicle videos on traffic roads, read the first 10,000 frames of the video, set the resolution of the output image to 640×640, output each frame in sequence in image format, obtain 10,000 infrared vehicle images, and annotate the location information of the vehicle targets in the obtained infrared vehicle images to create an infrared vehicle image dataset. The dataset contains 10,000 infrared vehicle images with a resolution of 640×640; (1.2) The infrared vehicle image dataset is divided into a training dataset and a test dataset in a ratio of 9:1. That is, 9000 infrared images are randomly selected from the dataset to form the training set, and the remaining 1000 infrared images form the test set. 3. According to claim 1, a method for infrared vehicle detection based on an improved YOLOv7 algorithm, characterized in that step 2 is specifically: (2.1) Abandon the backbone feature extraction network in the YOLOv7 algorithm and construct a new backbone feature extraction network Conv31 to replace the backbone feature extraction network in the YOLOv7 algorithm. Construct a new backbone feature extraction network Conv31, which contains 31 convolution blocks, and each convolution block has the following structure: The first convolutional block: contains a convolutional layer with an input channel of 1, an output channel of 32, a kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 32, and a LeakyReLU activation function layer with a complex slope of 0.1; The second convolutional block: contains a convolutional layer with 32 input channels, 16 output channels, a kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 16 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The third convolutional block: contains a convolutional layer with 16 input channels, 32 output channels, a kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The fourth convolutional block: contains a convolutional layer with 32 input channels, 32 output channels, a kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The fifth and seventh convolution blocks: contain a convolutional layer with 32 input channels, 64 output channels, a kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 64 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The sixth convolutional block: contains a convolutional layer with 64 input channels, 32 output channels, a kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 32 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The 8th convolutional block: contains a convolutional layer with 64 input channels, 64 output channels, a kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 64 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The 9th and 11th convolutional blocks: contain a convolutional layer with 64 input channels, 128 output channels, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 128 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The 10th convolutional block: contains a convolutional layer with 128 input channels, 64 output channels, a kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 64 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The 12th convolutional block: contains a convolutional layer with 128 input channels, 128 output channels, a kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 128 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The 13th and 15th convolutional blocks: contain a convolutional layer with 128 input channels, 256 output channels, a kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with 256 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The 14th convolutional block: contains a convolutional layer with 256 input channels, 128 output channels, a kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with 128 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The 16th convolution block, the 19th convolution block, the 21st convolution block, and the 23rd convolution block: contain a convolution layer with an input channel number of 256, an output channel number of 512, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1; The 17th convolutional block, the 20th convolutional block, and the 22nd convolutional block: contain a convolutional layer with an input channel number of 512, an output channel number of 256, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with a channel number of 256, and a LeakyReLU activation function layer with a complex slope of 0.1; The 18th convolutional block: contains a convolutional layer with 256 input channels, 256 output channels, a kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with 256 channels, and a LeakyReLU activation function layer with a complex slope of 0.1; The 24th convolution block, the 27th convolution block, the 29th convolution block, and the 31st convolution block: contain a convolution layer with an input channel number of 512, an output channel number of 1024, a convolution kernel size of 3×3, a stride of 1, and a padding of 1, a batch normalization layer with a channel number of 1024, and a LeakyReLU activation function layer with a complex slope of 0.1; The 25th, 28th, and 30th convolution blocks all contain a convolution layer with an input channel number of 1024, an output channel number of 512, a convolution kernel size of 1×1, a stride of 1, and a padding of 0, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1; The 26th convolutional block: contains a convolutional layer with an input channel number of 512, an output channel number of 512, a convolution kernel size of 1×1, a stride of 2, and a padding of 0, a batch normalization layer with a channel number of 512, and a LeakyReLU activation function layer with a complex slope of 0.1; (2.2) Connect the above 31 convolution blocks in sequence to obtain a new backbone feature extraction network Conv31 with the following structure: 1st convolution block -> 2nd convolution block -> 3rd convolution block -> 4th convolution block -> 5th convolution block -> 6th convolution block -> 7th convolution block -> 8th convolution block -> 9th convolution block -> 10th convolution block -> 11th convolution block -> 12th convolution block -> 13th convolution block -> 14th convolution block -> 15th convolution block -> 16th convolution block -> 17th convolution block -> 18th convolution block -> 19th convolution block -> 20th convolution block -> 21st convolution block -> 22nd convolution block -> 23rd convolution block -> 24th convolution block -> 25th convolution block -> 26th convolution block -> 27th convolution block -> 28th convolution block -> 29th convolution block -> 30th convolution block -> 31st convolution block; (2.3) Replace the backbone feature extraction network in the YOLOv7 algorithm with Conv31. 4. According to a method for infrared vehicle detection based on an improved YOLOv7 algorithm according to claim 1, it is characterized in that the step 3 is specifically: Connect the 16th convolutional block in the new backbone feature extraction network Conv31 obtained in step 2 to the first prediction branch of the YOLOv7 prediction network; Connect the 24th convolutional block in the new backbone feature extraction network Conv31 to the second prediction branch of the YOLOv7 prediction network; Connect the 31st convolutional block in the new backbone feature extraction network Conv31 to the 3rd prediction branch of the YOLOv7 prediction network. 5. An infrared vehicle detection method based on an improved YOLOv7 algorithm according to claim 4, characterized in that the connection relationship of the internal modules of the YOLOv7 prediction network is: The connection relationship between the modules of the first prediction branch is as follows: 16th convolution block->branch convolution block 1->Multi_Concat_Block1->RepConv1->Detection head 1; The connection relationship between the modules of the second prediction branch is as follows: 24th convolution block->branch convolution block 2->Multi_Concat_Block2->Multi_Concat_Block3->RepConv2->Detection head 2; The connection relationship between the modules of the third prediction branch is as follows: The 31st convolution block->Multi_Concat_Block4->RepConv3->Detection head 3; The connection relationship between the prediction branch modules is as follows: 31st convolution block->Upsampling convolution block 1->Upsampling layer 1->Multi_Concat_Block2->Upsampling convolution block 2->Upsampling layer 2->Multi_Concat_Block1; Multi_Concat_Block1->TransitionBlock1->Multi_Concat_Block2->TransitionBlock2->Multi_Concat_Block4. 6. According to claim 1, a method for infrared vehicle detection based on an improved YOLOv7 algorithm, characterized in that step 4 is specifically: (4.1) Set the training parameters: the number of training rounds is 200, the number of infrared vehicle images selected for one training is set to 16, the learning rate is set to 0.001, and the confidence threshold and IOU ignore threshold are both set to 0.5; (4.2) Input 16 images of 9000 infrared vehicle images in the training set into the Conv31-YOLOv7 model at a time. Each time, the offset value ( _tx , _ty , _tw , _th ) of the target bounding box relative to the labeled box and the target confidence p are output, where _tx is the offset value of the target bounding box relative to the labeled box in the x direction, _ty is the offset value of the target bounding box relative to the labeled box in the y direction, _tw is the offset value of the target bounding box relative to the width of the labeled box, and _th is the offset value of the target bounding box relative to the height of the labeled box; (4.3) The offset values ( _tx , _ty , _tw , _th ) are calculated using the following coordinate offset formula to obtain the position, width and height of the predicted box:

Among them, _{bx ,} by is the position of the prediction box, _cx , _cy is the position of the annotation box, _bw , _bh are the width and height of the prediction box, _pw , _ph are the width and height of the annotation box; (4.4) Substitute the predicted box position, width, height and target confidence ( _bx , _by , _bw , _bh , p) and the labeled box position, width, height and target confidence into the loss function to calculate the loss value, and use the mini-batch stochastic gradient descent algorithm to update its weights; (4.5) Repeat (4.2)-(4.4) until the loss value stabilizes and no longer decreases, then stop training to obtain a trained infrared vehicle detection model.

Images