CN111985451A - A UAV scene detection method based on YOLOv4 - Google Patents

Abstract

The invention provides a YOLOv4-based UAV scene detection method, comprising: S1, establishing a proprietary data set; dividing a training set and a test set according to a certain proportion; S2, establishing a network structure; S3, using a pre-training model, and The pre-training model sets specific training parameters to obtain the training model; S4, performs iterative training until the loss function converges, and obtains the UAV scene detection model; S5, uses the test set to test the model, and judges whether it meets the requirements, if If it does not meet the requirements, continue to step S4 until the test results meet the requirements; S6, output the UAV scene detection model that meets the requirements; S7, use the UAV scene detection model to perform target detection on the sequence images. Compared with the prior art, the invention occupies less memory, the average intersection ratio is increased by 5.26% after the improvement, the precision rate is increased by 3.30% compared with the original version, and the improved recall rate is increased by 1.08%.

Description

A UAV scene detection method based on YOLOv4

technical field

The invention relates to the field of unmanned aerial vehicles, in particular to a method for detecting unmanned aerial vehicle scenes based on YOLOv4.

Background technique

UAVs are small, flexible, and have a wide viewing angle. In recent years, they have been widely used in agricultural plant protection, disaster detection, security protection, and aerial video. Various deep learning schemes have greatly developed the accuracy and speed of target recognition, but most of the objects recognized by the target are from a flat perspective. Some models cannot be directly applied in the field of UAV image target recognition. The two-stage target detection framework represented by Faster-RCNN requires more hardware resources and is slow, which is not suitable for real-time scenarios.

In order to solve the problems of many small targets, low pixels, multi-scale and limited resources of the UAV hardware platform and high real-time performance in the UAV image, the present invention improves and trains a landing scene multi-target recognition based on the UAV image based on the YOLOv4 network. Model.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a UAV scene detection method based on YOLOv4, the main problem to be solved is: the existing target recognition model has a variety of scales, small size, and resolution in the UAV image. The detection effect of small targets with low rates is not good, and it is difficult to judge the problem of the scene where the drone is located.

For achieving the above-mentioned purpose, the present invention provides the following technical solutions:

The invention discloses a YOLOv4-based UAV scene detection method, comprising:

S1. Establish a proprietary data set; divide the training set and the test set according to a certain proportion;

S2, establish a network structure, the network structure is based on the improved YOLOv4 network, with CSPdarknet53 as the backbone network, the spatial pyramid pooling module and the path aggregation network module as the neck, and YOLOV3 as the head prediction output;

S3. First, use the ImageNet large data set to train the network structure obtained in step S2 to obtain a pre-training model, and then set specific training parameters for the network structure;

S4. Use the training set to iteratively train the pre-trained model until the loss function converges to obtain the UAV scene detection model;

S5. Use the above test set to test the UAV scene detection model, and judge whether it meets the requirements. If it does not meet the requirements, continue to step S4, and continue to perform iterative training until the test results meet the requirements;

S6. Output the UAV scene detection model that meets the requirements;

S7. Use the unmanned aerial vehicle scene detection model that meets the requirements in step S6 to perform target detection on the sequence images, and identify the scene where the unmanned aerial vehicle is located.

Further, in the step S1, the establishment of a dedicated data set includes the following steps:

S1.1. Collect basic data samples, the basic data samples include: pictures captured from videos captured by drones, and pictures captured from aerial photography data sets on the Internet; the pictures include: cars, boats, playgrounds , pictures of six kinds of targets of basketball court, bridge and port, and pictures of six kinds of targets that do not contain cars, boats, playgrounds, basketball courts, bridges and ports;

S1.2, label the target in the basic data sample, and process the label into the format required by the YOLO network, the label includes: category, center point abscissa, center point ordinate, target width and target length ;

S1.3, using a data enhancement method to expand the labeled pictures and the unlabeled pictures to obtain a proprietary data set, and the proprietary data set has a total of 1000 pictures.

Further, in the step S1, the ratio of the training set and the test set is: 9:1.

Further, in the step S3, the specific training parameters are: batch=64, the size of each image is 608×608, the batch subdivision=16, the maximum number of batches is 20000 times, and the initial learning rate is 0.0013 .

Further, in the step S4, the convergence value of the loss function is 0.5.

Further, in the step S5, the meeting the requirements refers to the performance evaluation of the UAV scene detection model, and the mAP@0.5 reaches 93.33% and above.

The beneficial effects of the present invention are:

The UAV scene detection model provided by the present invention occupies less memory than the RCNN series network, only needs 2G video memory, and can be used on the UAV airborne hardware platform and other equipment with insufficient hardware resources; the improved YOLOV4 is more average than the original network. The intersection and union are improved by 5.26%, the precision rate is increased by 3.30% compared with the original version, and the improved recall rate is increased by 1.08%.

Description of drawings

Figure 1 is a flowchart of the UAV scene detection method based on YOLOv4.

Figure 2 shows the YOLOv4 network framework diagram.

Figure 3 is a detailed structural diagram of the YOLOv4 network.

Figure 4 shows the relationship between the loss function and the number of model iterations and training.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The invention discloses a UAV scene detection method based on YOLOv4. Compared with the existing model, the UAV scene detection model provided in the present invention occupies less memory, only needs to occupy 2G video memory, and the model is trained as a whole. Performance has been better improved.

Example 1

Referring to Fig. 1, Fig. 2 and Fig. 3, the present embodiment 1 discloses a YOLOv4-based UAV scene detection method, including:

S1. Establish a proprietary data set; divide the training set and the test set according to the ratio of 9:1;

The data of the proprietary data set comes from basic data samples, which are obtained through two channels. The first channel is: intercepted pictures formed from videos captured by drones, and the second channel is intercepted from aerial photography data sets on the Internet. Pictures; the types of pictures are divided into two categories. The first category of pictures is pictures that contain at least one of the six types of objects, including cars, boats, playgrounds, basketball courts, bridges and ports. The second category of pictures is pictures that do not contain cars or boats. , playground, basketball court, bridge, port pictures.

Label the target in the above-mentioned picture, in this embodiment, the action of labeling the target in the picture can be to use the labelImg label tool or other label tools with the same function, and the file format output by the labelImg label tool is an xml file. , because the label file for YOLO training is in txt format, it is necessary to convert the xml file into txt format, which can be converted manually or in batches with tools. The specific labels include: category, abscissa of center point, ordinate of center point, Target width and target length.

Finally, the data enhancement method is used to expand the above-mentioned labeled pictures and unlabeled pictures to obtain a proprietary data set, which has a total of 1000 pictures.

S2. Establish a network structure. The network structure is based on the improved YOLOv4 network, with CSPdarknet53 as the backbone network, spatial pyramid pooling module and path aggregation network module as the neck, and YOLOV3 as the head prediction output; please refer to Figure 2 for the frame diagram of the network structure. and Figure 3;

Multi-channel (CSP) only directly performs convolution operations on a part of the feature map, and the convolution results are combined with another part of the original features, which can enhance the learning ability of the neural network, maintain accuracy while reducing weight, and reduce computational bottlenecks , reduce memory costs.

The Spatial Pyramid Pooling module integrates the Spatial Pyramid Matching Feature Method into a Convolutional Neural Network, and can produce a fixed-size output regardless of the size of the input image.

The path aggregation network module performs downsampling after upsampling, which improves the information flow path in the neural network and strengthens the feature pyramid.

S3. First, use the ImageNet large data set to train the network structure obtained in step S2 to obtain a pre-training model. The pre-training model contains a variety of object-related initialization feature parameters, and then set specific training parameters for the network structure. ; Specifically, the above specific training parameters are: the number of pictures sent to the network in each batch batch=64, the size of each picture is 608x608, the batch subdivision=16, in order to reduce the memory usage, the maximum number of batches is 20000 times, the initial learning rate is 0.0013.

S4. Input the training set to the pre-training model to perform iterative training until the loss function converges. The convergence value of the loss function is 0.5, and the UAV scene detection model is obtained. According to the shape of the training loss function according to Figure 4, the learning rate is set reasonably. Training takes up 2G of video memory, which saves computing resources compared to the two-stage target detection framework.

The overall performance indicators of the model obtained after training are shown in Table 1. The average intersection and union are improved by 5.26%, the precision rate is increased by 3.30% compared with the original version, the improved recall rate is increased by 1.08%, and mAP@0.5 reaches 93%. %.

Table 1

Performance accuracy recall average crossover ratio mAP@0.5 YOLOV4 0.91 0.93 0.76 0.89 Improve YOLOv4 0.94 0.94 0.80 0.93

S5. Use the above test set to test the UAV scene detection model, and judge whether it meets the requirements. If it does not meet the requirements, continue to step S4, and continue to perform iterative training until the test results meet the requirements; specifically, the requirements refer to mAP @0.5 achieves 93.33% and above.

S6. Output a UAV scene detection model that meets the requirements.

S7. Use the unmanned aerial vehicle scene detection model that meets the requirements in step S6 to perform target detection on the sequence images. Detect and identify the scene where the drone is located.

The parts that are not described in detail in the present invention are known techniques of those skilled in the art.

The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make many modifications and changes according to the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art through logical analysis, reasoning or limited experiments on the basis of the prior art according to the concept of the present invention shall fall within the protection scope determined by the claims.

Claims (6)

1. A method for unmanned aerial vehicle scene detection based on YOLOv4 is characterized by comprising the following steps:

s1, establishing a proprietary data set; dividing a training set and a test set according to a certain proportion;

s2, establishing a network structure, wherein the network structure is based on an improved YOLOv4 network, CSPdarknet53 is used as a main network, a spatial pyramid pooling module and a path aggregation network module are used as necks, and YOLOV3 is used as head prediction output;

s3, firstly, training the network structure obtained in the step S2 by using an ImageNet large-scale data set to obtain a pre-training model, and then setting specific training parameters for the network structure;

s4, carrying out iterative training on the pre-training model by using a training set until a loss function is converged to obtain an unmanned aerial vehicle scene detection model;

s5, testing the unmanned aerial vehicle scene detection model by using the test set, judging whether the unmanned aerial vehicle scene detection model meets the requirements, if not, continuing to perform the step S4, and continuing to perform iterative training until the test result meets the requirements;

s6, outputting an unmanned aerial vehicle scene detection model meeting the requirements;

and S7, carrying out target detection on the sequence images by using the unmanned aerial vehicle scene detection model meeting the requirements in the step S6, and identifying the scene where the unmanned aerial vehicle is located.

2. The method for detecting unmanned aerial vehicle scene based on YOLOv4 of claim 1, wherein in the step S1, the establishing the proprietary data set includes the following steps:

s1.1, acquiring basic data samples, wherein the basic data samples comprise: intercepting a picture formed by shooting a video by an unmanned aerial vehicle, and intercepting a picture in an aerial data set on a network;

the picture comprises: pictures containing six targets of an automobile, a ship, an playground, a basketball court, a bridge and a port, and pictures containing six targets of the automobile, the ship, the playground, the basketball court, the bridge and the port;

s1.2, labeling the target of the picture in the basic data sample, and processing the label into a format required by a YOLO network, wherein the label comprises: category, center point abscissa, center point ordinate, target width and target length;

s1.3, expanding the pictures which are subjected to the tags and the pictures which are not subjected to the tags by adopting a data enhancement method to obtain a proprietary data set, wherein the proprietary data set comprises 1000 pictures.

3. The method of claim 1, wherein in step S1, the ratio of the training set to the test set is: 9:1.

4. The method of claim 1, wherein the specific training parameters in step S3 are: the batch size is 64, the size of each graph is 608x608, the batch subdivision is 16, the maximum batch number is 20000, and the initial learning rate is 0.0013.

5. The method of claim 1, wherein in step S4, the convergence value of the loss function is 0.5.

6. The method of claim 1, wherein in step S5, the compliance is a performance evaluation of the drone scene detection model, and the mapp @0.5 is 93.33% or more.

Images