CN115147383A - Insulator state rapid detection method based on lightweight YOLOv5 model - Google Patents

Abstract

The invention discloses a method for rapidly detecting the state of an insulator based on a lightweight YOLOv5 model, and aims to solve the problems of low accuracy rate of insulator state detection and large detection network parameter quantity and calculated quantity. The insulator defect detection algorithm comprises the steps of collecting insulator images to form a data set; marking the insulator and the defect image thereof by using a Labelimg tool, and expanding a data set through data enhancement processing; introducing a lightweight network ShuffleNet V2-Stem to carry out lightweight improvement on the YOLOv5 model to form a YOLOv5-ShuffleNet V2S model; adding a small target detection layer; a CIoU loss function is adopted; training a lightweight improved YOLOv5 model; applying the improved model to an insulator defect data set; experimental results show that the lightweight YOLOv5 model enhances the capability of extracting image characteristic information, realizes the lightweight of the detection model on the premise of keeping the detection precision, and is more suitable for being deployed on an unmanned aerial vehicle platform to detect the state of the insulator.

Description

Insulator state rapid detection method based on lightweight YOLOv5 model

Technical Field

The invention belongs to the field of deep learning image processing, and relates to a method for rapidly detecting the state of an insulator based on a lightweight YOLOv5 model.

Background

Under the strategic pattern of 'west-east power transmission and north-south power supply' of the national power grid, the continuous improvement of the ultra-high voltage and high voltage transmission grade in China, the expansion of the length and the erection area of the transmission line, and the safe, stable and reliable operation of the power grid are very important. The safe and normal operation of the power transmission network is a crucial link for the safety of the power grid. Because the environment of the power transmission line is complex and changeable, power devices in the line are exposed in the natural environment all the year round, and faults such as corrosion, strand breakage, abrasion and the like are easily caused under the influence of external factors. The insulator is one of important devices of the power transmission line, and according to statistics, the number of accidents caused by insulator faults is more than half, so that great potential safety hazards are brought to stable operation of the power transmission line. According to the division of the power transmission line faults by national grid companies, the insulator defects are divided into three different damage grades of general damage, serious damage and emergency damage according to the damage degree, and the defect types are 83 in total. Therefore, it is necessary to monitor the working state of the insulators in the power transmission line in time and periodically perform inspection work on the power transmission line to check faulty insulators.

The power transmission line inspection mode comprises manual inspection, helicopter inspection, unmanned aerial vehicle inspection and inspection robot inspection. The traditional manual inspection is low in efficiency and difficult to ensure safety. Helicopter inspection can improve inspection efficiency, but the defects of high cost and low flexibility make the helicopter inspection incapable of being applied on a large scale. Along with the rapid development of artificial intelligence and the mature application of unmanned aerial vehicle technology, unmanned aerial vehicle patrols and examines and can use widely. Unmanned aerial vehicle patrols and examines and has safety, efficient advantage, utilizes the airborne camera equipment to shoot the power equipment in the transmission line, and whether the rethread is artifical to patrolling and examining the image identification and judging to have the trouble. However, the patrol image has the characteristics of large data with large volume and low value density, and the manual interpretation is easy to have misjudgment and missed judgment caused by visual fatigue. After deep learning is introduced, intelligent routing inspection becomes possible through the combination of a computer vision technology, an image processing technology and computing hardware.

The deep learning based target detection algorithm uses a convolutional neural network in the feature extraction module and is accelerated training using graphics cards and Batch processing (Batch Size). The method can be divided into a two-stage detection algorithm and a single-stage detection algorithm according to an implementation mechanism. The two-stage Detection algorithm is also called Object Detection algorithm (Object Detection Based on Regional delivery) Based on region naming. And generating a target candidate domain during detection, and predicting, classifying and identifying the target for the candidate domain. Classical algorithms are fast R-CNN, SPPNet, etc. The single-stage Detection algorithm is also called an End-to-End Based Object Detection algorithm (Object Detection Based on End to End Learning). The network can finish detection without generating a candidate area during detection, namely, the network directly outputs the spatial coordinates and the category information of the predicted object. The single-stage detection algorithm is based on regression of a bounding box and is a one-step in-place process. The single-stage detection network performs classification and bounding box regression while generating candidate boxes, and is characterized by high speed and low accuracy.

Compared with the traditional image processing method, the insulator detection method based on deep learning has the advantages that the detection precision is greatly improved, the generalization capability is strong, the requirement of the actual insulator inspection work on the detection precision cannot be met, in addition, the detection speed loss is serious, and the requirement of the insulator defect detection speed cannot be met.

Disclosure of Invention

Aiming at the problems of accuracy and speed of an insulator detection method, the invention provides a method for rapidly detecting the state of an insulator based on a lightweight YOLOv5 model.

In order to solve the technical problem, the invention adopts the following technical scheme:

a method for rapidly detecting the state of an insulator based on a lightweight YOLOv5 model is characterized by comprising the following steps: the method comprises the following steps:

step 1, collecting an insulator image forming data set;

step 2, labeling the data set by using a LabelImg labeling tool;

step 3, performing data enhancement processing on the acquired image to expand a data set;

step 4, introducing a lightweight network ShuffleNetV2-Stem as a backbone network of YOLOv5, and carrying out lightweight improvement on a YOLOv5 model to form a YOLOv5-ShuffleNetV2S model;

step 5, adding a small target detection layer in a feature fusion network of the lightweight YOLOv5 model;

step 6, optimizing a loss function, and taking the CIoU as the loss function of the lightweight YOLOv5 model;

step 7, training the improved network, setting a learning rate, a batch size and a total training round as training parameters, and training the light-weight Yolov5 model;

and 8, inputting the collected insulator image data set into the trained YOLOv5 model to obtain whether the input picture has a defective insulator and the position of the defect.

Further, the insulator image data set in step 1 comprises a defective insulator image and a complete insulator image.

Further, in the step 2, the data set is labeled to obtain an xml file conforming to the VOC data format, and the contents of the xml file include the image name, the image path, the height/width of the image, and the position and width/height information of the center point of the real frame.

Further, in the step 3, the data set is expanded by one or more data enhancement methods of adaptive contrast, rotation, random gray scale change, translation, clipping, color channel standardization and mix up.

Further, the concrete formula of the method for enhancing the Mixup data is as follows:

x＝λx _i +(1-λ)x _j

y＝λy _i +(1-λ)y _j

λ＝Beta(α,β)。

further, the YOLOv5 model after the improvement of the lightweight in the step 4 is a YOLOv5-ShuffleNetV2S model, and the YOLOv5-ShuffleNetV2S model is composed of a backbone network, a feature fusion network and a detection network; the main network consists of ShuffleNetv2 and Stem; the feature fusion network consists of CBL, CSP, upsampling and add.

Furthermore, the ShuffleNet2 network introduces grouping convolution and channel shuffling and mainly comprises two basic unit modules; one unit keeps the number of output channels the same as the number of input channels. The other unit is a down-sampling module which reduces the dimension of the feature map; one branch of the Stem module introduces a bottleneck layer, the number of channels is reduced, down sampling is carried out, and the other branch carries out maximum pooling on original input and splicing; and reconstructing the light ShuffleNet V2 and the Stem module to be used as a backbone network of the YOLOv 5.

Further, the small target detection layer added in step 5 is a process of adding 4 times down-sampling to the original input picture.

Further, in the step 6, CIoU is used as a loss function of the light-weight YOLOv5 model,

the regression of the target frame is better described from three aspects of the overlapping area, the distance of the central point and the aspect ratio, and the calculation formula is as follows:

further, the training of the lightweight YOLOv5 model in step 7 includes the following steps:

a. in the network training, the resolution of an input image is 640 × 640, and training is performed on a lightweight YOLOv5 model with depth _ multipl =0.33 and width _multiple = 0.50;

b. adopting an Adam optimizer, setting the initial learning rate to be 0.001, setting the batch size of model training to be 16, and setting the total training round to be 500 times;

c. after training is finished, storing the obtained weight file of the recognition model, and evaluating the performance of the model by using a test set;

d. and finally outputting the position frames for identifying the insulator and the defects thereof and the confidence coefficient of the corresponding category by the improved model.

Compared with the prior art, the technology provided reconstructs the Stem module and the ShuffleNet V2, enhances the capability of extracting image characteristic information, replaces the C3Net of the original YOLOv5 with the reconstructed ShuffleNet V2-Stem network as a background, obviously reduces the parameter quantity and the calculated quantity of the network, enhances the sensing capability of the network on the insulator defect by using the Mixup data enhancement and the CIoU loss function, increases a small target detection layer, enhances the detection precision of the network, and finally applies the small target detection layer to a self-made insulator sub data set for verification, thereby realizing the lightweight detection model.

Drawings

Fig. 1 is a network structure of the present invention, YOLOv5, which is a lightweight improvement.

Detailed Description

The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the following embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.

The present invention is further illustrated by the following examples, which are not to be construed as limiting the invention.

In order to solve the problems of low insulator state detection accuracy and large detection network parameter and calculation amount, the embodiment performs lightweight improvement on the existing YOLOv5 model, so that the lightweight of the detection model is realized on the premise of keeping the detection precision in the aspect of insulator defect detection. Firstly, collecting aerial images of insulators and defects thereof, carrying out data expansion on a data set, making an insulation data set of the experiment, then improving a network, wherein loss functions, a backbone network and a fusion network of the network are mainly modified, and the network structure of a lightweight improved YOLOv5 model is shown in figure 1. Secondly, the improved network is trained by the expanded data set. And finally, adjusting network parameters according to the experimental result.

As shown in fig. 1, in the network of the lightweight improved YOLOv5 model, an insulator sample picture is processed to obtain a three-channel picture with a fixed size, and the three-channel picture is input into a Stem module for processing, so that excessive information loss is prevented, and the work of reducing parameters is completed. And then entering a ShuffleNetv2 module to further reduce the calculated amount of the model and improve the running speed. At this time, the reconstruction of the trunk portion of the improved network is completed, and the trunk portion of the improved network better meets the requirement of light weight compared with the original YOLOv5 model. In the feature fusion network, a small target detection layer is added to balance the situation that targets are far and near in an insulator sample photo, and the detection precision of the small targets far away is improved. Correspondingly, a small target detection head is added to the output part correspondingly to finish the drawing of the small target anchor frame. The improved feature fusion network has better feature fusion capability, the feature map receptive field of the newly added small target detection layer is smaller, and the position information is relatively abundant, so that the detection precision of the model to the small target can be improved. Experimental results show that the light weight of the detection model is realized by adopting the light weight YOLOv5 on the premise of keeping the detection precision.

The embodiment is realized by the following technical scheme, and the insulator state rapid detection method based on the lightweight YOLOv5 model comprises the following steps:

1. an insulator image dataset is acquired containing a defective insulator image and a complete insulator image.

2. And labeling the data set by using a LabelImg labeling tool, wherein the labeled types are insulators and defects.

3. And carrying out data enhancement processing on the acquired image to expand the data set.

4. Introducing a lightweight network ShuffleNet V2-Stem as a backbone network of the YOLOv5, and carrying out lightweight improvement on the YOLOv5 model to form a YOLOv5-ShuffleNet V2S model.

Due to the limited computing power and memory resources of the mobile end platform, the performance of the GPU is far lower than that of the PC end, and the performance of the GPU is at least 1/10 lower than that of the PC end. In order to meet the application requirements of a mobile terminal and an embedded platform, some lightweight convolutional neural networks such as MobileNet and ShuffleNet are provided, and researchers improve the lightweight convolutional models to well balance the accuracy and the speed. Therefore, in order to realize the lightweight of the model, the structure of the YOLOv5 model needs to be adjusted and improved accordingly. The backbone network C3PNet parameters in the original YOLOv5 model are large, the model occupies a large amount of memory, the calculation complexity is high, and the calculation capacity requirement on hardware is high. In order to solve the problems, the lightweight network ShuffleNet V2-stem is adopted to replace the backbone network C3Net in the YOLOv5 model. The ShuffleNet V2 is a lightweight network improved by analyzing the defects of the ShuffleNet V1 and the MobileNet V2, and has the advantages of high precision and high speed. Integrating the lightweight ShuffleNet V2-Stem network into a YOLOv5 model to form a YOLOv5-ShuffleNet V2S model, and finishing the lightweight improvement of the YOLOv5 model. The YOLOv5-ShuffleNetV2S model can meet the detection precision, reduce the parameter quantity and the calculated quantity of the network, improve the detection speed of the model, ensure the balance between the accuracy of the model and the detection speed, minimize the volume of the network model and reduce the calculation capacity requirement of the model on hardware.

5. And adding a small target detection layer in the feature fusion network.

The small target detection effect of the YOLOv5 model is not good, one reason is that in insulator defect detection, most defect detection targets account for a small proportion of the whole image, and the down-sampling multiple of the YOLOv5 model is large, so that the characteristic information of the small target is difficult to learn by a deep characteristic diagram. Therefore, the method adds a 4-time down-sampling process to the original input picture on the basis of the light YOLOv5 model backbone network, namely, adds a small target detection layer, and detects the spliced shallow feature map and deep feature map by adding the small target detection layer. The original picture is subjected to 4-time down-sampling and then is sent to a feature fusion network to obtain a feature map with a new size, the feature map is small in receptive field, the position information is rich, and the detection effect of detecting a small target can be improved.

6. A loss function is optimized.

CIoU is taken as a bounding-box loss function LossCIoU of the lightweight improved YOLOv5 algorithm. The CIOU takes the distance, the overlapping rate, the scale and the punishment items between the real frame and the prediction frame into consideration, so that the regression of the prediction frame becomes more stable, the problems of divergence and the like in the training process can not occur like other loss functions, and the punishment factors take the length-width ratio of the prediction frame to the length-width ratio of the target frame into consideration, so that the model can be converged more quickly and better.

7. Training the improved network, setting a learning rate, a batch size and a total training turn as training parameters, and training the lightweight improved Yolov5 model;

8. and inputting the collected insulator image data set into a trained YOLOv5 model to obtain whether a defective insulator exists in an input picture and the position of the defect.

Specific examples of the insulator defect detection method based on the lightweight YOLOv5 are described as follows:

1. first, an insulator image is acquired to form a data set. The insulator defect data set used in this embodiment includes 281 porcelain insulators, 413 composite insulators and 694 pictures. Because the existing data set has fewer pictures and no target information, the task of target detection is difficult to complete, and therefore, preprocessing operations such as expansion, labeling and the like on the existing data set are very necessary. And marking the data set before the data set enters the network model training to obtain an xml file conforming to the VOC data format, wherein the content of the xml file comprises the image name, the image path, the height/width of the image, the position and the width/height of the center point of the real frame and the like. The data set is then augmented by adaptive contrast, rotation, random gray scale variation, translation, cropping, color channel normalization, mixup, and the like. The method is a data enhancement method based on a neighborhood risk principle, and the algorithm utilizes a linear interpolation mode to mix two samples and labels, namely randomly extracts two samples from a data set, carries out weighted summation on pixel values of the two samples according to certain weight, and carries out weighted summation on the labels corresponding to the two samples according to the same proportion, thereby expanding the distribution space of training data to a certain extent and improving the generalization capability of the model. The specific formula is as follows:

x＝λx _i +(1-λ)x _j

y＝λy _i +(1-λ)y _j

λ＝Beta(α,β)

wherein (x) _i ,y _i ) And (x) _j ,y _j ) Two samples are randomly extracted from the training data, x is the generated mixed picture, and y is the generated mixed label. λ is a weight, ranging from 0 to 1, obeying a Beta (α, β) distribution.

2. As shown in fig. 1, the YOLOv5 model is subjected to lightweight improvement, and the YOLOv5 model with the lightweight improvement is composed of a backbone network, a feature fusion network and a detection network. The backbone network consists of ShuffleNetv2 and Stem. The feature fusion network is composed of CBL, CSP, upsampling and add. In order to detect the position and the category of a target, characteristics need to be extracted from an image, and a backbone network carries out positioning and classification to capture the characteristics; the feature fusion network fuses features through the initial output features of the backbone network, and adapts to the size, so that the overall performance of the system structure is improved; the detection network receives the output of the feature fusion network, and each output feature maps the position of the bounding box of the output layer, the confidence of the object and the prediction of the object class.

In order to reduce the computational complexity of the model and meet the lightweight requirement of carrying the model to the mobile terminal device while ensuring accuracy, the embodiment proposes a ShuffleNetv2-Stem model to perform lightweight improvement on the yollov 5 backbone network based on the yollov 5 model. The shuffle net2 network introduces a Group Convolution (GC) and Channel Shuffle (CS) operation to reduce the amount of computation in network operation, so that it can be carried to the mobile end device. The ShuffleNet v2 network mainly comprises two basic unit modules, and the unit 1 ensures that the number of output channels is the same as that of input channels, thereby achieving the purpose of improving the speed. The unit 2 is a down-sampling module, and mainly plays a role in reducing the dimension of the feature map, so that the calculation amount of the network model is further reduced.

The Stem module increases the number of channels of the input image space dimension, performs a first down-sampling task, can enrich the feature layer and keep stronger insulator image feature expression capability, does not increase extra calculation amount, and is a module with lower cost. The main operation of the Stem module for reducing the parameter quantity is to introduce a bottleneck layer into one branch, reduce the channel quantity firstly, then carry out down sampling, and carry out maximum pooling on the original input and then splice the original input by the other branch, so that the aim of transmitting partial information in the input is to ensure that the final result still has enough semantic information on the basis of reducing the parameter quantity, and the excessive loss of the information cannot be caused. And reconstructing the light weight ShuffleNet V2 and Stem modules, and integrating the reconstructed network into a YOLOv5 model to form a YOLOv5-ShuffleNet V2S model.

And secondly, adding a 4-time down-sampling process in the feature fusion network part, and detecting the spliced shallow feature map and deep feature map by adding a small target detection layer. The original picture is subjected to 4-time down-sampling and then is sent to a feature fusion network to obtain a feature map with a new size, the feature map is small in receptive field, the position information is rich, and the detection effect of detecting a small target can be improved.

3. In order to further make up for the precision loss caused by light weight, a loss function is modified into a CIoU loss function on the basis of ShuffleNet V2-Stem and the addition of a small target detection layer. The improved model was applied to the homemade insulator data set for testing.

GIoU was used in raw YOLOv5 to calculate the positioning loss: unlike the original IoU, the GIoU not only focuses on the overlapping area between the real frame and the predicted frame, but also focuses on other non-overlapping regions, so the GIoU reflects the degree of overlap between the two better than the original IoU, but the GIoU always considers only one factor of the overlapping rate between the real frame and the predicted frame, and cannot describe the regression problem of the target frame well. When the prediction frames are inside the real frames and the sizes of the prediction frames are the same, the GIoU is degenerated to the IoU, and the positional relationship between the prediction frames cannot be distinguished. Selecting CIoU to replace GIoU as a loss function of the regression of the target frame, and better describing the regression of the target frame from three aspects of overlapping area, central point distance and aspect ratio, wherein the calculation formula is as follows:

4. training a reduced weight improved YOLOv5 model, wherein the resolution of an input image is 640 multiplied by 640, and the training is carried out on the reduced weight improved YOLOv5 model with depth _ multipl =0.33 and width _multiple = 0.50; adopting an Adam optimizer, setting the initial learning rate to be 0.001, setting the batch size of model training to be 16, and setting the total training round to be 500 times; after training is finished, storing the obtained weight file of the recognition model, and evaluating the performance of the model by using a test set; and finally outputting a position frame for identifying the insulator and the defects thereof and confidence degrees of corresponding categories by the lightweight improved YOLOv5 model.

And finally, inputting the collected insulator image data set into a trained YOLOv5 model to obtain whether a defect insulator exists in the input picture and the position of the defect.

Claims (10)

1. A method for rapidly detecting the state of an insulator based on a lightweight YOLOv5 model is characterized by comprising the following steps: the method comprises the following steps:

step 1, collecting an insulator image forming data set;

step 2, labeling the data set by using a LabelImg labeling tool;

step 3, performing data enhancement processing on the acquired image to expand a data set;

step 4, introducing a lightweight network ShuffleNetV2-Stem as a backbone network of YOLOv5, and carrying out lightweight improvement on a YOLOv5 model to form a YOLOv5-ShuffleNetV2S model;

step 5, adding a small target detection layer in a feature fusion network of the light-weight YOLOv5 model;

step 6, optimizing a loss function, and taking the CIoU as the loss function of the light-weight YOLOv5 model;

step 7, training the improved network, setting a learning rate, a batch size and a total training round as training parameters, and training the light-weight Yolov5 model;

and 8, inputting the collected insulator image data set into a trained lightweight YOLOv5 model to obtain whether a defect insulator exists in an input picture and the position of the defect.

2. The insulator state rapid detection method based on the lightweight YOLOv5 model according to claim 1, characterized in that: the insulator image data set in step 1 contains defective insulator images and complete insulator images.

3. The insulator state rapid detection method based on the lightweight YOLOv5 model according to claim 1, characterized in that: and in the step 2, the data set is labeled to obtain an xml file conforming to the VOC data format, wherein the content of the xml file comprises the image name, the image path, the height/width of the image and the position and width/height information of the center point of the real frame.

4. The insulator state rapid detection method based on the lightweight YOLOv5 model according to claim 1, characterized in that: and 3, expanding the data set by one or more data enhancement methods of self-adaptive contrast, rotation, random gray scale change, translation, cutting, color channel standardization and Mixup.

5. The insulator state rapid detection method based on the lightweight YOLOv5 model as claimed in claim 4, characterized in that: the concrete formula of the Mixup data enhancement method is as follows:

x＝λx _i +(1-λ)x _j

y＝λy _i +(1-λ)y _j

λ＝Beta(α,β)。

6. the insulator state rapid detection method based on the lightweight YOLOv5 model according to claim 1, characterized in that: the YOLOv5 model after the lightweight improvement in the step 4 is a YOLOv5-ShuffleNetV2S model, and the YOLOv5-ShuffleNetV2S model consists of a backbone network, a feature fusion network and a detection network; the backbone network consists of ShuffleNetv2 and Stem; the feature fusion network is composed of CBL, CSP, upsampling and add.

7. The insulator state rapid detection method based on the lightweight YOLOv5 model as claimed in claim 6, characterized in that: the ShuffleNet2 network introduces grouping convolution and channel shuffling and mainly comprises two basic unit modules; one unit keeps the number of output channels the same as the number of input channels. The other unit is a down-sampling module which reduces the dimension of the feature map; one branch of the Stem module introduces a bottleneck layer, the number of channels is reduced, down sampling is carried out, and the other branch carries out maximum pooling on original input and splicing; and reconstructing the light ShuffleNet V2 and the Stem module to be used as a backbone network of the YOLOv 5.

8. The insulator state rapid detection method based on the lightweight YOLOv5 model as claimed in claim 1, characterized in that: the small target detection layer added in the step 5 is a process of adding 4 times down-sampling to the original input picture.

9. The insulator state rapid detection method based on the lightweight YOLOv5 model according to claim 1, characterized in that: in the step 6, CIoU is taken as a loss function of the lightweight YOLOv5 model,

the regression of the target frame is better described from three aspects of the overlapping area, the distance of the central point and the aspect ratio, and the calculation formula is as follows:

10. the insulator state rapid detection method based on the lightweight YOLOv5 model as claimed in claim 1, characterized in that: the training of the lightweight YOLOv5 model in the step 7 comprises the following steps:

a. in the network training, the resolution of an input image is 640 × 640, and training is performed on a lightweight YOLOv5 model with depth _ multipl =0.33 and width _multiple = 0.50;

b. adopting an Adam optimizer, setting the initial learning rate to be 0.001, setting the batch size of model training to be 16, and setting the total training round to be 500 times;

c. after training is finished, storing the obtained weight file of the recognition model, and evaluating the performance of the model by using a test set;

d. the improved model finally outputs a position frame identifying the insulator and the defect thereof and confidence degrees of corresponding categories.

Images