CN111915558A

CN111915558A - Pin state detection method for high-voltage transmission line

Info

Publication number: CN111915558A
Application number: CN202010607242.2A
Authority: CN
Inventors: 曾志伟
Original assignee: Chengdu Sihan Technology Co ltd
Current assignee: Chengdu Sihan Technology Co ltd
Priority date: 2020-06-30
Filing date: 2020-06-30
Publication date: 2020-11-10
Anticipated expiration: 2040-06-30
Also published as: CN111915558B

Abstract

The invention belongs to the technical field of deep learning, and particularly relates to a pin state detection method for a high-voltage transmission line. The present invention focuses the pin status detection on a specific area of the assembled pin-rather than detecting the entire image as in the conventional method, automatically filtering the background area mostly without assembled pins. In addition, a characteristic pyramid network is constructed on the basis of a lightweight backbone network, and the multi-scale characteristics of the image are rapidly extracted unlike the traditional method which adopts a heavyweight network. The speed of pin state detection is faster, and it is also higher to detect the precision, and the interference killing feature is stronger.

Description

Pin state detection method for high-voltage transmission line

Technical Field

The invention belongs to the technical field of deep learning, and particularly relates to a pin state detection method for a high-voltage transmission line.

Background

The threaded fastener of the high-voltage transmission line comprises a bolt, a nut and a pin. The high-voltage transmission line has conductor galloping, and the iron tower also has high-frequency oscillation. These harsh operating environments lead to the possibility of loosening and falling off of the pins, which are key components for preventing the nut from falling off, and once the threaded fastener fails, serious faults of the power grid are directly caused. Therefore, an efficient and accurate pin state detection technology is designed, and the method is very important for guaranteeing safe and economic operation of a power grid.

Currently, pin state detection is mainly completed by manual discrimination. With the rise of the fields of computer vision and the like in recent years, a technology for realizing pin state judgment by applying an automatic detection algorithm on the basis of an unmanned aerial vehicle aerial image gradually appears. The first method for automatic pin state detection adopts a two-stage thought, wherein local Features of an image are extracted in the first stage by adopting Scale Invariant Feature Transform (SIFT), Aggregation Channel Features (ACF) and the like, a training set is constructed, a two-classifier is trained by algorithms such as a Support Vector Machine (SVM), Adaptive Boosting (AdaBoost) and the like, a threaded fastener in the image is detected by combining a sliding window technology, and a Non-Maximum Suppression (NMS) is used for eliminating a repeated detection frame. The detected threaded fastener area contains the pin as it is fitted over the portion of the bolt passing through the nut. The first stage also has a method of positioning threaded fasteners using histogram backprojection in combination with SIFT matching. In the second stage, a Convolutional Neural Network (CNN) is adopted to classify the threaded fasteners and judge whether the threaded fasteners contain or lack pins. The second method of automatic pin state detection adopts a single-stage idea, and integrates detection of a threaded fastener and discrimination of a pin state into one stage to complete the detection. A Feature Pyramid Network (FPN) is used as a backbone Network, and a standard convolution Network is strengthened by adding top-down hidden connections to construct a multi-scale Feature Pyramid with rich images. Two sub-networks are then added in parallel for each level of output of the FPN, one for predicting class probabilities of the object and one for regressing the bounding box of the object.

Because the shooting angle is variable, the aspect ratio of the threaded fastener on the image cannot be approximate to a fixed value, and the first method adopting the sliding window has to apply windows with various aspect ratios on an image pyramid to detect the threaded fasteners with different viewing angles and different distances. This is a huge calculation overhead, and the manually designed features are difficult to ensure that the classifier has sufficient accuracy in the actual application scenario. Because the proportion of the threaded fastener to the high-definition image shot by the unmanned aerial vehicle is very small, the second method is difficult to achieve practical detection precision under the conditions of available computing resources and acceptable time delay.

Disclosure of Invention

Aiming at the defects of the traditional pin state detection technology, the invention provides a cascaded pin state detection technology which can quickly and accurately detect a threaded fastener under the conditions of complex background and imaging and accurately judge whether the pin falls off or not.

In order to achieve the purpose, the invention adopts the following technical scheme:

a pin state detection method for a high-voltage transmission line comprises the following steps:

s1, constructing a training data set: shooting an image of the iron tower of the power transmission line by using an unmanned aerial vehicle, shooting weather conditions of the environment including sunny, rainy, snowy and foggy conditions, covering all types of wires, iron tower connecting pieces, insulators and iron tower connecting pieces, and marking external rectangles and categories of the connecting pieces and threaded fasteners by using a marking tool, wherein the categories include connecting pieces, threaded fasteners with pins and threaded fasteners lacking the pins; respectively taking the marked images as a connecting piece training data set and a threaded fastener training data set; copying the data set into two groups, wherein the first group removes the marking information of the threaded fastener and is used as the data set of the training connector detector; and a second group of subimages with the length and the width respectively being twice the length and the width of the connecting piece are taken from the original image by taking the connecting piece as a center to serve as training images of the threaded fastener detector.

S2, respectively constructing a connecting piece detection neural network and a threaded fastener neural network, defining the convolutional layer as Convn, defining the characteristic pyramid as Stagen, wherein n is the number of layers of the convolutional layer or the characteristic pyramid, and the connecting piece detection neural network sequentially comprises Conv1, a maximum pooling layer, Stage2, Stage3 and Stage 4; the tensor of the characteristic pyramid output is expressed in the form of channel number multiplied by height multiplied by width, the output of Stage4 is 256 multiplied by 19, the output of Stage3 is 256 multiplied by 38, and the output of Stage2 is 256 multiplied by 76; on the basis of the designed feature extraction backbone network, 3 network branches for executing target detection tasks are constructed and used for predicting the position, size and category of a target point by point on a feature output diagram; inputting an output of the Stage4 into a first network branch, wherein the first network branch comprises Stage5 and Conv2, the output of the first network branch is OC × 19 × 19, and OC ═ 3 × (5+ # classes), wherein "5" represents 5 indexes such as abscissa, ordinate, length, width, and existing confidence coefficient of a connector or a threaded fastener needing to be predicted, and # classes is the number of categories needing to be detected; expanding the resolution of a feature map by 2 times to 38 × 38 by using a nearest neighbor interpolation method for the output of Stage4, fusing the expanded resolution with the output of Stage3, and inputting the fused resolution into a second network branch, wherein the second network branch comprises Stage6 and Conv3, and the output of the second network branch is OC × 38 × 38; expanding the resolution of a feature map by 2 times to 76 × 76 by using a nearest neighbor interpolation method for the output of Stage3, fusing the feature map with the output of Stage2, and inputting the feature map into a third network branch, wherein the third network branch comprises Stage7 and Conv4, and the output of the second network branch is OC × 76 × 76; the structure of the threaded fastener neural network is the same as that of the connecting piece detection neural network;

s3, training the constructed neural network by adopting a training data set: training a connecting piece detection neural network by adopting a connecting piece training data set, training the threaded fastener neural network by adopting threaded fastener training data, wherein the training method is a batch random gradient descent method, a loss function is a mean square error and a binary cross entropy function, the final total loss is the sum of the losses of 3 network branches for executing a target detection task, and parameters of the network are updated by adopting a back propagation algorithm to obtain the trained connecting piece detection neural network and the threaded fastener neural network;

s4, inputting the power transmission line iron tower image acquired in real time into a trained connecting piece detection neural network to obtain a connecting piece area, defining the coordinates of the connecting piece area as (xmin, ymin, xmax, ymax), calculating the interested pin detection area as (xmin-x, ymin-y, xmax + x, ymax + y) according to the obtained coordinates, wherein x and y are offset in the horizontal direction and the vertical direction respectively, and the value is as follows: x is (xmax-xmin)/2, y is (ymax-ymin)/2; and inputting the interested pin detection area into the trained threaded fastener neural network to obtain a pin state detection result.

The invention has the beneficial effects that: the attention of pin state detection is focused on a specific area of the assembly pin instead of detecting the whole image like the traditional method, most background areas without the assembly pin are automatically filtered, the anti-interference capability of the algorithm is stronger, the processing speed of the algorithm is obviously improved, and the pin state detection precision is greatly improved. In addition, a characteristic pyramid network is constructed on the basis of a lightweight backbone network, a heavyweight network is not adopted like a traditional method, a more hardware-friendly basic unit is adopted to construct the network, and a wider and deeper network can be constructed on the premise of ensuring low algorithm complexity, which means that the characteristic expression capability of the network is stronger, the accuracy of detecting the state of the pin is higher, and the detection speed is faster.

Drawings

FIG. 1 is a flow chart of pin status detection;

FIG. 2 is a schematic diagram of a constructed feature network architecture;

FIG. 3 is a schematic diagram of the manner in which the area ratio of the intersection and union of rectangles is calculated;

FIG. 4 is a structural diagram of a lightweight network ShuffleNet V2.

Detailed Description

The invention is further illustrated with reference to the following figures and examples.

Examples

The processing flow of this example is as shown in fig. 1, data collection and annotation. The unmanned aerial vehicle is used for shooting images of the power transmission line iron tower, shooting environments including different weather conditions such as sunny weather, rain weather, snow weather and fog weather are needed, and all types of wires, iron tower connecting pieces, insulators and iron tower connecting pieces are covered. And marking the circumscribed rectangles and the categories of the connecting piece and the threaded fastener by using a marking tool, wherein the categories comprise the connecting piece, the threaded fastener with the pin and the threaded fastener without the pin.

And preprocessing pin data. From the labeled connector Region coordinates, a Region of Interest (ROI) pin condition detection is determined. Marking the area of the connecting piece as (xmin, ymin, xmax, ymax), setting (xmin-x, ymin-y, xmax + x, ymax + y) as the interested pin state detection area, wherein x and y are offset in the horizontal and vertical directions respectively, and the values are as follows: x is (xmax-xmin)/2, and y is (ymax-ymin)/2. And cutting out a pin state detection area from the original image according to the coordinates of the pin state detection area for a subsequent training task.

A connector detector for constructing a wire and an insulator and a pin state detector. The connecting pieces and the pins adopt the same network architecture. And (3) adopting a lightweight network ShuffleNet V2 as a backbone network, simultaneously adding FPN side branches to construct a characteristic pyramid of the image, and then outputting the added sub-networks from each level of the FPN for predicting the frame, confidence and class probability of the connecting piece or the pin. The specific backbone portion configuration of the backbone network is shown in table 1. Wherein Stagen (n ═ 1, 2.. 7) adopts the basic construction unit shown in fig. 4, firstly adopts the construction unit of fig. 4 left) to execute down-sampling, reduces the resolution of the feature map, and then calls several times to fig. 4 right) the unit of the structure strengthens the expression capability of the feature map with reduced resolution.

Table 1 backbone part configuration

Three levels of feature pyramids are constructed. The output of the last layer of Stage2, Stage3 and Stage4 of the backbone main branch is used as a reference feature map for constructing a feature pyramid with the resolution of 76 × 76, 38 × 38 and 19 × 19 respectively, and as shown in fig. 2, the pyramid features of three levels are set to be 256 channels. First, Stage4 is followed by 1 × 1 convolution to reduce the number of channels of a feature to 256, and then the resolution of the feature map is enlarged by 2 times to 38 × 38 by nearest neighbor interpolation, so that the size of the feature after processing is 256 × 38 × 38 (the tensor is expressed by the number of channels × height × width, the same applies hereinafter). Similarly, the output feature of Stage3 is reduced to 256 channels by 1 × 1 convolution to become a 256 × 38 × 38 feature, then the two equally large features are added point by point, and the fused features are processed by 3 × 3 convolution, so that the aliasing effect introduced by upsampling is weakened on the premise of keeping the number of channels unchanged. As to the way in which the features after Stage4 and Stage3 were fused, and the output features of Stage2 were fused, the above-described method was followed. The final feature pyramid outputs three levels of features: 256 × 19 × 19, 256 × 38 × 38, and 256 × 76 × 76.

On the basis of the backbone network designed above, the sub-networks performing the task of connector or pin detection are accessed from the output of each level of the feature pyramid. The three sub-networks consist of ShuffleNet V2 base cells, convolutional layers, and output layers, as shown in Table 2. Wherein Stage5, Stage6 and Stage7 are used for enhancing the characteristics of pyramid output, and Conv2, Conv3 and Conv4 are used for normalizing the channel number of the characteristics to the number of variables needing prediction. Setting each subnetwork to be responsible for predicting the 3-scale frame, the number of channels finally output by Conv2, Conv3 and Conv4 is 3 × (5+ # classes), where # classes is the number of classes to be detected.

Table 2 detection subnetwork configuration

Note: OC is 3 × (5+ # classes), # classes represents the number of classes that need to be predicted.

Training connector detectors and pin detectors. Due to the strategy of using cascade detection, the connector detector and the pin detector need to be trained separately. And measuring the Error of the predicted value and the actual value of the connector or pin circumscribed rectangle by Mean Square Error (MSE) and measuring the Error of the predicted value and the actual value of the connector or pin classified probability by Binary Cross Entropy (BCE). The insulator detector is trained by adopting a batch random Gradient Descent (SGD) method and combining an impulse mechanism. The initial learning rate is set to be 0.001, the impulse is set to be 0.9, the weight of the regularization term is 0.0005, a step-type learning rate updating strategy is adopted, the total iteration frequency is set to be 200, and the learning rate is reduced by 10 times when the total iteration frequency is 80% and 90%.

The performance of the method employed in this example was evaluated on the connector and pin data sets. And (4) respectively processing all images in the connector and pin data set by a connector detector and a pin detector to obtain predicted values of the connector and the pin. Since there may be multiple detection results for the same object, it is necessary to perform non-maximum suppression on the original prediction output and merge similar detection frames. And then, comparing the predicted value after NMS with the labeled value, and calculating indexes such as Average Precision (AP), processing speed and the like. The formula (1) is used to determine whether the detection result of the connector and the pin is correct, if the area ratio (Intersection over Union, IoU, see fig. 2) of the Intersection and the Union of the prediction frame and the actual frame of the connector and the pin is greater than 0.5, the detection result is considered to be correct, and the corresponding True Positive (TP) is accumulated as one, otherwise, the corresponding False Positive (FP) is accumulated as one. The accuracy of the test connectors and pins was calculated from TP and FP according to equation (2). The recall rate of the connector and pin detector was calculated according to equation (3). Where the sum of TP and FN is the total number of samples of connectors and pins in the labeled dataset. And then calculating the integral area of the P-R curve by precision and recall to obtain the AP.

When the detection model is actually deployed, the connector detector is firstly used for processing an original image, outputting a connector sub-region, and then acquiring a pin state detection region of interest according to the same method of the data preprocessing part. And calling a pin state detector for processing each ROI to judge whether the pin falls off or not.

Claims

1. A pin state detection method for a high-voltage transmission line is characterized by comprising the following steps:

s1, constructing a training data set: shooting an image of the iron tower of the power transmission line by using an unmanned aerial vehicle, shooting weather conditions of the environment including sunny, rainy, snowy and foggy conditions, covering all types of wires, iron tower connecting pieces, insulators and iron tower connecting pieces, and marking external rectangles and categories of the connecting pieces and threaded fasteners by using a marking tool, wherein the categories include connecting pieces, threaded fasteners with pins and threaded fasteners lacking the pins; respectively taking the marked images as a connecting piece training data set and a threaded fastener training data set;

s2, respectively constructing a connecting piece detection neural network and a threaded fastener neural network, defining the convolutional layer as Convn, defining the characteristic pyramid as Stagen, wherein n is the number of layers of the convolutional layer or the characteristic pyramid, and the connecting piece detection neural network sequentially comprises Conv1, a maximum pooling layer, Stage2, Stage3 and Stage 4; the tensor of the characteristic pyramid output is expressed in the form of channel number multiplied by height multiplied by width, the output of Stage4 is 256 multiplied by 19, the output of Stage3 is 256 multiplied by 38, and the output of Stage2 is 256 multiplied by 76; inputting an output of the Stage4 into a first network branch, wherein the first network branch comprises Stage5 and Conv2, the output of the first network branch is OC × 19 × 19, OC ═ 3 × (5+ # classes), 5 indicates 5 indexes needing prediction, and are respectively an abscissa, an ordinate, a length, a width and a confidence level of existence, and # classes is the number of classes needing detection; expanding the resolution of a feature map by 2 times to 38 × 38 by using a nearest neighbor interpolation method for the output of Stage4, fusing the expanded resolution with the output of Stage3, and inputting the fused resolution into a second network branch, wherein the second network branch comprises Stage6 and Conv3, and the output of the second network branch is OC × 38 × 38; expanding the resolution of a feature map by 2 times to 76 × 76 by using a nearest neighbor interpolation method for the output of Stage3, fusing the expanded resolution with the output of Stage2, and inputting the fused resolution into a third network branch, wherein the third network branch comprises Stage7 and Conv4, and the output of the second network branch is OC × 76 × 76; the structure of the threaded fastener neural network is the same as that of the connecting piece detection neural network;

s3, training the constructed neural network by adopting a training data set: training a connecting piece detection neural network by adopting a connecting piece training data set, training a threaded fastener neural network by adopting threaded fastener training data, wherein the training method is a batch random gradient descent method, a loss function is a mean square error and a binary cross entropy function, the final total loss is the sum of losses of a first network branch, a second network branch and a third network branch, and parameters of the network are updated by adopting a back propagation algorithm to obtain the trained connecting piece detection neural network and the trained threaded fastener neural network;