CN110618129A - Automatic power grid wire clamp detection and defect identification method and device - Google Patents

Abstract

The invention discloses a power grid wire clamp automatic detection and defect identification method, which comprises the following steps: acquiring a wire clamp picture through an image acquisition module; training a wire clamp picture to obtain the type of a wire clamp; classifying, deleting and selecting the wire clamp pictures; and identifying and outputting the fault wire clamp. The invention also discloses a device for automatically detecting the power grid wire clamp and identifying the defects. According to the invention, the image acquisition module is used for carrying out image acquisition on the wire clamp on the power transmission line, the fast R-CNN technology is used for training the wire clamp picture, the augmentation training technology is used, the defects of long time and slow model updating period caused by retraining new wire clamp characteristics are overcome, the overhaul cost is greatly saved for fault detection of the wire clamp, and the line patrol system is more efficient and intelligent.

Description

Automatic power grid wire clamp detection and defect identification method and device

Technical Field

The invention relates to the technical field of overhead transmission line safety, in particular to a method and a device for automatically detecting a power grid wire clamp and identifying defects.

Background

In an overhead transmission line, various transmission lines are staggered with dog teeth, and the transmission lines can be mutually wound, pulled and vibrated, so that the lines are easily broken, and the safe and reliable operation of the overhead transmission line is seriously threatened. The wire clamp is an important device in an overhead transmission line, can wrap a large number of transmission lines, plays a role in protecting and assisting in vibration reduction of the transmission lines, and greatly prolongs the service life of the transmission lines. But overhead transmission lines are used for long distance power transmission, during which different natural environments must be spanned. After experiencing long-time wind blowing and sun drying, the wire clamp can generate various defects, so that the wire clamp cannot play the original role. Therefore, in the process of routing inspection of the overhead transmission line, the defect detection of the wire clamp is important work of routing inspection. More importantly: the wire clamp is of various types, and a large number of different types of wire clamps need to be distinguished. At present overhead transmission line's the in-process of patrolling and examining, the manual work is patrolled and examined very consuming time, and patrols and examines personnel's work and has certain danger.

In recent years, with the rapid development of unmanned aerial vehicle technology, efficient and stable unmanned aerial vehicles are applied in other fields. Many electric power companies in China begin to adopt unmanned aerial vehicles to assist in inspection work. The picture of shooing through unmanned aerial vehicle acquires the fastener picture, carries out defect identification through the detection to the fastener picture. In the current target detection field, the application of a convolutional neural network enables the performance of target detection to have a great leap, fast R-CNN belongs to a two-stage target detection algorithm, and in the first step, propofol is extracted; and secondly, completing target detection through classification and regression, wherein the classification is to mark the category of the target of interest, and the regression is to correct a bounding box (a bounding box) of the object. YOLO and SSD belong to a single-stage detection algorithm, and target detection is directly realized through classification and regression; the Faster R-CNN belongs to a two-stage target detection algorithm and can achieve higher precision than a single-stage detection algorithm. The method can extract the characteristics of the target from the color and the texture of the target and from various characteristic multi-color channels, can filter part of noise and environmental interference through convolution, and keeps accurate recognition rate, which is significant for the fact that a wire clamp can still keep higher recognition rate under the condition of complex background.

The wire clamps are various in types, after the power grid wire clamp detection model is operated on line, new power grid wire clamps are always missed to be detected due to the fact that training data do not exist in the early stage, the fast R-CNN model is trained by using a large amount of wire clamp data, so that the model can be converged, the wire clamps in the picture to be detected can be fitted, and the purposes of detecting the wire clamps and classifying the wire clamps are achieved. However, in practical applications, a large number of clips with different characteristics, such as appearance and texture, are difficult to collect at one time, so that a clip with new characteristics appears, and the current solution is to train the currently collected data and the previously collected data together again.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a method and a device for automatically detecting and identifying the defects of the power grid wire clamp, which aim at performing augmentation training on a full connection layer aiming at a newly added category so as to realize the rapid update of a power grid wire clamp target detection model and the automatic identification of the defects of the power grid wire clamp.

The technical scheme adopted by the invention is as follows:

a power grid wire clamp automatic detection and defect identification method comprises the following steps:

acquiring a wire clamp picture through an image acquisition module;

training a wire clamp picture to obtain the type of a wire clamp;

classifying, deleting and selecting the wire clamp pictures;

and identifying and outputting the fault wire clamp.

The further technical scheme of the invention is as follows: obtain the fastener picture through image acquisition module, specifically include: adopt the shooting mode that unmanned aerial vehicle and camera device combined together, shoot the fastener picture that obtains the fastener on the transmission line.

The further technical scheme of the invention is as follows: training the wire clamp picture to obtain the wire clamp category; the method specifically comprises the following steps:

selecting a certain number of clip pictures as training samples;

and extracting the characteristics of the wire clamp by using a convolution kernel in Fast R-CNN to train to obtain the category of the wire clamp.

The further technical scheme of the invention is as follows: training the wire clamp picture to obtain the wire clamp category; further comprising: and performing augmentation training on the unclassified wire clamp pictures to obtain the wire clamp augmentation types.

Further, the wire clamp characteristics include wire clamp color, wire clamp profile, and wire clamp texture.

The further technical scheme of the invention is as follows: classifying, deleting and selecting the wire clamp pictures; the method specifically comprises the following steps:

generating a bounding box on the clip sheet through the area proposal network;

judging whether the inside of the surrounding frame is a target wire clamp or not;

if the target wire clamp is in the surrounding frame, the classification of the wire clamp picture is finished by distinguishing the classification through a classifier in Fast R-CNN.

The further technical scheme of the invention is as follows: identifying and outputting a fault wire clamp; the method specifically comprises the following steps:

training a sample picture of a fault point of the wire clamp to obtain the fault category of the wire clamp;

extracting the bounding box of the target wire clamp based on the convolution kernel in Fast R-CNN,

carrying out fault location on a target wire clamp in the surrounding frame, and identifying the fault type;

and outputting the fault position and the fault type.

The invention also provides a device for automatically detecting the power grid wire clamp and identifying the defects, which comprises the following components:

the image acquisition module is used for acquiring a wire clamp picture;

the wire clamp type training module is used for training a wire clamp picture to obtain a wire clamp type;

the wire clamp classifying module is used for classifying, deleting and selecting the wire clamp pictures;

and the wire clamp fault detection module is used for identifying and outputting the fault wire clamp.

The invention has the beneficial effects that:

the invention acquires the image of the wire clamp on the power transmission line through the image acquisition module, trains the wire clamp picture by adopting the Faster R-CNN technology to obtain the wire clamp type, realizes the classification of the wire clamp type, and simultaneously realizes the fault identification of the wire clamp device of the power transmission line, locates the fault point and detects the fault type through the training and identification of the fault wire clamp. The method adopts the augmentation training technology, and performs augmentation training on a part of the wire clamp pictures through the full-connection layer for the subsequently added wire clamp characteristic types, so that the defects of long time and slow model updating period caused by retraining new wire clamp characteristics are overcome, and the detection and identification efficiency is improved.

Drawings

FIG. 1 is a flow chart of a method for automatically detecting a power grid wire clamp and identifying defects, which is provided by the invention;

fig. 2 is a flow chart of the process of identifying and outputting a faulty wire clamp according to the present invention;

FIG. 3 is a flow chart of the Faster R-CNN operation proposed by the present invention;

FIG. 4 is a diagram of a Faster R-CNN network according to the present invention;

fig. 5 is a structural diagram of an automatic power grid wire clamp detection and defect identification device provided by the invention.

Detailed Description

The invention provides a method for automatically detecting a power grid wire clamp and identifying defects, which is used for researching a model augmentation training technology when a new power grid wire clamp target class appears. The fast R-CNN is a pre-stage and mid-stage structure taking a convolution layer and a pooling layer as models and can be collectively called a feature extraction part, a feature diagram in a high-dimensional matrix form is finally generated in the feature extraction part, generally, the feature diagram does not exceed four dimensions, and finally, the feature diagram is sent to a full connection layer for classification.

The target detection means that an interested target in an image is identified and the position and the size of the target are determined; the classification is mainly to label the image based on the content of the image, usually there is a fixed set of labels, and the established model must predict the label most suitable for the image.

The training model is to use a large amount of image data which are marked with what is the wire clamp to let the network "learn" what is the wire clamp and the position of the wire clamp. Namely: inputting data (marked pictures) into a neural network (each neuron inputs values for weighted accumulation and then inputs an activation function as an output value of the neuron) for forward propagation to obtain a score; inputting a loss function (regularization punishment and over-fitting prevention) into the score, comparing the score with an expected value to obtain errors, and judging the identification degree (the smaller the loss value is, the better the identification degree) through the errors if a plurality of scores are sums; determining gradient vectors by back propagation (back-derivatives, loss functions and each activation function in the neural network require, with the ultimate goal of minimizing errors); finally, each weight is adjusted through a gradient vector, and the error tends to 0 or the convergence trend is adjusted towards the score; the above process is repeated until the set number of times or the average value of the loss error no longer falls (lowest point).

Fully connected layers (FC) act as "classifiers" throughout the convolutional neural network. If we say that operations such as convolutional layers, pooling layers, and activation function layers map raw data to hidden layer feature space, the fully-connected layer serves to map the learned "distributed feature representation" to the sample label space. In CNN, full concatenation is often present in the last few layers for weighted summation of previously designed features. For example, fast R-CNN, the former convolution and pooling is equivalent to feature engineering, and the latter full join is equivalent to feature weighting.

The technical scheme provided by the invention has the following general idea:

the invention provides a method for automatically detecting and identifying a defect of a power grid wire clamp, which adopts a model augmentation training technology during the target classification of the power grid wire clamp, namely, new data is collected and then is not trained together with the previous data, but the newly collected data is utilized to augment and train a full connection layer aiming at newly added wire clamps with different characteristics, so that the model updating period is greatly shortened. The accuracy of automatic detection and defect identification of the power grid wire clamp is improved.

The foregoing is the core idea of the present application, and in order to make those skilled in the art better understand the scheme of the present application, the present application will be further described in detail with reference to the accompanying drawings. It should be understood that the specific features in the embodiments and examples of the present application are detailed description of the technical solutions of the present application, and are not limited to the technical solutions of the present application, and the technical features in the embodiments and examples of the present application may be combined with each other without conflict.

Example one

As shown in fig. 1, the invention provides a flow chart of a method for automatically detecting a power line clamp and identifying a defect.

Referring to fig. 1, a method for automatically detecting a power grid wire clamp and identifying defects includes the following steps:

step 101, acquiring a wire clamp picture through an image acquisition module;

step 102, training a wire clamp picture to obtain the type of a wire clamp;

103, classifying and deleting the clip pictures;

and 104, identifying and outputting the fault wire clamp.

In the embodiment of the invention, the image acquisition module is used for acquiring the image of the wire clamp on the power transmission line, the Faster R-CNN technology is used for training the wire clamp picture to obtain the wire clamp type, the classification of the wire clamp type is realized, and meanwhile, the fault identification of a wire clamp device of the power transmission line is realized, the fault point is positioned and the fault type is detected. The method adopts the augmentation training technology, and performs augmentation training on a part of the wire clamp pictures through the full-connection layer for the subsequently added wire clamp characteristic types, so that the defects of long time and slow model updating period caused by retraining new wire clamp characteristics are overcome, and the detection and identification efficiency is improved.

In step 101, the obtaining of the clip picture through the image acquisition module specifically includes: adopt the shooting mode that unmanned aerial vehicle and camera device combined together, shoot the fastener picture that obtains the fastener on the transmission line.

Specifically, shoot the transmission line picture under different backgrounds, different angles through unmanned aerial vehicle, increase the variety of sample to the generalization ability of better improvement model. And screening the shot pictures, and deleting the pictures without the wire clamps.

In step 102, training a wire clamp picture to obtain a wire clamp category; the method specifically comprises the following steps:

selecting a certain number of clip pictures as training samples;

and extracting the characteristics of the wire clamp by using a convolution kernel in Fast R-CNN to train to obtain the category of the wire clamp.

The method comprises the steps of firstly processing and training acquired clamp pictures, wherein a certain number of clamp pictures are obtained for the first time and are used as training samples, clamp characteristics are obtained through processing of the clamp pictures, so that clamp types are determined, undetermined clamp types can be added through subsequent augmentation training for the clamp pictures obtained through subsequent shooting, a certain number of clamp types are not limited, the clamp types can be determined according to the clamp pictures obtained for the first time, one part of the clamp types can be selected as the training samples, specifically, the actual design is taken as the standard, and optionally, clamps in partial images are used as the training samples, and characteristics such as clamp colors, profiles and textures are extracted through convolution kernels in Fast R-CNN and are used as characteristic vectors for training.

Training a wire clamp picture to obtain a wire clamp category; further comprising: and performing augmentation training on the unclassified wire clamp pictures to obtain the wire clamp augmentation types.

When the collected clip images do not exist, all the images do not need to be retrained again, and the training method based on the model augmentation training of the full-connection layer augmentation can be adopted. The specific method is to perform augmentation training on the full connection layer aiming at the newly added category on the newly collected data, namely performing model fine tuning on the originally trained model.

In the embodiment of the invention, the wire clamp pictures are classified, deleted and selected; the method specifically comprises the following steps:

generating a bounding box on the clip sheet through the area proposal network;

judging whether the inside of the surrounding frame is a target wire clamp or not;

if the target wire clamp is in the surrounding frame, the classification of the wire clamp picture is finished by distinguishing the classification through a classifier in Fast R-CNN.

During detection, surrounding frames which can be targets are generated on an image through the regional proposal network, the trained model can judge the surrounding frames, whether the surrounding frames are the target wire clamps or not is detected, the image detection method can be effectively suitable for various different types of wire clamps to position the image positions of the wire clamps, and the categories of the wire clamps can be distinguished through a classifier in Fast R-CNN.

Referring to fig. 2, a flow chart is provided for identifying and outputting the faulty wire clamp provided by the present invention;

as shown in fig. 2, identifying and outputting the faulty wire clamp specifically includes:

step 141, training a clamp fault point sample picture to obtain a clamp fault category;

142, extracting an enclosure of the target wire clamp through a convolution kernel based on Fast R-CNN;

step 143, carrying out fault location on the target wire clamp in the surrounding frame, and identifying the fault type;

and step 144, outputting the fault position and the fault type.

For the fault location of the wire clamp: the training process of the wire clamp fault is classified with the detection of the same wire clamp, and because the characteristics of the wire clamp are different from those of other wire clamps when the wire clamp has a rust fault or other faults, the surrounding frame where the wire clamp string fault is located can be detected by extracting the characteristics of the outline, the texture and the like of the wire clamp based on the convolution kernel in Fast R-CNN, and the fault position is located.

The wire clamp fault detection based on the Faster R-CNN is divided into two detection processes: and (4) wire clamp classification detection and wire clamp fault positioning. The work flow chart is as shown in fig. 1, and for the massive photos of the unmanned aerial vehicle transmission line aerial photography, most photos can not be used for detecting the wire clamp faults, so that the massive photos are intelligently screened and classified through wire clamp classification detection, and a foundation is laid for fault detection in the next step. The training positive samples of the wire clamp classification detection module are wire clamps of various categories, and the identification can be achieved in a complex background in an energy-saving mode. The method has the advantages that the classified screened photos are subjected to wire clamp fault detection and positioning, the training positive sample of the module is the fault point of the wire clamp, so that the method can be used for positioning the fault point under the rusty fault condition of the wire clamp, the fault point is formed by the surrounding frame in the graph of the fault wire clamp, the fault type is displayed, and the intelligent detection of the wire clamp fault is realized.

Fast R-CNN workflow diagram as shown in fig. 3, the fast R-CNN method includes 2 CNN networks: area proposed networks RPN (regional Proposal network) and Fast R-CNN detect networks. The RPN is a full convolution network, and the core idea is to directly generate a region proposal by using a convolution neural network, and generate a series of candidate frames with multi-scale and multi-aspect ratio in a picture. Fast R-CNN detects and identifies targets therein based on RPN extracted candidate boxes.

FIG. 4 is a diagram of the fast R-CNN network structure, and the specific detection process is as follows: the method comprises the steps that an input image firstly passes through a convolution layer, the size of the image is reduced and image features are extracted in depth through convolution and pooling, a feature map is formed on a last layer of convolution base layer, the feature map is deep convolution features of the input image, and deep features of similar objects are very close to each other; and the deep features of different objects are very different, namely the objects have good separability on the feature map. And the RPN network performs window sliding on the feature map, extracts a series of candidate frames and judges whether the window is a target or a background. And finally, detecting a network through Fast R-CNN, wherein the network also extracts the features of the feature map, the extracted region features and the RPN extracted candidate frame pass through an ROI posing layer, the ROI posing layer simultaneously collects the feature map and the candidate frame, extracts the region features, sends the region features into the subsequent two layers of full-connected layers, judges the category of the target through softmax, simultaneously displays the position of the image where the target is located, and corrects the detection frame, so that the position of the detection frame is more accurate. The Faster R-CNN method can be used for classifying and positioning, can be used for finding out the positions of the wire clamps and identifying the types of the wire clamps at the same time, can also be used for carrying out batch processing on the fault types of the wire clamps, and is efficient and intelligent.

Example two

Based on the same inventive concept as the automatic detection and defect identification method of the power grid wire clamp in the previous embodiment, the invention also provides an automatic detection and defect identification device of the power grid wire clamp.

Referring to fig. 5, an automatic detection and defect recognition device for a power grid wire clamp includes:

the image acquisition module 201 is used for acquiring a wire clamp picture;

the clamp type training module 202 is used for training a clamp picture to obtain a clamp type;

the wire clamp classifying module 203 is used for classifying, deleting and selecting the wire clamp pictures;

and the wire clamp fault detection module 204 is used for identifying and outputting a fault wire clamp.

Various changes and specific examples of the method for automatically detecting and identifying a power grid wire clamp in the first embodiment are also applicable to the device for automatically detecting and identifying a power grid wire clamp in the present embodiment, and through the foregoing detailed description of the method for automatically detecting and identifying a power grid wire clamp, those skilled in the art can clearly know the device for automatically detecting and identifying a power grid wire clamp in the present embodiment, so for the sake of brevity of the description, detailed descriptions are omitted here.

The above examples are typical examples of the present invention, but the embodiments of the present invention are not limited to the above examples. Other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principles of the invention are intended to be included within the scope of the invention.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Finally, it should be noted that: the above embodiments are only intended to illustrate the technical solution of the present invention and not to limit the same, and a person of ordinary skill in the art can make modifications or equivalents to the specific embodiments of the present invention with reference to the above embodiments, and such modifications or equivalents without departing from the spirit and scope of the present invention are within the scope of the claims of the present invention as set forth in the claims.

Claims (8)

1. A power grid wire clamp automatic detection and defect identification method is characterized by comprising the following steps:

acquiring a wire clamp picture through an image acquisition module;

training a wire clamp picture to obtain the type of a wire clamp;

classifying, deleting and selecting the wire clamp pictures;

and identifying and outputting the fault wire clamp.

2. The method according to claim 1, wherein the obtaining of the clip picture by the image acquisition module specifically comprises: adopt the shooting mode that unmanned aerial vehicle and camera device combined together, shoot the fastener picture that obtains the fastener on the transmission line.

3. The method of claim 1, wherein the training of the clip picture obtains a clip class; the method specifically comprises the following steps:

selecting a certain number of clip pictures as training samples;

and extracting the characteristics of the wire clamp by using a convolution kernel in Fast R-CNN to train to obtain the category of the wire clamp.

4. The method of claim 3, wherein the training of the clip picture obtains a clip class; further comprising: and performing augmentation training on the unclassified wire clamp pictures to obtain the wire clamp augmentation types.

5. The method of claim 3, wherein the clip characteristics include clip color, clip profile, clip texture.

6. The method of claim 1, wherein the classification and deletion of the clip pictures is performed; the method specifically comprises the following steps:

generating a bounding box on the clip sheet through the area proposal network;

judging whether the inside of the surrounding frame is a target wire clamp or not;

if the target wire clamp is in the surrounding frame, the classification of the wire clamp picture is finished by distinguishing the classification through a classifier in Fast R-CNN.

7. The method of claim 1, wherein a faulty wire clamp is identified and output; the method specifically comprises the following steps:

training a sample picture of a fault point of the wire clamp to obtain the fault category of the wire clamp;

extracting the bounding box of the target wire clamp based on the convolution kernel in Fast R-CNN,

carrying out fault location on a target wire clamp in the surrounding frame, and identifying the fault type;

and outputting the fault position and the fault type.

8. The method according to any one of claims 1 to 7, which provides an automatic detection and defect identification device for a power grid wire clamp, and is characterized by comprising the following steps:

the image acquisition module is used for acquiring a wire clamp picture;

the wire clamp type training module is used for training a wire clamp picture to obtain a wire clamp type;

the wire clamp classifying module is used for classifying, deleting and selecting the wire clamp pictures;

and the wire clamp fault detection module is used for identifying and outputting the fault wire clamp.