CN111723774A - A target recognition method for power transmission equipment based on UAV inspection - Google Patents

Abstract

The invention discloses a target identification method for power transmission equipment based on drone inspection. First, the drone collects image data, extracts the image of the target to be identified in the image data, establishes a target sample set, and then analyzes the images in the target sample set. Perform manual annotation to create a defect sample library of inspection images, and then use the defect sample library of inspection images as training samples to create a defect calibration model based on the Faster‑RCNN network, and then use drones to collect real-time inspection image data. The real-time inspection image data is input into the defect calibration model, and finally the target identification result is output and stored in a structured manner; the invention can carry out unified and intelligent management of massive UAV inspection images, and improve the automation at the tower level and the hardware level. Image classification ability, improve the accuracy of analysis, and greatly reduce the workload of manual analysis and processing.

Description

A target recognition method for power transmission equipment based on UAV inspection

technical field

The invention relates to the technical field of unmanned aerial vehicle inspection and power transmission equipment maintenance, in particular to a power transmission equipment target identification method based on unmanned aerial vehicle inspection.

Background technique

Transmission line inspection is a regular inspection of transmission lines and tower equipment. Defects found in the inspection process need to be recorded and eliminated in time. Power transmission line inspection can detect hidden dangers and faults in the operation of equipment in advance, and deal with the defects in time, thereby effectively preventing the occurrence of power grid faults and reducing the threats to the personal safety of residents, property safety and industrial generation. Therefore, in the transmission equipment It plays an important role in the daily maintenance work.

At present, the inspection of transmission lines usually adopts the manual inspection method, which not only consumes a lot of manpower, but also has low efficiency and low accuracy. With the application of unmanned aerial vehicles, the proportion of unmanned aerial vehicle inspection operations in the daily inspection work of transmission lines is increasing, and the line coverage has also been rapidly improved. It has brought a lot of obstacles to the later work of the inspection.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a target identification method for power transmission equipment based on UAV inspection, which can carry out unified intelligent management of massive UAV inspection images, and improve the automatic image classification ability at the tower level and the hardware level. Improve the accuracy of analysis, and greatly reduce the workload of manual analysis and processing.

The technical scheme adopted in the present invention is:

A target identification method for power transmission equipment based on UAV inspection, comprising the following steps:

A. Use drone inspection to capture the image data of the area where the power transmission equipment is located, extract the image of the target to be identified in the image data, and establish a target sample set;

Classify, reorganize, locate and randomly rename image data captured by drone inspections, and convert randomly named and disordered images into structured data;

The target to be identified includes N types, the image of the target i (i≤N) to be identified is collected to form a sample subset A _i , and the sample subset A1 to the sample subset A _N form a target sample set;

B. Manually label the images in the target sample set to create a defect sample library for inspection images;

C. Use the defect sample library of inspection images as training samples to create a defect calibration model based on Faster-RCNN network;

Use the training model based on Faster-RCNN network to detect the target to be identified in the training sample, use the neural network algorithm for training, and then use the YOLOV3 algorithm to create a defect calibration model;

D. Use drones to collect real-time inspection image data, and input real-time inspection image data into the defect calibration model;

E. The defect calibration model outputs the target recognition results and stores them in a structured manner.

Further, in the step B, the specific process of manually labeling the images in the target sample set is as follows: first label the content of the area in the image, and then label the hidden dangers and defects found in the area.

Further, the manual labeling process can calibrate the defects existing in the image by means of frame selection. During the calibration process, the zooming and dragging of the image can be realized, and the modification, deletion and addition of the calibration frame can be supported, and the defect information can be adjusted at the same time. Carry out structured storage and display; defect information includes the type and urgency of the defect.

Further, the specific process of training the model to detect the target to be identified in the training sample in step C is as follows:

c1. Use the candidate frame selection network RPN to determine the candidate frame of the training sample; the images in the candidate frame include three cases: including power transmission equipment, including non-power transmission equipment, including power transmission equipment and non-power transmission equipment;

c2, feature mapping; use the convolutional neural network to extract the features of the image in the candidate frame;

c3. Use a classifier to classify the extracted features, and determine whether the image in the candidate frame contains power transmission equipment;

c4. Area refinement, so that the edge of the candidate frame determined to be equipment tightly wraps the power transmission equipment, so that the area of the candidate frame is sufficient to accommodate the included power transmission equipment and the area is the smallest.

Further, the classifier in the step c3 adopts multi-classifier softmax.

The present invention has the following beneficial effects:

(1) Target recognition based on UAV inspection data analysis can make discrete and disordered image data move toward unified and orderly standardized management, and automatically rename images according to UAV inspection tasks to ensure inspection. The inspection images have use value and can be traced back to the history, realize the standardized operation of UAV inspection, and improve the standardized operation level of UAV inspection;

(2) Through intelligent operations such as analysis, clustering, identification, and archiving of UAV inspection data, the manual workload of image processing in the office can be reduced by at least 30%, greatly reducing the workload of personnel analysis and processing, and reducing Manual identification and processing of false positive rate and false negative rate, improve the accuracy and effectiveness of analysis, improve the efficiency and effectiveness of drone inspection work, and improve the coverage rate of inspection.

Detailed ways

The invention discloses a target identification method for power transmission equipment based on drone inspection, comprising the following steps:

A. Use drone inspection to capture the image data of the area where the power transmission equipment is located, extract the image of the target to be identified in the image data, and establish a target sample set;

Classify, reorganize, locate and randomly rename image data captured by drone inspections, and convert randomly named and disordered images into structured data;

The targets to be identified include N types, the images of the target i (i≤N) to be identified are collected to form a sample subset Ai, and the sample subsets A1 to _{A N} form a target sample set;

B. Manually label the images in the target sample set to create a defect sample library for inspection images;

C. Use the defect sample library of inspection images as training samples to create a defect calibration model based on Faster-RCNN network;

Use the training model based on Faster-RCNN network to detect the target to be identified in the training sample, use the neural network algorithm for training, and then use the YOLOV3 algorithm to create a defect calibration model;

D. Use drones to collect real-time inspection image data, and input real-time inspection image data into the defect calibration model;

E. The defect calibration model outputs the target recognition results and stores them in a structured manner.

In order to better understand the present invention, the technical solutions of the present invention are further described below with reference to specific embodiments.

A target identification method for power transmission equipment based on UAV inspection, comprising the following steps:

A. Use drone inspection to capture image data of the area where the power transmission equipment is located, extract the image of the target to be identified in the image data, and establish a target sample set.

The image data captured by the drone inspection is classified, reorganized, positioned and randomly renamed, and the randomly named and disordered images are converted into structured data.

The targets to be identified include N types. The images of the target i to be identified (i≤N) are collected to form a sample subset A _i , and the sample subsets A ₁ to _AN form a target sample set.

The targets to be identified in the inspection of transmission lines include tower bird nests, anti-vibration hammers, pins and other categories. The image data of each category constitutes its corresponding sample subset, and the sample subsets of all categories of targets to be identified constitute the target sample. set. When establishing the recognition model, a corresponding database is established for each classified target to be recognized.

B. Manually label the images in the target sample set to create a defect sample library for inspection images.

Among them, the specific process of manually labeling the images in the target sample set is as follows: first label the content of the area in the image, and then label the hidden dangers and defects found in the area.

The manual labeling process can calibrate the defects existing in the image by frame selection. During the calibration process, the image can be zoomed and dragged, and the calibration frame can be modified, deleted and added, and the defect information can be stored in a structured manner. and display; defect information includes the type and urgency of the defect.

In order to improve the accuracy and comprehensiveness of the training samples of the defect calibration model, the following principles need to be followed in the manual labeling process:

(1) All the whole and components of all the targets to be identified that appear in the image must be marked, and the marking cannot be omitted; for example, all the nut areas appearing in the pin area must be marked, and the marking cannot be missed.

(2) As long as there is a target to be identified in the image, no matter whether the target to be identified can be seen clearly, it must be marked; for example, if there is a position where the pin is installed in the image, it needs to be marked when the pin can be clearly displayed, due to occlusion and other reasons. It is not possible to show the pin in its entirety, but it also needs to be marked when there should be a complete pin at that location.

(3) For areas with large backgrounds and complex backgrounds, because there are too many interference items, the target to be identified cannot be clearly distinguished, so no labeling is required.

(4) When selecting the target to be identified, it needs to be completely framed, and interference items cannot be framed.

(5) The area of the calibration frame should be as small as possible under the premise that it can completely wrap the target to be identified.

C. Use the defect sample library of the inspection image as the training sample to create a defect calibration model based on the Faster-RCNN network; use the training model based on the Faster-RCNN network to detect the targets to be identified in the training samples, and use the neural network algorithm for training, Then use the YOLOV3 algorithm to create a defect calibration model.

Compared with Faster-RCNN, SSD has only one main neural network (base network), removes the RPN network, extracts some feature maps of various scales from the middle, adds a small convolution layer on the side to predict the position and type of objects, and then uses All positions and categories are integrated to output the final result. SSD runs faster because it removes a network. Compared with Faster-RCNN, some special target detection needs to be verified.

Among them, the specific process of training the model to detect the target to be identified in the training sample is as follows:

c1. Use the candidate frame selection network RPN to determine the candidate frame of the training sample; the images in the candidate frame include three cases: including power transmission equipment, including non-power transmission equipment, including power transmission equipment and non-power transmission equipment.

Since the position and scale of the device in the image are unknown, if the smooth window traversal method is used, it is necessary to use multiple scales to translate with various lengths, resulting in a huge number of windows to be determined, which in turn results in tedious calculations. The RPN method or the Selective Search method can only generate a small number of the most probable candidate boxes, so that the entire recognition process can be completed quickly. Relatively speaking, the RPN method is faster than the Selective Search method, and the candidate box search is more accurate.

c2. Feature mapping; use convolutional neural network to extract the features of the image in the candidate frame. Convolutional neural network is the most basic classification network of deep learning, and the classification performance and feature extraction accuracy are excellent.

c3. Use the multi-classifier softmax to classify the extracted features, and determine whether the image in the candidate frame contains power transmission equipment.

Softmax is a multi-classifier, which is different from binary classifiers such as logistic regression, which need to determine each device one by one (that is, determine whether it is device A, then determine whether it is device B, and so on), Softmax can complete the classification at one time, The computational performance and classification accuracy have been greatly improved.

c4. Area refinement, so that the edge of the candidate frame determined to be equipment tightly wraps the power transmission equipment, so that the candidate frame area is sufficient to accommodate the included power transmission equipment and the area is the smallest; because the original candidate area may only contain most of the equipment or except for the equipment. There are other background parts, so region refinement can improve the accuracy of the candidate regions.

D. Use drones to collect real-time inspection image data, and input real-time inspection image data into the defect calibration model.

E. The defect calibration model outputs the target recognition results and stores them in a structured manner.

At present, the main targets of transmission line inspections include transmission lines and tower equipment, etc. The following is some experimental data of the drone inspection of State Grid Shandong Electric Power Rizhao City Power Supply Company:

Tower bird's nest identification:

Using the deep learning image recognition and positioning algorithm Faster-RCNN, use a sample set of not less than 2000 tower images containing the bird's nest for training. After training, the trained recognition model is used to directly locate the bird's nest rectangular box as the output of the recognition result.

Recognition of the part of the anti-vibration hammer falling off:

Using the deep learning image recognition and positioning algorithm Faster-RCNN, using a sample set of not less than 2,000 tower images containing the anti-vibration hammer, to train the identification model of the whole anti-vibration hammer, the left side of the anti-vibration hammer, and the right side of the anti-vibration hammer, respectively, During the image recognition process, if the identified left hammer body of the anti-vibration hammer or the right hammer body of the anti-vibration hammer is not within the overall area of the anti-vibration hammer, it is determined that the part of the anti-vibration hammer has fallen off.

Anti-vibration hammer displacement identification:

Using the deep learning image recognition and positioning algorithm Faster-RCNN, using a sample set of not less than 2000 tower images containing the anti-vibration hammer to train the whole anti-vibration hammer, and identify the overall rectangular frame of the anti-vibration hammer in the image. The horizontal distance of the rectangular frame in the same height range is less than A pair of anti-vibration hammers of the width of the rectangular frame are judged to be displaced at least one.

Recognition of missing pins of nuts:

Using the deep learning image recognition and positioning algorithm Faster-RCNN, using a sample set of no less than 2000 images of towers and towers including hanging rings and hanging plates, etc., to train the pin recognition model, the connection fitting recognition model, and the nut recognition model. First identify the connecting hardware, then identify the nut, and then identify the pin. If there is only a nut but no pin, the pin is missing.

Cross arm nut missing identification:

Using the deep learning image recognition and positioning algorithm Faster-RCNN, using a sample set of not less than 2000 images of towers containing cross-arm nuts, train the overall recognition model of cross-arm nuts and nuts and bolts, first identify the whole nut and bolt, and identify the nut inside the whole. , if the number of nuts is less than two, the nuts are considered missing.

Through the above experiments, using the method of the present invention, the intelligent operations such as analysis, clustering, identification, and archiving of the drone inspection data of the State Grid Shandong Electric Power Rizhao City Power Supply Company can reduce the manual work of image processing in the internal industry. At least 30% of the results, the effect is excellent.

The present invention performs unified and intelligent management of massive drone inspection images, that is, applying artificial intelligence technology to improve the automatic image classification capability at the tower level and hardware level, and performs structured transformation on the defective objects in the images to realize defect data. The structured analysis of UAV greatly reduces the workload of manual analysis and processing, improves the accuracy of analysis, promotes the reform of UAV inspection mode and management mode, and helps to further promote the full coverage of UAV inspection operations.

Claims (5)

1. a power transmission equipment target identification method based on unmanned aerial vehicle inspection, is characterized in that: comprise the following steps: A. Use drone inspection to capture the image data of the area where the power transmission equipment is located, extract the image of the target to be identified in the image data, and establish a target sample set; Classify, reorganize, locate and randomly rename image data captured by drone inspections, and convert randomly named and disordered images into structured data; The targets to be identified include N types, the images of the target i (i≤N) to be identified are collected to form a sample subset A _i , and the sample subsets A ₁ to A _N form a target sample set; B. Manually label the images in the target sample set to create a defect sample library for inspection images; C. Use the defect sample library of inspection images as training samples to create a defect calibration model based on Faster-RCNN network; Use the training model based on Faster-RCNN network to detect the target to be identified in the training sample, use the neural network algorithm for training, and then use the YOLOV3 algorithm to create a defect calibration model; D. Use drones to collect real-time inspection image data, and input real-time inspection image data into the defect calibration model; E. The defect calibration model outputs the target recognition results and stores them in a structured manner. 2. The power transmission equipment target identification method based on UAV inspection according to claim 1, wherein the specific process of manually labeling the images in the target sample set in the step B is: The content is marked, and the hidden dangers and defects found in the area are marked. 3. The method for identifying targets for power transmission equipment based on UAV inspection according to claim 2, wherein the manual labeling process can demarcate defects existing in the image by frame selection, and in the calibration process, It can realize the zooming and dragging of the image, support the modification, deletion and addition of the calibration frame, and store and display the defect information in a structured manner; the defect information includes the type and urgency of the defect. 4. the power transmission equipment target identification method based on unmanned aerial vehicle inspection according to claim 1, is characterized in that: in step C, training model detects the specific process of target to be identified in training sample as follows: c1. Use the candidate frame selection network RPN to determine the candidate frame of the training sample; the images in the candidate frame include three cases: the first type: including power transmission equipment; the second type: including non-power transmission equipment; the third type: including power transmission equipment and non-power transmission equipment power transmission equipment; c2, feature mapping; use the convolutional neural network to extract the features of the image in the candidate frame; c3. Use a classifier to classify the extracted features, and determine whether the image in the candidate frame contains power transmission equipment; c4. Area refinement, so that the edge of the candidate frame determined to be equipment tightly wraps the power transmission equipment, so that the area of the candidate frame is sufficient to accommodate the included power transmission equipment and the area is the smallest. 5 . The method for identifying targets of power transmission equipment based on UAV inspection according to claim 4 , wherein the classifier in the step c3 adopts a multi-classifier softmax. 6 .