CN114529839A - Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system - Google Patents

Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system Download PDF

Info

Publication number
CN114529839A
CN114529839A CN202210123286.7A CN202210123286A CN114529839A CN 114529839 A CN114529839 A CN 114529839A CN 202210123286 A CN202210123286 A CN 202210123286A CN 114529839 A CN114529839 A CN 114529839A
Authority
CN
China
Prior art keywords
hardware
aerial vehicle
unmanned aerial
image
transmission line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210123286.7A
Other languages
Chinese (zh)
Inventor
郭志民
刘昊
李哲
田杨阳
卢明
梁允
张小斐
刘善峰
赵健
毛万登
王超
袁少光
王津宇
贺翔
耿俊成
李斌
许丹
陈岑
魏小钊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Henan Electric Power Co Ltd
Electric Power Research Institute of State Grid Henan Electric Power Co Ltd
Original Assignee
State Grid Henan Electric Power Co Ltd
Electric Power Research Institute of State Grid Henan Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Henan Electric Power Co Ltd, Electric Power Research Institute of State Grid Henan Electric Power Co Ltd filed Critical State Grid Henan Electric Power Co Ltd
Priority to CN202210123286.7A priority Critical patent/CN114529839A/en
Publication of CN114529839A publication Critical patent/CN114529839A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

A power transmission line hardware abnormity detection method and system for unmanned aerial vehicle routing inspection comprises the following steps: step 1, an unmanned aerial vehicle patrols and acquires a hardware defect image on a high-voltage transmission line; step 2, marking the hardware defect image obtained by the unmanned aerial vehicle inspection, and taking the marked hardware defect image as a sample set; step 3, improving based on a Faster RCNN model, and constructing an abnormality detection model; step 4, training the constructed anomaly detection model based on the sample set to obtain a trained anomaly detection model; and 5, using the trained abnormity detection model to perform abnormity detection processing on hardware fitting images on the high-voltage transmission line acquired by the unmanned aerial vehicle inspection, and outputting a detection result. The invention solves the problem of false detection in a complex environment by changing the method for extracting the features in the early detection stage and utilizing the relevance of the candidate region, and improves the detection rate and the detection accuracy of detecting the smaller hardware target in the image.

Description

Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system
Technical Field
The invention relates to the technical field of intelligent detection, in particular to a power transmission line hardware tool abnormity detection method for unmanned aerial vehicle inspection.
Background
To the unusual detection of the gold utensil on the high tension transmission line, the mode that traditional approach adopted the manual work to patrol and examine carries out the anomaly detection to it, however traditional manual work is patrolled and examined inefficiency, the security is low and labour cost is high, consequently unmanned aerial vehicle has become the instrument commonly used of high tension transmission line patrolling and examining in recent years, but unmanned aerial vehicle can only play the effect of data acquisition, want to really get rid of traditional manual detection, improve detection efficiency, need carry out real-time detection to the data that unmanned aerial vehicle gathered.
Detecting hardware by adopting an image recognition and detection mode of deep learning is a common detection means, for example, the prior art document 1(CN109344753A) discloses a method for identifying tiny hardware of a power transmission line based on an aerial image of deep learning, and the method carries out anti-shake and de-noising processing on the aerial image and establishes a tiny hardware identification image library; expanding the image library data by rotating, twisting and other methods; establishing an image tag library corresponding to the small hardware identification image library; establishing a small hardware fitting identification model by using a Faster R-CNN network, wherein the small hardware fitting identification model mainly comprises a feature extraction network, a regional suggestion network and a Fast R-CNN detection network; and (3) training the network by using aerial images in the image library, curing the deep neural network model according to whether the network parameters reach expected values during training, and storing the identified image information into a server for later-stage fine hardware fault detection.
However, for hardware detection on a high-voltage transmission line, a hardware target is small, a field background environment is complex, the hardware is seriously shielded, a small target under a complex background cannot be well identified in the prior art document 1, the detection effect is easily influenced by a shape-like interfering object on the ground or a building, and the conditions of missing detection and false detection are generally existed.
Therefore, the research on a reliable technology for carrying out abnormity detection on hardware fittings on the high-voltage transmission line in large batch has very important practical significance.
Disclosure of Invention
In order to solve the defects in the prior art, the invention aims to provide the power transmission line hardware abnormity detection method for unmanned aerial vehicle inspection, which is suitable for the abnormity detection problem of small hardware targets on the power transmission line, has good accurate detection rate, and improves the false detection condition in a complex scene.
The invention adopts the following technical scheme.
A power transmission line hardware abnormity detection method for unmanned aerial vehicle inspection comprises the following steps:
step 1, an unmanned aerial vehicle patrols and acquires a hardware fitting defect image on a high-voltage transmission line;
step 2, marking the hardware defect image obtained by the unmanned aerial vehicle inspection, and taking the marked hardware defect image as a sample set;
step 3, improving based on a fast RCNN model, and constructing an abnormality detection model by combining a self-attention mechanism;
step 4, training the constructed anomaly detection model based on a training mode and a sample set of transfer learning to obtain a trained anomaly detection model;
and 5, using the trained abnormity detection model to perform abnormity detection processing on hardware fitting images on the high-voltage transmission line acquired by the unmanned aerial vehicle inspection, and outputting a detection result.
Preferably, the step 2 further includes performing size transformation on the image to make the size of the image uniform to 512 × 512 pixels.
Preferably, the anomaly detection model is constructed based on a fast RCNN network, and includes a convolution layer, an RPN network, a ROI Pooling layer, and a fully-connected layer, and the fully-connected layer is used for classification regression calculation.
Preferably, the convolutional layer in the anomaly detection model uses a ResNet101 network as a residual network.
Preferably, the anomaly detection model further comprises a self-attention mechanism network, and the self-attention mechanism network is arranged behind the ROI Pooling layer.
Preferably, in step 4, the training of the anomaly detection model further includes:
training parameters trained on the PASAL VOC data set by the fast R-CNN official are used as initialization parameters of an abnormality detection model;
inputting the sample set into the initialized anomaly detection model to be trained for training;
and calculating the total loss of the anomaly detection model, and when the value of the total loss is not obviously reduced any more, indicating that the training is finished to obtain the trained anomaly detection model.
Preferably, the loss function includes a classification loss and a regression loss, wherein the classification loss is used for judging whether the anchor frame is positive or negative, and the regression loss is used for frame regression training.
Preferably, in the step 5, the detecting process of the input image by the anomaly detection model includes:
performing defect feature extraction on an input image by using a residual error network to obtain a feature map;
inputting the characteristic diagram into an RPN network to obtain a target area;
the target area is restrained by a non-maximum value to obtain an area suggestion frame;
transmitting the region suggestion frame into an ROI POOLING layer to obtain a region of interest;
the region of interest is transmitted into a self-attention mechanism frame for similarity measurement, and a feature map with the same size as the feature map of the region of interest is obtained;
and (4) carrying out classification regression on the characteristic graph output by the self-attention mechanism network after passing through the full connection layer, and outputting an abnormal detection result of the hardware fitting image.
Preferably, the abnormality detection model can detect a hardware part in the image and a confidence degree corresponding to the hardware, and determine whether the hardware is an abnormal hardware according to a preset confidence degree threshold, if so, frame the abnormal hardware with a labeling frame, and note a confidence value corresponding to the abnormal hardware at the side of the labeling frame, otherwise, no labeling is needed.
The invention also provides a power transmission line hardware tool abnormity detection system for unmanned aerial vehicle routing inspection, which comprises: the device comprises an image acquisition module, a size conversion module, a training module and a processing module;
the image acquisition module is used for acquiring an image of the power transmission line containing the hardware;
the size conversion module is used for carrying out size conversion on the image acquired by the image acquisition module so as to enable the image input into the processing module to be an image with uniform size;
the processing module comprises a constructed abnormity detection model and is used for carrying out abnormity detection on hardware in the input image and obtaining a detection result;
the training module can be used for training the abnormity detection model of the processing module by combining the image data of the image acquisition module.
Compared with the prior art, the invention has the following beneficial effects:
1. on the basis of small sample data, the method can transfer the training model parameters of the data set PASAL VOC data set to the model of the data set used by the method for transfer learning to serve as initialized parameters, so that the influence of a small amount of samples on model training is reduced, and the cost required by sample acquisition during training is reduced;
2. according to the method, the feature learning of the target object is enhanced before the classification and the judgment are carried out for the first time in the anomaly detection model, ResNet101 is used as a feature extraction network of a convolutional layer, the method for extracting the features in the early detection stage is changed, a residual error network with deeper network layers and without the problems of gradient disappearance, gradient explosion and the like is used for carrying out the feature extraction, and the phenomena of false detection and missed detection are reduced;
3. according to the method, before the second classification and discrimination in the anomaly detection model, the similarity measurement is carried out on the obtained candidate regions by adopting a self-attention mechanism, the problem of false detection in a complex environment is solved by utilizing the relevance of the candidate regions, and the detection rate and the detection accuracy of final detection are improved.
Drawings
Fig. 1 is an overall flow diagram of a power transmission line hardware abnormity detection method for unmanned aerial vehicle inspection according to the invention;
FIG. 2 is a schematic structural diagram of a ResNet101 network in the anomaly identification model of the present invention;
FIG. 3 is a schematic structural diagram of self-attack mechanism in the anomaly identification model of the present invention;
FIG. 4 is a schematic structural diagram of an improved fast RCNN network model according to the present invention;
FIG. 5 is a schematic structural diagram of an anomaly identification model according to the present invention;
fig. 6 is a schematic overall structure diagram of the anomaly detection system for unmanned aerial vehicle routing inspection of hardware on the high-voltage transmission line.
Detailed Description
The present application is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present application is not limited thereby.
As shown in fig. 1, the invention provides a power transmission line hardware abnormity detection method facing unmanned aerial vehicle inspection, which specifically comprises the following steps:
step 1, an unmanned aerial vehicle patrols and acquires a hardware fitting defect image on a high-voltage transmission line;
specifically, the number of the acquired hardware defect images in this embodiment is 500, and the images should include defects of hardware with different degrees and types.
Step 2, marking the hardware defect image obtained by the unmanned aerial vehicle inspection, and taking the marked hardware defect image as a sample set;
the marking can be carried out in a mode of manual marking or marking tools, the selectable marking tools comprise tools such as LabelImg, OpenCV and LabelIme, and the hardware part in the hardware defect image obtained by the unmanned aerial vehicle inspection is marked through marking.
Furthermore, in order to avoid excessive calculation amount during training, the size of the trained image can be converted into a uniform size, then labeling and dividing work is carried out, and the size-converted and labeled sample set image is used as training data.
Preferably, the size-changed images are all 512 by 512 pixels in size.
Step 3, improving based on a fast RCNN model, and constructing an abnormality detection model by combining a self-attention mechanism;
the anomaly detection model used by the method is improved and constructed based on a basic Faster RCNN model, the basic Faster RCNN model structure comprises a convolution layer, an RPN (resilient packet network), an ROI (region of interest) Pooling layer and a full-link layer, and the full-link layer is used for performing classification regression calculation.
Specifically, an vgg16 network is adopted in a convolutional layer of a basic FasterRCNN model, an anomaly detection model is constructed based on the FasterRCNN network model, a ResNet 01 network is used in the convolutional layer as a residual network, and a self-attention (self-attention) mechanism is added on the basis, so that the condition of missed detection and false detection is reduced in the aspects of improving feature quality and linking image context information respectively.
Specifically, the ResNet101 network enables the model to learn the feature information of the image well in the convolutional layer, and obtains abundant semantic information for the subsequent detection process. The underlying residual network actually uses a jump connection mode to perform convolution operation, as shown in fig. 2, and the residual unit is constructed in this way, so that the residual block is called structurally, and the calculation amount and the parameter amount are reduced. The specific structure of the ResNet101 network is shown in Table 1 below, and it can be seen that the ResNet101 network includes 5 layers of convolutions, 4 residual blocks plus the first layer of convolutions and the last fully connected layer, which is 101 layers, which does not include a pooling layer.
TABLE 1 concrete Structure of ResNet101 network
Figure BDA0003498599370000051
Figure BDA0003498599370000061
In computer vision, an attention mechanism (attention) can imitate the characteristics of human observation, pay attention to a desired object, and learn to focus on only desired features through learning, so that irrelevant information is ignored and only important information is focused. The convolution process can also achieve the effect of a partial attention mechanism, for example, for a high-level feature map of a classification network, the activated pixel points are exactly concentrated in the region relevant to the classification task. However, the convolution unit in convolution actually performs local region operation no matter how the late receptive field is increased, because it only focuses on the features of the regions adjacent to it, which results in that the contribution of other distant places to the current region is finally ignored. Non-Local blocks in Self-orientation in computer vision can then capture this long distance relationship: for a 2D image, the contribution value of any pixel in the image to the current pixel is obtained; for 3D video, it is the contribution of all pixels in all frames to the current frame pixel.
Furthermore, the method adds a self-attention mechanism behind an abnormal detection model ROI Pooling layer, measures the similarity between target candidate regions obtained in the same complex scene to obtain the contribution weight of a certain region to other regions, multiplies the weight by the self-attention mechanism and adds the weight, so that the score condition of the candidate region is re-evaluated, if the candidate region is a required target region, the score is increased, if the candidate region is not the required target region, the score is reduced, and finally, a more accurate detection result is achieved, so that the false detection condition in the complex scene is solved.
The self-association mechanism added in the invention uses the idea of NLP for reference, as shown in FIG. 4, FIG. 4 is the basic structure of self-association mechanism network, including Query, Key and Value. The feature diagram x of the convolution layer of the input self-attribute mechanism network is the output of the ROI Pooling layer. The self-Attention mechanism network comprises three branches, namely a first branchWkA second branch WqAnd a third branch WvFor the input x, the specific calculation steps of the input x through the self-attention mechanism are as follows:
according to the method, similarity measurement calculation is carried out according to the Query and each Key to obtain the weight, and the similarity measurement is carried out by using dot product operation:
Figure BDA0003498599370000062
wherein, Wq、
Figure BDA0003498599370000063
For the input convolved result, a similarity measure is performed by the convolution operation, dkFor the input characteristic dimension, the effect is to limit the scale of the value, preventing WjA situation other than 0, i.e. 1, occurs.
And performing normalization processing on the calculated weight, wherein the normalization processing can use a softmax function.
And carrying out weighted summation on the weight obtained by normalization and Value corresponding to Key to obtain the final W, wherein the calculation formula of W is as follows:
Figure BDA0003498599370000071
wherein, W represents the weight of each pixel point affected by other pixel points.
Adding the weight W of each pixel point influenced by other pixel points to the input x to obtain an output with the same dimension as the input, wherein the output is calculated as follows:
out＝Wv×W+x
wherein out represents the output of the input x after the self-actuation mechanism, and x is the input value.
Step 4, training the constructed anomaly detection model based on a training mode and a sample set of transfer learning to obtain a trained anomaly detection model;
specifically, the training of the anomaly detection model further includes:
training parameters trained on the PASAL VOC data set by the fast R-CNN official are used as initialization parameters of an abnormality detection model;
inputting the sample set into the initialized anomaly detection model to be trained for training;
and calculating the total loss of the anomaly detection model, and when the value of the total loss is not obviously reduced any more, indicating that the training is finished to obtain the trained anomaly detection model.
Wherein, since the data volume of the sample set is small, to prevent overfitting, the training batch is enlarged, and the batch _ size is set to 3. The invention sets 100 training rounds, the initial learning rate is 0.001, and the learning rate loss is 10% in every 25 training rounds.
Training parameters trained on the PASAL VOC data set by the fast R-CNN official are used as initialization parameters of the anomaly detection model used by the invention, and then training is continued on the basis, so that the training time is shortened. Inputting training data into the model for training, wherein the training process comprises the following steps: performing feature extraction on an input image by using a ResNet101 network to obtain a feature map; inputting the characteristic diagram into an RPN network to obtain a target area; calculating classification loss (Lcls) and regression loss (Lreg) of the RPN network, specifically, the loss L of the anomaly detection model includes classification loss LclsAnd regression loss LregThen the total loss L is calculated as follows:
Figure BDA0003498599370000081
and:
Figure BDA0003498599370000082
Figure BDA0003498599370000083
Figure BDA0003498599370000084
wherein i represents an anchor frame index, the anchor frame comprises an area suggestion frame and a real target frame, and piThe probability of classification is expressed as a positive class,
Figure BDA0003498599370000085
representing the corresponding real box value, t represents the aggressive bounding box,
Nclsindicates the size of the mini-batch in training, NregDenotes the number of anchor boxes, λ denotes the coefficient of equilibrium classification and regression, tiAn active anchor frame, i.e. an anchor frame with a target object,
Figure BDA0003498599370000086
then the frame is a real target frame corresponding to the active anchor frame; { x, y, w, h } is position coordinates of a target candidate region, which refers to an anchor frame containing a target object, { x, y } is anchor frame coordinates, and { w, h } is length and width.
Calculating IoU (overlapping degree) between the ith anchor frame and the real frame, judging whether the anchor frame is positive according to the size IoU, and when IoU between the ith anchor frame and the real frame is more than 0.7, considering the anchor frame to be positive, namely considering the anchor frame to be an object needing to be identified, and enabling the anchor frame to be an object needing to be identified at the moment
Figure BDA0003498599370000087
When IoU < 0.3, the anchor frame is considered negative and will not be paid attention to thereafter, which is the order
Figure BDA0003498599370000088
When 0.3 < IoU < 0.7, the corresponding anchor box is not involved in training.
As can be seen, the total loss function L includes two parts: loss of classification LclsAnd regression loss LregWherein the classification loss LclsFor distinguishing whether the anchor frame is positive or negative; regression loss LregAnd using soomth L1loss for frame regression training.
Since only the positive anchor frame, i.e., the regression of the anchor frame with the target object, is of interest in practical applications, the loss function is multiplied by
Figure BDA0003498599370000089
I.e., discarded if the anchor frame is not an anchor frame with a target object. Also in practical applications, to allow the total loss to be considered uniformly over 2 losses in the calculation, a parameter λ is used to balance NclsAnd NregThe difference between them is too large, and the parameter lambda is
Figure BDA0003498599370000091
And taking the rounded value. E.g. Ncls＝256，NregWhen it is 2500 hours, set up
Figure BDA0003498599370000092
Further, the process of training the anomaly detection model by using the sample set obtained in step 2 as training data specifically includes:
inputting the sample set into an anomaly detection model to be trained, and inhibiting a target area by the anomaly detection model through a non-maximum value to obtain an area suggestion frame; transmitting the region suggestion frame into an ROI POOLING layer to obtain a region of interest; the region of interest is transmitted into a self-attention mechanism frame, and a feature map with the same size as the feature map of the region of interest is obtained; and performing classification regression on the feature map output by the self-attention machine mechanism after passing through a full connection layer to obtain a final suggestion frame, and then performing classification loss and regression loss calculation, wherein a loss calculation formula is the same as the classification loss and regression loss calculation formula of the RPN, the loss and the loss of the RPN are added to be used as the total loss of training for training, and when the value of the total loss is not obviously reduced any more, the training is finished to obtain a trained anomaly detection model.
And 5, using the trained abnormity detection model to perform abnormity detection on hardware images obtained by the unmanned aerial vehicle inspection on the high-voltage transmission line, and outputting a detection result.
Wherein, the hardware fitting image obtained by the unmanned aerial vehicle inspection is subjected to size change, zoomed into an image with uniform size, and then input into an abnormality detection model, preferably,
specifically, the processing procedure of the anomaly detection model on the input image includes:
performing defect feature extraction on an input image by using a residual error network to obtain a feature map;
inputting the characteristic diagram into an RPN network to obtain a target area;
the target area is restrained by a non-maximum value to obtain an area suggestion frame;
transmitting the region suggestion frame into an ROI POOLING layer to obtain a region of interest;
transmitting the region of interest into a self-attention mechanism network for similarity measurement to obtain a feature map with the same size as the feature map of the region of interest;
and (4) carrying out classification regression on the characteristic graph output by the self-attention mechanism after passing through the full connection layer, and outputting an abnormal detection result of the hardware fitting image.
Specifically, the abnormality detection model can detect a hardware part in the image and a confidence degree corresponding to the hardware, judge whether the hardware is abnormal according to a preset confidence degree threshold, frame the abnormal hardware by using a labeling frame if the hardware is abnormal, and mark the confidence degree value corresponding to the abnormal hardware at the edge of the labeling frame, wherein the labeling is not needed if the hardware is abnormal.
The value range of the confidence coefficient is [0, 1], and the confidence coefficient threshold value set in the invention is 0.6, namely, the hardware with the confidence coefficient greater than 0.6 is judged as the abnormal hardware.
The higher the confidence value is, the higher the probability that the hardware is abnormal is indicated, and therefore the confidence value can be used as data for assisting the judgment of the inspector.
As shown in fig. 6, the present invention further provides an anomaly detection system for unmanned aerial vehicle routing inspection of hardware on a high voltage transmission line, and the above method for unmanned aerial vehicle routing inspection of transmission line hardware anomaly detection can be implemented based on the system, and the system specifically includes: the device comprises an image acquisition module, a size transformation module, a training module and a processing module.
The image acquisition module is used for acquiring an image of a power transmission line containing hardware fittings, and the image acquisition module can be an unmanned aerial vehicle for carrying out high-voltage power transmission line inspection in the embodiment;
the size transformation module is used for carrying out size transformation on the image acquired by the image acquisition module so as to enable the image input into the processing module to be an image with a uniform size;
the processing module comprises a constructed abnormity detection model and is used for carrying out abnormity detection on hardware in the input image and obtaining a detection result;
the training module can be used for training the abnormity detection model of the processing module by combining with the image data of the image acquisition module, so that accurate detection can be realized.
Simulation experiment:
in order to verify the beneficial effects of the invention, the following simulation experiments were performed: respectively adopting a basic Faster RCNN model (method 1), a Faster RCNN model (method 2) based on ResNet101 and an abnormity detection model (method 3) constructed by the invention to carry out abnormity detection on a high-voltage transmission line hardware image acquired by unmanned aerial vehicle inspection, and comparing the detection rate and the detection accuracy of a target object in the same hardware image by using three methods, wherein the obtained results are shown in the following table 2:
table 2: comparison table for detection rate and detection accuracy of hardware under different detection methods
Detection rate Rate of accuracy of detection
Method 1 77.3％ 74％
Method 2 81.3％ 80.5％
Method 3 84.6％ 83.1％
It can be seen that the detection of the basic fast RCNN model to the hardware has more missed detection and false detection conditions, and the analysis reasons include:
1. a plurality of interferents with the shapes and colors similar to the hardware fittings exist on the power transmission line, so that the condition of false detection is caused;
2. when the image background is complex, rusted hardware can be integrated with the background, and cannot be well distinguished in the identification process, so that the condition of missed detection is caused;
3. when some transmission lines are far away, hardware becomes very small in the image, so that the situation that small targets are missed in the identification process is caused.
Compared with the basic fast RCNN model, the fast RCNN model based on the ResNet101 improves the conditions of missed detection and false detection, but when the input image has serious interference, for example, a target object is blocked, the background is complex, the structure of the target object is similar, and the like, the detection accuracy is still reduced.
Aiming at the problem of false detection caused by interference, a mode of screening by connecting the context is adopted to eliminate false alarm. The method is characterized in that a self-attention mechanism is further added into a Faster RCNN model based on ResNet101, a candidate region for error detection is subjected to similarity association with the candidate region and then the weight is updated, and the weight of the candidate region is reduced after the candidate region is processed by the self-attention mechanism because the candidate region is different from the target object in shape, so that the candidate region cannot be mistakenly detected as the target object in subsequent detection, and the error detection condition caused by interference is eliminated when the abnormal detection model is predicted through secondary classification.
Compared with the prior art, the anomaly detection model constructed by the invention uses the ResNet101 network in the convolutional layer, namely, the feature learning of the target object is enhanced before the classification and the judgment are carried out for the first time, and richer semantic information is extracted, so that the model can better identify the target object in the subsequent detection.
The anomaly detection model constructed by the invention adopts a self-attention method to measure the similarity of the obtained candidate regions before the second classification discrimination, and solves the false detection problem in the complex environment by utilizing the relevance of the candidate regions.
The abnormity detection model constructed by the invention enhances the discrimination capability of the model at different stages of target detection, and experiments prove that the abnormity detection model is suitable for hardware abnormity detection under a complex background and can realize higher detection rate and detection accuracy.
The present applicant has described and illustrated embodiments of the present invention in detail with reference to the accompanying drawings, but it should be understood by those skilled in the art that the above embodiments are merely preferred embodiments of the present invention, and the detailed description is only for the purpose of helping the reader to better understand the spirit of the present invention, and not for limiting the scope of the present invention, and on the contrary, any improvement or modification made based on the spirit of the present invention should fall within the scope of the present invention.

Claims (10)

1. The utility model provides a power transmission line gold utensil anomaly detection method towards unmanned aerial vehicle patrols and examines, its characterized in that includes following step:
step 1, an unmanned aerial vehicle patrols and acquires a hardware fitting defect image on a high-voltage transmission line;
step 2, marking the hardware defect image obtained by the unmanned aerial vehicle inspection, and taking the marked hardware defect image as a sample set;
step 3, improving based on a fast RCNN model, and constructing an abnormality detection model by combining a self-attention mechanism;
step 4, training the constructed anomaly detection model based on a training mode and a sample set of transfer learning to obtain a trained anomaly detection model;
and 5, carrying out anomaly detection processing on hardware images on the high-voltage transmission line acquired by the unmanned aerial vehicle inspection by using the trained anomaly detection model, and outputting a detection result.
2. The method for detecting the abnormality of the power transmission line fitting for the unmanned aerial vehicle inspection according to claim 1,
in the step 2, the size conversion of the image is further included, so that the size of the image is unified to 512 × 512 pixels.
3. The method for detecting the abnormality of the power transmission line fitting for the unmanned aerial vehicle inspection according to claim 1,
the anomaly detection model is constructed based on a fast RCNN network and comprises a convolution layer, an RPN network, an ROI Pooling layer and a full-link layer, wherein the full-link layer is used for classification regression calculation.
4. The method for detecting the abnormality of the power transmission line fitting for the unmanned aerial vehicle inspection according to claim 3,
the convolutional layer in the anomaly detection model uses the ResNet101 network as a residual network.
5. The method for detecting the abnormality of the power transmission line hardware fitting for unmanned aerial vehicle inspection according to claim 3 or 4,
the anomaly detection model further comprises a self-attention mechanism network, and the self-attention mechanism network is arranged behind the ROI Pooling layer.
6. The method for detecting the abnormality of the power transmission line fitting for the unmanned aerial vehicle inspection according to claim 1,
in step 4, the training of the anomaly detection model further includes:
training parameters trained on a data set PASAL VOC data set by a Faster R-CNN official party are used as initialization parameters of an abnormality detection model;
inputting the sample set into the initialized anomaly detection model to be trained for training;
and calculating the total loss of the anomaly detection model, and when the value of the total loss is not obviously reduced any more, indicating that the training is finished to obtain the trained anomaly detection model.
7. The method for detecting the abnormality of the power transmission line fitting for the unmanned aerial vehicle inspection according to claim 1,
the loss function includes a classification loss and a regression loss, wherein the classification loss is used for judging whether the anchor frame is positive or negative, and the regression loss is used for frame regression training.
8. The method for detecting the abnormality of the power transmission line fitting for the unmanned aerial vehicle inspection according to claim 1,
in step 5, the detection process of the input image by the anomaly detection model includes:
performing defect feature extraction on an input image by using a residual error network to obtain a feature map;
inputting the characteristic diagram into an RPN network to obtain a target area;
the target area is restrained by a non-maximum value to obtain an area suggestion frame;
transmitting the region suggestion frame into an ROI POOLING layer to obtain a region of interest;
the region of interest is transmitted into a self-attention mechanism frame for similarity measurement, and a feature map with the same size as the feature map of the region of interest is obtained;
and (4) carrying out classification regression on the characteristic graph output by the self-attention mechanism network after passing through the full connection layer, and outputting an abnormal detection result of the hardware fitting image.
9. The method for detecting the abnormality of the power transmission line fitting for the unmanned aerial vehicle inspection according to claim 8,
the abnormal detection model can detect the hardware part in the image and the confidence degree corresponding to the hardware, judge whether the hardware is abnormal according to a preset confidence degree threshold value, frame the abnormal hardware by using a labeling frame if the hardware is abnormal, and mark the confidence degree value corresponding to the abnormal hardware at the edge of the labeling frame, wherein the labeling is not needed if the hardware is abnormal.
10. The utility model provides a power transmission line gold utensil anomaly detection system towards unmanned aerial vehicle patrols and examines, its characterized in that includes: the device comprises an image acquisition module, a size conversion module, a training module and a processing module;
the image acquisition module is used for acquiring an image of the power transmission line containing the hardware;
the size conversion module is used for carrying out size conversion on the image acquired by the image acquisition module so as to enable the image input into the processing module to be an image with uniform size;
the processing module comprises a constructed abnormity detection model and is used for carrying out abnormity detection on hardware in the input image and obtaining a detection result;
the training module can be used for training the abnormity detection model of the processing module by combining the image data of the image acquisition module.
CN202210123286.7A 2022-02-09 2022-02-09 Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system Pending CN114529839A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210123286.7A CN114529839A (en) 2022-02-09 2022-02-09 Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210123286.7A CN114529839A (en) 2022-02-09 2022-02-09 Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system

Publications (1)

Publication Number Publication Date
CN114529839A true CN114529839A (en) 2022-05-24

Family

ID=81622320

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210123286.7A Pending CN114529839A (en) 2022-02-09 2022-02-09 Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system

Country Status (1)

Country Link
CN (1) CN114529839A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758206A (en) * 2022-06-13 2022-07-15 武汉珈鹰智能科技有限公司 Steel truss structure abnormity detection method and device
CN115018788A (en) * 2022-06-02 2022-09-06 常州晋陵电力实业有限公司 Overhead line abnormity detection method and system based on intelligent robot
CN115272981A (en) * 2022-09-26 2022-11-01 山东大学 Cloud-edge co-learning power transmission inspection method and system
CN115436384A (en) * 2022-11-07 2022-12-06 国网山东省电力公司荣成市供电公司 Distribution box surface defect detection system and method based on unmanned aerial vehicle image

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115018788A (en) * 2022-06-02 2022-09-06 常州晋陵电力实业有限公司 Overhead line abnormity detection method and system based on intelligent robot
CN115018788B (en) * 2022-06-02 2023-11-14 常州晋陵电力实业有限公司 Overhead line abnormality detection method and system based on intelligent robot
CN114758206A (en) * 2022-06-13 2022-07-15 武汉珈鹰智能科技有限公司 Steel truss structure abnormity detection method and device
CN114758206B (en) * 2022-06-13 2022-10-28 武汉珈鹰智能科技有限公司 Steel truss structure abnormity detection method and device
CN115272981A (en) * 2022-09-26 2022-11-01 山东大学 Cloud-edge co-learning power transmission inspection method and system
CN115436384A (en) * 2022-11-07 2022-12-06 国网山东省电力公司荣成市供电公司 Distribution box surface defect detection system and method based on unmanned aerial vehicle image

Similar Documents

Publication Publication Date Title
CN110598736B (en) Power equipment infrared image fault positioning, identifying and predicting method
CN109543606B (en) Human face recognition method with attention mechanism
CN114529839A (en) Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system
CN112884064B (en) Target detection and identification method based on neural network
CN111611905B (en) Visible light and infrared fused target identification method
CN107194418B (en) Rice aphid detection method based on antagonistic characteristic learning
CN111523486B (en) Mechanical arm grabbing detection method based on improved CenterNet
CN110210335B (en) Training method, system and device for pedestrian re-recognition learning model
CN116883393B (en) Metal surface defect detection method based on anchor frame-free target detection algorithm
CN114821282A (en) Image detection model and method based on domain confrontation neural network
CN114821014A (en) Multi-mode and counterstudy-based multi-task target detection and identification method and device
CN115861210A (en) Transformer substation equipment abnormity detection method and system based on twin network
CN114092478B (en) Anomaly detection method
CN111950357A (en) Marine water surface garbage rapid identification method based on multi-feature YOLOV3
CN116205905B (en) Power distribution network construction safety and quality image detection method and system based on mobile terminal
CN117333948A (en) End-to-end multi-target broiler behavior identification method integrating space-time attention mechanism
CN117253192A (en) Intelligent system and method for silkworm breeding
CN107944453A (en) Based on Hu not bushing detection methods of bending moment and support vector machines
CN116704526A (en) Staff scanning robot and method thereof
CN115578325A (en) Image anomaly detection method based on channel attention registration network
CN111754459B (en) Dyeing fake image detection method based on statistical depth characteristics and electronic device
CN112465821A (en) Multi-scale pest image detection method based on boundary key point perception
CN113326873A (en) Method for automatically classifying opening and closing states of power equipment based on data enhancement
CN113506239B (en) Strip steel surface defect detection method based on cross-stage local network
CN117670755B (en) Detection method and device for lifting hook anti-drop device, storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination