CN116863358A - Method and system for identifying defects of power grid unmanned aerial vehicle inspection image insulator - Google Patents
Method and system for identifying defects of power grid unmanned aerial vehicle inspection image insulator Download PDFInfo
- Publication number
- CN116863358A CN116863358A CN202311042054.XA CN202311042054A CN116863358A CN 116863358 A CN116863358 A CN 116863358A CN 202311042054 A CN202311042054 A CN 202311042054A CN 116863358 A CN116863358 A CN 116863358A
- Authority
- CN
- China
- Prior art keywords
- insulator
- image
- main body
- images
- power grid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000012212 insulator Substances 0.000 title claims abstract description 351
- 230000007547 defect Effects 0.000 title claims abstract description 125
- 238000007689 inspection Methods 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012545 processing Methods 0.000 claims description 25
- 238000003709 image segmentation Methods 0.000 claims description 22
- 238000009413 insulation Methods 0.000 claims description 12
- 230000004927 fusion Effects 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000002950 deficient Effects 0.000 claims 2
- 230000000875 corresponding effect Effects 0.000 description 23
- 230000006870 function Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 238000012423 maintenance Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000013441 quality evaluation Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Abstract
A method and a system for identifying defects of an image insulator for inspection of a power grid unmanned plane relate to the field of image identification. In the method, the method comprises the following steps: acquiring an insulator image shot by the power grid unmanned aerial vehicle in one-time inspection; classifying all the insulator images, and dividing all the insulator images into effective insulator images and ineffective insulator images; dividing an insulator main body and an image background contained in each effective insulator image to obtain an insulator main body image corresponding to each effective insulator image; and carrying out defect identification on each insulator main body image, marking the insulator defect part in each insulator main body image, and generating an insulator defect image. By adopting the technical scheme provided by the application, when the insulator defect identification is carried out, the insulator main body and the image background are separated, so that the influence of the background on the insulator defect identification is reduced as much as possible, and the accuracy of the insulator defect identification is effectively improved.
Description
Technical Field
The application relates to the field of image recognition, in particular to a method and a system for recognizing defects of an image insulator for inspection of a power grid unmanned aerial vehicle.
Background
The traditional manual power inspection method is high in labor intensity, difficult in working condition and low in labor efficiency, line maintenance staff does not have favorable traffic advantages under the conditions of emergency faults and abnormal climates of the power grid, and common instruments or naked eyes are utilized to inspect facilities, so that certain potential safety hazards exist. Therefore, the current unmanned aerial vehicle-based power grid inspection becomes a mainstream mode of power grid inspection.
The unmanned aerial vehicle-based power grid inspection analyzes inspection images shot by the unmanned aerial vehicle of the power grid, and detects defects of the power grid, so that faults of the power grid are identified. In the unmanned aerial vehicle-based power grid inspection method, the most important steps are to effectively analyze and identify the acquired inspection image, and whether the power grid defect can be better identified from the inspection image or not directly determines the accuracy of the unmanned aerial vehicle-based power grid inspection.
At present, for the insulator defects in the power grid faults, as the problems of complex background, low background contrast, large background differences in different seasons in different areas and the like exist in the insulator images shot by the power grid unmanned aerial vehicle, the insulator defects in the insulator images cannot be well identified when the insulator images are analyzed, and the insulator defect identification accuracy is low.
Disclosure of Invention
In order to better identify insulator defects in an insulator image and effectively improve accuracy of insulator defect identification, the application provides a method and a system for identifying the insulator defects of an inspection image of a power grid unmanned aerial vehicle.
In a first aspect, the application provides a method for identifying defects of an image insulator for inspection of a power grid unmanned aerial vehicle, which comprises the following steps:
acquiring an insulator image shot by the power grid unmanned aerial vehicle in one-time inspection;
classifying all the insulator sub-images, and dividing all the insulator sub-images into effective insulator images and ineffective insulator images;
dividing an insulator main body and an image background contained in each effective insulator image to obtain an insulator main body image corresponding to each effective insulator image;
and carrying out defect identification on each insulator main body image, marking the insulator defect part in each insulator main body image, and generating an insulator defect image.
By adopting the technical scheme, the acquired original insulator image is subjected to image screening, an invalid insulator image with lower quality is removed, and the effective insulator image is subjected to segmentation on the insulator main body and the image background contained in the effective insulator image, so that the influence of the background on insulator image identification is eliminated. And the insulator main body image is taken as an object to identify the insulator defects, so that the accuracy of insulator defect identification is effectively improved.
Optionally, after classifying all the insulation sub-images, dividing all the insulation sub-images into an effective insulation sub-image and an ineffective insulation sub-image, the method further includes:
respectively acquiring first equipment position information corresponding to each invalid insulator image;
generating a re-inspection route according to the position information of each first device;
and sending the re-inspection route to the power grid unmanned aerial vehicle so that the power grid unmanned aerial vehicle can re-shoot the insulation sub-image of the power grid equipment at the position where the ineffective insulation sub-image is acquired according to the re-inspection route.
By adopting the technical scheme, for the invalid insulator image with poor part quality, the re-inspection route is planned based on the first identification position information corresponding to the invalid insulator image, so that the power grid unmanned aerial vehicle can inspect the power grid equipment where the invalid insulator image is acquired again, the image quality of the insulator image acquired at all the power grid equipment is ensured to meet the insulator defect identification requirement, and the insulators of the power grid equipment are comprehensively subjected to defect inspection.
Optionally, in classifying all the insulation sub-images, dividing all the insulation sub-images into an effective insulation sub-image and an ineffective insulation sub-image, specifically includes:
respectively obtaining the image quality scores of the insulator images;
classifying the insulator image whose image quality score is smaller than a set threshold value as a low-quality image, and classifying the insulator image whose image quality score is equal to or larger than the set threshold value as a high-quality image;
judging whether each high-quality image contains an insulator main body or not through a preset target identification model;
the high-quality image including the insulator main body is taken as the effective insulator image, and the high-quality image and the low-quality image not including the insulator main body are taken as the ineffective insulator image.
By adopting the technical scheme, the image quality of the insulator image is scored through the image quality scoring, and the insulator image with low quality scoring is removed. Whether the insulator image contains the insulator main body or not is judged through target identification, the insulator defect identification cannot be carried out on the insulator sub-image which does not contain the insulator main body, and the insulator image which does not contain the insulator main body is removed. Through two-step image screening, the image quality of the finally reserved insulator image is guaranteed to be good, insulator defect identification is carried out through the high-quality insulator image, and the accuracy of insulator defect identification is directly improved.
Optionally, the method for obtaining the insulator main body image corresponding to each effective insulator image includes that the insulator main body and the image background contained in each effective insulator image are divided, and specifically includes:
inputting the effective insulator image into an encoder module in a preset image segmentation model, and carrying out depth separable convolution on the effective insulator image to obtain first semantic features of the effective insulator image, wherein the first semantic features comprise a plurality of first semantic features;
transmitting each first semantic feature to a feature processing module in the image segmentation model, and performing feature calibration on each first semantic feature through a mixed attention structure in the feature processing module to convert each first semantic feature into a corresponding second semantic feature;
and respectively transmitting each second semantic feature to a decoder module in the image segmentation model, and respectively carrying out deconvolution and feature fusion on each second semantic feature to obtain the insulator main body image.
By adopting the technical scheme, the image segmentation is carried out on the effective insulator image through the image segmentation model, and the image segmentation model comprises an encoder module, a feature processing module and a decoder module. The encoder module performs first semantic feature extraction through depth separable convolution, so that the problems that a large amount of redundant calculation is generated and local receptive field is limited due to conventional convolution are avoided; the feature processing module introduces a mixed attention mechanism to convert the first semantic features into second semantic features, so that the function of highlighting useful features and suppressing redundant features is achieved; and the decoder carries out deconvolution and feature fusion on each second semantic feature so as to obtain a clear insulator main body image and complete background segmentation of the effective insulator image.
Optionally, in performing feature calibration on each first semantic feature through the mixed attention structure in the feature processing module, converting each first semantic feature into a corresponding second semantic feature, including:
inputting the first semantic features into a channel attention channel in the mixed attention structure, and acquiring the channel attention of the first semantic features;
inputting the first semantic features into a spatial attention channel in the mixed attention structure, and acquiring the spatial attention of the first semantic features;
acquiring joint attention weights of the first semantic features according to the channel attention and the space attention;
and carrying out matrix multiplication on the first semantic features and the joint attention weights to obtain the second semantic features.
By adopting the technical scheme, the mixed attention mechanism consists of two parallel branches of channel attention and space attention, and the first semantic features are recalibrated by learning the importance degree of the channel and space dimensions so as to achieve the effect of highlighting useful features and inhibiting redundant features.
Optionally, in performing defect recognition on each of the insulator main body images, marking an insulator defect position in each of the insulator main body images, and generating an insulator defect image, specifically including:
inputting each insulator main body image into a preset defect identification model, and positioning the insulator defect part in each insulator main body image;
and generating a defect marking frame to mark the defect part of the insulator in each insulator main body image and outputting the insulator defect image.
By adopting the technical scheme, the defect part of the insulator is definitely marked in the defect image of the insulator, so that related maintenance personnel can clearly acquire specific defect information of the insulator, and follow-up maintenance is facilitated.
Optionally, after performing defect recognition on each of the insulator main body images and marking an insulator defect part in each of the insulator main body images, generating an insulator defect image, the method further includes:
respectively acquiring second equipment position information corresponding to each insulator main body image;
and outputting an insulator defect report according to the second equipment position information and the insulator defect image.
By adopting the technical scheme, the second equipment information of the power grid equipment corresponding to the insulator defect image is obtained, and the insulator defect report is generated based on the second equipment information, so that relevant maintenance personnel can directly position the power grid equipment with defects in the maintenance process, and the follow-up maintenance efficiency is improved.
In a second aspect of the application, a system for identifying defects of an image insulator for inspection of a power grid unmanned aerial vehicle is provided, and comprises the following modules:
the insulator image acquisition module is used for acquiring an insulator image shot by the power grid unmanned aerial vehicle in one-time inspection;
the insulator sub-image classification module is used for classifying all the insulator sub-images and dividing all the insulator sub-images into effective insulator images and ineffective insulator images;
the insulator image segmentation module is used for segmenting an insulator main body and an image background contained in each effective insulator image to obtain an insulator main body image corresponding to each effective insulator image;
and the insulator defect marking module is used for carrying out defect identification on each insulator main body image, marking the insulator defect part in each insulator main body image and generating an insulator defect image.
In a third aspect of the application, an electronic device is provided;
the electronic equipment comprises a processor, a memory, a user interface and a network interface, wherein the memory is used for storing instructions, the user interface and the network interface are used for communicating with other equipment, and the processor is used for executing the instructions stored in the memory so that the electronic equipment can execute the method for identifying the defects of the image insulator of the power grid unmanned aerial vehicle inspection.
In a fourth aspect of the application, a computer readable storage medium is provided;
the computer readable storage medium stores instructions that, when executed, perform a method for identifying defects of an image insulator for inspection of a power grid unmanned aerial vehicle.
In summary, one or more technical solutions provided in the embodiments of the present application at least have the following technical effects or advantages:
1. for the obtained original insulator image, firstly, carrying out image screening on the insulator image, removing invalid insulator images with lower quality, and for the effective insulator image, dividing an insulator main body and an image background contained in the effective insulator image, so as to eliminate the influence of the background on the insulator image identification. And the insulator main body image is taken as an object to identify the insulator defects, so that the accuracy of insulator defect identification is effectively improved.
2. And carrying out image segmentation on the effective insulator image through an image segmentation model, wherein the image segmentation model comprises an encoder module, a feature processing module and a decoder module. The encoder module performs first semantic feature extraction through depth separable convolution, so that the problems that a large amount of redundant calculation is generated and local receptive field is limited due to conventional convolution are avoided; the feature processing module introduces a mixed attention mechanism to convert the first semantic features into second semantic features, so that the function of highlighting useful features and suppressing redundant features is achieved; and the decoder carries out deconvolution and feature fusion on each second semantic feature so as to obtain a clear insulator main body image and complete background segmentation of the effective insulator image.
3. And acquiring second equipment information of the power grid equipment corresponding to the insulator defect image, and generating an insulator defect report based on the second equipment information, so that relevant maintenance personnel can directly position the power grid equipment with defects in the overhaul process, and the follow-up overhaul efficiency can be improved.
Drawings
Fig. 1 is a schematic flow chart of a method for identifying defects of an image insulator for inspection of a power grid unmanned aerial vehicle according to an embodiment of the application.
Fig. 2 is a schematic diagram of a result of effective insulator image segmentation in the method for identifying defects of an image insulator for inspection of a power grid unmanned aerial vehicle according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of an image segmentation model according to an embodiment of the present application.
Fig. 4 is a schematic structural diagram of a feature processing module in an image segmentation model according to an embodiment of the present application.
Fig. 5 is a schematic structural diagram of a system for identifying defects of an image insulator for inspection of a power grid unmanned aerial vehicle according to an embodiment of the application.
Fig. 6 is a schematic structural diagram of an electronic device according to the disclosure.
Reference numerals illustrate: 501. an insulator image acquisition module; 502. an insulator image classification module; 503. an insulator image segmentation module; 504. an insulator defect marking module; 600. an electronic device; 601. a processor; 602. a communication bus; 603. a user interface; 604. a network interface; 605. a memory.
Description of the embodiments
In order that those skilled in the art will better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments.
In describing embodiments of the present application, words such as "for example" or "for example" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "such as" or "for example" in embodiments of the application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "or" for example "is intended to present related concepts in a concrete fashion.
In the description of embodiments of the application, the term "plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating an indicated technical feature. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
Referring to fig. 1, the application provides a method for identifying defects of an image insulator for inspection of a power grid unmanned aerial vehicle, which specifically comprises the following steps:
s1: and acquiring an insulator image shot by the power grid unmanned aerial vehicle in one-time inspection.
Specifically, the electric wire netting unmanned aerial vehicle is patrolled and examined the electric wire netting equipment of route of patrolling and examining according to the route of patrolling and examining that technical personnel planned, set up on the electric wire netting unmanned aerial vehicle and have shot equipment, shooting equipment can carry out image shooting to electric wire netting equipment, when electric wire netting unmanned aerial vehicle flies to the corresponding shooting point of electric wire netting equipment on the route of patrolling and examining, shooting equipment that sets up on the electric wire netting unmanned aerial vehicle will carry out image shooting to the specific part of electric wire netting equipment, obtain the equipment image that electric wire netting equipment corresponds, acquire the insulator image of electric wire netting equipment from these equipment images, accomplish the insulator image and acquire.
And when the acquired equipment images are shot by the power grid unmanned aerial vehicle, recording the place numbers of the power grid equipment corresponding to the shot equipment images, and acquiring the place numbers of the power grid equipment from the power grid unmanned aerial vehicle, and then acquiring the position information of the power grid equipment corresponding to the place numbers of the power grid equipment according to a place number rule preset by the power grid equipment. It can be appreciated that in this way, the device image can be correlated with the grid device and the device position information of the grid device, and when the device image is acquired, the device position information of the grid device at the position where the device image is captured can be found by the device image.
S2: all the insulator images are classified, and the all the insulator images are divided into effective insulator images and ineffective insulator images.
Specifically, after the insulator images shot during one-time inspection of the power grid unmanned aerial vehicle are obtained, image quality evaluation is carried out on all the insulator images respectively, and the insulator images are divided into effective insulator images and ineffective insulator images.
The image quality evaluation for the insulator image includes the steps of: firstly, respectively acquiring image quality scores of all the insulator images, and respectively comparing the image quality scores of all the insulator images with preset set thresholds; classifying the insulator image with the image quality score smaller than the set threshold value as a low-quality image, and classifying the insulator image with the image quality score larger than or equal to the set threshold value as a high-quality image; respectively carrying out target recognition on all the high-quality images, and judging whether the high-quality images contain an insulator main body or not; the high-quality image including the insulator main body is taken as an effective insulator image, and the high-quality image and the low-quality image not including the insulator main body are taken as ineffective insulator images.
Specifically, the image quality score of the insulator image is used for describing the identifiable degree of the insulator image, the higher the image quality score is, the clearer and the more identifiable the insulator image is, and on the contrary, the higher the image quality score is, the more blurred and the less identifiable the insulator image is. In a possible embodiment of the present application, the image quality score is obtained by an image quality evaluation algorithm, and the image quality score of the insulator image is obtained by calculating an insulator image by the image quality evaluation algorithm, where the image quality evaluation algorithm may be one or more of PSNR (peak signal to noise ratio), SSIM (structural similarity index), VIF (visual information fidelity) or NIQE (natural image quality evaluator), and specifically may be set by a person skilled in the art according to specific parameters of a photographing device set up on the power grid unmanned aerial vehicle.
The target recognition of the high-quality image is performed through a preset target recognition model, the target recognition model can process the input high-quality image and judge whether the high-quality image contains the insulator main body, the target recognition model can be a neural network model built based on algorithms such as CNN, RNN or Faster-RCNN, and the recognition of the main body target through the target recognition model is the prior art and is not repeated here.
By evaluating the image quality of the insulator image and identifying the insulator main body, an effective insulator image and an ineffective insulator image are divided in the insulator image. For invalid insulator images, the images may be poor in image quality or the images do not contain an insulator main body due to shooting reasons of the power grid unmanned aerial vehicle during the inspection of the power grid unmanned aerial vehicle, first equipment position information corresponding to the invalid insulator images is obtained, the inspection route of the power grid unmanned aerial vehicle is re-planned according to the first equipment position information, and the re-inspection route of the power grid unmanned aerial vehicle is determined, so that the power grid unmanned aerial vehicle shoots the electric grid equipment at the position where the invalid insulator image is obtained again according to the re-inspection route. Of course, during the re-inspection route planning, a technician can adaptively adjust the re-inspection route according to the specific image problem in the invalid insulator image, so that the power grid unmanned aerial vehicle can acquire equipment images with better quality when performing image shooting according to the re-inspection route.
S3: and dividing the insulator main body and the image background contained in each effective insulator image to obtain an insulator main body image corresponding to each effective insulator image.
Specifically, each effective insulator image is processed through a preset image segmentation model, and insulator main bodies and image backgrounds contained in each effective insulator image are segmented, so that insulator main body images corresponding to each effective insulator image are obtained.
Referring to fig. 2, the insulator main body image includes only the main body outline of the insulator main body, and does not include any image background. In the insulator main body image, the contained insulator main body is divided into white, and the rest parts are all black, so that the influence of the image background on insulator defect identification is eliminated.
Referring to fig. 3, the image segmentation model specifically includes an encoder module, a feature processing module, and a decoder module. For an input effective insulator image, firstly, an encoder module extracts multiple layers of first semantic features of the effective insulator image and transmits the first semantic features of each layer to a feature processing module; the feature processing module performs feature calibration on the first semantic features of each layer through the mixed attention structure built in the feature processing module, converts the first semantic features of each layer into corresponding second semantic features, and transmits the second semantic features to the decoder module; and finally, deconvoluting and feature fusion are carried out on each second semantic feature by the decoder module, and finally, the insulator main body image corresponding to the effective insulator image is obtained.
Specifically, the encoder module extracts the multi-layer first semantic features of the effective insulator image through a depth separable convolution (Depthwise Separable Convolution), which is a convolution operation commonly used in convolutional neural networks, and can effectively reduce the number of model parameters and the calculated amount and improve the calculation efficiency and accuracy of the model.
The depth separable convolution consists of two parts, namely depth convolution and point-by-point convolution, wherein convolution operation is firstly carried out on an effective insulator image, each convolution kernel only carries out convolution on a single channel of the effective insulator image, C input channels are subjected to single channel convolution through C convolution kernels to obtain C convolution results, and then the C convolution results are added according to channels to obtain a final depth convolution result; and performing point-by-point convolution operation on the depth convolution result, namely performing convolution operation on the output of each channel, and convolving each channel by adopting a convolution kernel of 1x1 to finally output multi-layer first semantic features of the effective insulator image.
Referring to fig. 4, the mixed attention structure in the feature processing module includes a channel attention channel and a spatial attention channel, the two attention channels are arranged in parallel, a first semantic feature output by the encoder module is input into the channel attention channel to acquire a channel attention of the first semantic feature, the first semantic feature is input into the spatial attention channel to acquire a spatial attention of the first semantic feature, the channel attention and the spatial attention are subjected to matrix multiplication to obtain a joint attention weight of the first semantic feature, and then the first semantic feature and the joint attention weight are subjected to matrix multiplication to convert the first semantic feature into a second semantic feature.
The channel attention channel can calculate the importance degree of each channel of the first semantic feature, and for the channel feature with larger influence on the result, the channel attention channel can spontaneously allocate larger weight to the channel feature, and for the channel with smaller relative effect, the channel attention channel can allocate smaller weight to ensure the accuracy of the output result; the spatial attention channel derives and calculates the first semantic features to determine which region in the first semantic features is important, and as with the channel attention channel, a larger weight is assigned to important regions and a smaller weight is assigned to relatively unimportant regions according to the calculation result. After the processing of the channel attention channel and the space attention channel, the first semantic features are subjected to feature calibration again, the main features in the first semantic features are enhanced, and the redundant features are restrained.
It should be noted that, before the image segmentation model processes the effective insulator image, the training set is derived from the historical insulator image, and is a rich training set, in a feasible embodiment of the present application, the image processing in various modes such as random adjustment of the size, random noise addition, rotation of the image, random translation, etc. is performed on the historical insulator image, so as to obtain more training samples.
S4: and carrying out defect identification on each insulator main body image, marking the insulator defect part in each insulator main body image, and generating an insulator defect image.
Specifically, processing each insulator main body image through a preset defect identification model, positioning insulator defect positions in each insulator main body image, generating a defect marking frame to mark the insulator defect positions in each insulator main body image, and outputting an insulator defect image.
The defect recognition model may be a YOLO target detection model, and in a preferred embodiment of the present application, YOLOv5 is used to perform insulator defect recognition on the insulator main body image.
After the insulator defect image corresponding to the effective insulator image is obtained, second equipment position information of the insulator defect image corresponding to the effective insulator image is obtained according to the equipment position information of the effective insulator image, and an insulator defect report is generated based on the second equipment position information.
The insulator defect report contains all insulator defect images, and meanwhile, second equipment position information corresponding to the insulator defect images is recorded beside the insulator defect images so as to display the equipment position information of the power grid equipment corresponding to the insulator defect images. Through the insulator defect report, the power grid maintainer can intuitively know the specific position of power grid equipment for operating the insulator defect, and based on the defect marking frame in the insulator defect image in the insulator defect report, the power grid maintainer can also know the specific insulator defect position of the power grid equipment, so that the maintenance efficiency of the power grid maintainer is improved.
Referring to fig. 5, the application further provides a system for identifying defects of the image insulator for inspection of the unmanned aerial vehicle of the power grid, which specifically comprises the following modules:
the insulator image acquisition module 501 is used for acquiring an insulator image shot by the power grid unmanned aerial vehicle in one inspection;
an insulator sub-image classification module 502, configured to classify all the insulator sub-images into an effective insulator image and an ineffective insulator image;
an insulator image segmentation module 503, configured to segment an insulator main body and an image background included in each effective insulator image, and obtain an insulator main body image corresponding to each effective insulator image;
and an insulator defect marking module 504, configured to identify defects of each insulator main body image, mark insulator defect positions in each insulator main body image, and generate an insulator defect image.
It should be noted that: in the device provided in the above embodiment, when implementing the functions thereof, only the division of the above functional modules is used as an example, in practical application, the above functional allocation may be implemented by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to implement all or part of the functions described above. In addition, the embodiments of the apparatus and the method provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the embodiments of the method are detailed in the method embodiments, which are not repeated herein.
The application also discloses an electronic device 600. Referring to fig. 6, fig. 6 is a schematic structural diagram of an electronic device 600 according to an embodiment of the present disclosure. The electronic device 600 may include: at least one processor 601, at least one network interface 604, a user interface 603, a memory 605, at least one communication bus 602.
Wherein the communication bus 602 is used to enable connected communications between these components.
The user interface 603 may include a Display screen (Display), a Camera (Camera), and the optional user interface 603 may further include a standard wired interface, a wireless interface.
The network interface 604 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.
Wherein the processor 601 may include one or more processing cores. The processor 601 connects various portions of the overall server using various interfaces and lines, performs various functions of the server and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 605, and invoking data stored in the memory 605. Alternatively, the processor 601 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 601 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 601 and may be implemented by a single chip.
The Memory 605 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 605 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 605 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 605 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, etc.; the storage data area may store data or the like involved in the above respective method embodiments. The memory 605 may also optionally be at least one storage device located remotely from the processor 601. Referring to fig. 5, an operating system, a network communication module, a user interface module, and an application program of a method for identifying defects of an image insulator for inspection of a power grid unmanned aerial vehicle may be included in a memory 605 as a computer storage medium.
In the electronic device 600 shown in fig. 6, the user interface 603 is mainly used for providing an input interface for a user, and acquiring data input by the user; and the processor 601 may be configured to invoke an application program in the memory 605 that stores a method for identifying defects in a power grid drone inspection image insulator, which when executed by the one or more processors 601, causes the electronic device 600 to perform the method as in one or more of the embodiments described above. It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all of the preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, such as a division of units, merely a division of logic functions, and there may be additional divisions in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some service interface, device or unit indirect coupling or communication connection, electrical or otherwise.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory 605. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory 605, including several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present application. Whereas the aforementioned memory 605 includes: various media capable of storing program codes, such as a U disk, a mobile hard disk, a magnetic disk or an optical disk.
The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. That is, equivalent changes and modifications are contemplated by the teachings of this disclosure, which fall within the scope of the present disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure.
This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a scope and spirit of the disclosure being indicated by the claims.
Claims (10)
1. The method for identifying the defects of the image insulator for inspection of the unmanned aerial vehicle of the power grid is characterized by comprising the following steps:
acquiring an insulator image shot by the power grid unmanned aerial vehicle in one-time inspection;
classifying all the insulator sub-images, and dividing all the insulator sub-images into effective insulator images and ineffective insulator images;
dividing an insulator main body and an image background contained in each effective insulator image to obtain an insulator main body image corresponding to each effective insulator image;
and carrying out defect identification on each insulator main body image, marking the insulator defect part in each insulator main body image, and generating an insulator defect image.
2. The method for identifying defects of an electrical network unmanned aerial vehicle inspection image insulator according to claim 1, wherein after classifying all the insulator sub-images, dividing all the insulator sub-images into an effective insulator image and an ineffective insulator image, further comprising:
respectively acquiring first equipment position information corresponding to each invalid insulator image;
generating a re-inspection route according to the position information of each first device;
and sending the re-inspection route to the power grid unmanned aerial vehicle so that the power grid unmanned aerial vehicle can re-shoot the insulation sub-image of the power grid equipment at the position where the ineffective insulation sub-image is acquired according to the re-inspection route.
3. The method for identifying the defects of the insulators of the inspection images of the unmanned aerial vehicle of the power grid according to claim 1, wherein in classifying all the insulator images, the insulator images are divided into effective insulator images and ineffective insulator images, specifically comprising:
respectively obtaining the image quality scores of the insulator images;
classifying the insulator image whose image quality score is smaller than a set threshold value as a low-quality image, and classifying the insulator image whose image quality score is equal to or larger than the set threshold value as a high-quality image;
judging whether each high-quality image contains an insulator main body or not through a preset target identification model;
the high-quality image including the insulator main body is taken as the effective insulator image, and the high-quality image and the low-quality image not including the insulator main body are taken as the ineffective insulator image.
4. The method for identifying the defects of the power grid unmanned aerial vehicle inspection image insulator according to claim 1, wherein the steps of dividing the insulator main body and the image background contained in each effective insulator image to obtain the insulator main body image corresponding to each effective insulator image specifically comprise the following steps:
inputting the effective insulator image into an encoder module in a preset image segmentation model, and carrying out depth separable convolution on the effective insulator image to obtain first semantic features of the effective insulator image, wherein the first semantic features comprise a plurality of first semantic features;
transmitting each first semantic feature to a feature processing module in the image segmentation model, and performing feature calibration on each first semantic feature through a mixed attention structure in the feature processing module to convert each first semantic feature into a corresponding second semantic feature;
and respectively transmitting each second semantic feature to a decoder module in the image segmentation model, and respectively carrying out deconvolution and feature fusion on each second semantic feature to obtain the insulator main body image.
5. The method for identifying defects of the power grid unmanned aerial vehicle inspection image insulator according to claim 4, wherein the method for identifying defects of the power grid unmanned aerial vehicle inspection image insulator is characterized in that the method for identifying the defects of the power grid unmanned aerial vehicle inspection image insulator comprises the steps of:
inputting the first semantic features into a channel attention channel in the mixed attention structure, and acquiring the channel attention of the first semantic features;
inputting the first semantic features into a spatial attention channel in the mixed attention structure, and acquiring the spatial attention of the first semantic features;
acquiring joint attention weights of the first semantic features according to the channel attention and the space attention;
and carrying out matrix multiplication on the first semantic features and the joint attention weights to obtain the second semantic features.
6. The method for identifying defects of an insulator in a power grid unmanned aerial vehicle inspection image according to claim 1, wherein in the steps of identifying defects of each insulator main body image, marking the defective parts of the insulator in each insulator main body image and generating an insulator defect image, the method specifically comprises the following steps:
inputting each insulator main body image into a preset defect identification model, and positioning the insulator defect part in each insulator main body image;
and generating a defect marking frame to mark the defect part of the insulator in each insulator main body image and outputting the insulator defect image.
7. The method for identifying defects of an insulator in a power grid unmanned aerial vehicle inspection image according to claim 1, wherein after identifying defects of each of the insulator main body images and marking the defective parts of the insulator in each of the insulator main body images, generating an insulator defect image, further comprises:
respectively acquiring second equipment position information corresponding to each insulator main body image;
and outputting an insulator defect report according to the second equipment position information and the insulator defect image.
8. An image insulator defect identification system is patrolled and examined to electric wire netting unmanned aerial vehicle, characterized in that, the system includes:
the insulator image acquisition module (501) is used for acquiring an insulator image shot by the power grid unmanned aerial vehicle in one-time inspection;
an insulator image classification module (502) for classifying all the insulator images into effective insulator images and ineffective insulator images;
an insulator image segmentation module (503) for segmenting an insulator main body and an image background contained in each effective insulator image to obtain an insulator main body image corresponding to each effective insulator image;
and the insulator defect marking module (504) is used for carrying out defect identification on each insulator main body image, marking the insulator defect part in each insulator main body image and generating an insulator defect image.
9. An electronic device comprising a processor (601), a memory (605), a user interface (603) and a network interface (604), the memory (605) being configured to store instructions, the user interface (603) and the network interface (604) being configured to communicate to other devices, the processor (601) being configured to execute the instructions stored in the memory (605) to cause the electronic device (600) to perform the method according to any of claims 1-7.
10. A computer readable storage medium storing instructions which, when executed, perform the method steps of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311042054.XA CN116863358A (en) | 2023-08-18 | 2023-08-18 | Method and system for identifying defects of power grid unmanned aerial vehicle inspection image insulator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311042054.XA CN116863358A (en) | 2023-08-18 | 2023-08-18 | Method and system for identifying defects of power grid unmanned aerial vehicle inspection image insulator |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116863358A true CN116863358A (en) | 2023-10-10 |
Family
ID=88234323
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311042054.XA Pending CN116863358A (en) | 2023-08-18 | 2023-08-18 | Method and system for identifying defects of power grid unmanned aerial vehicle inspection image insulator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116863358A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103941746A (en) * | 2014-03-29 | 2014-07-23 | 国家电网公司 | System and method for processing unmanned aerial vehicle polling image |
| CN113298789A (en) * | 2021-05-28 | 2021-08-24 | 国网陕西省电力公司电力科学研究院 | Insulator defect detection method and system, electronic device and readable storage medium |
| CN113379699A (en) * | 2021-06-08 | 2021-09-10 | 上海电机学院 | Transmission line insulator defect detection method based on deep learning |
| CN114332462A (en) * | 2021-12-31 | 2022-04-12 | 福州大学 | MRI segmentation method for integrating attention mechanism into cerebral lesion |
| CN114821174A (en) * | 2022-04-24 | 2022-07-29 | 西北工业大学 | Power transmission line aerial image data cleaning method based on content perception |
| CN114863236A (en) * | 2022-05-27 | 2022-08-05 | 浙江中烟工业有限责任公司 | Image target detection method based on double attention mechanism |
| CN114972248A (en) * | 2022-05-24 | 2022-08-30 | 广州市华奕电子科技有限公司 | Attention mechanism-based improved U-net liver tumor segmentation method |
| CN115619797A (en) * | 2022-10-24 | 2023-01-17 | 宁夏医科大学 | Lung image segmentation method of parallel U-Net network based on attention mechanism |
| CN115731242A (en) * | 2022-11-21 | 2023-03-03 | 电子科技大学 | Retina blood vessel segmentation method based on mixed attention mechanism and asymmetric convolution |
| CN116051589A (en) * | 2022-09-05 | 2023-05-02 | 河北师范大学 | Method and device for segmenting lung parenchyma and pulmonary blood vessels in CT image |
-
2023
- 2023-08-18 CN CN202311042054.XA patent/CN116863358A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103941746A (en) * | 2014-03-29 | 2014-07-23 | 国家电网公司 | System and method for processing unmanned aerial vehicle polling image |
| CN113298789A (en) * | 2021-05-28 | 2021-08-24 | 国网陕西省电力公司电力科学研究院 | Insulator defect detection method and system, electronic device and readable storage medium |
| CN113379699A (en) * | 2021-06-08 | 2021-09-10 | 上海电机学院 | Transmission line insulator defect detection method based on deep learning |
| CN114332462A (en) * | 2021-12-31 | 2022-04-12 | 福州大学 | MRI segmentation method for integrating attention mechanism into cerebral lesion |
| CN114821174A (en) * | 2022-04-24 | 2022-07-29 | 西北工业大学 | Power transmission line aerial image data cleaning method based on content perception |
| CN114972248A (en) * | 2022-05-24 | 2022-08-30 | 广州市华奕电子科技有限公司 | Attention mechanism-based improved U-net liver tumor segmentation method |
| CN114863236A (en) * | 2022-05-27 | 2022-08-05 | 浙江中烟工业有限责任公司 | Image target detection method based on double attention mechanism |
| CN116051589A (en) * | 2022-09-05 | 2023-05-02 | 河北师范大学 | Method and device for segmenting lung parenchyma and pulmonary blood vessels in CT image |
| CN115619797A (en) * | 2022-10-24 | 2023-01-17 | 宁夏医科大学 | Lung image segmentation method of parallel U-Net network based on attention mechanism |
| CN115731242A (en) * | 2022-11-21 | 2023-03-03 | 电子科技大学 | Retina blood vessel segmentation method based on mixed attention mechanism and asymmetric convolution |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020215985A1 (en) | Medical image segmentation method and device, electronic device and storage medium | |
| US11922626B2 (en) | Systems and methods for automatic detection and quantification of pathology using dynamic feature classification | |
| WO2019104780A1 (en) | Laser radar point cloud data classification method, apparatus and device, and storage medium | |
| CN111325739B (en) | Method and device for detecting lung focus and training method of image detection model | |
| CN111680746B (en) | Vehicle damage detection model training, vehicle damage detection method, device, equipment and medium | |
| KR20200013148A (en) | Method, system and computer program for providing defect analysis service of concrete structure | |
| EP3625696A1 (en) | Systems and methods for searching images | |
| CN108073898B (en) | Method, device and equipment for identifying human head area | |
| CN110853005A (en) | Immunohistochemical membrane staining section diagnosis method and device | |
| CN108830149B (en) | Target bacterium detection method and terminal equipment | |
| CN115546630A (en) | Construction site extraction method and system based on remote sensing image characteristic target detection | |
| CN112001317A (en) | Lead defect identification method and system based on semantic information and terminal equipment | |
| CN115239646A (en) | Defect detection method and device for power transmission line, electronic equipment and storage medium | |
| CN110807758A (en) | Method, device, equipment and storage medium for detecting uncovered area of heat preservation quilt | |
| CN113592839A (en) | Distribution network line typical defect diagnosis method and system based on improved fast RCNN | |
| CN111597939B (en) | High-speed rail line nest defect detection method based on deep learning | |
| CN111582278B (en) | Portrait segmentation method and device and electronic equipment | |
| CN116863358A (en) | Method and system for identifying defects of power grid unmanned aerial vehicle inspection image insulator | |
| CN111915565A (en) | Method for analyzing cracks of porcelain insulator of power transmission and transformation line in real time based on YOLACT algorithm | |
| CN108615235B (en) | Method and device for processing temporal ear image | |
| CN115471745A (en) | Network model and device for plant identification and electronic equipment | |
| CN113177566B (en) | Feature extraction model training method and device and computer equipment | |
| CN115311680A (en) | Human body image quality detection method and device, electronic equipment and storage medium | |
| CN113506290A (en) | Method and device for detecting defects of line insulator | |
| CN113516328A (en) | Data processing method, service providing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |