CN117612036A - Unmanned aerial vehicle inspection data defect automatic identification method, system, equipment and medium - Google Patents

Unmanned aerial vehicle inspection data defect automatic identification method, system, equipment and medium Download PDF

Info

Publication number
CN117612036A
CN117612036A CN202311512833.1A CN202311512833A CN117612036A CN 117612036 A CN117612036 A CN 117612036A CN 202311512833 A CN202311512833 A CN 202311512833A CN 117612036 A CN117612036 A CN 117612036A
Authority
CN
China
Prior art keywords
fault
equipment
aerial vehicle
unmanned aerial
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311512833.1A
Other languages
Chinese (zh)
Inventor
池建飞
朱鹏
唐洪良
夏红鑫
邬明亮
陈攀宇
陈悦
陈琴芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Zhejiang Electric Power Co Ltd Hangzhou Linping District Power Supply Co
Innovation And Entrepreneurship Center Of State Grid Zhejiang Electric Power Co ltd
State Grid Corp of China SGCC
Hangzhou Power Supply Co of State Grid Zhejiang Electric Power Co Ltd
Original Assignee
State Grid Zhejiang Electric Power Co Ltd Hangzhou Linping District Power Supply Co
Innovation And Entrepreneurship Center Of State Grid Zhejiang Electric Power Co ltd
State Grid Corp of China SGCC
Hangzhou Power Supply Co of State Grid Zhejiang Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Zhejiang Electric Power Co Ltd Hangzhou Linping District Power Supply Co, Innovation And Entrepreneurship Center Of State Grid Zhejiang Electric Power Co ltd, State Grid Corp of China SGCC, Hangzhou Power Supply Co of State Grid Zhejiang Electric Power Co Ltd filed Critical State Grid Zhejiang Electric Power Co Ltd Hangzhou Linping District Power Supply Co
Priority to CN202311512833.1A priority Critical patent/CN117612036A/en
Publication of CN117612036A publication Critical patent/CN117612036A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/17Terrestrial scenes taken from planes or by drones
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/70Labelling scene content, e.g. deriving syntactic or semantic representations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Remote Sensing (AREA)
  • Computational Linguistics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides an automatic identification method, system, equipment and medium for unmanned aerial vehicle inspection data defects, wherein the method comprises the steps of obtaining inspection image data of various power equipment in a target inspection area, and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module; acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label. The method can effectively reduce defect identification cost, provide missing data identification accuracy, and improve comprehensiveness, accuracy and high efficiency of fault marking while being convenient for timely finding hidden danger and problems of power grid equipment.

Description

Unmanned aerial vehicle inspection data defect automatic identification method, system, equipment and medium
Technical Field
The invention relates to the technical field of unmanned aerial vehicle inspection of power distribution networks, in particular to an automatic identification method, an automatic identification system, computer equipment and a storage medium for unmanned aerial vehicle inspection data defects.
Background
The distribution overhead line equipment has wide points, scattered regions and complex branch lines, and the increasing distribution line equipment brings huge pressure to the stable operation of the distribution network of a company. Compared with the power line, the power distribution line has relatively short distance, dense grids and complex branch line paths, and the traditional unmanned aerial vehicle line type radiation inspection mode of the power line is difficult to be suitable for.
At present, although preliminary application is realized on the basis of edge calculation for carrying out unmanned aerial vehicle inspection of distribution line equipment, the distribution line is intricate and dense, the background environment is dynamically complex, the practical problems of limited image acquisition angle, background dynamic interference, high autonomous flight route planning complexity and the like exist in the unmanned aerial vehicle operation process of the distribution line, the unmanned aerial vehicle accurate positioning navigation and equipment defect identification technology still needs deep attack and improvement, the defect identification accuracy is low, the later stage is completed through manual secondary audit, time and labor are wasted, the efficiency is low, the identification result is influenced by the subjectivity of the professional level of personnel, the accuracy is not ensured, hidden danger and problem of power grid equipment are difficult to discover in time, and reliable guarantee cannot be provided for the safe operation of a power grid.
Disclosure of Invention
The invention aims to provide an automatic identification method for unmanned aerial vehicle inspection data defects, which is characterized in that a fault identification model which is rich in fault samples and has representativeness is established, a fault identification mode for automatic fault detection and marking is trained through image data set, a fault target is marked through a method of introducing context information marking by combining equipment information, equipment history operation data and image information is sampled, the application defects of high defect identification cost and low accuracy of the existing unmanned aerial vehicle inspection data of a distribution line are overcome, the defect identification cost can be effectively reduced, the defect identification accuracy is provided, hidden danger and problem of power grid equipment can be found in time conveniently, the comprehensiveness, the accuracy and the high efficiency of the fault marking can be improved through comprehensive utilization of the equipment information, the history operation data and the image information, and solid guarantee is provided for safe and stable operation of a transformer substation.
In order to achieve the above object, it is necessary to provide an automatic identification method, system, computer device and storage medium for unmanned aerial vehicle inspection data defects, aiming at the above technical problems.
In a first aspect, an embodiment of the present invention provides a method for automatically identifying defects of inspection data of an unmanned aerial vehicle, where the method includes the following steps:
Acquiring inspection image data of various power equipment in a target inspection area, and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; the inspection image data comprises various fault type image data and normal image data;
training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module;
acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label.
Further, the step of constructing the unmanned aerial vehicle inspection image data set according to the inspection image data includes:
screening the inspection image data according to a preset fault detection requirement to obtain inspection image sample data;
information labeling is carried out on each inspection image sample data according to a preset image labeling tool and preset labeling content, and an unmanned aerial vehicle inspection image data set is obtained; the preset labeling content comprises equipment information, shooting information and fault information; the device information includes a device model number and a device manufacturer; the fault information includes a fault type, a fault region location, and a fault region shape.
Further, the step of training a preset target detection model according to the unmanned aerial vehicle inspection image dataset to obtain a fault identification model includes:
inputting the unmanned aerial vehicle inspection image data set into a fault detection module in the preset target detection model to detect a fault target, and obtaining a fault target detection result; the fault target detection result comprises a fault target and a corresponding fault type;
inputting the fault target detection result into a fault labeling module in the preset target detection model to label fault information, so as to obtain a fault prediction result;
and carrying out iterative updating on parameters of the preset target detection model according to comparison analysis of the fault prediction result of each inspection image sample data in the unmanned aerial vehicle inspection image data set and corresponding preset labeling content to obtain the fault identification model.
Further, the step of inputting the fault target detection result into the fault labeling module in the preset target detection model to label fault information, and obtaining a fault prediction result includes:
acquiring the position of a corresponding image fault area according to the fault target;
Acquiring corresponding fault marking information according to the fault target and a pre-constructed equipment information base;
and marking the fault marking information at the position of the fault area of the image to obtain the fault prediction result.
Further, the step of obtaining the corresponding image fault region position according to the fault target includes:
acquiring a corresponding fault pixel region according to the fault target;
creating a blank marking mask on the fault pixel region; the shape of the blank mark mask is the same as that of the fault pixel region;
and selecting a corresponding pixel region filling mode according to the shape of the blank marking mask to fill and draw the blank marking mask, so as to obtain the position of the image fault region.
Further, the fault annotation information comprises equipment information, equipment position information, fault description information and equipment historical operation data;
the step of obtaining corresponding fault labeling information according to the fault target and a pre-constructed equipment information base comprises the following steps:
acquiring corresponding fault equipment information according to the fault target;
and searching the pre-constructed equipment information base according to the fault equipment information to acquire corresponding equipment position information, fault description information and equipment history operation data.
Further, the step of constructing the device information base includes:
acquiring operation and maintenance document data of various electric equipment, and adopting a preset natural language processing model to perform named entity identification on the operation and maintenance document data to obtain equipment operation and maintenance key information; the equipment operation and maintenance key information comprises equipment models, equipment manufacturers, equipment positions and fault descriptions corresponding to various fault types;
obtaining continuous operation records of various power equipment, and analyzing the continuous operation history records to obtain corresponding equipment history operation data; the continuous operation record comprises equipment operation states, equipment temperatures and equipment currents at all historical moments within a preset duration range; the equipment history operation data comprises equipment history states and corresponding equipment operation characteristics;
and according to the equipment model, carrying out data association on the equipment operation and maintenance key information and the equipment historical operation data, and establishing the equipment information base.
In a second aspect, an embodiment of the present invention provides an automatic identification system for a defect of inspection data of an unmanned aerial vehicle, where the system includes:
the data acquisition module is used for acquiring inspection image data of various power equipment in the target inspection area and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; the inspection image data comprises various fault type image data and normal image data;
The model construction module is used for training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module;
the fault identification module is used for acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label.
In a third aspect, embodiments of the present invention further provide a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method when executing the computer program.
In a fourth aspect, embodiments of the present invention also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above method.
The method comprises the steps of obtaining inspection image data of various power equipment in a target inspection area, constructing an unmanned aerial vehicle inspection image data set according to the inspection image data, training a preset target detection model sequentially comprising a fault detection module and a fault marking module according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model, obtaining an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding technical scheme of fault identification results comprising fault types and fault area marks. Compared with the prior art, the unmanned aerial vehicle inspection data defect automatic identification method can effectively reduce defect identification cost and provide missing data identification accuracy, is convenient for finding hidden danger and problems of power grid equipment in time, and can improve comprehensiveness, accuracy and high efficiency of fault marking by comprehensively utilizing equipment information, historical operation data and image information, so that firm guarantee is provided for safe and stable operation of a transformer substation.
Drawings
Fig. 1 is a schematic diagram of an application scenario of an automatic identification method for defects of unmanned aerial vehicle inspection data in an embodiment of the invention;
fig. 2 is a schematic flow chart of an automatic identification method for defects of inspection data of an unmanned aerial vehicle in an embodiment of the invention;
FIG. 3 is a schematic diagram of a failed pixel region acquired according to a failure target in an embodiment of the invention;
FIG. 4 is a schematic diagram of a blank annotation mask created from a defective pixel area in an embodiment of the invention;
FIG. 5 is a schematic diagram of the position of an image fault area obtained by filling and drawing a blank mark mask in an embodiment of the invention;
fig. 6 is a schematic structural diagram of an automatic identification system for defects of inspection data of an unmanned aerial vehicle in an embodiment of the invention;
fig. 7 is an internal structural view of a computer device in an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantageous effects of the present application more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples, and it should be understood that the examples described below are only illustrative of the present invention and are not intended to limit the scope of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The unmanned aerial vehicle inspection data defect automatic identification method provided by the invention can be understood as considering that the unmanned aerial vehicle automatic inspection data defect identification accuracy is low due to the fact that the existing unmanned aerial vehicle inspection of the distribution network line is complicated due to the fact that lines are staggered and dense, background environment is dynamic, and the like, manual auxiliary inspection is needed, the efficiency is low, the cost is high, the accuracy of an identification result is not guaranteed, hidden danger and problems of power grid equipment are difficult to find in time through the inspection result, and the application situation that reliable guarantee cannot be provided for safe operation of a power grid is caused. The method can be applied to a terminal or a server as shown in fig. 1, wherein the terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server can be implemented by a stand-alone server or a server cluster formed by a plurality of servers. The server can carry out automatic fault detection and comprehensive and accurate fault marking by adopting the unmanned aerial vehicle inspection data defect automatic identification method provided by the invention according to actual application requirements, and the correspondingly obtained fault identification result is used for subsequent research of the server or is transmitted to a terminal for a terminal user to check and analyze; the following embodiment will explain the automatic identification method of the unmanned aerial vehicle inspection data defect.
In one embodiment, as shown in fig. 2, there is provided a method for automatically identifying defects of inspection data of an unmanned aerial vehicle, including the steps of:
s11, acquiring inspection image data of various power equipment in a target inspection area, and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; the inspection image data comprises various fault type image data and normal image data; the inspection image data can be understood as photographs taken by an electric power inspection personnel when faults are detected, corresponding equipment types, fault types and other data, and can be selected from different electric power equipment and environmental conditions including a transformer substation, a power transmission line, a power transmission tower and the like, and in order to ensure that sources of samples are representative, various possible fault conditions can be covered, and abundant fault types such as equipment damage, looseness, foreign matter invasion and the like can be acquired as far as possible;
considering that the collected original inspection image data may include low-quality invalid data which does not utilize fault detection, such as repetition, data missing, and the like, in order to improve the reliability of the unmanned plane inspection image data set for automatic inspection model training, in this embodiment, the obtained inspection image data are preferably screened and reasonably labeled; specifically, the step of constructing the unmanned aerial vehicle inspection image data set according to the inspection image data includes:
Screening the inspection image data according to a preset fault detection requirement to obtain inspection image sample data; the preset fault detection requirement can be formulated according to an actual application scene, and is not particularly limited herein; in principle, only the obtained inspection image sample data needs to be ensured to comprise high-quality data of different power equipment in normal states and various fault types under different environmental conditions as far as possible, and a specific screening mode is not limited herein;
information labeling is carried out on each inspection image sample data according to a preset image labeling tool and preset labeling content, and an unmanned aerial vehicle inspection image data set is obtained; the preset labeling content comprises equipment information, shooting information and fault information; the device information includes a device model number and a device manufacturer; the fault information comprises a fault type, a fault area position and a fault area shape; the preset image marking tool can be understood as a tool for marking the position and the shape of a fault region in each fault image sample, and in practical application, fault objects and fault types to be marked can be selected according to requirements, for example, target detection and rectangular bounding box marking are required, the target type diversity is less, and when the position and the size of a target are mainly required to be marked, the LabelImg image marking tool can be used; the method has the advantages that target detection, rectangular bounding boxes and polygonal labeling are required, the variety of target types is large, flexible shape labeling tools such as polygonal labeling, key point or characteristic point labeling, labeling of custom attributes or metadata and the like are required, and an image labeling tool RectLabel can be used; the method comprises the steps that target detection, rectangular bounding boxes and polygonal labeling are needed, diversified target shapes and areas are supported, polygonal labeling or segmentation masks are used for accurately labeling, and an image labeling tool VoTT is used under the condition that example-level target segmentation is needed; considering that an image annotation tool of an unmanned aerial vehicle inspection scene needs to support various image annotation types, such as a rectangular frame, a polygon, a key point annotation, a custom attribute annotation, a mask and the like, and has the task requirements of a convenient interaction interface and flexible expansibility, the embodiment preferably uses a RectLabel tool to carry out information annotation on inspection image sample data;
It should be noted that, the above information labeling process can also record important information such as equipment type, shooting environment, shooting time and the like related to each image, provide reference for subsequent labeling work, and facilitate the arrangement and archiving of the whole data set, so that a rich and representative image data set is obtained, powerful data support is provided for subsequent training of an automatic fault detection and labeling algorithm, and meanwhile, the data set can be orderly and easy to maintain and manage, and reliable support is provided for the whole unmanned aerial vehicle inspection work.
S12, training a preset target detection model according to the unmanned aerial vehicle inspection image dataset to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module; the fault detection module in the preset target detection model may be understood as a module with an image recognition function, and may be implemented by using existing methods such as target detection, image segmentation or anomaly detection, for example, convolutional neural network (Convolutional Neural Network, CNN), single Shot MultiBox Detector (SSD), you Only Look Once (YOLO), U-Net, mask R-CNN, support vector machine (Support Vector Machine, SVM), SIFT (Scale-Invariant Feature Transform), SURF (speed-Up Robust Features), and the like, which are not limited herein;
The fault labeling module can be understood as a key improvement of a fault recognition model, and is mainly used for fusing equipment information, equipment historical operation data and image information in a manner of introducing context information so as to ensure the comprehensiveness and the accuracy of labeling information while guaranteeing high-efficiency labeling, provide powerful support for fault positioning analysis and further provide firm guarantee for safe and stable operation of a transformer substation; it should be noted that, in this embodiment, the obtained unmanned aerial vehicle inspection image dataset is divided into two parts, namely a training set and a testing set, the training set is used for training a fault recognition model including automatic fault detection and automatic fault labeling, and the testing set is used for evaluating the accuracy of fault detection and labeling results;
specifically, the step of training a preset target detection model according to the unmanned aerial vehicle inspection image dataset to obtain a fault identification model includes:
inputting the unmanned aerial vehicle inspection image data set into a fault detection module in the preset target detection model to detect a fault target, and obtaining a fault target detection result; the fault target detection result comprises a fault target and a corresponding fault type;
Inputting the fault target detection result into a fault labeling module in the preset target detection model to label fault information, so as to obtain a fault prediction result; the fault information labeling can be understood as a process of labeling a fault area and related information on a corresponding inspection image based on a fault target detection result output by the fault detection module; specifically, the step of inputting the fault target detection result into the fault labeling module in the preset target detection model to label fault information, and obtaining a fault prediction result includes:
acquiring the position of a corresponding image fault area according to the fault target; the image fault region position can be understood as generating a fault region with a shape corresponding to the fault type on the image through a labeling algorithm; specifically, the step of obtaining the corresponding image fault region position according to the fault target includes:
acquiring a corresponding fault pixel region according to the fault target; the fault pixel region can be understood as region position information obtained according to the position coordinates (pixel coordinates of the center of the fault region) of the fault target obtained by the fault detection module and the corresponding fault type; for example, if the object is described in a bounding box or polygonal area, it may be converted into a corresponding pixel area; if the key points or the contour information exists, the key points or the contour information can be converted into corresponding pixel areas according to a related algorithm;
Creating a blank marking mask on the fault pixel region; the shape of the blank mark mask is the same as that of the fault pixel region; the blank labeling mask is understood to be a mask with all initial pixel values set to zero, that is, the pixel values of the fault pixel areas obtained in the above steps are all cleared, the size of the blank labeling mask should be the same as that of the original image, so as to ensure that each pixel position has corresponding labeling information, and a corresponding data structure can be provided by adopting a related programming language to represent a matrix, for example, if a Python language algorithm is used, the blank labeling mask can be represented by using a data structure in a NumPy library, and the details are not described herein;
selecting a corresponding pixel region filling mode according to the shape of the blank marking mask to fill and draw the blank marking mask, so as to obtain the position of the image fault region; the pixel region filling mode can be selected according to the shape of the blank marking mask and the fault target position information, for example, if the blank marking mask is in a rectangular boundary box shape, the pixel region (width is w and height is h) around the pixel coordinates (x and y) at the center of the fault region is set to be a non-zero value; if the pixel area is in a polygonal shape, filling the pixel area corresponding to the scanning line filling algorithm according to a given polygonal vertex coordinate array; if the key point or the outline shape is the key point or the outline shape, the key point or the outline information can be converted into a continuous pixel area by using the technologies such as linear interpolation, closed curve fitting and the like, and filling is carried out on a blank mark mask; the following description will be given by taking a label for generating a rectangular Bounding Box (Bounding Box) as an example:
Assuming that the target image I is input (size w×h= 3*3), the labeling result needs to be output: m (binary mask representing a fault, the same size as the input image), pixel coordinates (x=1, y=2) at the center of the fault region and fault type fault_type; the process of correspondingly acquiring the location of the image failure area can be understood as: firstly, according to a fault target, a corresponding fault pixel region, namely a binary mask region shown in fig. 3, is obtained; resetting the pixel value of the obtained fault pixel region to obtain a blank marking mask shown in fig. 4; then, filling and drawing the blank marking mask, so that the marking mask shown in fig. 5, namely the position of the image fault region, can be obtained;
acquiring corresponding fault marking information according to the fault target and a pre-constructed equipment information base; the fault marking information comprises equipment information, equipment position information, fault description information and equipment history operation data; specifically, the step of obtaining corresponding fault labeling information according to the fault target and a pre-constructed equipment information base includes:
acquiring corresponding fault equipment information according to the fault target; wherein the fault equipment information comprises equipment model numbers, equipment manufacturers and the like;
Searching the pre-constructed equipment information base according to the fault equipment information to acquire corresponding equipment position information, fault description information and equipment history operation data; the equipment information base can be understood as a database which can index information such as corresponding equipment manufacturers, equipment positions, fault descriptions corresponding to various fault types, continuous operation records of equipment and the like according to equipment signals; specifically, the step of constructing the equipment information base includes:
acquiring operation and maintenance document data of various electric equipment, and adopting a preset natural language processing model to perform named entity identification on the operation and maintenance document data to obtain equipment operation and maintenance key information; the equipment operation and maintenance key information comprises equipment models, equipment manufacturers, equipment positions and fault descriptions corresponding to various fault types; the operation and maintenance document data of various power equipment can be understood as maintenance reports, fault records, work logs, technical documents and the like related to the power equipment, and the documents generally contain detailed descriptions of equipment states, fault conditions and maintenance records; in addition, the method can also comprise automatically acquiring document materials and the like in the power system through network retrieval, such as using a web crawler or a data acquisition tool (such as Beautiful Soup, scrapy and the like in Python);
The process of obtaining the key information of the operation and maintenance of the equipment can be understood as follows: by adopting a natural language processing technology, each acquired operation and maintenance document can be scanned by means of a pre-trained Named Entity Recognition (NER) model, and important information directly related to equipment is accurately recognized and extracted from the operation and maintenance document, wherein the important information comprises equipment types, names of various parts and other named entities, for example, for a given maintenance report or fault record, the related equipment types and the specific parts possibly having problems can be rapidly and accurately recognized by the aid of the pre-trained natural language processing model; meanwhile, a professional vocabulary of the equipment and the components is constructed by calling all spare part names in the existing warehouse system, and entities related to the power equipment can be quickly searched and identified in the text through the vocabulary, so that key information can be locked; the specific keyword or phrase required to be extracted through the natural language processing technology can be determined according to the actual task requirement, for example, in fault detection, the keyword can comprise a fault type, a device name and the like, and a regular expression (Regular Expression, regex or RegExp) is used for matching formatted information such as a device number, a fault code and the like; the dependency syntax analysis tool is then used to identify dependencies between words in the sentence to help extract key information, such as the following description, which may be included in a maintenance report: by means of entity identification, the entity in the transformer A12 can be marked: device name entity: "Transformer A12", fault type entity: the two pieces of key information are very important for fault detection, provide specific description and occurrence position of faults, and provide basis for subsequent processing and decision-making; an extraction model may then be used to extract key information from the above description of the equipment failure; it should be noted that, here, a support vector machine (Support Vector Machine, SVM), a cyclic neural network (Recurrent Neural Network, RNN), a Long Short-Term Memory network (LSTM), a convolutional neural network (Convolutional Neural Network, CNN) or the like may be used to perform sequence modeling on the text to extract key information, so that entities related to the power equipment can be efficiently extracted from a large amount of text information, and a solid foundation is laid for subsequent fault identification and labeling work;
Obtaining continuous operation records of various power equipment, and analyzing the continuous operation history records to obtain corresponding equipment history operation data; the continuous operation record comprises equipment operation states, equipment temperatures and equipment currents at all historical moments within a preset duration range; the equipment history operation data comprises equipment history states and corresponding equipment operation characteristics;
according to the equipment model, carrying out data association on the equipment operation and maintenance key information and the equipment historical operation data, and establishing the equipment information base;
the equipment information base constructed through the method steps can comprise detailed information of each piece of electric equipment, such as equipment model, manufacturer, operation time and geographical position information of the equipment, including important data such as longitude and latitude, belonging substations and the like, and the information can be called at any time in the labeling process, so that powerful support is provided for accurate labeling; meanwhile, the continuous recording and analysis of the parameters such as the running state, the temperature, the current and the like of each power device are stored to obtain the running characteristics and the history state of the device, so that the long-term running condition of the device can be known, and clues can be provided for fault labeling; for example, a device may exhibit abnormal current fluctuations in historical data, which may mean that the device is potentially problematic and needs to be of particular concern;
Labeling the fault labeling information at the position of the fault region of the image to obtain the fault prediction result; the fault marking information in the fault prediction result can be understood as all information which is obtained by fusing equipment information, equipment historical operation information and image fault identification information and can be presented on a marking interface, wherein the equipment information including equipment model and equipment manufacturer is directly displayed, and the corresponding equipment historical operation information can be checked by clicking the corresponding equipment information, so that the operation condition of equipment can be conveniently known;
according to the comparison analysis of the fault prediction result of each inspection image sample data in the unmanned aerial vehicle inspection image data set and the corresponding preset labeling content, performing iterative updating on the parameters of the preset target detection model to obtain the fault identification model; after the fault prediction result is obtained through the method, the fault prediction result can be compared with the real labeling information (real label) of the corresponding inspection image sample data, the accuracy and the understandability of fault labeling are optimized by considering the characteristics and the surrounding background information of the target, the labeled image data set is utilized for verification and evaluation, the consistency of the automatic fault labeling result and the manual labeling is compared, and the model parameters are iteratively updated according to the evaluation result, wherein the specific process is as follows:
Randomly selecting a part of inspection image samples to obtain labeling results according to a fault recognition model, inviting professionals or field experts to manually evaluate the samples, taking the evaluation results as standards, comparing the automatic labeling results with the manual evaluation results, calculating evaluation indexes such as accuracy, recall rate and the like, determining the performance of the fault recognition model by using the indexes, carrying out detailed error analysis on samples inconsistent with the manual evaluation results, determining common error types and reasons of automatic labeling, and adjusting and optimizing an automatic labeling algorithm according to the error analysis results, such as adjusting model parameters, updating feature engineering or adopting a higher-level model, and re-labeling a data set by using the optimized and updated model so as to ensure the accuracy of the labeling results, and repeating the steps until the performance of the fault recognition model reaches a satisfactory level;
it should be noted that, in this embodiment, the fault labeling may further perform function expansion by adding an interactive interface, where an original image and a labeling result on the fault labeling interface are displayed side by side, and the specific design of the interactive interface is as follows: 1. providing a segmentation view which can be freely adjusted in size and position, so that an operator can simultaneously view an original image, an automatic labeling result and a manually corrected labeling; 2. providing intelligent prompt and search; 3. providing an intelligent prompt function, and rapidly completing annotation according to the existing fault labels and keywords; 4. supporting a keyword-based search function so that an operator can quickly find and edit a specific annotation item; 5. interaction tool and mode selection: providing a plurality of interactive tools, including rectangular frames, polygons or masks with adjustable sizes, so as to adapt to marking requirements of different fault types; 6. marking modes, such as drawing, modifying or deleting modes, can be switched to flexibly adapt to different operation requirements in the marking process; 7. gesture and shortcut key support: support common gesture operations such as zoom, pan and rotate to make finer operations when labeling large-size images; 8. providing a configurable shortcut key to accelerate the labeling process and improve the efficiency of the operation; 9. marking quality evaluation and correction: providing a real-time marking quality evaluation index, such as IoU (Intersection over Union) or Dice coefficient, so that an operator can know the accuracy of a marking result and can manually adjust marking boundaries or shapes to carry out fine correction and improvement when required; 10. labeling version control and collaboration support: allowing the labeling results of different versions to be stored, and providing version comparison and rollback functions to track changes in the labeling process; 11. supporting multi-user cooperation, allowing multiple operators to annotate the same image at the same time, and providing a conflict resolution mechanism and annotation functions; 12. export and preservation: providing a plurality of export format options, such as common labeling formats (such as Pascal VOC, YOLO and the like) or custom formats, so as to facilitate subsequent model training and data analysis; 13. supporting automatic save and restore functions to prevent label data loss caused by unexpected interface closing or temporary power failure; the interface can display an original image and an automatic fault detection result, provide a user interaction tool (such as a drawing frame, a polygon, a shade and the like) to manually correct or add labels, and simultaneously can be used for manually rechecking or manually intervening, timely collect feedback of a user, optimize the interface according to the feedback and improve user experience.
S13, acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the unmanned aerial vehicle inspection image to be identified can be understood as image data which is acquired in real time through the unmanned aerial vehicle inspection and needs defect data identification in actual application, the corresponding fault identification result comprises a fault type and a fault area mark, the corresponding acquisition process can refer to the related description in the training process of the fault identification model, and the description is omitted here;
the embodiment of the application trains the preset target detection model sequentially comprising the fault detection module and the fault labeling module according to the unmanned aerial vehicle inspection image data set after acquiring the inspection image data comprising various fault type image data and normal image data of various power equipment in the target inspection area and constructing the unmanned aerial vehicle inspection image data set according to the inspection image data, obtains the fault recognition model, acquires the unmanned aerial vehicle inspection image to be recognized, inputs the unmanned aerial vehicle inspection image to be recognized into the fault recognition model for fault recognition and labeling, obtains the corresponding scheme comprising the fault recognition result of fault type and fault area labeling, effectively solves the problem that the unmanned aerial vehicle automatic inspection data defect recognition accuracy is lower due to the factors of line miscompaction, dynamic complexity of background environment and the like in the existing distribution network line unmanned aerial vehicle inspection, the system has the advantages that manual auxiliary checking is needed, the efficiency is low, the cost is high, the accuracy of the identification result is not guaranteed, hidden danger and problem of power grid equipment are difficult to find in time through the checking result, the application defect that reliable guarantee cannot be provided for safe operation of the power grid is caused, the checking efficiency is remarkably improved through combining front-end identification and edge calculation fusion to the distribution network equipment body identification and positioning photographing technology, the 5G networking technology, front-end identification and calculation analysis and the image identification technology, the electric power checking task is completed through equipment with high safety, low strength, reliable quality, low cost and high automation level, the defect identification cost is effectively reduced, missing data identification accuracy is provided, hidden danger and problem of the power grid equipment can be found in time conveniently, and the comprehensiveness of fault marking can be improved through comprehensive utilization of equipment information, historical operation data and image information Accuracy and high efficiency provide solid guarantee for the safe and stable operation of the transformer substation.
Although the steps in the flowcharts described above are shown in order as indicated by arrows, these steps are not necessarily executed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders.
In one embodiment, as shown in fig. 6, there is provided an automatic identification system for unmanned aerial vehicle inspection data defects, the system comprising:
the data acquisition module 1 is used for acquiring inspection image data of various power equipment in a target inspection area and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; the inspection image data comprises various fault type image data and normal image data;
the model construction module 2 is used for training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module;
the fault identification module 3 is used for acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label.
The specific limitation of the unmanned aerial vehicle inspection data defect automatic identification system can be referred to the limitation of the unmanned aerial vehicle inspection data defect automatic identification method, and the corresponding technical effects can be obtained equally, and are not repeated here. All or part of each module in the unmanned aerial vehicle inspection data defect automatic identification system can be realized by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
Fig. 7 shows an internal structural diagram of a computer device, which may be a terminal or a server in particular, in one embodiment. As shown in fig. 7, the computer device includes a processor, a memory, a network interface, a display, a camera, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by the processor to realize the automatic identification method of the unmanned aerial vehicle inspection data defects. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those of ordinary skill in the art that the architecture shown in fig. 7 is merely a block diagram of some of the architecture relevant to the present application and is not intended to limit the computer device on which the present application may be implemented, and that a particular computing device may include more or fewer components than shown, or may combine certain components, or have the same arrangement of components.
In one embodiment, a computer device is provided comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method when the computer program is executed.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, implements the steps of the above method.
In summary, the automatic identification method and system for the unmanned aerial vehicle inspection data defects provided by the embodiment of the invention realize the technical scheme that the automatic identification method for the unmanned aerial vehicle inspection data defects obtains inspection image data comprising various fault type image data and normal image data of various power equipment in a target inspection area, after an unmanned aerial vehicle inspection image data set is constructed according to the inspection image data, a preset target detection model sequentially comprising a fault detection module and a fault labeling module is trained according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model, the unmanned aerial vehicle inspection image to be identified is obtained, the unmanned aerial vehicle inspection image to be identified is input into the fault identification model to carry out fault identification and labeling, and the corresponding technical scheme comprising fault type and fault area labeling fault identification results is obtained.
In this specification, each embodiment is described in a progressive manner, and all the embodiments are directly the same or similar parts referring to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments. It should be noted that, any combination of the technical features of the foregoing embodiments may be used, and for brevity, all of the possible combinations of the technical features of the foregoing embodiments are not described, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples represent only a few preferred embodiments of the present application, which are described in more detail and are not thereby to be construed as limiting the scope of the invention. It should be noted that modifications and substitutions can be made by those skilled in the art without departing from the technical principles of the present invention, and such modifications and substitutions should also be considered to be within the scope of the present application. Therefore, the protection scope of the patent application is subject to the protection scope of the claims.

Claims (10)

1. An automatic identification method for unmanned aerial vehicle inspection data defects is characterized by comprising the following steps:
acquiring inspection image data of various power equipment in a target inspection area, and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; the inspection image data comprises various fault type image data and normal image data;
training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module;
acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label.
2. The method for automatically identifying defects of inspection data of an unmanned aerial vehicle according to claim 1, wherein the step of constructing an unmanned aerial vehicle inspection image dataset from the inspection image data comprises:
screening the inspection image data according to a preset fault detection requirement to obtain inspection image sample data;
Information labeling is carried out on each inspection image sample data according to a preset image labeling tool and preset labeling content, and an unmanned aerial vehicle inspection image data set is obtained; the preset labeling content comprises equipment information, shooting information and fault information; the device information includes a device model number and a device manufacturer; the fault information includes a fault type, a fault region location, and a fault region shape.
3. The method for automatically identifying defects of unmanned aerial vehicle inspection data according to claim 2, wherein the step of training a preset target detection model according to the unmanned aerial vehicle inspection image dataset to obtain a fault identification model comprises the steps of:
inputting the unmanned aerial vehicle inspection image data set into a fault detection module in the preset target detection model to detect a fault target, and obtaining a fault target detection result; the fault target detection result comprises a fault target and a corresponding fault type;
inputting the fault target detection result into a fault labeling module in the preset target detection model to label fault information, so as to obtain a fault prediction result;
and carrying out iterative updating on parameters of the preset target detection model according to comparison analysis of the fault prediction result of each inspection image sample data in the unmanned aerial vehicle inspection image data set and corresponding preset labeling content to obtain the fault identification model.
4. The method for automatically identifying defects of unmanned aerial vehicle inspection data according to claim 3, wherein the step of inputting the fault target detection result into a fault labeling module in the preset target detection model to label fault information and obtain a fault prediction result comprises the steps of:
acquiring the position of a corresponding image fault area according to the fault target;
acquiring corresponding fault marking information according to the fault target and a pre-constructed equipment information base;
and marking the fault marking information at the position of the fault area of the image to obtain the fault prediction result.
5. The unmanned aerial vehicle inspection data defect automatic identification method of claim 4, wherein the step of acquiring the corresponding image fault region location according to the fault target comprises:
acquiring a corresponding fault pixel region according to the fault target;
creating a blank marking mask on the fault pixel region; the shape of the blank mark mask is the same as that of the fault pixel region;
and selecting a corresponding pixel region filling mode according to the shape of the blank marking mask to fill and draw the blank marking mask, so as to obtain the position of the image fault region.
6. The unmanned aerial vehicle inspection data defect automatic identification method of claim 5, wherein the fault annotation information comprises device information, device location information, fault description information and device history operation data;
the step of obtaining corresponding fault labeling information according to the fault target and a pre-constructed equipment information base comprises the following steps:
acquiring corresponding fault equipment information according to the fault target;
and searching the pre-constructed equipment information base according to the fault equipment information to acquire corresponding equipment position information, fault description information and equipment history operation data.
7. The unmanned aerial vehicle inspection data defect automatic identification method according to claim 6, wherein the step of constructing the equipment information base comprises:
acquiring operation and maintenance document data of various electric equipment, and adopting a preset natural language processing model to perform named entity identification on the operation and maintenance document data to obtain equipment operation and maintenance key information; the equipment operation and maintenance key information comprises equipment models, equipment manufacturers, equipment positions and fault descriptions corresponding to various fault types;
obtaining continuous operation records of various power equipment, and analyzing the continuous operation history records to obtain corresponding equipment history operation data; the continuous operation record comprises equipment operation states, equipment temperatures and equipment currents at all historical moments within a preset duration range; the equipment history operation data comprises equipment history states and corresponding equipment operation characteristics;
And according to the equipment model, carrying out data association on the equipment operation and maintenance key information and the equipment historical operation data, and establishing the equipment information base.
8. An unmanned aerial vehicle inspection data defect automatic identification system, the system comprising:
the data acquisition module is used for acquiring inspection image data of various power equipment in the target inspection area and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; the inspection image data comprises various fault type image data and normal image data;
the model construction module is used for training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module;
the fault identification module is used for acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
CN202311512833.1A 2023-11-14 2023-11-14 Unmanned aerial vehicle inspection data defect automatic identification method, system, equipment and medium Pending CN117612036A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311512833.1A CN117612036A (en) 2023-11-14 2023-11-14 Unmanned aerial vehicle inspection data defect automatic identification method, system, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311512833.1A CN117612036A (en) 2023-11-14 2023-11-14 Unmanned aerial vehicle inspection data defect automatic identification method, system, equipment and medium

Publications (1)

Publication Number Publication Date
CN117612036A true CN117612036A (en) 2024-02-27

Family

ID=89957103

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311512833.1A Pending CN117612036A (en) 2023-11-14 2023-11-14 Unmanned aerial vehicle inspection data defect automatic identification method, system, equipment and medium

Country Status (1)

Country Link
CN (1) CN117612036A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118072250A (en) * 2024-04-16 2024-05-24 武汉人云智物科技有限公司 Target detection method and device based on hydropower plant video monitoring system
CN118071123A (en) * 2024-04-19 2024-05-24 季华实验室 Power line inspection unmanned aerial vehicle regulation and control method and related equipment
CN118314574A (en) * 2024-04-28 2024-07-09 江苏中天互联科技有限公司 Fault information labeling method and related equipment
CN118586895A (en) * 2024-08-06 2024-09-03 华能山东发电有限公司烟台发电厂 Equipment maintenance management system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118072250A (en) * 2024-04-16 2024-05-24 武汉人云智物科技有限公司 Target detection method and device based on hydropower plant video monitoring system
CN118071123A (en) * 2024-04-19 2024-05-24 季华实验室 Power line inspection unmanned aerial vehicle regulation and control method and related equipment
CN118314574A (en) * 2024-04-28 2024-07-09 江苏中天互联科技有限公司 Fault information labeling method and related equipment
CN118586895A (en) * 2024-08-06 2024-09-03 华能山东发电有限公司烟台发电厂 Equipment maintenance management system

Similar Documents

Publication Publication Date Title
CN117612036A (en) Unmanned aerial vehicle inspection data defect automatic identification method, system, equipment and medium
CN109635666B (en) Image target rapid detection method based on deep learning
US11657373B2 (en) System and method for identifying structural asset features and damage
CN113255614A (en) RPA flow automatic generation method and system based on video analysis
CN109858414A (en) A kind of invoice piecemeal detection method
CN110910360B (en) Positioning method of power grid image and training method of image positioning model
CN112102443A (en) Marking system and marking method suitable for substation equipment inspection image
CN113656477A (en) Method for verifying and fusing multi-source heterogeneous data of homeland space
CN116823793A (en) Device defect detection method, device, electronic device and readable storage medium
CN115586536A (en) Forest resource investigation monitoring system and method based on laser point cloud
CN114913376A (en) Image-based defect automatic identification method, device and system and storage medium
CN118245560A (en) Efficient electronic chart data organization method
CN117635859B (en) Quick map drawing method and system based on GIS data
CN116932499B (en) Method for modifying labels of DWG format files in batches
CN113779261A (en) Knowledge graph quality evaluation method and device, computer equipment and storage medium
CN117708720A (en) Equipment fault diagnosis system based on knowledge graph
Vieth Point cloud classification and segmentation of catenary systems
CN113409305B (en) Optical cable resource digital management method
Sheng et al. A Method and Implementation of Transmission Line's Key Components and Defects Identification Based on YOLO
KR100941771B1 (en) Method and apparatus for updating gis data
CN117372510B (en) Map annotation identification method, terminal and medium based on computer vision model
CN115390853B (en) Structured analysis method, system, terminal and storage medium for multi-source process file
Edwards Digital Twin Framework and Auto-Linking for Management of Legacy Assets
KR102086112B1 (en) Apparatus for updating digital mockup of nuclear power plant and method thereof
Liu Intelligent surveying and geographic information data updating method based on ArcGIS server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination