CN117612036A - Methods, systems, equipment and media for automatic identification of defects in drone inspection data - Google Patents

Abstract

The invention provides an automatic identification method, system, equipment and medium for unmanned aerial vehicle inspection data defects, wherein the method comprises the steps of obtaining inspection image data of various power equipment in a target inspection area, and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module; acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label. The method can effectively reduce defect identification cost, provide missing data identification accuracy, and improve comprehensiveness, accuracy and high efficiency of fault marking while being convenient for timely finding hidden danger and problems of power grid equipment.

Description

Methods, systems, equipment and media for automatic identification of defects in drone inspection data

Technical field

The invention relates to the technical field of drone inspection of distribution network, and in particular to a method, system, computer equipment and storage medium for automatic identification of defects in drone inspection data.

Background technique

Distribution overhead lines have many equipment points, are geographically dispersed, and have complex branch lines. The growing number of distribution line equipment puts tremendous pressure on the stable operation of the company's distribution network. Compared with power lines, distribution lines have relatively short distances, dense grids, and complex branch lines, making it difficult to apply the traditional power line drone "line" radiation inspection model.

At present, although drone inspections of distribution line equipment based on edge computing have been initially applied, the distribution network lines are intricate and dense, and the background environment is dynamically complex. There are limitations in image acquisition angles during drone operations on distribution lines. Realistic problems such as background dynamic interference and high complexity of autonomous flight route planning have resulted in UAV precise positioning and navigation and equipment defect identification technology still need to be in-depth research and improvement. The accuracy of defect identification is low, and it needs to be completed through manual secondary review in the later stage. Not only is it time-consuming and labor-intensive, but the efficiency is low, the identification results are also affected by the subjectivity of personnel's professional level, and the accuracy cannot be guaranteed. It is difficult to detect hidden dangers and problems in power grid equipment in a timely manner, and cannot provide reliable guarantee for the safe operation of the power grid.

Contents of the invention

The purpose of the present invention is to provide a method for automatic identification of defects in UAV inspection data, by establishing a rich and representative image data set of fault samples to train a fault identification model for automatic fault detection and annotation, and sampling equipment information. , the introduction of contextual information annotation method that combines equipment historical operation data and image information to annotate fault targets, solves the application defects of high cost and low accuracy in defect identification of existing distribution line drone inspection data, and can effectively reduce defects. Identifying costs and providing missing data identification accuracy facilitates the timely discovery of hidden dangers and problems in power grid equipment. It can also improve the comprehensiveness, accuracy and efficiency of fault annotation by comprehensively utilizing equipment information, historical operating data and image information. It provides a solid guarantee for the safe and stable operation of the substation.

In order to achieve the above purpose, it is necessary to provide an automatic identification method, system, computer equipment and storage medium for drone inspection data defects in view of the above technical problems.

In a first aspect, embodiments of the present invention provide a method for automatically identifying defects in drone inspection data. The method includes the following steps:

Obtain inspection image data of various types of power equipment in the target inspection area, and construct a UAV inspection image data set based on the inspection image data; the inspection image data includes various fault type image data and normal image data;

According to the UAV inspection image data set, the preset target detection model is trained to obtain a fault identification model; the preset target detection model includes a fault detection module and a fault annotation module in sequence;

Obtain the UAV inspection image to be identified, input the UAV inspection image to be identified into the fault identification model for fault identification and annotation, and obtain the corresponding fault identification result; the fault identification result includes the fault type and Fault area labeling.

Further, the step of constructing a drone inspection image data set based on the inspection image data includes:

According to the preset fault detection requirements, filter the inspection image data to obtain inspection image sample data;

According to the preset image annotation tool and the preset annotation content, each inspection image sample data is information annotated to obtain the drone inspection image data set; the preset annotation content includes equipment information, shooting information and fault information. ; The equipment information includes equipment model and equipment manufacturer; the fault information includes fault type, fault area location and fault area shape.

Further, the step of training a preset target detection model based on the drone inspection image data set to obtain a fault identification model includes:

Input the UAV inspection image data set into the fault detection module in the preset target detection model to detect fault targets, and obtain fault target detection results; the fault target detection results include fault targets and corresponding fault types;

Input the fault target detection results into the fault annotation module in the preset target detection model to annotate fault information to obtain fault prediction results;

According to the comparison and analysis of the fault prediction results of each inspection image sample data in the drone inspection image data set and the corresponding preset annotation content, the parameters of the preset target detection model are iteratively updated to obtain the Describe the fault identification model.

Further, the step of inputting the fault target detection result into the fault annotation module in the preset target detection model to annotate fault information and obtain the fault prediction result includes:

According to the fault target, obtain the corresponding image fault area location;

Obtain the corresponding fault annotation information according to the fault target and the pre-built equipment information database;

The fault annotation information is marked at the fault area position of the image to obtain the fault prediction result.

Further, the step of obtaining the corresponding image fault area location according to the fault target includes:

According to the fault target, obtain the corresponding fault pixel area;

Create a blank label mask on the faulty pixel area; the shape of the blank label mask is the same as the shape of the faulty pixel area;

According to the shape of the blank annotation mask, a corresponding pixel area filling method is selected to fill and draw the blank annotation mask to obtain the position of the image fault area.

Further, the fault labeling information includes equipment information, equipment location information, fault description information and equipment historical operation data;

The step of obtaining corresponding fault annotation information based on the fault target and the pre-built equipment information database includes:

According to the fault target, obtain the corresponding fault equipment information;

According to the faulty equipment information, the pre-built equipment information database is searched to obtain corresponding equipment location information, fault description information and equipment historical operation data.

Further, the steps of constructing the device information database include:

Obtain the operation and maintenance documentation of various power equipment, and use a preset natural language processing model to perform named entity recognition on the operation and maintenance documentation to obtain key equipment operation and maintenance information; the key equipment operation and maintenance information includes equipment model, Equipment manufacturer, equipment location, and fault descriptions corresponding to various fault types;

Obtain the continuous operation records of various power equipment, analyze the continuous operation history records, and obtain the corresponding historical operation data of the equipment; the continuous operation records include the equipment operation status and equipment operation status at each historical moment within the preset time range. Temperature and equipment current; the equipment historical operation data includes equipment historical status and corresponding equipment operation characteristics;

According to the equipment model, data correlation is performed between the equipment operation and maintenance key information and the equipment historical operation data, and the equipment information database is established.

In a second aspect, embodiments of the present invention provide an automatic identification system for defects in UAV inspection data. The system includes:

The data acquisition module is used to obtain inspection image data of various types of power equipment in the target inspection area, and construct a UAV inspection image data set based on the inspection image data; the inspection image data includes various Fault type image data and normal image data;

A model building module, configured to train a preset target detection model based on the drone inspection image data set to obtain a fault identification model; the preset target detection model includes a fault detection module and a fault annotation module in sequence;

A fault identification module is used to obtain the drone inspection image to be identified, and input the drone inspection image to be identified into the fault identification model to perform fault identification and labeling, and obtain the corresponding fault identification result; the fault The identification results include fault type and fault area annotation.

In a third aspect, embodiments of the present invention also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above method is implemented. A step of.

In a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

The above-mentioned application provides a method, system, computer equipment and storage medium for automatic identification of defects in drone inspection data. Through the method, it is possible to obtain image data of various types of power equipment in the target inspection area, including various fault types. and normal image data, and after constructing the UAV inspection image data set based on the inspection image data, the preset targets including the fault detection module and the fault labeling module are sequentially based on the UAV inspection image data set. The detection model is trained to obtain a fault identification model, and the drone inspection image to be identified is obtained, and the drone inspection image to be identified is input into the fault identification model for fault identification and labeling, and the corresponding fault type and fault area are obtained. Technical solution for labeled fault identification results. Compared with the existing technology, this method of automatic identification of defects in drone inspection data can effectively reduce the cost of defect identification and provide the accuracy of missing data identification, which not only facilitates the timely discovery of hidden dangers and problems in power grid equipment, but also enables comprehensive utilization of Equipment information, historical operating data and image information improve the comprehensiveness, accuracy and efficiency of fault annotation, providing a solid guarantee for the safe and stable operation of the substation.

Description of drawings

Figure 1 is a schematic diagram of the application scenario of the automatic identification method of drone inspection data defects in the embodiment of the present invention;

Figure 2 is a schematic flow chart of a method for automatically identifying defects in drone inspection data in an embodiment of the present invention;

Figure 3 is a schematic diagram of a faulty pixel area obtained according to a faulty target in an embodiment of the present invention;

Figure 4 is a schematic diagram of a blank annotation mask created based on a faulty pixel area in an embodiment of the present invention;

Figure 5 is a schematic diagram of the location of the image fault area obtained by filling and drawing the blank label mask in the embodiment of the present invention;

Figure 6 is a schematic structural diagram of the automatic identification system for UAV inspection data defects in the embodiment of the present invention;

Figure 7 is an internal structural diagram of a computer device in an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and beneficial effects of the present application more clear, the present invention will be further described in detail below in conjunction with the accompanying drawings and examples. Obviously, the embodiments described below are part of the embodiments of the present invention and are only used for to illustrate the invention but not to limit the scope of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present invention.

The automatic identification method of UAV inspection data defects provided by the present invention can be understood as taking into account that the UAV inspection data of existing distribution network lines is intricate and dense, and the background environment is dynamically complex, which leads to the identification of UAV automatic inspection data defects. The accuracy rate is low, manual-assisted review is required, the efficiency is low, the cost is high, and the accuracy of the identification results cannot be guaranteed, making it difficult to timely discover hidden dangers and problems in power grid equipment through inspection results, and cannot provide reliable guarantee for the safe operation of the power grid. Application status, and proposed a fault recognition model for automated fault detection and annotation by establishing a rich and representative image data set of fault samples, and sampling the device information, device historical operation data and image information. An automatic identification method for distribution network line fault drone inspection by introducing contextual information annotation method to automatically annotate fault targets. This method can be applied to a terminal or server as shown in Figure 1, where the terminal can be but is not limited to various personal computers, laptops, smart phones, tablets and portable wearable devices. The server can be an independent server or a A server cluster composed of multiple servers is implemented. The server can use the automatic identification method of drone inspection data defects provided by the present invention to perform automated fault detection and comprehensive and accurate fault labeling according to actual application requirements, and use the corresponding fault identification results for subsequent research on the server, or transmit them to The terminal is provided for terminal users to view and analyze; the following embodiments will describe in detail the automatic identification method of drone inspection data defects of the present invention.

In one embodiment, as shown in Figure 2, a method for automatic identification of defects in drone inspection data is provided, including the following steps:

S11. Obtain inspection image data of various types of power equipment in the target inspection area, and construct a UAV inspection image data set based on the inspection image data; the inspection image data includes image data of various fault types and normal image data; among which, inspection image data can be understood as photos taken by power inspection personnel when they detect faults, as well as data such as corresponding equipment models and fault types, which can be derived from different power equipment and environmental conditions, including substations. , transmission lines, transmission towers, etc., and in order to ensure that the source of the sample is representative, it can cover various possible fault situations and obtain as many fault types as possible, such as equipment damage, looseness, foreign matter intrusion, etc.;

Considering that the original inspection image data collected may include duplication, missing data and other low-quality invalid data that does not utilize fault detection, in order to improve the reliability of the UAV inspection image data set used for automatic detection model training, this embodiment Preferably, each acquired inspection image data is screened and reasonably annotated; specifically, the step of constructing a UAV inspection image data set based on the inspection image data includes:

According to the preset fault detection requirements, the inspection image data is filtered to obtain the inspection image sample data; among them, the preset fault detection requirements can be formulated according to the actual application scenario, and there are no specific limitations here; in principle, it only needs to ensure The obtained inspection image sample data should include high-quality data on the normal status of different power equipment under different environmental conditions and various fault types as much as possible. The specific screening method is not limited here;

According to the preset image annotation tool and the preset annotation content, each inspection image sample data is information annotated to obtain the drone inspection image data set; the preset annotation content includes equipment information, shooting information and fault information. ; The equipment information includes equipment model and equipment manufacturer; the fault information includes fault type, fault area location and fault area shape; wherein, the preset image annotation tool can be understood as being used to classify faults in each fault image sample A tool for labeling the location and shape of an area. In practical applications, you can select according to the fault objects and fault types that need to be labeled. For example, target detection and rectangular bounding box labeling are required, and the target types are less diverse, so the target mainly needs to be labeled. The LabelImg image annotation tool can be used when the location and size of For point annotation, if you need custom attributes or metadata annotation, you can use the image annotation tool RectLabel; if you need target detection, rectangular bounding box and polygon annotation, it supports diverse target shapes and areas, and you can use polygon annotation or segmentation mask. When precise annotation of membranes is required and instance-level target segmentation is required, the image annotation tool VoTT can be used; image annotation tools considering drone inspection scenarios need to support multiple image annotation types, such as rectangular boxes, polygons, key Point annotation, custom attribute annotation, masking, etc., and has a convenient interactive interface and flexible scalability task requirements. In this embodiment, the RectLabel tool is preferably used to annotate inspection image sample data;

It should be noted that the above information annotation process can also record important information such as the equipment type, shooting environment, and shooting time involved in each image, providing reference for subsequent annotation work and facilitating the organization and archiving of the entire data set, so that in Obtain a rich and representative image data set, which not only provides strong data support for subsequent training of automated fault detection and annotation algorithms, but also makes the data set orderly and easy to maintain and manage, providing a basis for the entire drone patrol. Provide reliable support for inspection work.

S12. According to the UAV inspection image data set, train the preset target detection model to obtain a fault identification model; the preset target detection model includes a fault detection module and a fault labeling module in sequence; wherein, the preset target The fault detection module in the detection model can be understood as a module with image recognition function, which can be implemented using existing target detection, image segmentation or anomaly detection methods, such as Convolutional Neural Network (CNN), Single Shot MultiBox Detector (SSD), You Only Look Once (YOLO), U-Net, Mask R-CNN, Support Vector Machine (SVM), SIFT (Scale-InvariantFeature Transform), SURF (Speeded-Up Robust Features) Etc., there is no specific limit here;

The above fault annotation module can be understood as a key improvement of the fault identification model. It is mainly used to integrate equipment information, equipment historical operation data and image information by introducing contextual information to ensure efficient annotation while ensuring The comprehensiveness and accuracy of the annotated information provide strong support for fault location analysis, thereby providing a solid guarantee for the safe and stable operation of the substation; it should be noted that in this embodiment, the obtained UAV inspection image data set is divided into training There are two parts: set and test set. The training set is used to train a fault recognition model including automatic fault detection and automatic fault annotation. The test set is used to evaluate the accuracy of fault detection and annotation results;

Specifically, the step of training a preset target detection model based on the UAV inspection image data set to obtain a fault identification model includes:

Input the UAV inspection image data set into the fault detection module in the preset target detection model to detect fault targets, and obtain fault target detection results; the fault target detection results include fault targets and corresponding fault types;

The fault target detection results are input into the fault annotation module in the preset target detection model for fault information annotation to obtain fault prediction results; where the fault information annotation can be understood as based on the fault target detection results output by the fault detection module, The process of labeling the fault area and related information on the corresponding inspection image; specifically, inputting the fault target detection result into the fault labeling module in the preset target detection model to label the fault information to obtain a fault prediction The resulting steps include:

According to the fault target, the corresponding image fault area position is obtained; wherein the image fault area position can be understood as generating a fault area on the image with a shape corresponding to the fault type through an annotation algorithm; specifically, according to the fault target , the steps to obtain the corresponding image fault area location include:

According to the fault target, the corresponding fault pixel area is obtained; wherein the fault pixel area can be understood as the area obtained according to the position coordinates of the fault target (pixel coordinates of the center of the fault area) and the corresponding fault type obtained by the fault detection module Position information; for example, if there is a bounding box or polygon area to describe the target, it can be converted into the corresponding pixel area; if there is key point or contour information, it can also be converted into the corresponding pixel area according to relevant algorithms;

Create a blank label mask on the faulty pixel area; the shape of the blank label mask is the same as the shape of the faulty pixel area; wherein the blank label mask can be understood as a mask with all initial pixel values set to zero. Mask, that is, clear all pixel values in the faulty pixel area obtained in the above steps. The size of the blank labeling mask should be the same as the original image to ensure that each pixel position has corresponding labeling information. Specifically, relevant programming languages can be used to provide corresponding Data structure to represent the matrix. For example, if you use Python language algorithm, you can use the data structure in the NumPy library to represent it, which will not be described in detail here;

According to the shape of the blank annotation mask, the corresponding pixel area filling method is selected to fill and draw the blank annotation mask to obtain the position of the image fault area; wherein the pixel area filling method can be based on the shape of the blank annotation mask. and fault target location information to select, for example, if it is a rectangular bounding box shape, set the pixel area (width w, height h) around the pixel coordinates (x, y) located in the center of the fault area to non-zero values; if If it is a polygon shape, use the scan line filling algorithm to fill the corresponding pixel area according to the given polygon vertex coordinate array; if it is a key point or contour shape, you can consider using techniques such as linear interpolation and closed curve fitting to fill the key points. The point or contour information is converted into a continuous pixel area and filled on the blank label mask; the following is an example of generating a rectangular bounding box (Bounding Box) label:

Assume that the target image I is input (size is W*H=3*3), and the labeling result needs to be output: M (a binary mask representing the fault, the size is the same as the input image), the pixel coordinate of the center of the fault area (x=1, y=2) and fault type fault_type; then the corresponding process of obtaining the position of the image fault area can be understood as: first, according to the fault target, obtain the corresponding fault pixel area, that is, the binary mask area shown in Figure 3; and then obtain the The pixel value of the faulty pixel area is cleared to obtain the blank annotation mask shown in Figure 4; then, the blank annotation mask is filled and drawn to obtain the annotation mask shown in Figure 5, which is the location of the image fault area;

According to the fault target and the pre-built equipment information database, the corresponding fault labeling information is obtained; wherein the fault labeling information includes equipment information, equipment location information, fault description information and equipment historical operation data; specifically, according to The steps for obtaining the corresponding fault annotation information based on the fault target and the pre-built equipment information database include:

According to the fault target, obtain the corresponding faulty equipment information; wherein the faulty equipment information includes equipment model and equipment manufacturer, etc.;

According to the faulty equipment information, search the pre-built equipment information database to obtain the corresponding equipment location information, fault description information and equipment historical operation data; wherein, the equipment information database can be understood as indexing to the corresponding equipment according to the equipment signal A database of manufacturer, equipment location, fault descriptions corresponding to various fault types, and equipment continuous operation records; specifically, the construction steps of the equipment information database include:

Obtain the operation and maintenance documentation of various power equipment, and use a preset natural language processing model to perform named entity recognition on the operation and maintenance documentation to obtain key equipment operation and maintenance information; the key equipment operation and maintenance information includes equipment model, Equipment manufacturer, equipment location, and fault descriptions corresponding to various fault types; among them, the operation and maintenance documentation of various power equipment can be understood as maintenance reports, fault records, work logs, technical documents, etc. related to power equipment. Documents usually include detailed descriptions of equipment status, fault conditions, and maintenance records; in addition, they can also include retrieval through the network, such as using web crawlers or data collection tools (such as Beautiful Soup, Scrapy in Python, etc.) to automatically obtain the power system Internal documentation, etc.;

The process of obtaining the above key equipment operation and maintenance information can be understood as follows: using natural language processing technology, the pre-trained named entity recognition (NER) model can be used to scan the obtained operation and maintenance documents, and accurately identify and extract them. Important information directly related to the equipment, including named entities such as equipment model and each component name. For example, for a given maintenance report or fault record, the equipment involved can be quickly and accurately identified through a preset natural language processing model. type, as well as the specific parts that may have problems; at the same time, by calling the names of all spare parts in the existing warehouse system, a professional vocabulary of equipment and parts can be quickly searched and identified in the text. Entities related to power equipment to help lock in key information; specific keywords or phrases that need to be extracted through natural language processing technology can be determined based on actual task requirements. For example, in fault detection, keywords can include "fault type", "Device name", etc., use regular expressions (Regex or RegExp) to match formatted information such as device numbers, fault codes, etc.; then, use dependency syntax analysis tools to identify the dependencies between each vocabulary in the sentence , thus helping to extract key information. For example, a maintenance report may contain the following description: "Transformer A12 has a short-circuit fault." Through entity recognition, the entities in it can be marked: Equipment name entity: "Transformer A12" , Fault type entity: "Short circuit", these two key information are very important for fault detection. They provide a specific description and location of the fault, which provide a basis for subsequent processing and decision-making; then an extraction model can be used To extract key information from the above equipment fault description; it should be noted that Support Vector Machine (SVM), Recurrent Neural Network (RNN), Long Short-term Memory Network (Long Short-term Memory Network) can be used here. Term Memory, LSTM) or Convolutional Neural Network (CNN), etc. perform sequence modeling on text to extract key information, which can efficiently extract entities related to power equipment from a large amount of text information, providing the basis for subsequent Lay a solid foundation for fault identification and labeling work;

Obtain the continuous operation records of various power equipment, analyze the continuous operation history records, and obtain the corresponding historical operation data of the equipment; the continuous operation records include the equipment operation status and equipment operation status at each historical moment within the preset time range. Temperature and equipment current; the equipment historical operation data includes equipment historical status and corresponding equipment operation characteristics;

According to the equipment model, perform data correlation on the equipment operation and maintenance key information and the equipment historical operation data, and establish the equipment information database;

The equipment information database constructed through the above method steps can include detailed information of each power equipment, such as equipment model, manufacturer, operation time, geographical location information of the equipment, including important data such as longitude and latitude and the substation to which it belongs. It can be used in the annotation process. This information can be retrieved at any time, thus providing strong support for accurate labeling; at the same time, it also stores the equipment operating characteristics and equipment historical status obtained through continuous recording and analysis of the operating status, temperature, current and other parameters of each power equipment, not only It helps to understand the long-term operating status of the equipment and can also provide clues for labeling faults; for example, if a piece of equipment shows abnormal current fluctuations in historical data, this may mean that there is a potential problem with the equipment and requires special attention;

The fault annotation information is marked on the fault area of the image to obtain the fault prediction result; where the fault annotation information in the fault prediction result can be understood as combining equipment information, equipment historical operation information and image fault identification information. Fusion can present all the information on the annotation interface, which directly displays the device information including the device model and device manufacturer. By clicking on the corresponding device information, you can view the corresponding historical operation information of the device, making it easier to understand the operation of the device. situation;

According to the comparison and analysis of the fault prediction results of each inspection image sample data in the drone inspection image data set and the corresponding preset annotation content, the parameters of the preset target detection model are iteratively updated to obtain the The fault identification model is described above; among them, after the fault prediction result is obtained through the above method, it can be compared with the real annotation information (real label) of the corresponding inspection image sample data, and by considering the characteristics of the target and the surrounding background information , optimize the accuracy and understandability of fault annotation, use annotated image data sets for verification and evaluation, compare the consistency of automated fault annotation results with manual annotation, and iteratively update model parameters based on the evaluation results. The specific process is as follows :

Randomly select a part of the inspection image samples to obtain the annotation results based on the fault recognition model, and invite professionals or domain experts to manually evaluate these samples, use their evaluation results as the standard, compare the automatic annotation results with the manual evaluation results, and calculate Evaluation indicators such as accuracy and recall, use these indicators to determine the performance of the fault identification model, conduct detailed error analysis for samples that are inconsistent with the manual evaluation results, and determine the common error types and causes of automatic labeling, and based on the error analysis results , adjust and optimize the automatic labeling algorithm, for example, adjust model parameters, update feature engineering, or use a more advanced model, and use the optimized and updated model to re-label the data set to ensure the accuracy of the labeling results, repeat the above steps , until the performance of the fault identification model reaches a satisfactory level;

It should be noted that the fault annotation in this embodiment can also be functionally expanded by adding an interactive interface. The original image and annotation results are displayed side by side on the fault annotation interface. The specific interactive interface is designed as follows: 1. It can be freely provided Adjust the size and position of the split view, allowing the operator to view the original image, automatic annotation results and manually corrected annotations at the same time; 2. Provide intelligent prompts and search; 3. Provide intelligent prompt functions, based on existing fault tags and keywords Quickly complete annotations; 4. Support keyword-based search function so that operators can quickly find and edit specific annotation items; 5. Interactive tools and mode selection: Provide a variety of interactive tools, including resizable rectangular boxes, polygons Or mask, etc., to adapt to the labeling needs of different fault types; 6. The labeling mode can be switched, such as drawing, modifying or deleting mode, to flexibly adapt to different operating requirements in the labeling process; 7. Gesture and shortcut key support: support commonly used Gesture operations, such as zooming, panning and rotating, for more precise operations when annotating large-size images; 8. Provide configurable shortcut keys to speed up the annotation process and improve the efficiency of the operation; 9. Annotation quality assessment and correction: Provide real-time annotation quality evaluation indicators, such as IoU (Intersection over Union) or Dice coefficient, allowing operators to understand the accuracy of annotation results, and support manual adjustment of annotation boundaries or shapes to make subtle corrections and corrections when needed. Improvement; 10. Annotation version control and collaboration support: allows to save different versions of annotation results, and provides version comparison and rollback functions to track changes in the annotation process; 11. Supports multi-user collaboration, allowing multiple operators to annotate at the same time The same image, and provides conflict resolution mechanism and annotation annotation functions; 12. Export and save: Provides multiple export format options, such as common annotation formats (such as PascalVOC, YOLO, etc.) or custom formats to facilitate subsequent model training and data Analysis; 13. Support automatic save and restore functions to prevent the loss of annotation data caused by accidental closing of the interface or temporary power outage; that is, through the fault annotation interactive interface combined with human-computer interaction and algorithm assistance, efficient and accurate fault annotation can be achieved. The interface should be able to display original images and automated fault detection results, and provide user interaction tools (such as drawing boxes, polygons and masks, etc.) to manually correct or add annotations. It can also be used for manual review or manual intervention to collect user data in a timely manner. feedback, and optimize the interface based on the feedback to improve user experience.

S13. Obtain the inspection image of the drone to be identified, and input the inspection image of the drone to be identified into the fault identification model for fault identification and annotation, and obtain the corresponding fault identification result; wherein, the drone to be identified is Inspection images can be understood as image data collected in real time by inspection drones in practical applications and required for defect data identification. The corresponding fault identification results include fault type and fault area annotation. The corresponding acquisition process can refer to the above fault identification. The relevant description of the model training process will not be repeated here;

The embodiment of this application obtains inspection image data including various fault type image data and normal image data of various types of power equipment in the target inspection area, and constructs a drone inspection image data set based on the inspection image data. Based on the UAV inspection image data set, a preset target detection model including a fault detection module and a fault annotation module is trained to obtain a fault recognition model, and the UAV inspection image to be identified is obtained, and the UAV inspection image to be identified is The machine inspection image is input into the fault recognition model for fault identification and annotation, and the corresponding fault identification result including fault type and fault area annotation is obtained. This solution effectively solves the problem of drone inspection of existing distribution network lines due to intricate and dense lines and background. Factors such as complex environmental dynamics lead to low accuracy in identifying defects in UAV automatic inspection data, which requires manual review. The efficiency is low, the cost is high, and the accuracy of the identification results cannot be guaranteed, making it difficult to detect the power grid in a timely manner through the inspection results. Equipment hidden dangers and problems, application defects that cannot provide reliable guarantee for the safe operation of the power grid, through the integration of front-end identification and edge computing, distribution network equipment body identification and positioning photography technology, 5G network connection technology, front-end identification and calculation analysis, image recognition Technology will significantly improve the efficiency of inspections. Equipment with high safety, low intensity, reliable quality, low cost and high automation level can complete power inspection tasks, effectively reduce the cost of defect identification and provide missing data identification accuracy to facilitate timely discovery. In addition to identifying hidden dangers and problems in power grid equipment, it can also improve the comprehensiveness, accuracy and efficiency of fault annotation by comprehensively utilizing equipment information, historical operating data and image information, providing a solid guarantee for the safe and stable operation of substations.

It should be noted that although each step in the above flowchart is shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders.

In one embodiment, as shown in Figure 6, an automatic identification system for drone inspection data defects is provided. The system includes:

Data acquisition module 1 is used to obtain inspection image data of various types of power equipment in the target inspection area, and construct a UAV inspection image data set based on the inspection image data; the inspection image data includes various fault type image data and normal image data;

Model building module 2 is used to train a preset target detection model based on the drone inspection image data set to obtain a fault identification model; the preset target detection model includes a fault detection module and a fault annotation module in sequence;

The fault identification module 3 is used to obtain the drone inspection image to be identified, and input the drone inspection image to be identified into the fault identification model for fault identification and labeling, and obtain the corresponding fault identification result; Fault identification results include fault type and fault area labeling.

Regarding the specific limitations of the automatic identification system of UAV inspection data defects, please refer to the limitations on the automatic identification method of UAV inspection data defects mentioned above. The corresponding technical effects can also be obtained equally, and will not be repeated here. Each module in the above-mentioned automatic identification system for drone inspection data defects can be realized in whole or in part through software, hardware and their combinations. Each of the above modules may be embedded in or independent of the processor of the computer device in the form of hardware, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

Figure 7 shows an internal structure diagram of a computer device in one embodiment. The computer device may specifically be a terminal or a server. As shown in Figure 7, the computer device includes a processor, memory, network interface, display, camera and input device connected through a system bus. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media and internal memory. The non-volatile storage medium stores operating systems and computer programs. This internal memory provides an environment for the execution of operating systems and computer programs in non-volatile storage media. The network interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements an automatic identification method for defects in drone inspection data. The display screen of the computer device may be a liquid crystal display or an electronic ink display. The input device of the computer device may be a touch layer covered on the display screen, or may be a button, trackball or touch pad provided on the computer device shell. , it can also be an external keyboard, trackpad or mouse, etc.

Those of ordinary skill in the art can understand that the structure shown in Figure 7 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Specific computing equipment It is possible to include more or fewer components than shown in the figures, or to combine certain components, or to have the same arrangement of components.

In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

In one embodiment, a computer-readable storage medium is provided, a computer program is stored thereon, and when the computer program is executed by a processor, the steps of the above method are implemented.

In summary, embodiments of the present invention provide a method and system for automatic identification of defects in UAV inspection data. The method and system for automatic identification of defects in UAV inspection data realize the acquisition of various types of power equipment in the target inspection area, including Inspection image data of fault type image data and normal image data, and after constructing a UAV inspection image data set based on the inspection image data, the UAV inspection image data set includes a fault detection module and fault annotation in sequence. The module's preset target detection model is trained to obtain a fault identification model, and the drone inspection image to be identified is obtained, and the drone inspection image to be identified is input into the fault identification model for fault identification and labeling, and the corresponding A technical solution for fault identification results marked by fault types and fault areas. This method combines front-end identification and edge computing to integrate distribution network equipment ontology identification and positioning photography technology, 5G network connection technology, front-end identification and calculation analysis, and image recognition technology. Significantly improve the efficiency of inspections, use equipment with high safety, low intensity, reliable quality, low cost and high automation level to complete power inspection tasks, effectively reduce the cost of defect identification and provide missing data identification accuracy to facilitate timely discovery of power grid equipment While identifying hidden dangers and problems, it can also improve the comprehensiveness, accuracy and efficiency of fault annotation by comprehensively utilizing equipment information, historical operating data and image information, thus providing a solid guarantee for the safe and stable operation of the substation.

Each embodiment in this specification is described in a progressive manner. The same or similar parts of each embodiment can be directly referred to each other. Each embodiment focuses on its differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment. It should be noted that the technical features of the above embodiments can be combined in any way. To simplify the description, all possible combinations of the technical features in the above embodiments are not described. However, as long as the combination of these technical features does not If there is any contradiction, it should be considered to be within the scope of this manual.

The above-described embodiments only express several preferred embodiments of the present application. The descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that, for those of ordinary skill in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of this application shall be subject to the protection scope of the claims.

Claims (10)

1. An automatic identification method for unmanned aerial vehicle inspection data defects is characterized by comprising the following steps:

acquiring inspection image data of various power equipment in a target inspection area, and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; the inspection image data comprises various fault type image data and normal image data;

training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module;

acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label.

2. The method for automatically identifying defects of inspection data of an unmanned aerial vehicle according to claim 1, wherein the step of constructing an unmanned aerial vehicle inspection image dataset from the inspection image data comprises:

screening the inspection image data according to a preset fault detection requirement to obtain inspection image sample data;

Information labeling is carried out on each inspection image sample data according to a preset image labeling tool and preset labeling content, and an unmanned aerial vehicle inspection image data set is obtained; the preset labeling content comprises equipment information, shooting information and fault information; the device information includes a device model number and a device manufacturer; the fault information includes a fault type, a fault region location, and a fault region shape.

3. The method for automatically identifying defects of unmanned aerial vehicle inspection data according to claim 2, wherein the step of training a preset target detection model according to the unmanned aerial vehicle inspection image dataset to obtain a fault identification model comprises the steps of:

inputting the unmanned aerial vehicle inspection image data set into a fault detection module in the preset target detection model to detect a fault target, and obtaining a fault target detection result; the fault target detection result comprises a fault target and a corresponding fault type;

inputting the fault target detection result into a fault labeling module in the preset target detection model to label fault information, so as to obtain a fault prediction result;

and carrying out iterative updating on parameters of the preset target detection model according to comparison analysis of the fault prediction result of each inspection image sample data in the unmanned aerial vehicle inspection image data set and corresponding preset labeling content to obtain the fault identification model.

4. The method for automatically identifying defects of unmanned aerial vehicle inspection data according to claim 3, wherein the step of inputting the fault target detection result into a fault labeling module in the preset target detection model to label fault information and obtain a fault prediction result comprises the steps of:

acquiring the position of a corresponding image fault area according to the fault target;

acquiring corresponding fault marking information according to the fault target and a pre-constructed equipment information base;

and marking the fault marking information at the position of the fault area of the image to obtain the fault prediction result.

5. The unmanned aerial vehicle inspection data defect automatic identification method of claim 4, wherein the step of acquiring the corresponding image fault region location according to the fault target comprises:

acquiring a corresponding fault pixel region according to the fault target;

creating a blank marking mask on the fault pixel region; the shape of the blank mark mask is the same as that of the fault pixel region;

and selecting a corresponding pixel region filling mode according to the shape of the blank marking mask to fill and draw the blank marking mask, so as to obtain the position of the image fault region.

6. The unmanned aerial vehicle inspection data defect automatic identification method of claim 5, wherein the fault annotation information comprises device information, device location information, fault description information and device history operation data;

the step of obtaining corresponding fault labeling information according to the fault target and a pre-constructed equipment information base comprises the following steps:

acquiring corresponding fault equipment information according to the fault target;

and searching the pre-constructed equipment information base according to the fault equipment information to acquire corresponding equipment position information, fault description information and equipment history operation data.

7. The unmanned aerial vehicle inspection data defect automatic identification method according to claim 6, wherein the step of constructing the equipment information base comprises:

acquiring operation and maintenance document data of various electric equipment, and adopting a preset natural language processing model to perform named entity identification on the operation and maintenance document data to obtain equipment operation and maintenance key information; the equipment operation and maintenance key information comprises equipment models, equipment manufacturers, equipment positions and fault descriptions corresponding to various fault types;

obtaining continuous operation records of various power equipment, and analyzing the continuous operation history records to obtain corresponding equipment history operation data; the continuous operation record comprises equipment operation states, equipment temperatures and equipment currents at all historical moments within a preset duration range; the equipment history operation data comprises equipment history states and corresponding equipment operation characteristics;

And according to the equipment model, carrying out data association on the equipment operation and maintenance key information and the equipment historical operation data, and establishing the equipment information base.

8. An unmanned aerial vehicle inspection data defect automatic identification system, the system comprising:

the data acquisition module is used for acquiring inspection image data of various power equipment in the target inspection area and constructing an unmanned aerial vehicle inspection image data set according to the inspection image data; the inspection image data comprises various fault type image data and normal image data;

the model construction module is used for training a preset target detection model according to the unmanned aerial vehicle inspection image data set to obtain a fault identification model; the preset target detection model sequentially comprises a fault detection module and a fault labeling module;

the fault identification module is used for acquiring an unmanned aerial vehicle inspection image to be identified, inputting the unmanned aerial vehicle inspection image to be identified into the fault identification model for fault identification and marking, and obtaining a corresponding fault identification result; the fault identification result comprises a fault type and a fault region label.

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.