CN115082813A - Detection method, unmanned aerial vehicle, detection system and medium - Google Patents

Detection method, unmanned aerial vehicle, detection system and medium Download PDF

Info

Publication number
CN115082813A
CN115082813A CN202210735434.0A CN202210735434A CN115082813A CN 115082813 A CN115082813 A CN 115082813A CN 202210735434 A CN202210735434 A CN 202210735434A CN 115082813 A CN115082813 A CN 115082813A
Authority
CN
China
Prior art keywords
image
transmission line
power transmission
training
defect detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210735434.0A
Other languages
Chinese (zh)
Inventor
孙鹏飞
杜君
姜帆
郭飞
孟伟
李慧
刘立宗
刘洋
高源�
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Smartchip Microelectronics Technology Co Ltd
Beijing Smartchip Semiconductor Technology Co Ltd
Original Assignee
Beijing Smartchip Microelectronics Technology Co Ltd
Beijing Smartchip Semiconductor Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Smartchip Microelectronics Technology Co Ltd, Beijing Smartchip Semiconductor Technology Co Ltd filed Critical Beijing Smartchip Microelectronics Technology Co Ltd
Priority to CN202210735434.0A priority Critical patent/CN115082813A/en
Publication of CN115082813A publication Critical patent/CN115082813A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation

Abstract

The application discloses a detection method of a power transmission line, an unmanned aerial vehicle, a detection system and a storage medium. The detection method of the power transmission line comprises the following steps: patrolling along a preset path to obtain a patrolling image of the power transmission line; and detecting the inspection image through a defect detection model to determine an abnormal target image of the power transmission line, wherein the defect detection model is obtained by Yolov5 network training. According to the detection method of the power transmission line, the inspection image of the power transmission line is detected in real time through the defect detection model, so that foreign matters and defects on the power transmission line are identified, the detection precision and the detection speed of the foreign matters and the defects are high, and technical reference can be provided for power grid inspection personnel to carry out obstacle removal work of the power transmission line.

Description

Detection method, unmanned aerial vehicle, detection system and medium
Technical Field
The present application relates to the field of image processing, and in particular, to a method and a system for detecting a power transmission line, an unmanned aerial vehicle, and a non-volatile computer-readable storage medium.
Background
The detection methods for the abnormal target of the power transmission line are roughly divided into two types: manual inspection and target detection based on deep learning. Due to the reasons of geographical environment, weather, high maintenance difficulty and the like, the requirement of safety monitoring of the running state of the power transmission line is difficult to meet through manual inspection.
At present, the common power transmission line abnormal target detection based on deep learning is to arrange a plurality of cameras around a power transmission line for video shooting, then upload videos to a cloud end, identify the videos through a deep learning detection network established by the cloud end, send alarm information and transmit the alarm information back. However, the data volume uploaded to the cloud is too large and contains a large amount of invalid data information, so that a large amount of resources are occupied, and the identified foreign matters and defect targets cause detection delay due to insufficient bandwidth or poor signals, so that abnormity cannot be timely handled, packet loss risks exist, and unknown potential safety hazards are easily reserved.
Disclosure of Invention
In view of this, the present application provides a method for detecting a power transmission line, an unmanned aerial vehicle, a system for detecting a power transmission line, and a non-volatile computer-readable storage medium.
The detection method of the power transmission line provided by the embodiment of the application is used for the unmanned aerial vehicle, and the detection method comprises the following steps:
patrolling along a preset path to obtain a patrolling image of the power transmission line;
and detecting the inspection image through a defect detection model to determine an abnormal target image of the power transmission line, wherein the defect detection model is obtained by Yolov5 network training.
In some embodiments, the detecting the inspection image through the defect detection model to determine the abnormal target image of the power transmission line includes:
preprocessing the inspection image to obtain a preprocessed image;
slicing the preprocessed image and extracting features to obtain a feature image;
performing branch fusion processing on the characteristic image to obtain a characteristic enhanced image;
and according to the characteristic reinforced image modeling, detecting an abnormal target image of the power transmission line.
In some embodiments, the slicing the preprocessed image and the feature extraction to obtain the feature image comprise:
processing each preprocessed image slice through a Focus module to obtain a characteristic image;
and performing convolution, residual error and pooling on the characteristic image through a CSP module to extract the characteristic image.
In certain embodiments, the detection method further comprises:
and sending the abnormal target image to a server.
In certain embodiments, the detection method further comprises:
generating an alarm signal according to the abnormal target image;
and sending the alarm signal and/or the abnormal target image to an application terminal.
In some embodiments, the training step of the defect detection model trained by the Yolov5 network includes:
acquiring a historical image about the power transmission line;
marking the position information and the category information of the foreign matters and the defects in the historical image to construct a data set;
training a Yolov5 network according to the data set to modify weight parameters of the Yolov5 network;
and under the condition that the test result of the Yolov5 network meets the requirement of accuracy, taking the modified Yolov5 network as the defect detection model.
In certain embodiments, the training of the Yolov5 network to modify the weight parameters of the Yolov5 network according to the dataset comprises:
dividing the data set into a training set and a test set;
training the Yolov5 network according to the training set to obtain a training result;
calculating a loss value of the Yolov5 network through an objective loss function based on the training results and the test set;
and correcting the weight parameters of the Yolov5 network according to the loss value of the Yolov5 network.
In some embodiments, the training the Yolov5 network according to the training set to obtain a training result includes:
performing data enhancement processing on the training set to obtain an enhanced training set;
training the Yolov5 network according to the enhanced training set to obtain the training result.
In some embodiments, the target loss function comprises a localization loss function, a classification loss function, and a confidence loss function;
the positioning loss function is:
Figure BDA0003715156150000021
wherein B represents a prediction box, B gt Representing a real box, C representing containing B gt Intersection ratio with the minimum convex closed box of B, IoU;
the classification loss function is:
Figure BDA0003715156150000031
c denotes the class, p denotes the probability of predicting the class, S denotes the mesh size,
Figure BDA0003715156150000032
indicating that if there is a target at the jth box of the ith grid, its value is 1, otherwise it is 0;
the confidence loss function is:
Figure BDA0003715156150000033
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003715156150000034
indicating that if there is no target at the jth box of the ith grid, its value is 1, otherwise it is 0, C i And representing the confidence, if the bounding box in the grid is responsible for predicting a certain object, the value is 0, otherwise.
In certain embodiments, the detection method further comprises:
and performing incremental learning on the defect detection model according to the target abnormal image to optimize the defect detection model.
In certain embodiments, the detection method further comprises:
quantizing the defect detection model through a preset quantization algorithm;
and sending the defect detection model subjected to quantization processing to the unmanned aerial vehicle.
In certain embodiments, the preset quantization algorithm comprises INT8 quantization.
The unmanned aerial vehicle that this application embodiment provided includes:
the image acquisition module is used for acquiring an inspection image of the power transmission line along a predetermined path in an inspection way;
and the defect detection module is used for detecting the inspection image through a defect detection model to determine an abnormal target image of the power transmission line, and the defect detection model is obtained by Yolov5 network training.
The detection system for the power transmission line comprises an unmanned aerial vehicle and a server, wherein the unmanned aerial vehicle is communicated with the server, the unmanned aerial vehicle is wirelessly communicated with the server, the server is used for acquiring a historical image about the power transmission line, marking position information and category information of foreign matters and defects in the historical image to construct a data set, training a Yolov5 network according to the data set to correct weight parameters of the Yolov5 network, and taking the corrected Yolov5 network as a defect detection model and sending the defect detection model to the unmanned aerial vehicle under the condition that a test result of the Yolov5 network meets an accuracy requirement;
the unmanned aerial vehicle is used for receiving the defect detection model, patrolling along a preset path to obtain a patrolling image of the power transmission line, and detecting the patrolling image through the defect detection model to determine an abnormal target image of the power transmission line.
In some embodiments, the detection system includes an application terminal, the application terminal is in communication with the server, the drone is further configured to send the abnormal target image to the server, the server is further configured to generate an alarm signal according to the abnormal target image and send the alarm signal and the abnormal target image to the application terminal, and the application terminal is configured to display the alarm signal and/or the abnormal target image.
The non-transitory computer-readable storage medium containing a computer program according to an embodiment of the present application, when the computer program is executed by a processor, causes the processor to execute the method for detecting a power transmission line or execute the method.
According to the detection method, the unmanned aerial vehicle, the detection system and the readable storage medium of the power transmission line, the inspection image of the power transmission line is detected in real time through the defect detection model, so that foreign matters and defects on the power transmission line are identified, the foreign matter detection precision and the detection speed are high, and technical reference can be provided for power grid inspection personnel to carry out obstacle removal work of the power transmission line.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flow chart of a method for detecting a power transmission line according to some embodiments of the present disclosure;
fig. 2 is a block schematic diagram of a drone according to certain embodiments of the present application;
fig. 3 is a schematic flow chart of a method for detecting a power transmission line according to some embodiments of the present disclosure;
FIGS. 4-5 are block schematic diagrams of a defect detection model according to certain embodiments of the present application;
fig. 6-8 are schematic flow charts of methods for detecting a power transmission line according to certain embodiments of the present disclosure;
fig. 9 is a schematic flow chart of a method for detecting a power transmission line according to some embodiments of the present disclosure;
FIG. 10 is a block diagram of a server in accordance with certain embodiments of the present application;
FIGS. 11-15 are schematic flow charts of detection methods according to certain embodiments of the present application;
fig. 16 is a schematic view of a scene of a detection system of a power transmission line according to some embodiments of the present disclosure.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
The transmission circuit is one of the most important energy Internet in China, and the safe and stable operation of the transmission circuit is the foundation for guaranteeing the smoothness of national economy. At present, the abnormal target detection of the power transmission line is roughly divided into manual inspection and target detection based on deep learning.
For the manual inspection mode, when inspection is performed, external equipment is needed for detection, such as a telescope, sometimes even tower climbing detection is needed, so that potential safety hazards existing in the power transmission line are inspected, any factors which can interfere the normal operation of the power transmission line are eliminated, and the stability of the power transmission line is guaranteed. Due to the fact that the landforms are various, the environment of the power transmission line is very complex, the efficiency is low only through manual inspection, a large amount of labor force needs to be consumed, and the manual inspection speed is low. If hidden dangers which have great influences on the safety of the power transmission line are not detected in time, great influences on the life of residents, the operation of factories and the social stability can be caused.
The common power transmission line abnormal target detection based on deep learning is that video shooting is carried out by deploying a plurality of cameras around a power transmission line, then videos are uploaded to a cloud end, a deep learning detection network established by the cloud end is used for identification, alarm information is sent out, and then the videos are transmitted back. However, the data volume uploaded to the cloud end is too large, a large amount of invalid data information is contained, a large amount of resources are occupied, and the detection delay is caused by the fact that the identified foreign matters and defect targets are insufficient in bandwidth or poor in signal, so that abnormality cannot be timely processed, and if the data is transmitted to the cloud end and lost, unknown potential safety hazards may be reserved.
Although unmanned aerial vehicles are used for inspecting power transmission lines at present, most unmanned aerial vehicles also serve as cameras, data are stored in the SD card, and then identification and judgment are carried out after inspection is finished. Detection models are deployed on a small number of unmanned aerial vehicles, but the models need to be optimized continuously along with the continuous increase of the types of abnormal targets, and the models are iterated continuously, so that the existing unmanned aerial vehicle identification modules cannot be realized. Meanwhile, the existing detection model deployed by the unmanned aerial vehicle generally directly uses a large model trained by a cloud server, and the normal operation of foreign matters and a defect detection model is difficult to support on the embedded edge device due to the fact that the performance of the device is not matched with the model, such as low computing power and low power consumption.
In view of this, referring to fig. 1, an application embodiment provides a method for detecting a power transmission line, where the method includes:
01, polling along a preset path to obtain a polling image of the power transmission line;
02, detecting the inspection image through a defect detection model to determine an abnormal target image of the power transmission line, wherein the defect detection model is obtained by Yolov5 network training.
Referring to fig. 2, the present embodiment provides an unmanned aerial vehicle 10. The drone 10 includes an image acquisition module 110 and a defect detection module 120. Wherein 01 may be implemented by the obtaining module 110, and 02 may be implemented by the defect detecting module 120. Alternatively, the obtaining module 110 may be configured to obtain the inspection image of the power transmission line along the predetermined path. The defect detection module 120 may be configured to detect the inspection image through a defect detection model to determine an abnormal target image of the power transmission line, where the defect detection model is obtained by Yolov5 network training.
According to the detection method and the unmanned aerial vehicle 10, the inspection image of the power transmission line is detected in real time through the defect detection model, so that foreign matters and defects on the power transmission line are identified, the foreign matter detection precision and the detection speed are high, and technical reference can be provided for power grid inspection personnel to carry out obstacle removal work of the power transmission line.
Specifically, the drone 10 also includes a controller 130. The controller 130 may include a development kit, and the developer obtains various data on the unmanned aerial vehicle 10 by calling an interface specified in the development kit, and executes corresponding calculation and processing through a software logic and algorithm framework designed by the developer, and generates a corresponding control instruction to control the unmanned aerial vehicle 10 to execute a corresponding action, so as to implement automatic flight of the unmanned aerial vehicle 10 along a predetermined path and fixed-point shooting of the unmanned aerial vehicle 10. Therefore, the unmanned aerial vehicle 10 can realize the routing inspection of the power transmission line along the preset path, shoot at the path designated position through the image acquisition module, obtain a plurality of routing inspection images, shoot a plurality of routing inspection images at the image acquisition module, and send the routing inspection images to the defect detection module 120.
The defect detection module is preset with a lightweight defect detection model, and the defect detection model can detect foreign matters and defects of the power transmission line in real time according to the inspection image acquired by the image acquisition module, so that an abnormal target image of the power transmission line is output.
The defect detection model can be obtained by training a server according to a Yolov5 network. That is, in this application, the training of defect detection model is realized by the server, and the rethread server sends the defect detection model that trains for unmanned aerial vehicle 10 to unmanned aerial vehicle 10 can realize the detection to transmission line according to the defect detection model.
The Yolov5 network is an object recognition and positioning algorithm based on a deep neural network, and has the characteristics of high running speed, capability of being used for a real-time system and the like. The Yolov5 network comprises four algorithm models, namely Yolov5s, Yolov5m, Yolov5l and Yolov5x, and in the application, the Yolov5m algorithm model is sampled for training to obtain a defect detection model.
The abnormal target image may include, but is not limited to, an image in which an abnormality occurs around the power transmission line or a defect exists in the power transmission line itself, and the like. For example, an image of a tower crane, a crane, smoke, fire, and the like around the power transmission line and an image of aging of an insulator of the power transmission line itself may be used as the abnormal target image.
Referring to fig. 3, in some embodiments, step 02 includes the sub-steps of:
021, preprocessing the inspection image to obtain a preprocessed image;
022, slicing the preprocessed image and extracting features to obtain a feature image;
023, performing branch fusion processing on the characteristic image to obtain a characteristic enhanced image;
024, according to the characteristic strengthening image modeling, detecting the abnormal target image of the transmission line.
Referring to fig. 2, in some embodiments, the substeps 021-. Or the defect detection module 120 may be configured to perform preprocessing on the inspection image to obtain a preprocessed image, slice the preprocessed image and extract features to obtain a feature image, perform branch fusion processing on the feature image to obtain a feature-enhanced image, and perform modeling according to the feature-enhanced image to detect an abnormal target image of the power transmission line.
The preprocessing may include scaling, normalization, and the like, for example, scaling the input inspection image to the input size of the network and performing normalization.
Referring to fig. 4 and 5, in particular, the defect detection model includes four parts, i.e., an input terminal, a Backbone network (Backbone network), a Neck network and an output terminal, where substep 021 is implemented at the input terminal of the defect detection model, substep 022 is implemented at the Backbone network of the defect detection model, substep 023 is implemented at the Neck network of the defect detection model, and substep 023 is implemented at the input terminal of the defect detection model.
Further, the backbone network comprises a Focus module and a CSP module, that is, the Focus module and the CSP module realize slicing the preprocessed image and feature extraction to obtain a feature image.
The CSP module comprises a minimum convolution unit CBL, a residual error structure CSP1_ X and a Spatial Pyramid Pooling Structure (SPP), wherein each CBL comprises a convolutional layer Conv, a Batch Normalization (BN) and an activation function Leaky relu, and the CBL is a minimum unit for extracting characteristic information. In the residual error structure CSP1_ X, X represents the number of residual error structures, and the addition of the residual error structures can prevent the gradient from disappearing when the deep network carries out reverse propagation, and the obtained feature granularity is finer. The SPP structure can effectively avoid the problems of image distortion and the like caused by image region cutting and scaling operation, and realizes the extraction of local features and global features.
The NECK network adopts CSP2_ X, wherein X represents the number of CBL contained in the network, CSP2_ X not only divides the output of the pure CBL into two branches, but also concat the branch, thereby enhancing the fusion capability of the characteristics and enriching the characteristic information.
Referring to fig. 6, in some embodiments, step 022 includes the sub-steps of:
0221, processing each preprocessed image slice through a Focus module to obtain a multi-channel input segmentation image;
0222, performing convolution, residual error and pooling on the segmented image by a CSP module to extract a characteristic image.
In certain embodiments, substeps 0221 and 0222 may be implemented by defect detection module 120. Alternatively, the defect detection module 120 may be configured to perform a segmentation process on each preprocessed image slice by the Focus module to obtain a multi-channel input segmented image, and perform a convolution, residual and pooling process on the segmented image by the CSP module to extract a feature image.
Specifically, a Focus module extracts a pixel value from every other pixel in each preprocessed image to obtain four pictures, the four pictures are complementary to each other, W, H information is concentrated in a channel space, an input channel is expanded by 4 times, namely the spliced pictures are changed into 12 channels relative to an original RGB three-channel mode, and finally, the obtained new pictures are subjected to convolution operation to finally obtain a segmented image without information loss.
For example, in some examples, an original 640 × 640 × 3 image is input into a Focus module, and is first changed into a 320 × 320 × 12 feature map by a slicing operation, and then is subjected to a convolution operation, and finally becomes a 320 × 320 × 32 segmented image.
Referring to fig. 7, in some embodiments, the detection method further includes:
and 03, sending the abnormal target image to a server.
In certain embodiments, the drone 10 further includes an image transmission module 140. Step 03 may be implemented by the image transmission module 140, or the image transmission module 140 may be configured to send the abnormal target image to the server.
Therefore, the server can optimize the defect detection model according to the abnormal target image, or generate an alarm signal according to the abnormal target image to prompt relevant personnel.
Referring to fig. 8, in some embodiments, the detection method further includes:
04, generating an alarm signal according to the abnormal target image;
and 05, sending an alarm signal and/or an abnormal target image to the application terminal.
In some embodiments, step 04 may be implemented by the controller 130, and step 05 may be implemented by the image transmission module 140, or the controller 130 may be configured to generate an alarm signal according to the abnormal target image, and the image transmission module 140 may be configured to send the alarm signal and/or the abnormal target image to the application terminal.
For example, in some examples, the drone 10 may send an alert signal to an application terminal, and for example, in some examples, the drone 10 may send an alert signal and an abnormal target image to an application terminal. In some examples, for example, the drone 10 may send an anomalous target image to the application terminal.
The warning signal (warning signal) is a signal for giving out information on the existence of a trouble, an accident, or other danger to a person by using a stimulus such as light, sound, mechanical, electrical, or odor. According to different senses acting on human, the method is divided into visual warning signals, auditory warning signals, tactile warning signals, olfactory warning signals and the like.
Therefore, related personnel can timely process and maintain the power transmission line according to the abnormal target image received by the application terminal, and the safety of the power transmission line is ensured.
Referring to fig. 9, in some embodiments, the training step of the defect detection model trained by the Yolov5 network includes:
11, acquiring historical images about the power transmission line;
marking the position information and the category information of foreign matters and defects in the historical images to construct a data set;
13, training the Yolov5 network according to the data set to correct the weight parameters of the Yolov5 network;
14, under the condition that the test result of the Yolov5 network meets the accuracy requirement, taking the modified Yolov5 network as a defect detection model.
Referring to fig. 10, the embodiment of the present application provides a server 20, where the training process of the defect detection model may be implemented by the server 20, and the server 20 includes an information transmission module 210, a model training module 220, and a model updating module 230. Wherein, step 11 can be implemented by the information transmission module 210, steps 12 and 13 can be implemented by the model training module 220, and step 14 can be implemented by the model updating module 230.
Alternatively, the information transmission module 210 may be configured to obtain a history image about the power transmission line; the model training module 220 may be configured to label the location information and the category information of the foreign objects and the defects in the historical images to construct a data set; training the Yolov5 network according to the data set to correct the weight parameters of the Yolov5 network; the model updating module 230 may be configured to use the modified Yolov5 network as a defect detection model when a test result of the Yolov5 network meets an accuracy requirement.
In the detection method and the server 20 of the embodiment of the application, the server 20 is used for constructing the data set according to the position information and the category information of the foreign matters and the defects in the historical images, the defect detection model is obtained by training the Yolov5 network through the data set, the defect detection model can be sent to the unmanned aerial vehicle, the unmanned aerial vehicle can directly patrol and examine the power transmission line according to the defect detection model, and therefore the foreign matter detection precision and the detection speed of the unmanned aerial vehicle are effectively improved.
Server 20 can communicate with foretell unmanned aerial vehicle to obtain the historical image about transmission line from unmanned aerial vehicle, the foreign matter can be tower crane, firework, flame etc. and the defect can be that the insulator is ageing etc..
Referring to fig. 11, in some embodiments, step 13 includes:
131, dividing the data set into a training set and a testing set;
132, training the Yolov5 network according to the training set to obtain a training result;
133, calculating a loss value of the Yolov5 network through a target loss function based on the training result and the test set;
134, correcting the weight parameters of the Yolov5 network according to the loss value of the Yolov5 network.
In some embodiments, the substeps 131-134 may be implemented by the model training module 220. Alternatively, the model training module 220 may be used to divide the data set into a training set and a test set; training the Yolov5 network according to the training set to obtain a training result; calculating a loss value of the Yolov5 network through a target loss function based on the training result and the test set; and correcting the weight parameters of the Yolov5 network according to the loss value of the Yolov5 network.
The training set was used as a training sample for the Yolov5 network, and the test set was used as a validation sample for the Yolov5 network. It should be noted that the target loss function includes a localization loss function, a classification loss function, and a confidence loss function;
the target loss function is calculated by the formula:
Loss(obj)＝L GIoU +L CLASS +L OBJ
wherein Loss (obj) represents the target loss function, L GIoU Representing the localization loss function, L CLASS A classification loss function is represented.
The calculation formula of the positioning loss function is as follows:
Figure BDA0003715156150000091
wherein B represents a prediction box, B gt Representing a real box, C representing containing B gt Intersection ratio with the minimum convex closed box of B, IoU;
the formula for the classification loss function is:
Figure BDA0003715156150000101
c denotes the class, p denotes the probability of predicting the class, S denotes the mesh size,
Figure BDA0003715156150000102
indicating that if there is a target at the jth box of the ith grid, the value is 1, otherwise, it is 0;
the confidence loss function is calculated as:
Figure BDA0003715156150000103
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003715156150000104
indicating that if there is no target at the jth box of the ith grid, its value is 1, otherwise it is 0, C i And representing the confidence, if the bounding box in the grid is responsible for predicting a certain object, the value is 0, otherwise.
Referring to FIG. 12, in some embodiments, substep 132 comprises:
1321, performing data enhancement processing on the training set to obtain an enhanced training set;
1322, the Yolov5 network is trained according to the enhanced training set to obtain training results.
In certain embodiments, substeps 1321 and 1322 may be implemented by model training module 220. In other words, the model training module 220 may be configured to perform data enhancement processing on the training set to obtain an enhanced training set, and train the Yolov5 network according to the enhanced training set to obtain a training result.
Data enhancement processing may include, but is not limited to, including Mosaic data enhancement, adaptive anchor frame calculation, and adaptive picture scaling, among others. The Mosaic data enhancement is formed by splicing four pictures, so that each image can contain a small target at a higher probability, and the detection of the small target is promoted.
Therefore, the detection data set is greatly enriched by performing data enhancement processing on the training set, particularly, many small targets are added by random scaling, the GPU can be reduced, the training of the Yolov5 network is facilitated, and the robustness of the model is improved.
Referring to fig. 13, in some embodiments, the detection method further includes:
15, receiving a target abnormal image of the unmanned aerial vehicle;
and 16, performing incremental learning on the defect detection model according to the target abnormal image to optimize the defect detection model.
In some embodiments, step 15 may be implemented by information transfer module 210 and step 16 may be implemented by model training module 220. Alternatively, the information transmission module 210 may be configured to receive a target abnormal image of the drone; the model training module 220 may be configured to perform incremental learning on the defect detection model based on the target anomaly image to optimize the defect detection model.
In this embodiment, the information transmission module 210 may implement wireless communication with the drone through WI-FI or 5G, etc.
It should be noted that incremental learning means that a learning system can continuously learn new knowledge from new samples and can store most of the previously learned knowledge. On one hand, the increment learning does not need to store historical data, so that the occupation of storage space is reduced, and on the other hand, the increment learning fully utilizes historical training results in the current sample training, so that the time of subsequent training is obviously reduced.
Therefore, incremental learning is carried out on the defect detection model through the target abnormal image, the requirements of the defect detection model on time and space are reduced, and the defect detection model can better meet various actual scene requirements of routing inspection.
Referring to fig. 14, in some embodiments, the detection method further includes:
17, generating an alarm signal according to the abnormal target image;
and 18, sending an alarm signal and/or an abnormal target image to the application terminal.
In some embodiments, steps 17 and 18 may be implemented by information transfer module 210. Alternatively, the information transmission module 210 may be configured to generate an alarm signal according to the abnormal target image, and send the alarm signal and/or the abnormal target image to the application terminal.
Specifically, the information transmission module 210 may implement communication with the application terminal through a TCP network protocol or an IP network protocol, so as to send the alarm signal and/or the abnormal target image to the application terminal.
For example, in some examples, the information transmission module 210 may send an alert signal to the application terminal, and in yet other examples, the information transmission module 210 may send the alert signal and the abnormal target image to the application terminal. For another example, in some examples, the information transmission module 210 may send the anomaly target image to the application terminal.
Therefore, related personnel can timely process and maintain the power transmission line according to the abnormal target image obtained by the application terminal, and the safety of the power transmission line is ensured.
Referring to fig. 15, in some embodiments, the detection method further includes:
and 19, carrying out quantization processing on the defect detection model through a preset quantization algorithm.
And 20, sending the defect detection model after the quantization processing to the unmanned aerial vehicle.
In some embodiments, step 17 may be implemented by model training module 220 and step 18 may be implemented by information transfer module 210. Alternatively, the model training module 220 may be configured to perform a quantization process on the defect detection model through a predetermined quantization algorithm. The information transmission module 210 may be configured to send the defect detection model after quantization to the drone.
It can be understood that, for the deep learning model, not only the calculated amount is very large, the stored model also occupies resources, and in order to realize the real-time operation of the defect detection model, dedicated computing platforms such as a GPU are needed, so that the defect detection model can be conveniently deployed in the unmanned aerial vehicle and adapted to the performance of the unmanned aerial vehicle by processing the defect detection model in a quantitative manner.
The preset quantization algorithm may quantize for INT 8. That is, the defect detection model is quantized by INT8 quantization.
Please refer to fig. 16, an embodiment of the present application further provides a detection system for a power transmission line, which is applied to inspection of the power transmission line, where the detection system includes an unmanned aerial vehicle and a server, and the unmanned aerial vehicle and the server can implement wireless communication through 5G or Wi-Fi.
The server is used for acquiring a historical image of the power transmission line, marking position information and category information of foreign matters and defects in the historical image to construct a data set, training the Yolov5 network according to the data set to correct weight parameters of the Yolov5 network, and taking the corrected Yolov5 network as a defect detection model and sending the defect detection model to the unmanned aerial vehicle under the condition that a test result of the Yolov5 network meets the requirement of accuracy;
the unmanned aerial vehicle is used for receiving the defect detection model, patrols and examines along the predetermined route and obtains the patrol image of the power transmission line, and detects the patrol image through the defect detection model to determine the abnormal target image of the power transmission line.
The detection system of this application embodiment, through position information and the classification information to foreign matter and defect in the historical image in order to establish the data set to obtain the defect detection model to Yolov5 network training through the data set, send the defect detection model to unmanned aerial vehicle again, make unmanned aerial vehicle can directly patrol and examine transmission line according to the defect detection model, so, effectively improved unmanned aerial vehicle's foreign matter detection precision and detection speed.
In some embodiments, the detection system includes an application terminal, the application terminal communicates with the server, the drone is further configured to send an abnormal target image to the server, the server is further configured to generate an alarm signal according to the abnormal target image and send the alarm signal and the abnormal target image to the application terminal, and the application terminal is configured to display the alarm signal and/or the abnormal target image.
It should be noted that the application terminal includes applications such as monitoring alarm information, remote monitoring, transmission line system visual analysis, and the like, and can locally monitor the operation state of the transmission line in real time.
Therefore, related personnel can timely process and maintain the power transmission line according to the abnormal target image obtained by the application terminal, and the safety of the power transmission line is ensured.
The embodiment of the application also provides a nonvolatile computer readable storage medium, and the readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the computer program causes the processors to execute the detection method of the power transmission line.
In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware or any other combination. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Video Disk (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), among others.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (16)

1. A detection method of a power transmission line is characterized by comprising the following steps:
patrolling along a preset path to obtain a patrolling image of the power transmission line;
and detecting the inspection image through a defect detection model to determine an abnormal target image of the power transmission line, wherein the defect detection model is obtained by Yolov5 network training.
2. The inspection method according to claim 1, wherein the inspecting the inspection image through a defect detection model to determine an abnormal target image of the power transmission line comprises:
preprocessing the inspection image to obtain a preprocessed image;
slicing the preprocessed image and extracting features to obtain a feature image;
performing branch fusion processing on the characteristic image to obtain a characteristic enhanced image;
and according to the characteristic reinforced image modeling, detecting an abnormal target image of the power transmission line.
3. The detection method according to claim 2, wherein the slicing the preprocessed image and the feature extraction to obtain the feature image comprise:
processing each preprocessed image slice through a Focus module to obtain a multi-channel input segmentation image;
and performing convolution, residual error and pooling on the segmented image through a CSP module to extract the characteristic image.
4. The detection method according to claim 1, further comprising:
and sending the abnormal target image to a server.
5. The detection method according to claim 1, further comprising:
generating an alarm signal according to the abnormal target image;
and sending the alarm signal and/or the abnormal target image to an application terminal.
6. The detection method according to claim 1, wherein the training step of the defect detection model trained by a Yolov5 network comprises:
acquiring a historical image about the power transmission line;
marking the position information and the category information of the foreign matters and the defects in the historical image to construct a data set;
training a Yolov5 network according to the data set to modify weight parameters of the Yolov5 network;
and under the condition that the test result of the Yolov5 network meets the requirement of accuracy, taking the modified Yolov5 network as the defect detection model.
7. The detection method according to claim 6, wherein the training of the Yolov5 network according to the data set to modify the weight parameters of the Yolov5 network comprises:
dividing the data set into a training set and a test set;
training the Yolov5 network according to the training set to obtain a training result;
calculating a loss value of the Yolov5 network through an objective loss function based on the training results and the test set;
and correcting the weight parameters of the Yolov5 network according to the loss value of the Yolov5 network.
8. The detection method according to claim 7, wherein the training the Yolov5 network according to the training set to obtain a training result comprises:
performing data enhancement processing on the training set to obtain an enhanced training set;
training the Yolov5 network according to the enhanced training set to obtain the training result.
9. The detection method of claim 7, wherein the objective loss function comprises a localization loss function, a classification loss function, and a confidence loss function;
the positioning loss function is:
Figure FDA0003715156140000021
wherein B represents a prediction box, B gt Representing a real box, C representing containing B gt Intersection ratio with the minimum convex closed box of B, IoU;
the classification loss function is:
Figure FDA0003715156140000022
c denotes the class, p denotes the probability of predicting the class, S denotes the mesh size,
Figure FDA0003715156140000023
indicating that if there is a target at the jth box of the ith grid, its value is 1, otherwise it is 0;
the confidence loss function is:
Figure FDA0003715156140000024
wherein the content of the first and second substances,
Figure FDA0003715156140000031
indicating that if there is no target at the jth box of the ith grid, its value is 1, otherwise it is 0, C i Representing a confidence level.
10. The detection method according to claim 1, further comprising:
and performing incremental learning on the defect detection model according to the target abnormal image to optimize the defect detection model.
11. The detection method according to claim 1, further comprising:
quantizing the defect detection model through a preset quantization algorithm;
and sending the defect detection model subjected to quantization processing to the unmanned aerial vehicle.
12. The detection method according to claim 11, wherein the preset quantization algorithm comprises INT8 quantization.
13. An unmanned aerial vehicle, comprising:
the image acquisition module is used for acquiring an inspection image of the power transmission line along a predetermined path in an inspection way;
and the defect detection module is used for detecting the patrol inspection image through a defect detection model to determine an abnormal target image of the power transmission line, and the defect detection model is obtained by Yolov5 network training.
14. The detection system of the power transmission line is characterized by comprising an unmanned aerial vehicle and a server, wherein the unmanned aerial vehicle is in wireless communication with the server, the server is used for acquiring a historical image about the power transmission line, marking position information and category information of foreign matters and defects in the historical image to construct a data set, training a Yolov5 network according to the data set to correct weight parameters of the Yolov5 network, and taking the corrected Yolov5 network as a defect detection model and sending the defect detection model to the unmanned aerial vehicle under the condition that a test result of the Yolov5 network meets an accuracy requirement;
the unmanned aerial vehicle is used for receiving the defect detection model, patrolling along a preset path to obtain a patrolling image of the power transmission line, and detecting the patrolling image through the defect detection model to determine an abnormal target image of the power transmission line.
15. The detection system according to claim 14, wherein the detection system comprises an application terminal, the application terminal is in communication with the server, the drone is further configured to send the abnormal target image to the server, the server is further configured to generate an alarm signal according to the abnormal target image and send the alarm signal and the abnormal target image to the application terminal, and the application terminal is configured to display the alarm signal and/or the abnormal target image.
16. A non-transitory computer-readable storage medium containing a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the method of detecting a power transmission line of any one of claims 1-12.
CN202210735434.0A 2022-06-27 2022-06-27 Detection method, unmanned aerial vehicle, detection system and medium Pending CN115082813A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210735434.0A CN115082813A (en) 2022-06-27 2022-06-27 Detection method, unmanned aerial vehicle, detection system and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210735434.0A CN115082813A (en) 2022-06-27 2022-06-27 Detection method, unmanned aerial vehicle, detection system and medium

Publications (1)

Publication Number Publication Date
CN115082813A true CN115082813A (en) 2022-09-20

Family

ID=83255911

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210735434.0A Pending CN115082813A (en) 2022-06-27 2022-06-27 Detection method, unmanned aerial vehicle, detection system and medium

Country Status (1)

Country Link
CN (1) CN115082813A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115545198A (en) * 2022-11-25 2022-12-30 成都信息工程大学 Edge intelligent collaborative inference method and system based on deep learning model
CN115953652A (en) * 2023-03-15 2023-04-11 广东电网有限责任公司肇庆供电局 Batch normalization layer pruning method, device, equipment and medium for target detection network

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115545198A (en) * 2022-11-25 2022-12-30 成都信息工程大学 Edge intelligent collaborative inference method and system based on deep learning model
CN115545198B (en) * 2022-11-25 2023-05-26 成都信息工程大学 Edge intelligent collaborative inference method and system based on deep learning model
CN115953652A (en) * 2023-03-15 2023-04-11 广东电网有限责任公司肇庆供电局 Batch normalization layer pruning method, device, equipment and medium for target detection network

Similar Documents

Publication Publication Date Title
CN108830837B (en) Method and device for detecting steel ladle corrosion defect
CN110084165B (en) Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation
CN115082813A (en) Detection method, unmanned aerial vehicle, detection system and medium
CN109544870B (en) Alarm judgment method for intelligent monitoring system and intelligent monitoring system
CN112688434A (en) Monitoring and early warning method and device for power transmission and distribution line, computer equipment and medium
CN113361473B (en) Image processing method, model training method, image processing device, model training apparatus, storage medium, and program
CN113095160B (en) Power system personnel safety behavior identification method and system based on artificial intelligence and 5G
CN111401437A (en) Deep learning-based power transmission channel hidden danger early warning grade analysis method
CN116310922A (en) Petrochemical plant area monitoring video risk identification method, system, electronic equipment and storage medium
CN112016380B (en) Wild animal monitoring method and system
CN112967248A (en) Method, apparatus, medium, and program product for generating defect image sample
CN115083229B (en) Intelligent recognition and warning system of flight training equipment based on AI visual recognition
CN115294048A (en) Foreign matter detection method, device, equipment and storage medium
CN115187880A (en) Communication optical cable defect detection method and system based on image recognition and storage medium
CN114445782A (en) Power transmission line image acquisition system based on edge AI and Beidou short messages
CN112926415A (en) Pedestrian avoiding system and pedestrian monitoring method
CN114373162A (en) Dangerous area personnel intrusion detection method and system for transformer substation video monitoring
CN113569956A (en) Mountain fire disaster investigation and identification method based on AI algorithm
CN113705442A (en) Outdoor large-board advertising picture monitoring and identifying system and method
CN113780178A (en) Road detection method, road detection device, electronic equipment and storage medium
AU2021106241A4 (en) System and method for aerial power lines measurement using computer vision and unmanned aerial vehicle
CN115375855B (en) Engineering project visualization method and device, electronic equipment and readable medium
CN114818880B (en) Method and system for automatically identifying key operation flow of YOLOv3 railway
CN115620496A (en) Fault alarm method, device, equipment and medium applied to power transmission line
CN117935129A (en) Method, device, equipment and storage medium for detecting power distribution network line disaster damage

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination