CN112395972B - Unmanned aerial vehicle image processing-based insulator string identification method for power system - Google Patents
Unmanned aerial vehicle image processing-based insulator string identification method for power system Download PDFInfo
- Publication number
- CN112395972B CN112395972B CN202011276698.1A CN202011276698A CN112395972B CN 112395972 B CN112395972 B CN 112395972B CN 202011276698 A CN202011276698 A CN 202011276698A CN 112395972 B CN112395972 B CN 112395972B
- Authority
- CN
- China
- Prior art keywords
- insulator string
- image
- ellipse
- unmanned aerial
- aerial vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000012212 insulator Substances 0.000 title claims abstract description 161
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012545 processing Methods 0.000 title claims abstract description 17
- 230000008569 process Effects 0.000 claims abstract description 9
- 238000005457 optimization Methods 0.000 claims abstract description 6
- 230000011218 segmentation Effects 0.000 claims description 14
- 238000005265 energy consumption Methods 0.000 abstract description 16
- 238000004364 calculation method Methods 0.000 abstract description 8
- 238000007689 inspection Methods 0.000 abstract description 5
- 230000005540 biological transmission Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 230000006854 communication Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 206010033799 Paralysis Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Economics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the field of industrial Internet of things and edge calculation, in particular to an insulator string identification method of an electric power system based on unmanned aerial vehicle image processing. The method comprises the following steps: 1) The unmanned aerial vehicle segments the H channel image by adopting an OTSU algorithm with a 3 threshold value to obtain a segmented result image; 2) Processing the result image by the unmanned aerial vehicle to obtain a zero-order moment, a first-order moment and a second-order moment of the result image; acquiring parameter information of an ellipse covered by the insulator string; 3) Estimating a suspected insulator string region through an iterative optimization algorithm according to the parameter information of the insulator string covering ellipse, and estimating the possible direction of the insulator string; 4) Transmitting the detected image containing the insulator string and the estimated direction thereof back to the ground server for processing; 5) And the ground server rotates the image until the insulator string is in a horizontal state, and the insulator string in the image is identified by using a Faster RCNN network. The invention solves the problem of faster energy consumption in the inspection process of the unmanned aerial vehicle, and prolongs the flight time of the unmanned aerial vehicle.
Description
Technical Field
The invention relates to the field of industrial Internet of things and edge calculation, in particular to an insulator string identification method of an electric power system based on unmanned aerial vehicle image processing.
Background
Along with the development of unmanned aerial vehicle technology, unmanned aerial vehicle's application in industry thing networking is paid more and more attention to. The unmanned aerial vehicle not only can improve the flexibility of target monitoring, but also can assist in completing the communication process, thereby effectively expanding the application range of the industrial Internet of things. The limited energy of unmanned aerial vehicles presents a great challenge to their use in industrial internet of things. The flight time of the unmanned aerial vehicle is mainly related to the weight, flight time, data transmission energy consumption and task calculation energy consumption of the unmanned aerial vehicle.
In an electric power system, the safety and reliability of a transmission line have a crucial effect on the smooth operation of power transmission. The insulator string is used as a key component for insulating and supporting the transmission line, and once the insulator string is defective, the whole transmission line can be in a paralysis state. The insulator string identification and the defect detection are core parts of the inspection of the transmission line in the power system, and have very important practical significance, while the insulator string identification is the basis of the defect detection of the insulator string, and the accuracy of the defect detection is directly determined by the quality of the identification result.
Transmitting real-time video captured by the unmanned aerial vehicle to a ground server for analysis consumes a lot of energy. If the unmanned aerial vehicle can execute some data analysis tasks, the real-time video transmission is only carried out under the necessary condition, and the data transmission energy consumption is greatly reduced. Moreover, the insulator string has a very large aspect ratio compared to a general natural object, and when the target is identified by using the deep neural network, all possible directions of the target need to be detected, which is very large for the energy consumption of calculation. For high aspect ratio target detection based on deep neural networks, no effective solution has been proposed to reduce computational power consumption.
Disclosure of Invention
In the unmanned aerial vehicle inspection process, whether the unmanned aerial vehicle performs all image analysis processes or transmits all real-time videos to a ground server, a large amount of energy can be consumed, and therefore the flight time of the unmanned aerial vehicle is shortened. Aiming at the defects of the prior art, the invention provides a method for identifying an insulator string of an electric power system based on unmanned aerial vehicle image processing, which is used for accurately positioning the insulator string at a server side, and solves the problem of huge energy consumption of target calculation for identifying a large length-width ratio by the existing deep neural network, thereby prolonging the flight time of an unmanned aerial vehicle.
The technical scheme adopted by the invention for achieving the purpose is as follows: an identification method of an insulator string of an electric power system based on unmanned aerial vehicle image processing comprises the following steps:
1) The unmanned aerial vehicle acquires an image of a scene by carrying a camera, converts the image into an image of an HSV space from an RGB space, and segments an H channel image by adopting an OTSU algorithm of a 3 threshold value to obtain a segmented result image of the OTSU algorithm;
2) The unmanned aerial vehicle processes the result image to obtain zero-order moment, first-order moment and second-order moment of the result image; acquiring parameter information of the insulator string covering ellipse according to the zero-order moment, the first-order moment and the second-order moment;
3) Estimating a suspected insulator string region according to parameter information of the ellipse covered by the insulator string and through an iterative optimization algorithm, and estimating a possible direction of the insulator string according to the suspected insulator string region;
4) Transmitting the detected image containing the insulator string and the estimated direction of the insulator string area back to the ground server for processing;
5) After receiving the image and the direction information of the insulator string sent by the unmanned aerial vehicle, the ground server rotates the image to the direction of the horizontal state of the insulator string, and then identifies the insulator string in the image by using a Faster RCNN network.
The step 2) specifically comprises the following steps:
first, the result image R after segmentation using OTSU algorithm * Is selected as the initial seed region R 0 ；
Result image R * X, y), wherein x and y represent pixels in the horizontal and vertical directions in the image, respectively;
from the resulting image R * The pixel point (x, y) in (a) acquires the result image R * The zero order, first order, and second order of (a), is represented by:
zero order moment M 00 ：
First moment M 10 And M 01 ：
Second moment M 20 、M 02 And M 11 ：
The method comprises the steps of obtaining parameter information of an insulator string covering ellipse according to zero-order moment, first-order moment and second-order moment, specifically
Obtaining an initial seed region R according to the formula (1) and the formula (2) 0 The center point (X, Y) of the overlay ellipse is:
(X,Y)＝(M 10 /M 00 ,M 01 /M 00 )(4)
obtaining the length of the major axis of the insulator string coverage ellipse according to the formula (3):
obtaining the short axis length of the insulator string coverage ellipse according to the formula (3):
obtaining the direction of the corresponding insulator string according to the formula (1) (2) (3):
wherein X is C Is the abscissa of the ellipse center point on the image, Y C Is the ordinate of the ellipse center point on the image.
The parameter information of the insulator string comprises: initial seed region R 0 The center point of the ellipse, the major axis length L, the minor axis length S and the direction O.
In step 3), the estimating the suspected insulator string region by the iterative optimization algorithm specifically includes:
step (1): by maximizing this algorithm, an iterative objective function T is obtained, namely:
wherein R is * Is a result image segmented by using an OTSU algorithm, E is a coverage ellipse of a suspected insulator string region, |E| is the area of the ellipse, |R * Inverted E| is the area of the resulting image covered by ellipse E;
step (2): estimating a coverage ellipse E of a suspected insulator string region through the parameter information of the insulator string; and obtain a coverage ellipse E and a result image R of the suspected insulator string region * The area of the resulting image covered by ellipse E, i.e. the overlap region R F ；
Step (3): based on the area |E| and the overlapping region R of the ellipse F Obtaining a filling rate F;
step (4): according to the filling rate F and the overlapping region R F Obtaining a continuously iterated objective function T;
step (5): starting to decrease according to the value of the objective function T iterated continuously, and judging that the aspect ratio of the current E is larger than a threshold value, wherein the current E is possibly an insulator string; conversely, if the aspect ratio of E is less than the set point, then it is not an insulator string.
Estimating the possible direction of the insulator string according to the suspected insulator string region in the step 3), specifically:
when only one insulator string exists in the image, the direction of the insulator string in the image is directly acquired;
when a plurality of insulator strings exist in the image, a direction set of the insulator strings in the image is obtained through a direction estimation algorithm.
When a plurality of insulator strings exist in an image, a direction set of the insulator strings in the image is obtained through a direction estimation algorithm, specifically:
A. setting an initial direction candidate set Ori as an empty set, wherein the number of Ori set items is defined as |ori|;
B. converting the shot insulator string image from an RGB space to an HSV space to obtain an H channel image;
C. obtaining a segmentation threshold t= { T using a 3-threshold OTSU algorithm 1 ,t 2 ,t 3 }；
D. Segmentation of the H-channel image using T yields the region set r= { R 1 ,r 2 ,r 3 ,r 4 -where r 1 ≤t 1 ，t 1 ＜r 2 ≤t 2 ，t 2 ＜r 3 ≤t 3 ，t 3 ＜r 4 Wherein t is 1 ,t 2 ,t 3 Three thresholds respectively segmented by the OTSU algorithm;
E. obtaining r by using the obtained connected domain function 4 Removing the region with the noise point area smaller than 100 to obtain a connected region set C;
F. arranging the items in C in descending order of area to obtain a region set R= { R Fk K=1,..k, where K is the total number of regions in R, R Fk K overlapping regions in the region set R.
In the step 5), the insulator string is analyzed according to the parameter information of the insulator string and the estimated possible direction of the insulator string, and the direction of the insulator string is determined specifically as follows:
when k.ltoreq.K and the number of terms of |Ori| < 6, the following loop is performed:
obtaining each R in the region set R by the objective function T Fk The filling rate F, the length L of the long axis, the length S of the short axis, the direction O and the center point (X, Y) of the insulator string, and determining the direction set of the insulator string.
The method for determining the direction set of the insulator string specifically comprises the following steps:
when L/S > 4 and M 00 At > 100
Judging the current R Fk Direction O of (2) k When the difference from all the terms in Ori is greater than 15 °, then the direction of the current insulator string is added to the candidate direction set Ori, where ori= { O k -a }; otherwise, if the difference between the direction of the insulator string and the term in Ori is less than or equal to 15 degrees, the term in the original candidate direction set is still kept unchanged.
The invention has the following beneficial effects and advantages:
the invention provides a method for identifying an insulator string in a power system based on cloud edge fusion, which can solve the problem of faster energy consumption in the unmanned aerial vehicle inspection process well and prolong the flight time of the unmanned aerial vehicle by roughly detecting the insulator string at the edge end and estimating the direction of the insulator string and precisely positioning the insulator string at the cloud server end.
Drawings
FIG. 1 is a block diagram of the overall architecture of the present invention;
FIG. 2 is a sample image of 3 different channels of the present invention;
fig. 3 is a sample of the coarse identification of an insulator string of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to the appended drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The invention may be embodied in many other forms than described herein and similarly modified by those skilled in the art without departing from the spirit or scope of the invention, which is therefore not limited to the specific embodiments disclosed below.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
In the present invention, navigation and obstacle avoidance of the drone relies on GPS signals and visual analysis. And only when the suspected target is detected, the real-time video is transmitted back to the cloud server side for further analysis. On the basis, a novel insulator string identification frame is provided.
As shown in fig. 1, the proposed framework comprises two steps. The first step is to roughly detect the insulator string and estimate the possible direction of the insulator string; if one or more insulator strings are detected in the image, the image and the estimated direction are sent back to the ground server, and then the accurate detection of the insulator strings is performed by adopting the fast RCNN algorithm.
For the same insulator string, the colors of all insulators are the same, and the insulator string can be segmented according to the color information. As shown in fig. 2, different channel images of one image are given, from left to right, as RGB images, R channel, G channel, B channel, H channel, S channel, and V channel images, respectively. The color of the string of insulators in these three figures is different. As can be seen from fig. 2, the H-channel image is more suitable for segmentation using a threshold-based segmentation algorithm. In the invention, the threshold value is calculated by adopting an OTSU algorithm, and then an insulator chain in the image is segmented by adopting a 3-threshold OTSU algorithm.
Although the insulator string region can be obtained by the method, the segmentation algorithm based on the threshold value is sensitive to background noise under the condition of complex background. The present invention thus proposes a new area filtering method. The core idea of the method is that the pixels of the insulator string region are more densely distributed than the background noise, and the overall shape is close to a rectangle. Also, for a rectangle, when the aspect ratio is large, it may be approximated as an ellipse. Based on the above analysis, the present invention estimates the suspected insulator string region by maximizing the following equation.
Wherein R is * Is to use OTSU algorithmThe result after segmentation, E is the covered ellipse of the suspected insulator string region, |E| is the area of the ellipse, |R * And E| is the area of the segmentation result covered by ellipse E.
The objective function T may be optimized iteratively using the following algorithm:
first, the result R is segmented * Is selected as the initial seed region R 0 ；
Next, R 0 The center point (X, Y), the long axis length L, the short axis length S, and the direction O of the image can be calculated according to the zero-order moment, the first-order moment, and the second-order moment of the image, and the corresponding calculation formulas are as follows:
zero order moment M 00 ：
First moment M 10 And M 01 ：
Second moment M 20 、M 02 And M 11 ：
The method comprises the steps of obtaining parameter information of an insulator string covering ellipse according to zero-order moment, first-order moment and second-order moment, specifically
Obtaining an initial seed region R according to the formula (1) and the formula (2) 0 The center point (X, Y) of the overlay ellipse is:
(X,Y)＝(M 10 /M 00 ,M 01 /M 00 ) (4)
obtaining the length of the major axis of the insulator string coverage ellipse according to the formula (3):
obtaining the short axis length of the insulator string coverage ellipse according to the formula (3):
obtaining the direction of the corresponding insulator string according to the formula (1) (2) (3):
wherein X is C Is the abscissa of the ellipse center point on the image, Y C Is the ordinate of the ellipse center point on the image.
Then, using the center points (X, Y), L, S and O, the coverage ellipse E is estimated;
thereafter, according to R 0 ＝R * E is inverted to obtain R * Overlapping region R with E 0 ；
Finally, according to f= |r 0 Calculating the filling rate F by I/E;
at this time, the objective function T can be calculated according to F 0 And I is calculated. This process is iterated until T begins to decrease. By using the algorithm, the region E with larger area and higher filling rate can be obtained at the same time.
If the aspect ratio of E is large, it may be an insulator string. Conversely, if the aspect ratio of E is small, even close to 1, it should not be an insulator string. The result shows that compared with the direct use of a deep learning method with higher computational complexity, the method can be used for screening out images possibly containing insulator strings at lower computational cost.
Considering that there may be a plurality of insulator strings in the image, the following method for acquiring the possible directions of the plurality of insulator strings through a direction estimation algorithm is proposed. Since the number of insulator strings in an actual scene is relatively small and some insulator strings are in the same direction, 6 main directions are sufficient for most scenes. Thus, the present invention yields up to 6 possible directions.
In this algorithm, L/S > 4 is because the length of the insulator string is typically much greater than the width of the insulator sheet, so if the L/S of the region obtained using the optimization algorithm described above is small, such as near 1, that region should not be the insulator string region. At the same time M 00 The meaning of > 100 is when M 00 When < 100, the small noise point can be removed by using a method of removing the small connected domain.
Input: RGB image Img.
A. Setting an initial direction candidate set Ori as an empty set, wherein the number of Ori set items is defined as |ori|;
B. converting the shot insulator string image from an RGB space to an HSV space to obtain an H channel image;
C. obtaining a segmentation threshold t= { T using a 3-threshold OTSU algorithm 1 ,t 2 ,t 3 }；
D. Segmentation of the H-channel image using T yields the region set r= { R 1 ,r 2 ,r 3 ,r 4 -where r 1 ≤t 1 ，t 1 ＜r 2 ≤t 2 ，t 2 ＜r 3 ≤t 3 ，t 3 ＜r 4 Wherein t is 1 ,t 2 ,t 3 Three thresholds respectively segmented by the OTSU algorithm;
E. obtaining r by using the obtained connected domain function 4 Removing the region with the noise point area smaller than 100 to obtain a connected region set C;
F. arranging the items in C in descending order of area to obtain a region set R= { R Fk K=1,..k, where K is the total number of regions in R, R Fk K overlapping regions in the region set R.
In the step 5), the insulator string is analyzed according to the parameter information of the insulator string and the estimated possible direction of the insulator string, and the direction of the insulator string is determined specifically as follows:
when k.ltoreq.K and the number of terms of |Ori| < 6, the following loop is performed:
obtaining each R in the region set R by the objective function T Fk Is used for the filling rate F of the (c),the length L of the long axis, the length S of the short axis, the direction O and the center point (X, Y) of the insulator string, and determining the direction set of the insulator string.
The method for determining the direction set of the insulator string specifically comprises the following steps:
when L/S > 4 and M 00 At > 100
Judging the current R Fk Direction O of (2) k When the difference from all the terms in Ori is greater than 15 °, then the direction of the current insulator string is added to the candidate direction set Ori, where ori= { O k -a }; otherwise, if the difference between the direction of the insulator string and the term in Ori is less than or equal to 15 degrees, the term in the original candidate direction set is still kept unchanged.
And (3) outputting: the direction candidate set Ori.
Finally, the image is rotated to the direction that the insulator string is in the horizontal state through the direction candidate set Ori, and then the insulator string in the image is identified by using the Faster RCNN network.
Fig. 3 shows an example of a coarse identification of an insulator string, wherein fig. 3 (d) is the result of algorithm 2, with different insulator string regions highlighted in different colors. As can be seen from fig. 3 (d), the direction estimation algorithm proposed by the present invention can successfully identify the insulator string in the image, thereby estimating the direction of the insulator string.
When the unmanned aerial vehicle recognizes that one or more insulator strings exist in the image, the corresponding image and the estimated direction are transmitted back to the cloud server for further analysis. At the server side, the insulator string image is rotated to the horizontal direction according to the estimated direction, then the insulator string is identified by using an improved fast RCNN deep learning framework, the proportion of the candidate area network in the fast RCNN is changed to 1:8,1:4,1:2,1:1,2:1,4:1,8:1, the scale is changed to 64,128,256 and 512, the experiment is carried out on ubuntu18.04, python3.6 and rtx2080ti, the deep learning framework uses caffe, the initial learning rate is set to 0.001, the weight attenuation coefficient is set to 0.0005 and the dynamic value is set to 0.9 in the training stage.
According to the characteristic of larger length-width ratio of the insulator string, the RPN part in the target detection algorithm Faster RCNN is modified:
the ratio of the RPN network is changed from original 1:1,1:2 and 2:1: 1:8,1:4,1:2,1:1,2:1,4:1,8:1, the dimensions were changed from original 128,256,512 to 64,128,256 and 512.
Example 1:
by the identification method, the power consumption of the unmanned aerial vehicle is evaluated, and the method comprises the following steps:
currently, in the power inspection process, the unmanned aerial vehicle needs to transmit the shot video and real-time position data of the unmanned aerial vehicle back to the cloud server. By adopting the method provided by the invention, when no target or obstacle appears, the unmanned aerial vehicle only needs to send the position data back to the operator, so that the operator can control the flight path by using the electronic map. When a suspected target is found using the coarse recognition algorithm, no one has the opportunity to transmit the video back to the operator. And then an operator can control the unmanned aerial vehicle to shoot a detailed video or photo of the suspected target, and further analyze the suspected target by utilizing a fine recognition algorithm of the cloud server. Therefore, if the method provided by the invention is adopted, the energy consumption ecl of the unmanned aerial vehicle is modeled as follows:
E col ＝(F×W+K+I 1 +mean(C B ))×T+mean(C I )×t
f is energy consumption of unit weight in unit time in the flight process of the unmanned aerial vehicle, W is weight of the unmanned aerial vehicle, K is energy consumption of common calculation tasks in unit time, I 1 Is the energy consumption of the rough image recognition, mean (C B ) Is the average energy consumption (e.g., location data transmission) of the underlying communication, mean (C I ) The average energy consumption of the transmitted image in unit time is T is the total flight time, and T is the time of image transmission after the suspected target is found.
For comparison, a method of transmitting all videos and images to an operator in real time for intelligent analysis is called a centralized method, and a method of completely performing intelligent analysis by the unmanned aerial vehicle is called a decentralized method. For a centralized approach, unmanned energy may be consumed E cen Modeling is as follows:
E cen ＝(F×W+K+mean(C B )+mean(C I ))×T
in addition, for the decentralized method, unmanned energy may be consumed E dec Modeling is as follows:
E dec ＝(F×(W+S)+K+I all +mean(C B ))×T
where S is the quality of the extra special hardware supporting deep learning computation, I all Is the energy consumption per unit time to run the entire object recognition algorithm.
In order to compare the method of the present invention with the centralized method and the decentralized method, respectively, the difference between the method of the present invention and the other two methods is calculated, respectively:
E col -E cen ＝I 1 ×T+mean(C I )×(t-T)
E col -E dec ＝-(I 2 +F×S)×T+mean(C I )×t
wherein I is 2 Representing the energy consumption in the accurate recognition of the object.
Typically T < T, so:
E col -E cen ≈(I 1 -mean(C I ))×T
E col -E dec ＝-(I 2 +F×S)×T
the method of the invention only carries out simple linear conversion, segmentation based on threshold value and pixel statistics operation, thus the calculation amount of rough identification is small, thus I 1 Very small. And, the typical video data transfer power consumption of the unmanned aerial vehicle is about 1W, thus E in the above formula col -E cen ＜0。
From E col -E dec It can also be seen from this formula that the method proposed by the present invention is superior to the discretization method, and is embodied as- (I) 2 +I 3 The +FxS) xT is less than 0, so that the identification method of the insulator string of the power system based on unmanned aerial vehicle image processing has the effect of reducing energy consumption.
The algorithm provided by the invention can well identify the insulator string in the image, and has great advantages in the aspects of accuracy and recall rate.
The above embodiments are only for illustrating the technical aspects of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, it should be understood by those of ordinary skill in the art that: modifications and equivalents may be made to the specific embodiments of the invention without departing from the spirit and scope of the invention, which is intended to be covered by the claims.
Claims (6)
1. The method for identifying the insulator string of the power system based on unmanned aerial vehicle image processing is characterized by comprising the following steps of:
1) The unmanned aerial vehicle acquires an image of a scene by carrying a camera, converts the image into an image of an HSV space from an RGB space, and segments an H channel image by adopting an OTSU algorithm of a 3 threshold value to obtain a segmented result image of the OTSU algorithm;
2) The unmanned aerial vehicle processes the result image to obtain zero-order moment, first-order moment and second-order moment of the result image; acquiring parameter information of the insulator string covering ellipse according to the zero-order moment, the first-order moment and the second-order moment;
the step 2) specifically comprises the following steps:
first, the result image R after segmentation using OTSU algorithm * Is selected as the initial seed region R 0 ；
Result image R * X, y), wherein x and y represent pixels in the horizontal and vertical directions in the image, respectively;
from the resulting image R * The pixel point (x, y) in (a) acquires the result image R * The zero order, first order, and second order of (a), is represented by:
zero order moment M 00 ：
First moment M 10 And M 01 ：
Second moment M 20 、M 02 And M 11 ：
According to the zero-order moment, the first-order moment and the second-order moment, the parameter information of the insulator string coverage ellipse is obtained, specifically:
obtaining an initial seed region R according to the formula (1) and the formula (2) 0 The center point (X, Y) of the overlay ellipse is:
(X,Y)＝(M 10 /M 00 ,M 01 /M 00 )(4)
obtaining the length of the major axis of the insulator string coverage ellipse according to the formula (3):
obtaining the short axis length of the insulator string coverage ellipse according to the formula (3):
obtaining the direction of the corresponding insulator string according to the formula (1) (2) (3):
wherein X is C Is the abscissa of the ellipse center point on the image, Y C Is the ordinate of the ellipse center point in the image;
3) Estimating a suspected insulator string region according to parameter information of the ellipse covered by the insulator string and through an iterative optimization algorithm, and estimating a possible direction of the insulator string according to the suspected insulator string region;
the estimating the suspected insulator string region through the iterative optimization algorithm specifically comprises the following steps:
step (1): by maximizing this algorithm, an iterative objective function T is obtained, namely:
wherein R is * Is a result image segmented by using an OTSU algorithm, E is a coverage ellipse of a suspected insulator string region, |E| is the area of the ellipse, |R * Inverted E| is the area of the resulting image covered by ellipse E;
step (2): estimating a coverage ellipse E of a suspected insulator string region through the parameter information of the insulator string; and obtain a coverage ellipse E and a result image R of the suspected insulator string region * The area of the resulting image covered by ellipse E, i.e. the overlap region R F ；
Step (3): based on the area |E| and the overlapping region R of the ellipse F Obtaining a filling rate F;
step (4): according to the filling rate F and the overlapping region R F Obtaining a continuously iterated objective function T;
step (5): starting to decrease according to the value of the objective function T iterated continuously, and judging that the aspect ratio of the current E is larger than a threshold value, wherein the current E is possibly an insulator string; conversely, if the aspect ratio of E is less than the set value, then it is not an insulator string;
4) Transmitting the detected image containing the insulator string and the estimated direction of the insulator string area back to the ground server for processing;
5) After receiving the image and the direction information of the insulator string sent by the unmanned aerial vehicle, the ground server rotates the image to the direction of the horizontal state of the insulator string, and then identifies the insulator string in the image by using a Faster RCNN network.
2. A base according to claim 1The method for identifying the insulator string of the power system for unmanned aerial vehicle image processing is characterized in that the parameter information of the insulator string comprises the following steps: initial seed region R 0 The center point of the ellipse, the major axis length L, the minor axis length S and the direction O.
3. The method for identifying the insulator string of the electric power system based on the unmanned aerial vehicle image processing according to claim 1, wherein the estimating the possible direction of the insulator string according to the suspected insulator string region in the step 3) is specifically as follows:
when only one insulator string exists in the image, the direction of the insulator string in the image is directly acquired;
when a plurality of insulator strings exist in the image, a direction set of the insulator strings in the image is obtained through a direction estimation algorithm.
4. The method for identifying the insulator strings of the power system based on unmanned aerial vehicle image processing according to claim 3, wherein when a plurality of insulator strings exist in an image, a direction set of the insulator strings in the image is obtained through a direction estimation algorithm, specifically:
A. setting an initial direction candidate set Ori as an empty set, wherein the number of Ori set items is defined as |ori|;
B. converting the shot insulator string image from an RGB space to an HSV space to obtain an H channel image;
C. obtaining a segmentation threshold t= { T using a 3-threshold OTSU algorithm 1 ,t 2 ,t 3 }；
D. Segmentation of the H-channel image using T yields the region set r= { R 1 ,r 2 ,r 3 ,r 4 -where r 1 ≤t 1 ，t 1 ＜r 2 ≤t 2 ，t 2 ＜r 3 ≤t 3 ，t 3 ＜r 4 Wherein t is 1 ,t 2 ,t 3 Three thresholds respectively segmented by the OTSU algorithm;
E. obtaining r by using the obtained connected domain function 4 Is to make the noise point areaRemoving the region smaller than 100 to obtain a connected domain set C;
F. arranging the items in C in descending order of area to obtain a region set R= { R Fk K=1,..k, where K is the total number of regions in R, R Fk K overlapping regions in the region set R.
5. The method for identifying the insulator string of the electric power system based on the unmanned aerial vehicle image processing according to claim 3, wherein in the step 5), the insulator string is analyzed according to the parameter information of the insulator string and the estimated possible direction of the insulator string, and the direction of the insulator string is determined specifically as follows:
when k.ltoreq.K and the number of terms of |Ori| < 6, the following loop is performed:
obtaining each R in the region set R by the objective function T Fk The filling rate F, the length L of the long axis, the length S of the short axis, the direction O and the center point (X, Y) of the insulator string, and determining the direction set of the insulator string.
6. The method for identifying the insulator string of the power system based on unmanned aerial vehicle image processing according to claim 5, wherein the determining the direction set of the insulator string is specifically as follows:
when L/S > 4 and M 00 At > 100
Judging the current R Fk Direction O of (2) k When the difference from all the terms in Ori is greater than 15 °, then the direction of the current insulator string is added to the candidate direction set Ori, where ori= { O k -a }; otherwise, if the difference between the direction of the insulator string and the term in Ori is less than or equal to 15 degrees, the term in the original candidate direction set is still kept unchanged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011276698.1A CN112395972B (en) | 2020-11-16 | 2020-11-16 | Unmanned aerial vehicle image processing-based insulator string identification method for power system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011276698.1A CN112395972B (en) | 2020-11-16 | 2020-11-16 | Unmanned aerial vehicle image processing-based insulator string identification method for power system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112395972A CN112395972A (en) | 2021-02-23 |
| CN112395972B true CN112395972B (en) | 2023-07-14 |
Family
ID=74599516
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011276698.1A Active CN112395972B (en) | 2020-11-16 | 2020-11-16 | Unmanned aerial vehicle image processing-based insulator string identification method for power system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112395972B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255434B (en) * | 2021-04-08 | 2023-12-19 | 淮阴工学院 | Apple identification method integrating fruit characteristics and deep convolutional neural network |
| CN114581742B (en) * | 2021-10-26 | 2023-01-24 | 中科慧远视觉技术(北京)有限公司 | Linearity-based connected domain clustering fusion method, device, system and medium |
| CN114460086B (en) * | 2021-12-15 | 2023-08-15 | 国网河北省电力有限公司电力科学研究院 | Detection method of inclined tension insulator string |
| CN116846789B (en) * | 2023-09-01 | 2023-11-14 | 国网四川省电力公司信息通信公司 | Operation and maintenance management system for communication link |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504696A (en) * | 2014-12-16 | 2015-04-08 | 江南大学 | Embedded parallel optimization method for image salient region detection |
| CN105740843A (en) * | 2016-03-02 | 2016-07-06 | 成都翼比特自动化设备有限公司 | Insulator complete segmentation algorithm based on image recognition technology |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105389555A (en) * | 2015-11-06 | 2016-03-09 | 天津航天中为数据系统科技有限公司 | Hidden danger target analysis method and system based on aerial image |
| CN105549603B (en) * | 2015-12-07 | 2018-08-28 | 北京航空航天大学 | A kind of Intelligent road inspection control method of multi-rotor unmanned aerial vehicle |
| CN107507172A (en) * | 2017-08-08 | 2017-12-22 | 国网上海市电力公司 | Merge the extra high voltage line insulator chain deep learning recognition methods of infrared visible ray |
| CN107610128A (en) * | 2017-09-26 | 2018-01-19 | 山东鲁能智能技术有限公司 | The method for inspecting and device of a kind of oil level indicator |
| US11017266B2 (en) * | 2018-05-09 | 2021-05-25 | Figure Eight Technologies, Inc. | Aggregated image annotation |
| CN110222683A (en) * | 2019-06-11 | 2019-09-10 | 云南电网有限责任公司曲靖供电局 | A kind of quick-fried defect recognition localization method of electric transmission line isolator component based on depth convolutional neural networks |
-
2020
- 2020-11-16 CN CN202011276698.1A patent/CN112395972B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504696A (en) * | 2014-12-16 | 2015-04-08 | 江南大学 | Embedded parallel optimization method for image salient region detection |
| CN105740843A (en) * | 2016-03-02 | 2016-07-06 | 成都翼比特自动化设备有限公司 | Insulator complete segmentation algorithm based on image recognition technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112395972A (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112395972B (en) | Unmanned aerial vehicle image processing-based insulator string identification method for power system | |
| CN108665487B (en) | Transformer substation operation object and target positioning method based on infrared and visible light fusion | |
| CN111259706B (en) | Lane line pressing judgment method and system for vehicle | |
| CN104966304B (en) | Multi-target detection tracking based on Kalman filtering and nonparametric background model | |
| CN111784633B (en) | Insulator defect automatic detection algorithm for electric power inspection video | |
| CN110268190A (en) | A kind of underground pipe gallery leakage detection method based on the processing of static Infrared Thermogram | |
| CN112488071B (en) | Method, device, electronic equipment and storage medium for extracting pedestrian features | |
| CN111797653A (en) | Image annotation method and device based on high-dimensional image | |
| CN111814686A (en) | Vision-based power transmission line identification and foreign matter invasion online detection method | |
| CN111259704B (en) | Training method of dotted lane line endpoint detection model | |
| KR102270834B1 (en) | Method and system for recognizing marine object using hyperspectral data | |
| CN111353496B (en) | Real-time detection method for infrared dim targets | |
| CN106462975A (en) | Method and apparatus for object tracking and segmentation via background tracking | |
| CN113255590A (en) | Defect detection model training method, defect detection method, device and system | |
| Luo et al. | Stationary vehicle detection in aerial surveillance with a UAV | |
| CN114926456A (en) | Rail foreign matter detection method based on semi-automatic labeling and improved deep learning | |
| CN113284144A (en) | Tunnel detection method and device based on unmanned aerial vehicle | |
| CN107274374B (en) | Smoke monitoring method based on computer vision technology | |
| Funde et al. | Object detection and tracking approaches for video surveillance over camera network | |
| CN115761647A (en) | Computer vision-based method and system for detecting motion trail of object in transformer substation | |
| CN112069997B (en) | Unmanned aerial vehicle autonomous landing target extraction method and device based on DenseHR-Net | |
| CN111798516B (en) | Method for detecting running state quantity and analyzing errors of bridge crane equipment | |
| CN113487559A (en) | Method for detecting opening and closing states of disconnecting switch of transformer substation in real time | |
| CN112883836A (en) | Video detection method for deformation of underground coal mine roadway | |
| CN111667507A (en) | Method for tracking vehicle track on highway |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |