CN112229845A - Unmanned aerial vehicle high-precision winding tower intelligent inspection method based on visual navigation technology - Google Patents
Unmanned aerial vehicle high-precision winding tower intelligent inspection method based on visual navigation technology Download PDFInfo
- Publication number
- CN112229845A CN112229845A CN202011083101.1A CN202011083101A CN112229845A CN 112229845 A CN112229845 A CN 112229845A CN 202011083101 A CN202011083101 A CN 202011083101A CN 112229845 A CN112229845 A CN 112229845A
- Authority
- CN
- China
- Prior art keywords
- tower
- aerial vehicle
- unmanned aerial
- identification model
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000005516 engineering process Methods 0.000 title claims abstract description 14
- 230000000007 visual effect Effects 0.000 title claims abstract description 13
- 238000004804 winding Methods 0.000 title description 2
- 230000007547 defect Effects 0.000 claims abstract description 49
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 14
- 238000012360 testing method Methods 0.000 claims abstract description 13
- 238000001514 detection method Methods 0.000 claims abstract description 9
- 238000012549 training Methods 0.000 claims abstract description 8
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 4
- 239000012212 insulator Substances 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 241001155433 Centrarchus macropterus Species 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 125000006850 spacer group Chemical group 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C39/00—Aircraft not otherwise provided for
- B64C39/02—Aircraft not otherwise provided for characterised by special use
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02G—INSTALLATION OF ELECTRIC CABLES OR LINES, OR OF COMBINED OPTICAL AND ELECTRIC CABLES OR LINES
- H02G1/00—Methods or apparatus specially adapted for installing, maintaining, repairing or dismantling electric cables or lines
- H02G1/02—Methods or apparatus specially adapted for installing, maintaining, repairing or dismantling electric cables or lines for overhead lines or cables
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Remote Sensing (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Radar, Positioning & Navigation (AREA)
- Immunology (AREA)
- Biochemistry (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Aviation & Aerospace Engineering (AREA)
- Astronomy & Astrophysics (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses an unmanned aerial vehicle high-precision tower intelligent inspection method based on a visual navigation technology, which comprises the following steps: establishing a linear tower and non-linear tower type database and an important part image data set; randomly dividing an image data set of an important part into a sample data set and a test data set; establishing a convolutional neural network model based on a yolo v4 target detection algorithm, training the convolutional neural network model by using a sample data set, and respectively establishing an inspection image front-end component identification model and an inspection image defect identification model; generating an automatic flight route of the unmanned aerial vehicle through the point cloud and the unmanned aerial vehicle control device; the unmanned aerial vehicle flies according to the automatic flight route, the type of each tower is identified, and important parts on the towers are photographed by using the inspection image front-end part identification model until the flight route is finished; and the inspection image defect identification model identifies the defects of the shot image. The invention can obviously improve the inspection efficiency.
Description
Technical Field
The invention belongs to the technical field of power grid equipment state detection, and particularly relates to an unmanned aerial vehicle high-precision pylon intelligent inspection method based on a visual navigation technology.
Background
In recent years, unmanned aerial vehicle inspection becomes an important inspection means of power transmission lines, and inspection benefit and quality are remarkably improved compared with traditional manual inspection. The intelligent inspection mode represented by the unmanned aerial vehicle is good in autonomy and high in quality, and the inspection safety and reliability are greatly improved. The unmanned aerial vehicle and the inspection robot are used for replacing the traditional manual inspection mode, and the development trend is achieved.
The problems of low autonomous level, low fault identification precision, short endurance mileage and the like of the line patrol unmanned aerial vehicle seriously restrict the popularization and application of the unmanned aerial vehicle. The unmanned aerial vehicle line patrol technology is widely applied to line patrol operation of a power transmission line, but line patrol videos and images are checked manually in the actual operation process, so that the unmanned aerial vehicle line patrol system is large in labor workload, low in monitoring efficiency and untimely in police control, and timeliness, effectiveness and comprehensiveness of unmanned aerial vehicle line patrol results cannot be guaranteed.
Unmanned aerial vehicle transmission line patrols and examines mainly for artifical manual control unmanned aerial vehicle carries out the operation, and the flier is subaerial manual control unmanned aerial vehicle, closely shoots the photo image of gathering each position of shaft tower, and the collection process is chaotic, causes easily to miss the phenomenon of clapping, the mistake claps, patrols and examines efficiency and quality and rely on the technique and the responsibility heart of flier completely. The unmanned aerial vehicle has low intelligent degree, high requirement on the professional level of a flyer, short endurance mileage and the like, and the popularization and the application of the unmanned aerial vehicle are seriously restricted.
Disclosure of Invention
The invention provides a high-precision tower-around intelligent inspection method of an unmanned aerial vehicle based on a visual navigation technology, which aims at the problems that the intelligent degree of the unmanned aerial vehicle is low and the image acquisition cannot be realized accurately with high quality, and effectively solves the problem that the unmanned aerial vehicle inspects the high-quality and high-precision image acquisition.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
an unmanned aerial vehicle high-precision tower intelligent inspection method based on a visual navigation technology comprises the following steps:
s1, establishing a linear tower and non-linear tower type database, collecting standard images and defect images of all important parts, and marking the important parts in the standard images and the defect images by target frames to form an important part image data set;
s2, processing the images in the important part image data set established in the step S1, and randomly dividing the important part image data set into a sample data set and a test data set;
s3, establishing a convolutional neural network model based on a yolo v4 target detection algorithm, and training the convolutional neural network model by using the sample data set in the step S2 to respectively establish an inspection image front-end component identification model and an inspection image defect identification model;
s4, generating an automatic flight path of the unmanned aerial vehicle through the point cloud and the unmanned aerial vehicle control device;
s5, the unmanned aerial vehicle flies according to the automatic flight route generated in the step S4, the type of each tower is identified according to the type database of the linear tower and the non-linear tower in the step S1 at each tower, and then the inspection image front-end part identification model established in the step S3 is used for shooting images of all important parts on all towers until the flight route is finished;
and S6, utilizing the inspection image defect identification model established in the step S3 to perform defect identification on the image shot in the step S5 and outputting a defect identification image.
In step S1, the database of the types of the linear tower and the non-linear tower includes images of the linear tower and the non-linear tower, and a shooting route of an important component on the linear tower; the important parts comprise insulator strings, anti-vibration hammers, spacing rods and grading rings.
The step S3 includes the following steps:
s3.1, setting parameters through a parameter configuration file, wherein the parameters comprise an initial learning rate, a learning step length, image learning times and a discrimination precision threshold;
s3.2, selecting a yolo v4 single-stage target detection algorithm, and initializing a convolutional neural network model by using a parameter configuration file;
s3.3, performing iterative training on the convolutional neural network model by using the sample data set in the step S2 to respectively establish an inspection image front-end component identification model and an inspection image defect identification model;
s3.4, respectively carrying out model precision test on the inspection image front-end component identification model and the inspection image defect identification model by using the test data set, and carrying out overlapping degree calculation on a target frame of a judgment result of the inspection image front-end component identification model and the inspection image defect identification model and a target frame in the test data set to finally obtain the average accuracy of the models;
and S3.5, comparing the average accuracy rates of the models calculated in the step S3.4 with the discrimination accuracy threshold set in the step S3.1 respectively, executing the step S4 if the average accuracy rates of the models are greater than the discrimination accuracy threshold, otherwise, returning to the step S3.3.
The calculation formula of the model average accuracy rate is as follows:
in the formula, AP represents the average accuracy of the model, P represents the accuracy, and r represents the recall rate.
The step S4 includes the following steps:
s4.1, controlling the unmanned aerial vehicle by the unmanned aerial vehicle control device, taking pictures of tower images of all towers by the unmanned aerial vehicle, correspondingly recording position data of the towers, and generating an LAS point cloud by taking pictures at multiple angles;
the location data includes longitude, latitude, and altitude.
S4.2, judging whether the tower is a tangent tower or not according to the LAS point cloud generated in the step S4.1, if so, calling the type database of the straight tower and the non-straight tower in the step S1 to generate a tangent tower flight route, and if not, manually dotting by using an unmanned aerial vehicle control device, and simultaneously recording position data of a manual dotting position to generate a non-tangent tower flight route;
and S4.3, generating an automatic flight path according to the position data of the tower in the step S4.1 and the linear tower flight path and the non-linear tower flight path in the step S4.2.
The method for identifying the defects by the inspection image defect identification model comprises the following steps:
s6.1, detecting the input images by the inspection image defect identification model, and acquiring at least 3 scales of characteristic maps corresponding to each image according to the images;
s6.2, the inspection image defect identification model predicts a target window of each feature map, and removes redundant windows in the target window by using a non-maximum suppression algorithm;
and S6.3, identifying the target window obtained in the step S6.2 after the redundant window is removed by using a defect identification model of the inspection image, and outputting a corresponding defect identification image.
The invention has the beneficial effects that: the intelligent level and the inspection efficiency of the inspection process are obviously improved, and the method is a basis for realizing one-key inspection; the inspection work specification of the unmanned aerial vehicle is standardized, the randomness and the workload of manually operating the unmanned aerial vehicle to shoot pictures are reduced, and the intelligent inspection of the unmanned aerial vehicle is realized; the safety and the efficiency of unmanned aerial vehicle inspection are improved, and manpower, material resources and financial resources are greatly saved. Compare with traditional manually operation unmanned aerial vehicle mode of patrolling and examining, greatly improved and patrolled and examined operation autonomy, automation and intelligent level around the tower, made unmanned aerial vehicle patrol and examine the operation security higher, efficiency is higher, easy popularization nature is stronger around the tower, alleviates fortune and examines personnel intensity of labour, reduces fortune dimension cost by a wide margin. The method can accurately capture and high-precision snapshot key targets, avoid missed or wrong snapshots caused by the problems of disturbance, focusing and the like of the airplane due to environmental factors, ensure the image quality, remarkably improve the automatic operation level of national network patrol inspection, and has important significance for obtaining higher economic, safety and social benefits for power grid management.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of the present invention.
FIG. 2 is a P-R graph.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
To facilitate a better understanding of this patent, a brief description of yolo v4s is now provided:
the Yolo v4 provides a CSPDarkNet53 feature extraction network and a feature enhancement network SPP in a main network part, and the CSPDarkNet53 feature extraction network is used for solving the problem that a large amount of reasoning calculation is needed in the traditional network structure. In the feature fusion part, PANet replaces FPN in YOLOv3 to be used as a parameter aggregation method, parameter aggregation is carried out from different trunk layers aiming at different detector levels, the original PANet method is modified, and tensor connection (concat) is used for replacing original shortcut connection (shortcut connection); the SPP structure is doped, after the last feature layer of CSPdark 53 is subjected to three activation convolutions, the SPP structure is processed by utilizing the maximum pooling of four different scales respectively, so that the receptive field can be greatly increased, and the most significant contextual features can be separated. The maximum pooled nuclei sizes were 13x13, 9x9, 5x5, 1x1, respectively.
Yolo v4 can be applied to a fast target detection system in a real working environment, and can be optimized in parallel without deliberately pursuing a theoretically low computational load (BFLOP). At the same time, the yolo v4 algorithm can be easily trained and can yield a better result.
An unmanned aerial vehicle high-precision tower intelligent inspection method based on a visual navigation technology is disclosed, and as shown in figure 1, the method comprises the following steps:
and S1, establishing a linear tower and non-linear tower type database, collecting standard images and all defect images of important parts, and marking all the important parts in the standard images and the defect images by target frames to form an important part image data set.
The linear tower and non-linear tower type database comprises images of the linear tower and the non-linear tower and shooting routes of important parts on the linear tower; the important parts comprise insulator strings, vibration dampers, spacers and grading rings, and the target frame marks indicate the outlines of the important parts, so that the important parts can be recognized quickly during later-stage model training. When the unmanned aerial vehicle flies to a certain tower, the tower is photographed, the tower type is accurately identified by comparing the photographed tower image with the tower type database, and then important parts on the tower are photographed according to the tower type.
And S2, processing the images in the important part image data set established in the step S1, and randomly dividing the important part image data set into a sample data set and a test data set.
The processing of the image in the important part image data set refers to the normalization of the image to obtain an image sample with a size of M × M.
S3, establishing a convolutional neural network model based on a yolo v4 target detection algorithm, training the convolutional neural network model by using the sample data set in the step S2, and respectively establishing an inspection image front-end component recognition model and an inspection image defect recognition model, wherein the method comprises the following steps:
s3.1, setting parameters through a parameter configuration file, wherein the parameters comprise an initial learning rate, a learning step length, image learning times and a discrimination precision threshold;
s3.2, selecting a yolo v4 single-stage target detection algorithm, and initializing a convolutional neural network model by using a parameter configuration file;
s3.3, performing iterative training on the convolutional neural network model by using the sample data set in the step S2 to respectively establish an inspection image front-end component identification model and an inspection image defect identification model;
s3.4, respectively carrying out model precision test on the inspection image front-end component identification model and the inspection image defect identification model by using the test data set, and respectively carrying out overlapping degree calculation on a target frame of a judgment result of the two models and a real target frame in the test data set to finally obtain the average accuracy of the models corresponding to the two models;
the calculation formula of the model average accuracy rate is as follows:
in the formula, AP represents the average accuracy of the models, P represents the accuracy, the false alarm condition output by the two models can be measured, r represents the recall rate, and the practical level of the two models can be measured;
the calculation formula of the accuracy P is as follows:
P＝TP/(TP+FP)；
in the formula, TP represents the number of correctly classified defect targets, FP represents the number of mistakenly considered defect targets by background interference;
the calculation formula of the recall rate is as follows:
P＝TP/(TP+FN)；
in the formula, FN represents the number of erroneous divisions of the defect object into the background.
The average model accuracy AP can measure the comprehensive performance of the two models and is obtained by calculating the area under a P-R curve formed by the recall rate R and the accuracy P.
And S3.5, comparing the average accuracy rates of the models calculated in the step S3.4 with the discrimination accuracy threshold set in the step S3.1 respectively, executing the step S4 if the average accuracy rates of the models are greater than the discrimination accuracy threshold, otherwise, returning to the step S3.3.
S4, generating an automatic flight path of the unmanned aerial vehicle through the point cloud and the unmanned aerial vehicle control device, comprising the following steps:
s4.1, controlling the unmanned aerial vehicle by the unmanned aerial vehicle control device, taking pictures of all towers by using a camera device on the unmanned aerial vehicle, correspondingly recording position data of the towers, and generating an LAS point cloud by taking pictures at multiple angles;
the location data includes longitude, latitude, and altitude.
S4.2, judging whether the tower is a tangent tower or not according to the LAS point cloud generated in the step S4.1, if so, calling the type database of the straight tower and the non-straight tower in the step S1 to generate a tangent tower flight route, and if not, manually dotting by using an unmanned aerial vehicle control device, and simultaneously recording position data of a manual dotting position to generate a non-tangent tower flight route;
the manual dotting means that a flyer manually controls the unmanned aerial vehicle to fly by using a remote controller, and sets a patrol and examine shooting viewpoint, namely a navigation point, namely the hovering position of the unmanned aerial vehicle, so that images of important parts can be conveniently shot.
And S4.3, generating an automatic flight path according to the position data of the tower, the linear tower flight path and the non-linear tower flight path in the step S4.1.
And S5, the unmanned aerial vehicle flies according to the automatic flight route generated in the step S4, the type of each tower is identified by using the linear tower and non-linear tower type database in the step S1 at each tower, and then the inspection image front-end part identification model established in the step S3 is used for carrying out accurate image shooting on important parts on the towers until the flight route is finished.
After the unmanned aerial vehicle identifies the pole tower, the unmanned aerial vehicle flies to each navigation point and hovers, important parts on the pole tower are identified through the inspection image front end part identification model, accurate photographing is achieved by utilizing the inspection image front end part identification model, and after all important parts on the pole tower are photographed by the unmanned aerial vehicle, the unmanned aerial vehicle flies to the next pole tower to be identified and photographed until all flight routes are flown.
And S6, utilizing the inspection image defect identification model established in the step S3 to identify defects of all the images shot in the step S5 and outputting a defect identification image.
The method for identifying the defects by the inspection image defect identification model comprises the following steps:
s6.1, detecting the input images by the inspection image defect identification model, and acquiring at least 3 scales of characteristic maps corresponding to each image according to the images;
the feature maps of the 3 scales are respectively as follows: eat1- > (256 × 52), feat2- > (512 × 26), feat3- > (1024 × 13);
s6.2, predicting a target window of each feature map, and removing redundant windows in the target window by using a non-maximum suppression algorithm;
the method for predicting the target window of each feature map comprises the following steps: scaling the feature map, dividing the feature map into S-S grids, and predicting B frames for each grid, so that one feature map corresponds to S-B target windows.
And S6.3, identifying the target window obtained in the step S6.2 after the redundant window is removed by using a defect identification model of the inspection image, and outputting a corresponding defect identification image.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (6)
1. An unmanned aerial vehicle high-precision tower intelligent inspection method based on a visual navigation technology is characterized by comprising the following steps:
s1, establishing a linear tower and non-linear tower type database, collecting standard images and defect images of all important parts, and marking the important parts in the standard images and the defect images by target frames to form an important part image data set;
s2, processing the images in the important part image data set established in the step S1, and randomly dividing the important part image data set into a sample data set and a test data set;
s3, establishing a convolutional neural network model based on a yolo v4 target detection algorithm, and training the convolutional neural network model by using the sample data set in the step S2 to respectively establish an inspection image front-end component identification model and an inspection image defect identification model;
s4, generating an automatic flight path of the unmanned aerial vehicle through the point cloud and the unmanned aerial vehicle control device;
s5, the unmanned aerial vehicle flies according to the automatic flight route generated in the step S4, the type of each tower is identified according to the type database of the linear tower and the non-linear tower in the step S1 at each tower, and then the inspection image front-end part identification model established in the step S3 is used for shooting images of all important parts on all towers until the flight route is finished;
and S6, utilizing the inspection image defect identification model established in the step S3 to perform defect identification on the image shot in the step S5 and outputting a defect identification image.
2. The unmanned aerial vehicle high-precision intelligent patrol inspection method based on the visual navigation technology as claimed in claim 1, wherein in step S1, the database of the types of the linear towers and the non-linear towers comprises images of the linear towers and the non-linear towers and shooting routes of important parts on the linear towers; the important parts comprise insulator strings, anti-vibration hammers, spacing rods and grading rings.
3. The unmanned aerial vehicle high-precision pylon intelligent inspection method based on the visual navigation technology as claimed in claim 1 or 2, wherein the step S3 comprises the following steps:
s3.1, setting parameters through a parameter configuration file, wherein the parameters comprise an initial learning rate, a learning step length, image learning times and a discrimination precision threshold;
s3.2, selecting a yolo v4 single-stage target detection algorithm, and initializing a convolutional neural network model by using a parameter configuration file;
s3.3, performing iterative training on the convolutional neural network model by using the sample data set in the step S2 to respectively establish an inspection image front-end component identification model and an inspection image defect identification model;
s3.4, respectively carrying out model precision test on the inspection image front-end component identification model and the inspection image defect identification model by using the test data set, and carrying out overlapping degree calculation on a target frame of a judgment result of the inspection image front-end component identification model and the inspection image defect identification model and a target frame in the test data set to finally obtain the average accuracy of the models;
and S3.5, comparing the average accuracy rates of the models calculated in the step S3.4 with the discrimination accuracy threshold set in the step S3.1 respectively, executing the step S4 if the average accuracy rates of the models are greater than the discrimination accuracy threshold, otherwise, returning to the step S3.3.
4. The unmanned aerial vehicle high-precision pylon intelligent inspection method based on the visual navigation technology as claimed in claim 3, wherein the calculation formula of the model average accuracy rate is as follows:
in the formula, AP represents the average accuracy of the model, P represents the accuracy, and r represents the recall rate.
5. The unmanned aerial vehicle high-precision pylon intelligent inspection method based on the visual navigation technology as claimed in claim 3, wherein the step S4 includes the following steps:
s4.1, controlling the unmanned aerial vehicle by the unmanned aerial vehicle control device, taking pictures of tower images of all towers by the unmanned aerial vehicle, correspondingly recording position data of the towers, and generating an LAS point cloud by taking pictures at multiple angles;
the location data includes longitude, latitude, and altitude.
S4.2, judging whether the tower is a tangent tower or not according to the LAS point cloud generated in the step S4.1, if so, calling the type database of the straight tower and the non-straight tower in the step S1 to generate a tangent tower flight route, and if not, manually dotting by using an unmanned aerial vehicle control device, and simultaneously recording position data of a manual dotting position to generate a non-tangent tower flight route;
and S4.3, generating an automatic flight path according to the position data of the tower in the step S4.1 and the linear tower flight path and the non-linear tower flight path in the step S4.2.
6. The unmanned aerial vehicle high-precision pylon intelligent inspection method based on the visual navigation technology, according to claim 1, wherein the method for identifying defects by the inspection image defect identification model is as follows:
s6.1, detecting the input images by the inspection image defect identification model, and acquiring at least 3 scales of characteristic maps corresponding to each image according to the images;
s6.2, the inspection image defect identification model predicts a target window of each feature map, and removes redundant windows in the target window by using a non-maximum suppression algorithm;
and S6.3, identifying the target window obtained in the step S6.2 after the redundant window is removed by using a defect identification model of the inspection image, and outputting a corresponding defect identification image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011083101.1A CN112229845A (en) | 2020-10-12 | 2020-10-12 | Unmanned aerial vehicle high-precision winding tower intelligent inspection method based on visual navigation technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011083101.1A CN112229845A (en) | 2020-10-12 | 2020-10-12 | Unmanned aerial vehicle high-precision winding tower intelligent inspection method based on visual navigation technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112229845A true CN112229845A (en) | 2021-01-15 |
Family
ID=74112077
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011083101.1A Pending CN112229845A (en) | 2020-10-12 | 2020-10-12 | Unmanned aerial vehicle high-precision winding tower intelligent inspection method based on visual navigation technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112229845A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113095253A (en) * | 2021-04-20 | 2021-07-09 | 池州学院 | Insulator detection method for unmanned aerial vehicle to inspect power transmission line |
| CN113191389A (en) * | 2021-03-31 | 2021-07-30 | 中国石油大学(华东) | Submarine pipeline autonomous inspection method and device based on optical vision technology |
| CN113296537A (en) * | 2021-05-25 | 2021-08-24 | 湖南博瑞通航航空技术有限公司 | Electric power unmanned aerial vehicle inspection method and system based on electric power tower model matching |
| CN113298035A (en) * | 2021-06-17 | 2021-08-24 | 上海红檀智能科技有限公司 | Unmanned aerial vehicle electric power tower detection and autonomous cruise method based on image recognition |
| CN114049573A (en) * | 2021-11-09 | 2022-02-15 | 上海建工四建集团有限公司 | Method for supervising safety construction of village under construction residence |
| CN114397306A (en) * | 2022-03-25 | 2022-04-26 | 南方电网数字电网研究院有限公司 | Power grid grading ring hypercomplex category defect multi-stage model joint detection method |
| CN115953486A (en) * | 2022-12-30 | 2023-04-11 | 国网电力空间技术有限公司 | Automatic coding method for direct-current T-shaped tangent tower component inspection image |
Citations (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295655A (en) * | 2016-08-03 | 2017-01-04 | 国网山东省电力公司电力科学研究院 | A kind of transmission line part extraction method patrolling and examining image for unmanned plane |
| CN107480730A (en) * | 2017-09-05 | 2017-12-15 | 广州供电局有限公司 | Power equipment identification model construction method and system, the recognition methods of power equipment |
| CN107729808A (en) * | 2017-09-08 | 2018-02-23 | 国网山东省电力公司电力科学研究院 | A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection |
| CN107944412A (en) * | 2017-12-04 | 2018-04-20 | 国网山东省电力公司电力科学研究院 | Transmission line of electricity automatic recognition system and method based on multilayer convolutional neural networks |
| CN108037133A (en) * | 2017-12-27 | 2018-05-15 | 武汉市智勤创亿信息技术股份有限公司 | A kind of power equipments defect intelligent identification Method and its system based on unmanned plane inspection image |
| CN108365557A (en) * | 2018-02-24 | 2018-08-03 | 广东电网有限责任公司肇庆供电局 | A kind of method and system of unmanned plane fining inspection transmission line of electricity |
| CN108597053A (en) * | 2018-04-25 | 2018-09-28 | 北京御航智能科技有限公司 | Shaft tower and channel targets identification based on image data and neural network and defect diagnostic method |
| WO2018195955A1 (en) * | 2017-04-28 | 2018-11-01 | 深圳市大疆创新科技有限公司 | Aircraft-based facility detection method and control device |
| CN109671174A (en) * | 2018-12-20 | 2019-04-23 | 北京中飞艾维航空科技有限公司 | A kind of pylon method for inspecting and device |
| CN109978820A (en) * | 2019-01-31 | 2019-07-05 | 广州中科云图智能科技有限公司 | Unmanned plane course line acquisition methods, system and equipment based on laser point cloud |
| CN110133440A (en) * | 2019-05-27 | 2019-08-16 | 国电南瑞科技股份有限公司 | Electric power unmanned plane and method for inspecting based on Tower Model matching and vision guided navigation |
| CN110413003A (en) * | 2019-07-31 | 2019-11-05 | 广东电网有限责任公司 | Inspection method, device, equipment and the computer readable storage medium of transmission line of electricity |
| CN110647813A (en) * | 2019-08-21 | 2020-01-03 | 成都携恩科技有限公司 | Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography |
| CN110689531A (en) * | 2019-09-23 | 2020-01-14 | 云南电网有限责任公司电力科学研究院 | Automatic power transmission line machine inspection image defect identification method based on yolo |
| CN110687904A (en) * | 2019-12-09 | 2020-01-14 | 广东科凯达智能机器人有限公司 | Visual navigation routing inspection and obstacle avoidance method for inspection robot |
| CN110705397A (en) * | 2019-09-19 | 2020-01-17 | 东南大学 | YOLOv3 pruning identification insulator defect method suitable for small field sample amount |
| CN110703800A (en) * | 2019-10-29 | 2020-01-17 | 国网江苏省电力有限公司泰州供电分公司 | Unmanned aerial vehicle-based intelligent identification method and system for electric power facilities |
| CN110727288A (en) * | 2019-11-13 | 2020-01-24 | 昆明能讯科技有限责任公司 | Point cloud-based accurate three-dimensional route planning method for power inspection |
| CN110796107A (en) * | 2019-11-04 | 2020-02-14 | 南京北旨智能科技有限公司 | Power inspection image defect identification method and system and power inspection unmanned aerial vehicle |
| CN110827251A (en) * | 2019-10-30 | 2020-02-21 | 江苏方天电力技术有限公司 | Power transmission line locking pin defect detection method based on aerial image |
| CN111080634A (en) * | 2019-12-23 | 2020-04-28 | 北京新松融通机器人科技有限公司 | Transformer appearance defect identification method based on inspection robot and Cascade RCNN algorithm |
| CN111157530A (en) * | 2019-12-25 | 2020-05-15 | 国网宁夏电力有限公司电力科学研究院 | Unmanned aerial vehicle-based safety detection method for power transmission line |
| CN111179249A (en) * | 2019-12-30 | 2020-05-19 | 南京南瑞信息通信科技有限公司 | Power equipment detection method and device based on deep convolutional neural network |
| CN111311570A (en) * | 2020-02-12 | 2020-06-19 | 江苏方天电力技术有限公司 | Transmission line key device defect identification method based on unmanned aerial vehicle inspection |
| CN111311569A (en) * | 2020-02-12 | 2020-06-19 | 江苏方天电力技术有限公司 | Pole tower defect identification method based on unmanned aerial vehicle inspection |
| CN111353413A (en) * | 2020-02-25 | 2020-06-30 | 武汉大学 | Low-missing-report-rate defect identification method for power transmission equipment |
| CN111476756A (en) * | 2020-03-09 | 2020-07-31 | 重庆大学 | Method for identifying casting DR image loose defects based on improved YO L Ov3 network model |
| CN111489339A (en) * | 2020-04-08 | 2020-08-04 | 北京交通大学 | Method for detecting defects of bolt spare nuts of high-speed railway positioner |
| CN111723774A (en) * | 2020-07-03 | 2020-09-29 | 国网山东省电力公司日照供电公司 | Target identification method for power transmission equipment based on unmanned aerial vehicle inspection |
-
2020
- 2020-10-12 CN CN202011083101.1A patent/CN112229845A/en active Pending
Patent Citations (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295655A (en) * | 2016-08-03 | 2017-01-04 | 国网山东省电力公司电力科学研究院 | A kind of transmission line part extraction method patrolling and examining image for unmanned plane |
| WO2018195955A1 (en) * | 2017-04-28 | 2018-11-01 | 深圳市大疆创新科技有限公司 | Aircraft-based facility detection method and control device |
| CN107480730A (en) * | 2017-09-05 | 2017-12-15 | 广州供电局有限公司 | Power equipment identification model construction method and system, the recognition methods of power equipment |
| CN107729808A (en) * | 2017-09-08 | 2018-02-23 | 国网山东省电力公司电力科学研究院 | A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection |
| CN107944412A (en) * | 2017-12-04 | 2018-04-20 | 国网山东省电力公司电力科学研究院 | Transmission line of electricity automatic recognition system and method based on multilayer convolutional neural networks |
| CN108037133A (en) * | 2017-12-27 | 2018-05-15 | 武汉市智勤创亿信息技术股份有限公司 | A kind of power equipments defect intelligent identification Method and its system based on unmanned plane inspection image |
| CN108365557A (en) * | 2018-02-24 | 2018-08-03 | 广东电网有限责任公司肇庆供电局 | A kind of method and system of unmanned plane fining inspection transmission line of electricity |
| CN108597053A (en) * | 2018-04-25 | 2018-09-28 | 北京御航智能科技有限公司 | Shaft tower and channel targets identification based on image data and neural network and defect diagnostic method |
| CN109671174A (en) * | 2018-12-20 | 2019-04-23 | 北京中飞艾维航空科技有限公司 | A kind of pylon method for inspecting and device |
| CN109978820A (en) * | 2019-01-31 | 2019-07-05 | 广州中科云图智能科技有限公司 | Unmanned plane course line acquisition methods, system and equipment based on laser point cloud |
| CN110133440A (en) * | 2019-05-27 | 2019-08-16 | 国电南瑞科技股份有限公司 | Electric power unmanned plane and method for inspecting based on Tower Model matching and vision guided navigation |
| CN110413003A (en) * | 2019-07-31 | 2019-11-05 | 广东电网有限责任公司 | Inspection method, device, equipment and the computer readable storage medium of transmission line of electricity |
| CN110647813A (en) * | 2019-08-21 | 2020-01-03 | 成都携恩科技有限公司 | Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography |
| CN110705397A (en) * | 2019-09-19 | 2020-01-17 | 东南大学 | YOLOv3 pruning identification insulator defect method suitable for small field sample amount |
| CN110689531A (en) * | 2019-09-23 | 2020-01-14 | 云南电网有限责任公司电力科学研究院 | Automatic power transmission line machine inspection image defect identification method based on yolo |
| CN110703800A (en) * | 2019-10-29 | 2020-01-17 | 国网江苏省电力有限公司泰州供电分公司 | Unmanned aerial vehicle-based intelligent identification method and system for electric power facilities |
| CN110827251A (en) * | 2019-10-30 | 2020-02-21 | 江苏方天电力技术有限公司 | Power transmission line locking pin defect detection method based on aerial image |
| CN110796107A (en) * | 2019-11-04 | 2020-02-14 | 南京北旨智能科技有限公司 | Power inspection image defect identification method and system and power inspection unmanned aerial vehicle |
| CN110727288A (en) * | 2019-11-13 | 2020-01-24 | 昆明能讯科技有限责任公司 | Point cloud-based accurate three-dimensional route planning method for power inspection |
| CN110687904A (en) * | 2019-12-09 | 2020-01-14 | 广东科凯达智能机器人有限公司 | Visual navigation routing inspection and obstacle avoidance method for inspection robot |
| CN111080634A (en) * | 2019-12-23 | 2020-04-28 | 北京新松融通机器人科技有限公司 | Transformer appearance defect identification method based on inspection robot and Cascade RCNN algorithm |
| CN111157530A (en) * | 2019-12-25 | 2020-05-15 | 国网宁夏电力有限公司电力科学研究院 | Unmanned aerial vehicle-based safety detection method for power transmission line |
| CN111179249A (en) * | 2019-12-30 | 2020-05-19 | 南京南瑞信息通信科技有限公司 | Power equipment detection method and device based on deep convolutional neural network |
| CN111311570A (en) * | 2020-02-12 | 2020-06-19 | 江苏方天电力技术有限公司 | Transmission line key device defect identification method based on unmanned aerial vehicle inspection |
| CN111311569A (en) * | 2020-02-12 | 2020-06-19 | 江苏方天电力技术有限公司 | Pole tower defect identification method based on unmanned aerial vehicle inspection |
| CN111353413A (en) * | 2020-02-25 | 2020-06-30 | 武汉大学 | Low-missing-report-rate defect identification method for power transmission equipment |
| CN111476756A (en) * | 2020-03-09 | 2020-07-31 | 重庆大学 | Method for identifying casting DR image loose defects based on improved YO L Ov3 network model |
| CN111489339A (en) * | 2020-04-08 | 2020-08-04 | 北京交通大学 | Method for detecting defects of bolt spare nuts of high-speed railway positioner |
| CN111723774A (en) * | 2020-07-03 | 2020-09-29 | 国网山东省电力公司日照供电公司 | Target identification method for power transmission equipment based on unmanned aerial vehicle inspection |
Non-Patent Citations (2)
| Title |
|---|
| LIU XINYUN ET AL.: "Insulator Detection in Aerial Images Based on Faster Regions with Convolutional Neural Network", 《2018 IEEE 14TH INTERNATIONAL CONFERENCE ON CONTROL AND AUTOMATION (ICCA)》 * |
| LIU XINYUN ET AL.: "Insulator Detection in Aerial Images Based on Faster Regions with Convolutional Neural Network", 《2018 IEEE 14TH INTERNATIONAL CONFERENCE ON CONTROL AND AUTOMATION (ICCA)》, 23 August 2018 (2018-08-23), pages 1082 - 1086 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113191389A (en) * | 2021-03-31 | 2021-07-30 | 中国石油大学(华东) | Submarine pipeline autonomous inspection method and device based on optical vision technology |
| CN113191389B (en) * | 2021-03-31 | 2022-10-11 | 中国石油大学(华东) | Submarine pipeline autonomous inspection method and device based on optical vision technology |
| CN113095253A (en) * | 2021-04-20 | 2021-07-09 | 池州学院 | Insulator detection method for unmanned aerial vehicle to inspect power transmission line |
| CN113095253B (en) * | 2021-04-20 | 2023-05-23 | 池州学院 | Insulator detection method for unmanned aerial vehicle inspection transmission line |
| CN113296537A (en) * | 2021-05-25 | 2021-08-24 | 湖南博瑞通航航空技术有限公司 | Electric power unmanned aerial vehicle inspection method and system based on electric power tower model matching |
| CN113296537B (en) * | 2021-05-25 | 2024-03-12 | 湖南博瑞通航航空技术有限公司 | Electric power unmanned aerial vehicle inspection method and system based on electric power pole tower model matching |
| CN113298035A (en) * | 2021-06-17 | 2021-08-24 | 上海红檀智能科技有限公司 | Unmanned aerial vehicle electric power tower detection and autonomous cruise method based on image recognition |
| CN114049573A (en) * | 2021-11-09 | 2022-02-15 | 上海建工四建集团有限公司 | Method for supervising safety construction of village under construction residence |
| CN114049573B (en) * | 2021-11-09 | 2024-07-09 | 上海建工四建集团有限公司 | Safety construction supervision method for rural residential building |
| CN114397306A (en) * | 2022-03-25 | 2022-04-26 | 南方电网数字电网研究院有限公司 | Power grid grading ring hypercomplex category defect multi-stage model joint detection method |
| CN115953486A (en) * | 2022-12-30 | 2023-04-11 | 国网电力空间技术有限公司 | Automatic coding method for direct-current T-shaped tangent tower component inspection image |
| CN115953486B (en) * | 2022-12-30 | 2024-04-12 | 国网电力空间技术有限公司 | Automatic encoding method for inspection image of direct-current T-shaped tangent tower part |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112229845A (en) | Unmanned aerial vehicle high-precision winding tower intelligent inspection method based on visual navigation technology | |
| CN110084165B (en) | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation | |
| CN108037770B (en) | Unmanned aerial vehicle power transmission line inspection system and method based on artificial intelligence | |
| CN108416294B (en) | Fan blade fault intelligent identification method based on deep learning | |
| CN108022235B (en) | Method for identifying defects of key components of high-voltage transmission iron tower | |
| CN112906620B (en) | Unmanned aerial vehicle-assisted insulator fault detection method and device and electronic equipment | |
| CN112115927B (en) | Intelligent machine room equipment identification method and system based on deep learning | |
| CN106851229B (en) | Security and protection intelligent decision method and system based on image recognition | |
| CN111652835A (en) | Method for detecting insulator loss of power transmission line based on deep learning and clustering | |
| CN104239899A (en) | Electric transmission line spacer identification method for unmanned aerial vehicle inspection | |
| CN112950634A (en) | Method, equipment and system for identifying damage of wind turbine blade based on unmanned aerial vehicle routing inspection | |
| CN112349057A (en) | Deep learning-based indoor smoke and fire detection method | |
| CN116310891A (en) | Cloud-edge cooperative transmission line defect intelligent detection system and method | |
| CN116258980A (en) | Unmanned aerial vehicle distributed photovoltaic power station inspection method based on vision | |
| CN114035604A (en) | Video monitoring and unmanned aerial vehicle air-ground linkage abnormal target detection method | |
| Hao et al. | Detection of bird nests on power line patrol using single shot detector | |
| CN115171045A (en) | YOLO-based power grid operation field violation identification method and terminal | |
| CN112598666B (en) | Cable tunnel anomaly detection method based on convolutional neural network | |
| CN109614888B (en) | Deep learning defect detection model training method based on overhead transmission line defect auxiliary data set | |
| CN111241905A (en) | Power transmission line nest detection method based on improved SSD algorithm | |
| CN113610009A (en) | Flood disaster unmanned aerial vehicle image information extraction system | |
| CN117612044A (en) | Method for inspecting transmission line insulator by unmanned aerial vehicle in complex scene | |
| CN114596244A (en) | Infrared image identification method and system based on visual processing and multi-feature fusion | |
| CN116403162B (en) | Airport scene target behavior recognition method and system and electronic equipment | |
| CN117293697A (en) | Substation floater processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210115 |