CN111275015A - Unmanned aerial vehicle-based power line inspection electric tower detection and identification method and system - Google Patents
Unmanned aerial vehicle-based power line inspection electric tower detection and identification method and system Download PDFInfo
- Publication number
- CN111275015A CN111275015A CN202010130319.1A CN202010130319A CN111275015A CN 111275015 A CN111275015 A CN 111275015A CN 202010130319 A CN202010130319 A CN 202010130319A CN 111275015 A CN111275015 A CN 111275015A
- Authority
- CN
- China
- Prior art keywords
- tower
- aerial vehicle
- unmanned aerial
- neural network
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention discloses an unmanned aerial vehicle-based electric power line inspection electric tower detection and identification method and system, which are suitable for a pre-trained tower identification neural network model and comprise the following steps: controlling the unmanned aerial vehicle to automatically inspect according to a visual positioning method; collecting images in the unmanned aerial vehicle inspection process, generating synthetic data according to the images, and transmitting the synthetic data to a trained tower recognition neural network model; and the tower identification neural network model generates belief mapping based on the synthetic data, identifies the tower according to the belief mapping and outputs a final identification result. According to the invention, the pole tower in the image acquired in the unmanned aerial vehicle inspection process is automatically identified through the pole tower identification neural network model, and the unmanned aerial vehicle is controlled to automatically inspect through the visual positioning method, so that the whole process does not need human intervention, the detection precision is high, the labor cost is saved, and the working efficiency is greatly improved.
Description
Technical Field
The invention relates to the technical field of machine learning, in particular to a method and a system for detecting and identifying an electric power line patrol tower based on an unmanned aerial vehicle.
Background
At present, an unmanned aerial vehicle is widely applied to the technical field of power line patrol, and aims to realize autonomous positioning of the unmanned aerial vehicle during power line patrol and stable hovering operation around a transmission tower; therefore, detection and identification of the transmission tower are necessary. In terms of artificial intelligence, in order for a robot to work safely and efficiently with humans, the robot must be aware of the surrounding environment. One aspect of this awareness is the knowledge of the three-dimensional position and orientation of objects in the scene, commonly referred to as a 6 degree-of-freedom pose. This knowledge is important for performing simulated learning of object pick and place, transferring or observing someone handling the object from one person. The posture of the target object is estimated through deep learning, the three-dimensional postures of the disordered objects can be deduced from a single RGB image in real time, and the robot can operate the objects.
The existing method for detecting the transmission tower by the line patrol unmanned aerial vehicle mainly depends on GPS coordinate positioning of the unmanned aerial vehicle, the GPS information is utilized to determine the coordinate of the transmission tower, an operator identifies the electric wire tower from an image through an image acquired by the unmanned aerial vehicle, the operator is required to always control the unmanned aerial vehicle, the efficiency is low, and the situations of wrong identification and missed identification are easy to occur.
To sum up, when adopting unmanned aerial vehicle to patrol the transmission tower among the prior art, need artificially to control unmanned aerial vehicle and artificially discern the shaft tower, there is the technical problem of inefficiency.
Disclosure of Invention
The invention provides an unmanned aerial vehicle-based electric power line patrol tower detection and identification method and system, and solves the technical problem that in the prior art, when an unmanned aerial vehicle is adopted to patrol a transmission tower, the unmanned aerial vehicle needs to be manually operated and the tower needs to be manually identified, so that the efficiency is low.
The invention provides an unmanned aerial vehicle-based power line inspection electric tower detection and identification method, which is suitable for a pre-trained tower identification neural network model and comprises the following steps:
controlling the unmanned aerial vehicle to automatically inspect according to a visual positioning method;
collecting images in the unmanned aerial vehicle inspection process, generating synthetic data according to the images, and transmitting the synthetic data to a trained tower recognition neural network model;
and the tower identification neural network model generates belief mapping based on the synthetic data, identifies the tower according to the belief mapping, outputs a final identification result and records the current position of the unmanned aerial vehicle.
Preferably, the training process of the tower recognition neural network model is as follows:
applying a random illumination condition to the tower, and collecting each frame of image of each direction of the tower by adopting two cameras;
generating synthetic data of each frame of image based on the acquired image of each frame;
and inputting the synthetic data of each frame of image into the tower recognition neural network model, and training the tower recognition neural network model to obtain the trained tower recognition neural network model.
Preferably, the synthesized data of each frame image includes an RGB image, a depth map, a segmentation map, and a data markup file.
Preferably, the data marker file comprises the size of the collected picture, position coordinates and orientation matrix information of the double cameras in a world coordinate system, coordinate information of the tower, attitude information of the tower and size information of the tower.
Preferably, the tower recognition neural network model comprises a feature extraction layer, a first 3 × 3 convolutional layer, a second 3 × 3 convolutional layer, a belief mapping layer and a vector layer, wherein the feature extraction layer is a VGG-19 convolutional neural network, and each layer is connected through a relu function.
Preferably, the working principle of the tower recognition neural network model is as follows:
the VGG-19 convolutional neural network extracts 512-dimensional image features from the synthetic data;
the first 3 x 3 convolutional layer reduces the dimension of the image feature from 512 to 256;
the second 3 x 3 convolutional layer reduces the dimension of the image feature from 256 to 128;
and the belief mapping layer and the vector layer generate belief mapping from 128-dimensional image characteristics, and the tower is identified according to the belief mapping.
Preferably, the L2 loss function is used in belief maps as well as in vector fields.
Preferably, the principle of the visual localization method is as follows:
acquiring position information and attitude information of the unmanned aerial vehicle;
and solving the speed, the angular speed and the acceleration of the unmanned aerial vehicle according to the position information and the attitude information.
Preferably, the speed and the angular velocity of the unmanned aerial vehicle are solved by adopting a least square fitting method or a Kalman filtering method, and the acceleration is solved by carrying out difference on the speed.
A power line inspection electric tower detection and identification system based on an unmanned aerial vehicle comprises the unmanned aerial vehicle, a synthetic data generation module, a tower identification neural network model module, a tower identification module, an unmanned aerial vehicle control module and a positioning module; the unmanned aerial vehicle is provided with two cameras;
the unmanned aerial vehicle is used for collecting images in the inspection process and transmitting the images to the synthetic data generation module;
the synthetic data generation module is used for generating synthetic data according to the images acquired by the unmanned aerial vehicle and outputting the synthetic data to the pole and tower recognition neural network model module;
the pole and tower recognition neural network model module is used for providing a pole and tower recognition neural network model, and the pole and tower recognition neural network model generates belief mapping based on synthetic data;
the pole and tower identification module is used for identifying the pole and tower according to the belief mapping and outputting a final identification result;
the unmanned aerial vehicle control module is used for controlling the unmanned aerial vehicle to automatically inspect according to a visual positioning method;
the positioning module is used for recording the position information of the unmanned aerial vehicle.
According to the technical scheme, the embodiment of the invention has the following advantages:
according to the embodiment of the invention, the tower identification neural network model is used for automatically identifying the tower in the image acquired in the inspection process of the unmanned aerial vehicle, and the unmanned aerial vehicle is controlled to automatically inspect by the visual positioning method, so that the whole process is free from manual intervention, the detection precision is high, the labor cost is saved, the technical problem of low efficiency caused by manual operation of the unmanned aerial vehicle and manual identification of the tower when the unmanned aerial vehicle is used for inspecting the transmission tower in the prior art is solved, and the working efficiency is greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is also possible for those skilled in the art to obtain other drawings based on the drawings without inventive exercise.
Fig. 1 is a method flowchart of a method and a system for detecting and identifying an electric power line patrol pylon based on an unmanned aerial vehicle according to an embodiment of the present invention.
Fig. 2 is a system structure diagram of a power line patrol electric tower detection and identification method and system based on an unmanned aerial vehicle according to an embodiment of the present invention.
Fig. 3 is a flowchart of a machine vision module of the method and system for detecting and identifying the power line patrol pylon based on the unmanned aerial vehicle according to the embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method and a system for detecting and identifying a power line inspection tower based on an unmanned aerial vehicle, which are used for solving the technical problem of low efficiency caused by the fact that when the unmanned aerial vehicle is adopted to inspect the power transmission tower, the unmanned aerial vehicle needs to be manually operated and the tower needs to be manually identified.
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Referring to fig. 1, fig. 1 is a flowchart of a method and a system for detecting and identifying an electric power line patrol tower based on an unmanned aerial vehicle according to an embodiment of the present invention.
The method for detecting and identifying the power line inspection tower based on the unmanned aerial vehicle is suitable for a pre-trained tower identification neural network model, and comprises the following steps of:
the unmanned aerial vehicle is controlled to automatically inspect according to the visual positioning method, a worker is not required to control the unmanned aerial vehicle in the inspection process, and the whole inspection process is independently controlled by the unmanned aerial vehicle, so that the labor cost is reduced.
Collecting images in the unmanned aerial vehicle inspection process, generating synthetic data according to the images, and transmitting the synthetic data to a trained tower recognition neural network model;
and the tower identification neural network model generates belief mapping based on the synthetic data, identifies the tower according to the belief mapping, outputs a final identification result and records the current position of the unmanned aerial vehicle.
As a preferred embodiment, the training process of the tower recognition neural network model is as follows:
random illumination condition is applyed to the shaft tower, consider the particularity of shaft tower material, because the shaft tower is silvery white metal material, high reflectivity has, so the shaft tower surface colour is influenced by illumination more obviously in reality, in order to overcome the interference that this factor of illumination detected the shaft tower, can make the shaft tower more reasonable more lifelike through applying random illumination condition, be close to reality more, the variety of the data of follow-up collection has also been improved, adopt two cameras to gather every frame image in shaft tower each position, distance between the two cameras keeps 60 cm.
The composite data of each frame of image is generated at the speed of 50-100Hz by asynchronous and multithread sequential frame grabbing based on each frame of image, thereby improving the processing speed of the image and reducing the time cost.
And inputting the synthetic data of each frame of image into the tower recognition neural network model, and training the tower recognition neural network model to obtain the trained tower recognition neural network model.
As a preferred embodiment, the composite data of each frame image includes RGB image, depth map, segmentation map and data mark file, and one frame data includes 8 files since it is a dual camera acquisition.
As a preferred embodiment, the data marking file comprises the size of the collected picture, the position coordinates and the orientation matrix information of the double cameras in the world coordinate system, the coordinate information of the tower, the attitude information of the tower and the size information of the tower.
As a preferred embodiment, the tower identification neural network model comprises a feature extraction layer, a first 3 x 3 convolutional layer, a second 3 x 3 convolutional layer, a belief mapping layer and a vector layer, wherein the feature extraction layer is a VGG-19 convolutional neural network and is connected between every two layers through a relu function.
As a preferred embodiment, the operation principle of the tower recognition neural network model is as follows:
the VGG-19 convolutional neural network extracts 512-dimensional image features from the synthetic data; the image feature extraction is completed by the first ten layers of the VGG-19 convolutional neural network, and the image features are extracted by utilizing the first 10 layers of the VGG-19, so that the extraction efficiency is improved.
The first 3 x 3 convolutional layer reduces the dimension of the image feature from 512 to 256;
the second 3 x 3 convolution layer reduces the dimensionality of the image feature from 256 to 128, the 128-dimensional image feature is prepared for a first stage comprising three 3 x 128 layers and one 1 x 512 layer, plus one 1 x 9 belief map layer and one 1 x 6 vector field layer, the remaining five stages are identical to the first stage, unless the remaining five stages receive 153-dimensional input (128+16+ 9) consisting of five 7 x 128 layers and one 1 x 9 or one 1 x 16 layer.
The belief mapping layer and the vector layer generate belief maps from 128-dimensional image features, with 9 belief maps, each corresponding to 8 projected vertices of the input object 3D bounding box, and the other corresponding to the centroid. Description of the drawings: the 3D bounding box is a rectangular bounding box of the input object, the mass center is the center of the rectangular bounding box, and the 3D bounding box has the function of visualizing the real-time position coordinates and the posture information of the input object. Similarly, the direction from 8 vertices to the corresponding centroid is represented by 8 vector fields, and after the belief mapping is obtained, the individual objects need to be extracted from the belief mapping. Compared with other methods, objects need to be listed one by one in a complex architecture or program, the method of the embodiment relies on a simple post-processing step, local peaks above a threshold are searched in belief mapping, and then a greedy allocation algorithm is used for associating a projection vertex with a detected centroid. For each vertex, the latter step compares the vector field computed at the vertex to the direction from the vertex to each centroid, assigning the vertex to the closest centroid within some angular threshold of the vector. Once the vertices of each object instance are determined, the pose of the object is retrieved using the PnP algorithm, a commonly used pose estimation method.
To avoid the problem of gradient vanishing in the network, a loss is calculated at the output of each stage, and a L2 loss function is used for the belief map and the vector field.
As a preferred embodiment, the principle of the visual localization method is as follows:
in the initial stage of visual navigation, the approximate position of the central point of the downward-looking image of the unmanned aerial vehicle in the reference image is determined by utilizing the position and attitude angle information acquired by the machine vision system, searching and matching are carried out in the nearby area, the corresponding relation between image features is obtained, and the position and attitude information of the camera can be solved according to the photographic mark theory in the photogrammetry. And then determining the coordinates and the posture of the unmanned aerial vehicle according to the relative position relation between the camera and the unmanned aerial vehicle.
Acquiring position information and attitude information of the unmanned aerial vehicle;
usually, a central projection model is used as a camera model, and a real-time image is generally acquired and shot by an unmanned aerial vehicle and expressed by (x)i,k,yi,k) Sit of feature i in kth real time graphTarget, use (x)i,k,yi,k,zi,k) Representing the three-dimensional coordinates of the corresponding point in the reference image, the central projection model can be represented as:
wherein K represents the internal parameter matrix of the camera, including the effective focal length, the main focus and other parameters which can be calibrated accurately in advance, and R represents the attitude angle (w) of the camerai,φi,ki) S represents a scale factor, which can be eliminated in the calculation process.
And solving the speed, the angular speed and the acceleration of the unmanned aerial vehicle according to the position information and the attitude information.
The flight path of the unmanned aerial vehicle can be fitted by using some curves, the unmanned aerial vehicle always keeps cruising at a constant speed in the actual flight process, the motion in a short time can be regarded as linear motion, and the speed and the angular speed of the unmanned aerial vehicle are obtained by using a parameter fitting method in the implementation. Suppose that the horizontal coordinate of the Ti unmanned aerial vehicle at a certain moment is (X) in the geographical coordinate system pointed in the northeasti,Yi) Assuming that the flying height of the unmanned aerial vehicle is unchanged, the unmanned aerial vehicle keeps flying in a straight line in a short time, and assuming that the north-facing speed is vnAnd the north motion of the unmanned plane can be written as:
solving the equation can obtain the north direction velocity vnThe east velocity v can be obtained by the same formulaEThen, the course angle of the drone is:
as a preferred embodiment, the speed and the angular velocity of the unmanned aerial vehicle are solved by using a least square fitting method or a kalman filtering method, and the acceleration is solved by differentiating the speed.
As shown in fig. 2, an unmanned aerial vehicle-based power line patrol electric tower detection and identification system includes an unmanned aerial vehicle 201, a synthetic data generation module 202, a tower identification neural network model module 203, a tower identification module 204, an unmanned aerial vehicle control module 205, and a positioning module 206; the unmanned aerial vehicle 201 is provided with two cameras;
the unmanned aerial vehicle 201 is used for acquiring images in the inspection process and transmitting the images to the synthetic data generation module;
the synthetic data generation module 202 is used for generating synthetic data according to the images acquired by the unmanned aerial vehicle and outputting the synthetic data to the tower recognition neural network model module 203;
the pole and tower recognition neural network model module 203 is used for providing a pole and tower recognition neural network model, and the pole and tower recognition neural network model generates belief mapping based on synthetic data;
the pole and tower identification module 204 is used for identifying the pole and tower according to the belief mapping and outputting a final identification result;
the unmanned aerial vehicle control module 205 is used for controlling an unmanned aerial vehicle to automatically inspect according to a visual positioning method;
the positioning module 206 is configured to record position information of the drone.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units can refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Example 2
In embodiment 2, a plug-in is developed by the UE 4-based system, and it is possible to generate a large amount of high-quality synthesized data including RGB images, depth maps, segmentation maps, and tag data files for each frame. By utilizing asynchronous, multi-threaded sequential frame grabbing, the plug-in generates data at a speed of 50-100Hz, which is much faster than the default UE4 screen capture function. In addition to the composition data, the plug-in also includes different components, such as a scene management component, a virtual camera component; by configuring their properties, highly randomized images can be generated. Such randomization includes lighting, object and camera position, pose, texture and interference, and camera path following, among others. These components enable researchers to easily create stochastic scenarios to train deep neural networks.
By importing the model tower into the UE4 scene, each frame of image of the model tower at various orientations in the realistic scene can be captured and synthetic data generated at high speed using the automatically transferred virtual camera provided by the plug-in. Wherein the virtual camera is composed of a left camera and a right camera, the distance between the left camera and the right camera is maintained at 60cm (this is the coordinate distance in the UE4, and the specific value is set by the size of the object), and the captured frame data includes: the RGB image, the depth map, the segmentation map and the data mark file, since they are captured by the left and right cameras, one frame data includes 8 files.
The synthetic data also automatically generates two profiles: camera _ settings.json and object _ settings.json are: json is a configuration file of the UE4 virtual camera, containing information such as captured picture sizes of some cameras, and position coordinates and orientation matrices of the cameras in the world coordinate system; json is a configuration file for capturing objects, containing information such as class name, coordinates, posture and size of the objects.
In consideration of the particularity of the material of the power tower, the power tower is made of silver white metal and has high reflectivity, so that the influence of illumination on the surface color is obvious in the scene of the UE 4. In order to overcome the interference of illumination factor on the tower detection, random illumination components are added in the UE4 scene when the data is synthesized, and the random illumination components can randomly change the illumination intensity and the position of the light source at a fast speed. Or different illumination intensities are adjusted at different positions of the scene, and a certain amount of data are respectively generated for later training.
Example 3
In mainstream unmanned aerial vehicle positioning systems, the main implementation method is to utilize various sensors, for example: laser range finder, ultrasonic sensor, electromagnetic detection sensor etc. it is bigger to receive the surrounding environment interference to utilize the sensor to fix a position, and especially various electromagnetic interference can influence the performance of sensor when unmanned aerial vehicle electric power patrols the line, and replaces these sensors with machine vision module to realize that unmanned aerial vehicle's autonomic flight location can avoid these interferences, and its working process is shown in fig. 3:
in this embodiment, the microcomputer adopts a newly released TX2 development board for deep learning of NVIDIA, integrates a plurality of frames and environments for deep learning, installs the development board on the unmanned aerial vehicle, receives image signals from an onboard camera, and then builds a tower recognition neural network model on the development board, so that the posture estimation and recognition of a three-dimensional object can be realized, and the autonomous positioning of the unmanned aerial vehicle during power line patrol and the detection of the tower can be realized.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus, and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is merely a logical division, and in actual implementation, there can be other divisions, for example, multiple units or components can be combined or integrated into another system, or some features can be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection can be an indirect coupling or communication connection of some interfaces, devices or units, and can be in an electric, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, i.e. may be located in one place, or may also be distributed over a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, can be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributing to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium, and including instructions for causing a computer device (which can be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, and an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments can still be modified, or some technical features of the foregoing embodiments can be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. The utility model provides an electric power inspection tower detection and identification method based on unmanned aerial vehicle, which is characterized in that, is applicable to the pole tower recognition neural network model trained in advance, includes the following steps:
controlling the unmanned aerial vehicle to automatically inspect according to a visual positioning method;
collecting images in the unmanned aerial vehicle inspection process, generating synthetic data according to the images, and transmitting the synthetic data to a trained tower recognition neural network model;
and the tower identification neural network model generates belief mapping based on the synthetic data, identifies the tower according to the belief mapping, outputs a final identification result and records the current position of the unmanned aerial vehicle.
2. The unmanned aerial vehicle-based power line patrol tower detection and identification method according to claim 1, wherein the training process of the tower identification neural network model is as follows:
applying a random illumination condition to the tower, and collecting each frame of image of each direction of the tower by adopting two cameras;
generating synthetic data of each frame of image based on the acquired image of each frame;
and inputting the synthetic data of each frame of image into the tower recognition neural network model, and training the tower recognition neural network model to obtain the trained tower recognition neural network model.
3. The unmanned aerial vehicle-based power line patrol electric tower detection and identification method according to claim 2, wherein the synthetic data of each frame of image comprises an RGB image, a depth map, a segmentation map and a data mark file.
4. The unmanned aerial vehicle-based power line patrol tower detection and identification method according to claim 3, wherein the data marker file comprises size of collected pictures, position coordinates and orientation matrix information of the dual cameras in a world coordinate system, coordinate information of the tower, attitude information of the tower and size information of the tower.
5. The unmanned aerial vehicle-based power line patrol tower detection and identification method according to claim 4, wherein the tower identification neural network model comprises a feature extraction layer, a first 3 x 3 convolutional layer, a second 3 x 3 convolutional layer, a belief mapping layer and a vector layer, wherein the feature extraction layer is a VGG-19 convolutional neural network and is connected between each layer through a relu function.
6. The unmanned aerial vehicle-based power line patrol tower detection and identification method according to claim 5, wherein the tower identification neural network model has the following working principle:
the VGG-19 convolutional neural network extracts 512-dimensional image features from the synthetic data;
the first 3 x 3 convolutional layer reduces the dimension of the image feature from 512 to 256;
the second 3 x 3 convolutional layer reduces the dimension of the image feature from 256 to 128;
and the belief mapping layer and the vector layer generate belief mapping from 128-dimensional image characteristics, and the tower is identified according to the belief mapping.
7. The unmanned aerial vehicle-based power patrol tower detection and identification method according to claim 6, wherein an L2 loss function is used in a belief map and a vector field.
8. The unmanned aerial vehicle-based power line patrol electric tower detection and identification method according to claim 7, wherein the principle of the visual positioning method is as follows:
acquiring position information and attitude information of the unmanned aerial vehicle;
and solving the speed, the angular speed and the acceleration of the unmanned aerial vehicle according to the position information and the attitude information.
9. The unmanned aerial vehicle-based power line patrol electric tower detection and identification method according to claim 8, characterized in that the speed and angular velocity of the unmanned aerial vehicle are solved by adopting least square fitting or Kalman filtering method, and the acceleration is solved by differentiating the speed.
10. A power line inspection electric tower detection and identification system based on an unmanned aerial vehicle is characterized by comprising the unmanned aerial vehicle, a synthetic data generation module, a tower identification neural network model module, a tower identification module, an unmanned aerial vehicle control module and a positioning module; the unmanned aerial vehicle is provided with two cameras;
the unmanned aerial vehicle is used for collecting images in the inspection process and transmitting the images to the synthetic data generation module;
the synthetic data generation module is used for generating synthetic data according to the images acquired by the unmanned aerial vehicle and outputting the synthetic data to the pole and tower recognition neural network model module;
the pole and tower recognition neural network model module is used for providing a pole and tower recognition neural network model, and the pole and tower recognition neural network model generates belief mapping based on synthetic data;
the pole and tower identification module is used for identifying the pole and tower according to the belief mapping and outputting a final identification result;
the unmanned aerial vehicle control module is used for controlling the unmanned aerial vehicle to automatically inspect according to a visual positioning method;
the positioning module is used for recording the position information of the unmanned aerial vehicle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010130319.1A CN111275015A (en) | 2020-02-28 | 2020-02-28 | Unmanned aerial vehicle-based power line inspection electric tower detection and identification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010130319.1A CN111275015A (en) | 2020-02-28 | 2020-02-28 | Unmanned aerial vehicle-based power line inspection electric tower detection and identification method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111275015A true CN111275015A (en) | 2020-06-12 |
Family
ID=71004060
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010130319.1A Pending CN111275015A (en) | 2020-02-28 | 2020-02-28 | Unmanned aerial vehicle-based power line inspection electric tower detection and identification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111275015A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111913484A (en) * | 2020-07-30 | 2020-11-10 | 杭州电子科技大学 | Path planning method of transformer substation inspection robot in unknown environment |
| CN112068591A (en) * | 2020-08-25 | 2020-12-11 | 中国南方电网有限责任公司超高压输电公司天生桥局 | Unmanned aerial vehicle for automatic inspection of power transmission line, control method and device and storage medium |
| CN112464881A (en) * | 2020-12-11 | 2021-03-09 | 国网辽宁省电力有限公司抚顺供电公司 | Unmanned aerial vehicle optical remote sensing autonomous inspection method for power transmission line |
| CN113222838A (en) * | 2021-05-07 | 2021-08-06 | 国网山西省电力公司吕梁供电公司 | Unmanned aerial vehicle autonomous line patrol method based on visual positioning |
| CN113762132A (en) * | 2021-09-01 | 2021-12-07 | 国网浙江省电力有限公司金华供电公司 | Automatic classification and automatic naming system for unmanned aerial vehicle inspection image |
| CN114582040A (en) * | 2022-05-05 | 2022-06-03 | 中国长江三峡集团有限公司 | Intelligent inspection system and method for wind power generation equipment |
| CN115953486A (en) * | 2022-12-30 | 2023-04-11 | 国网电力空间技术有限公司 | Automatic coding method for direct-current T-shaped tangent tower component inspection image |
| CN116012742A (en) * | 2023-03-24 | 2023-04-25 | 广东数字生态科技有限责任公司 | Power transmission tower detection model configuration method and device and unmanned aerial vehicle |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102829785A (en) * | 2012-08-30 | 2012-12-19 | 中国人民解放军国防科学技术大学 | Air vehicle full-parameter navigation method based on sequence image and reference image matching |
| CN103246203A (en) * | 2013-04-23 | 2013-08-14 | 东南大学 | GPS (Global Position System) based prediction method for speed state of micro quad-rotor unmanned aerial vehicle |
| CN107392901A (en) * | 2017-07-24 | 2017-11-24 | 国网山东省电力公司信息通信公司 | A kind of method for transmission line part intelligence automatic identification |
| CN107729808A (en) * | 2017-09-08 | 2018-02-23 | 国网山东省电力公司电力科学研究院 | A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection |
| CN110048878A (en) * | 2019-03-08 | 2019-07-23 | 国网河南省电力公司郑州供电公司 | A kind of transmission line of electricity three is across Visualized management system |
| CN110133440A (en) * | 2019-05-27 | 2019-08-16 | 国电南瑞科技股份有限公司 | Electric power unmanned plane and method for inspecting based on Tower Model matching and vision guided navigation |
| US20190355150A1 (en) * | 2018-05-17 | 2019-11-21 | Nvidia Corporation | Detecting and estimating the pose of an object using a neural network model |
-
2020
- 2020-02-28 CN CN202010130319.1A patent/CN111275015A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102829785A (en) * | 2012-08-30 | 2012-12-19 | 中国人民解放军国防科学技术大学 | Air vehicle full-parameter navigation method based on sequence image and reference image matching |
| CN103246203A (en) * | 2013-04-23 | 2013-08-14 | 东南大学 | GPS (Global Position System) based prediction method for speed state of micro quad-rotor unmanned aerial vehicle |
| CN107392901A (en) * | 2017-07-24 | 2017-11-24 | 国网山东省电力公司信息通信公司 | A kind of method for transmission line part intelligence automatic identification |
| CN107729808A (en) * | 2017-09-08 | 2018-02-23 | 国网山东省电力公司电力科学研究院 | A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection |
| US20190355150A1 (en) * | 2018-05-17 | 2019-11-21 | Nvidia Corporation | Detecting and estimating the pose of an object using a neural network model |
| CN110048878A (en) * | 2019-03-08 | 2019-07-23 | 国网河南省电力公司郑州供电公司 | A kind of transmission line of electricity three is across Visualized management system |
| CN110133440A (en) * | 2019-05-27 | 2019-08-16 | 国电南瑞科技股份有限公司 | Electric power unmanned plane and method for inspecting based on Tower Model matching and vision guided navigation |
Non-Patent Citations (1)
| Title |
|---|
| 闫冬 等: "基于图像处理的输电线路无人机巡检的研究", 《计算机科学与探索》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111913484A (en) * | 2020-07-30 | 2020-11-10 | 杭州电子科技大学 | Path planning method of transformer substation inspection robot in unknown environment |
| CN111913484B (en) * | 2020-07-30 | 2022-08-12 | 杭州电子科技大学 | Path planning method of transformer substation inspection robot in unknown environment |
| CN112068591A (en) * | 2020-08-25 | 2020-12-11 | 中国南方电网有限责任公司超高压输电公司天生桥局 | Unmanned aerial vehicle for automatic inspection of power transmission line, control method and device and storage medium |
| CN112464881A (en) * | 2020-12-11 | 2021-03-09 | 国网辽宁省电力有限公司抚顺供电公司 | Unmanned aerial vehicle optical remote sensing autonomous inspection method for power transmission line |
| CN113222838A (en) * | 2021-05-07 | 2021-08-06 | 国网山西省电力公司吕梁供电公司 | Unmanned aerial vehicle autonomous line patrol method based on visual positioning |
| CN113762132A (en) * | 2021-09-01 | 2021-12-07 | 国网浙江省电力有限公司金华供电公司 | Automatic classification and automatic naming system for unmanned aerial vehicle inspection image |
| CN114582040A (en) * | 2022-05-05 | 2022-06-03 | 中国长江三峡集团有限公司 | Intelligent inspection system and method for wind power generation equipment |
| CN115953486A (en) * | 2022-12-30 | 2023-04-11 | 国网电力空间技术有限公司 | Automatic coding method for direct-current T-shaped tangent tower component inspection image |
| CN115953486B (en) * | 2022-12-30 | 2024-04-12 | 国网电力空间技术有限公司 | Automatic encoding method for inspection image of direct-current T-shaped tangent tower part |
| CN116012742A (en) * | 2023-03-24 | 2023-04-25 | 广东数字生态科技有限责任公司 | Power transmission tower detection model configuration method and device and unmanned aerial vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111275015A (en) | Unmanned aerial vehicle-based power line inspection electric tower detection and identification method and system | |
| CN110568447B (en) | Visual positioning method, device and computer readable medium | |
| CN108401461A (en) | Three-dimensional mapping method, device and system, cloud platform, electronic equipment and computer program product | |
| CN108898676B (en) | Method and system for detecting collision and shielding between virtual and real objects | |
| CN109579843A (en) | Multirobot co-located and fusion under a kind of vacant lot multi-angle of view build drawing method | |
| CN106600627A (en) | Rigid body motion capturing method and system based on mark point | |
| CN112652016B (en) | Point cloud prediction model generation method, pose estimation method and pose estimation device | |
| US20210327287A1 (en) | Uav path planning method and device guided by the safety situation, uav and storage medium | |
| CN106826833A (en) | Independent navigation robot system based on 3D solid cognition technologies | |
| CN108303994B (en) | Group control interaction method for unmanned aerial vehicle | |
| CN105955308A (en) | Aircraft control method and device | |
| CN110176032A (en) | A kind of three-dimensional rebuilding method and device | |
| CN110567441B (en) | Particle filter-based positioning method, positioning device, mapping and positioning method | |
| CN109996728A (en) | Method and system for analog vision data | |
| CN109166181A (en) | A kind of mixing motion capture system based on deep learning | |
| CN114004977A (en) | Aerial photography data target positioning method and system based on deep learning | |
| Sobel et al. | Camera calibration for tracked vehicles augmented reality applications | |
| CN113848931A (en) | Agricultural machinery automatic driving obstacle recognition method, system, equipment and storage medium | |
| Natraj et al. | Vision based attitude and altitude estimation for UAVs in dark environments | |
| US20210156710A1 (en) | Map processing method, device, and computer-readable storage medium | |
| Spicer et al. | Producing usable simulation terrain data from UAS-collected imagery | |
| CN105930766A (en) | Unmanned plane | |
| CN111402324A (en) | Target measuring method, electronic equipment and computer storage medium | |
| Hao | An Autonomous Navigation Algorithm for Monocular Visual Recognition | |
| Symeonidis et al. | A UAV Video Data Generation Framework for Improved Robustness of UAV Detection Methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200612 |