CN117423018A - Power equipment detection system and method based on unmanned aerial vehicle acquisition multi-mode source fusion - Google Patents
Power equipment detection system and method based on unmanned aerial vehicle acquisition multi-mode source fusion Download PDFInfo
- Publication number
- CN117423018A CN117423018A CN202311501965.4A CN202311501965A CN117423018A CN 117423018 A CN117423018 A CN 117423018A CN 202311501965 A CN202311501965 A CN 202311501965A CN 117423018 A CN117423018 A CN 117423018A
- Authority
- CN
- China
- Prior art keywords
- visible light
- image
- power equipment
- target
- light image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 54
- 238000000034 method Methods 0.000 title claims abstract description 15
- 230000004927 fusion Effects 0.000 title claims abstract description 13
- 230000009466 transformation Effects 0.000 claims abstract description 20
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000012549 training Methods 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims description 33
- 238000006243 chemical reaction Methods 0.000 claims description 26
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 10
- 238000003708 edge detection Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 10
- 239000012212 insulator Substances 0.000 description 9
- 230000005540 biological transmission Effects 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000001931 thermography Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Remote Sensing (AREA)
- Biodiversity & Conservation Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a power equipment detection system and method based on unmanned aerial vehicle acquisition multi-mode source fusion. Firstly, collecting paired infrared and visible light images of target power equipment by using an unmanned plane, marking the visible light images according to the outline of the target equipment, and then performing model training; and respectively inputting the paired images into a trained model and an infrared preprocessing module to obtain target edge contour images of the two images and coordinate information in a target visible light image, inputting the data into a registration and coordinate transformation module to obtain position information of equipment in the infrared image, extracting target temperature information according to the information, and detecting the equipment by judging the temperature. The invention fully utilizes the convenience of unmanned aerial vehicle acquisition, optimizes the identification of the small target power equipment and the complex background power equipment in the unmanned aerial vehicle shooting scene, obviously improves the image registration and target identification effect, and improves the overall detection precision and effect.
Description
Technical Field
The invention belongs to the technical field of power equipment detection, and particularly relates to a power equipment fault detection system and method based on unmanned aerial vehicle collected image data and infrared thermal imaging and visible light fusion.
Background
Along with the promotion of the modern construction process in China, the electric power demand is continuously increased, the scale of the power distribution network is also continuously enlarged, and the potential safety hazard of electric power equipment is also increased. Therefore, the safety of the power transmission equipment is ensured while the distribution efficiency of the power system is improved. The transmission line is an important component in a power distribution system and is an important guarantee for safe transmission of electric energy.
The unmanned aerial vehicle technology is widely applied to the fields of monitoring, detecting and rescuing and the like, but a detection system matched with an unmanned aerial vehicle for monitoring power transmission line equipment is deficient, the unmanned aerial vehicle technology brings acquisition convenience, and simultaneously, some problems are also caused, for example, the problems that the resolution ratio of images acquired by unmanned aerial vehicle high-altitude operation is high, the targets are small and the images are difficult to identify are solved, and the shooting angle of the unmanned aerial vehicle determines a large number of irrelevant backgrounds in the images, so that challenges are brought to image identification and image registration.
Disclosure of Invention
Aiming at the defects of the existing system, the invention provides a power equipment detection system based on unmanned aerial vehicle acquisition multi-mode source fusion in order to fully utilize the advantages of unmanned aerial vehicle acquisition and monitoring technology.
The system provided by the invention comprises the following modules: the system comprises an unmanned aerial vehicle data acquisition module, a visible light image processing module, a small target re-identification module, an image preprocessing module, an image and coordinate conversion registration module and an equipment temperature detection module.
Unmanned aerial vehicle data acquisition module: the unmanned aerial vehicle carries infrared and visible light cameras, and images of shooting target power equipment are collected and comprise visible light images and infrared images.
Visible light image processing module: training and predicting a target detection model, and marking the acquired visible light image of the target power equipment, wherein the marking method is close to the target contour; inputting the marked visible light image into the improved Yolov5 target detection model for training to obtain a target power equipment detection model in the visible light image; the model can identify the target power equipment in the visible light image, acquire the position information of the target power equipment in the visible light image, and the position information is used for the image and coordinate conversion registration module to subsequently coordinate and position the target power equipment in the infrared image.
Small target re-recognition module: the small target re-identification module is a module for optimizing the problem that small target power equipment in an image shot by the unmanned aerial vehicle is difficult to detect and identify by a Yolov5 model of the visible light image processing module, the module further identifies the small target power equipment in the visible light by utilizing the obtained position information of the visible light image processing module and the original visible light image, the identification rate of the small target power equipment is improved, the position of the target power equipment is obtained, and the used model is the Yolov5 model in the visible light image processing module.
An image preprocessing module: and acquiring edge contour information of the target power equipment in the infrared image and the visible light image by using an image edge extraction algorithm, removing non-target power equipment information, and using the edge contour information for a subsequent image and coordinate conversion registration module.
The edge extraction algorithm of the image is realized through a Holistic-Nested Edge Detection model.
The HED (Holistic-Nested Edge Detection) module is used for removing noise, and the obtained contour image is used for an image and coordinate conversion registration module.
Further, HED (Holistcally-Nested Edge Detection) is a deep learning model for edge detection. The HED model is based on the idea of Convolutional Neural Networks (CNNs) and multi-level feature extraction. It captures edge information in an image by simultaneously utilizing features of different levels. The HED model includes a pre-trained VGGNet as a feature extractor and generates the final edge image by fusing features at different levels. The model is trained using a pixel-by-pixel binary cross entropy loss function to predict edge locations in the image to a maximum extent.
The image and coordinate conversion registration module: the method comprises the steps of performing image registration on a visible light image and a corresponding infrared image thereof, inputting the preprocessed infrared image and the corresponding visible light image into a Brisk algorithm to obtain matched characteristic points in two pictures, and then obtaining a transformation matrix for registering the infrared image to the position of the visible light image by using the characteristic points; applying the transformation matrix to the position information of the target power equipment obtained by the visible light image processing module, and finally obtaining the position information of the target power equipment in the infrared image;
the equipment temperature detection module: the infrared image is provided with temperature information, the module is used for detecting the position information of the target power equipment in the infrared image obtained by the image and coordinate conversion registration module, so that the temperature information of the target power equipment is obtained, and the temperature information is compared with the equipment early-warning temperature information to obtain a detection result.
Further, the main idea of the BRISK algorithm in the image-coordinate conversion registration module is as follows: feature descriptors are generated using a rotation distribution binary pattern RDB and scale invariance is achieved using multi-layer grading. And extracting the matched characteristic points in the infrared image and the visible light image by using a BRISK algorithm. And (3) for the extracted characteristic points, carrying out characteristic point matching by adopting a FLANN algorithm to obtain a transformation matrix.
Further, the transformation matrix is applied to the position information of the target power equipment in the visible light image, which is obtained in the visible light image processing module, namely, the transformation matrix is subjected to dot multiplication with the position information, so that the position information of the target power equipment in the infrared image is obtained.
The invention has the following beneficial effects:
the system fully utilizes the convenience of unmanned aerial vehicle acquisition, optimizes the identification of small target power equipment and complex background power equipment in an unmanned aerial vehicle shooting scene, and has obvious improvement on image registration and target identification effect, and overall detection precision and effect are improved.
Drawings
FIG. 1 is a schematic diagram of the system architecture and operation flow of the present invention;
fig. 2 is an illustration of a system of the present invention.
Detailed Description
The system provided by the invention comprises the following modules:
firstly, collecting paired infrared and visible light images of target power equipment by using an unmanned plane, marking the visible light images according to the outline of the target equipment, and then performing model training; and respectively inputting the paired images into a trained model and an infrared preprocessing module to obtain target edge contour images of the two images and coordinate information in a target visible light image, inputting the data into a registration and coordinate transformation module to obtain position information of equipment in the infrared image, extracting target temperature information according to the information, and detecting the equipment by judging the temperature. The invention fully utilizes the convenience of unmanned aerial vehicle acquisition, optimizes the identification of the small target power equipment and the complex background power equipment in the unmanned aerial vehicle shooting scene, obviously improves the image registration and target identification effect, and improves the overall detection precision and effect.
The power equipment detection system based on unmanned aerial vehicle acquisition multi-mode source fusion comprises an unmanned aerial vehicle data acquisition module, a visible light image processing module, a small target re-identification module, an image preprocessing module, an image and coordinate conversion registration module and an equipment temperature detection module;
unmanned aerial vehicle data acquisition module: the unmanned aerial vehicle carries infrared and visible light cameras, and images of shooting target power equipment, including visible light images and infrared images, are collected;
visible light image processing module: training and predicting a target detection model, and marking the acquired visible light image of the target power equipment, wherein the marking method is close to the target contour; inputting the marked visible light image into the improved Yolov5 target detection model for training to obtain a target power equipment detection model in the visible light image; the target power equipment detection model can identify target power equipment in the visible light image, and acquire position information of the target power equipment in the visible light image, wherein the position information is used for an image and coordinate conversion registration module, and the position of the target power equipment in the infrared image is positioned in a subsequent matching mode;
small target re-recognition module: the device comprises a module for optimizing the problem that electric equipment with a target smaller than a set threshold parameter in an image shot by an unmanned aerial vehicle is difficult to detect and identify by a Yolov5 target detection model of a visible light image processing module, wherein the module further identifies small target electric equipment in a visible light image by utilizing position information obtained by the visible light image processing module and an original visible light image, improves the identification rate of the small target electric equipment, obtains the position of the target electric equipment, and the model used in the module is also the Yolov5 target detection model in the visible light image processing module;
an image preprocessing module: acquiring edge contour information of target power equipment in an infrared image and a visible light image by using an image edge extraction algorithm, removing non-target power equipment information, and using the edge contour information for a subsequent image and coordinate conversion registration module;
the image and coordinate conversion registration module: the method comprises the steps of performing image registration on a visible light image and a corresponding infrared image thereof, inputting the preprocessed infrared image and the corresponding visible light image into a Brisk algorithm to obtain matched characteristic points in two pictures, and then obtaining a transformation matrix for registering the infrared image to the position of the visible light image by using the characteristic points; applying the transformation matrix to the position information of the target power equipment obtained by the visible light image processing module, and finally obtaining the position information of the target power equipment in the infrared image;
the equipment temperature detection module: the infrared image is provided with temperature information, the module is used for detecting the position information of the target power equipment in the infrared image obtained by the image and coordinate conversion registration module, so that the temperature information of the target power equipment is obtained, and the temperature information is compared with the equipment early-warning temperature information to obtain a detection result.
Further, in the small target re-recognition module, the recognition of the small target device is improved in two points, and in the first point, for the recognized target, the processing module cuts a region with a specified size near the target and re-recognizes the region, and by enlarging the region, the probability of recognition failure and recognition error can be reduced. And the second point is that the processing module divides the high-resolution image into areas with fixed sizes, the target detection model respectively carries out target recognition on the small areas, and the recognized results are screened and combined.
The image preprocessing module is used for extracting the outline of the infrared image; the main contour of the power equipment under the complex background can be extracted by adopting the Holistcally-Nested Edge Detection, the effect of this step is to remove noise, and the obtained contour image is used for an image and coordinate conversion registration module.
HED (Holisically-Nested Edge Detection) is a deep learning model for edge detection. The HED model is based on the idea of Convolutional Neural Networks (CNNs) and multi-level feature extraction. It captures edge information in an image by simultaneously utilizing features of different levels. The HED model includes a pre-trained VGGNet as a feature extractor and generates the final edge image by fusing features at different levels. The model is trained using a pixel-by-pixel binary cross entropy loss function to predict edge locations in the image to a maximum extent.
The image and coordinate conversion registration module is used for acquiring the position information of the target power equipment corresponding to the visible light image in the infrared image; and extracting image characteristic points from the preprocessed visible light image and the preprocessed infrared image, namely the contour image by using a Brisk algorithm.
The primary idea of BRISK is to use a rotation distribution binary pattern RDB to generate feature descriptors and use multi-layer grading to achieve scale invariance. The characteristic points in the two images are extracted through the steps. For the extracted feature points, a FLANN algorithm is adopted to match the feature points. The matched characteristic points can obtain a transformation matrix from the visible light image to the infrared image through a Homography algorithm, and the transformation matrix is applied to the position information of the equipment in the image obtained in the visible light image processing module, so that the position information of the equipment in the infrared image can be obtained.
The position information is input into a device temperature detection module, the detection module extracts temperature information at the position information in the infrared image of the device, compares the temperature information with temperature information of the working operation of the device to obtain information of whether the device is faulty or not, and prompts maintenance personnel of the power device in a warning mode for the device with fault risk.
Furthermore, the invention also provides a power equipment detection method based on unmanned aerial vehicle acquisition multi-mode source fusion, which comprises the following specific implementation steps:
step 1, unmanned aerial vehicle data acquisition, unmanned aerial vehicle carries infrared and visible light camera, gathers shooting target power equipment's image, including visible light image and infrared image.
Step 2, visible light image processing, which comprises training and predicting a target detection model, and marking the acquired visible light image of the target power equipment, wherein the marking method is to be close to the target contour; inputting the marked visible light image into the improved Yolov5 target detection model for training to obtain a target power equipment detection model in the visible light image; the model can identify the target power equipment in the visible light image, acquire the position information of the target power equipment in the visible light image, and the position information is used for the image and coordinate conversion registration module to subsequently coordinate and position the target power equipment in the infrared image.
And step 3, preprocessing the image, namely acquiring edge contour information of target power equipment in the infrared image and the visible light image by using an image edge extraction algorithm, removing non-target power equipment information, and using the edge contour information for a subsequent image and coordinate conversion registration module.
Step 4, performing image registration on the visible light image and the corresponding infrared image, inputting the infrared image and the corresponding visible light image into a Brisk algorithm, and obtaining a transformation matrix for registering the infrared image to the position of the visible light image; applying the transformation matrix to the position information of the target power equipment obtained by the visible light image processing module, and finally obtaining the position information of the target power equipment in the infrared image;
and 5, detecting the temperature of the equipment, wherein the infrared image is provided with temperature information, and the module is used for detecting the position information of the target power equipment in the infrared image obtained by the image and coordinate conversion registration module, so as to obtain the temperature information of the target power equipment, and comparing the temperature information with the equipment early-warning temperature information to obtain a detection result.
Further, prior to the preprocessing in step 3, the small target may be re-identified: and the small target re-identification module is used for optimizing the problem that the power equipment with the target smaller than the set threshold value in the image shot by the unmanned aerial vehicle is difficult to detect and identify by a Yolov5 model in the visible light image processing process, the small target power equipment in the visible light is further identified by utilizing the obtained position information of the visible light image processing module and the original visible light image, the identification rate of the small target power equipment is improved, the position of the target power equipment is obtained, and the used model is the Yolov5 model in the visible light image processing module.
The invention, in part not described in detail, is within the skill of those skilled in the art.
The team performs experimental verification on the system performance, and collects an experimental data set, which is mainly 430 image data collected in an actual power transmission line of a power supply company in a Fenghua district of Zhejiang province in the national network, and specifically comprises the following categories:
1) Visible light image data of the circuit device: 215 pieces of visible light image data of the transmission line mainly overhauled by insulators, grounding wires, towers and the like.
2) Circuit device thermal imaging image count: 215 pieces of thermal imaging image data of the transmission line mainly overhauled by insulators, grounding wires, towers and the like. The system detects the insulator targets in the experimental data images, and the recognition rate and the registration rate of the insulator targets are tested and counted by the system, so that the following results are obtained.
TABLE 1 actual Effect of the System
| Recognition rate% | Registration rate |
| 90.1 | 86.9 |
Table 1 shows the effect of the system on identifying and detecting the insulator of the power equipment image acquired by the unmanned aerial vehicle. The calculation formula of the recognition rate is as follows:
r represents insulator recognition rate, N s Indicating the number of insulators identified by the system, N t The higher the identification rate is, the better the identification effect of the system is.
The registration rate is calculated by the following formula:
wherein R represents recognition rate, O c Representing the size of a pixel area obtained after registering the pixel area of a target in visible light into an infrared image, O r The actual pixel area of the target in the infrared image is represented, if the ratio is larger, the registration rate of the target and the actual pixel area is better, and the temperature accuracy of final extraction is higher. Both will ultimately affect the efficiency and accuracy of the system in detecting targets.
It can be seen from table 1 that the system has a certain practical value. Fig. 2 is an actual demonstration of the system, in which the system boxes the insulator position and shows the insulator temperature information.
Claims (6)
1. The power equipment detection system based on the unmanned aerial vehicle acquisition multi-mode source fusion is characterized by comprising an unmanned aerial vehicle data acquisition module, a visible light image processing module, a small target re-identification module, an image preprocessing module, an image and coordinate conversion registration module and an equipment temperature detection module;
unmanned aerial vehicle data acquisition module: the unmanned aerial vehicle carries infrared and visible light cameras, and images of shooting target power equipment, including visible light images and infrared images, are collected;
visible light image processing module: training and predicting a target detection model, and marking the acquired visible light image of the target power equipment, wherein the marking method is close to the target contour; inputting the marked visible light image into the improved Yolov5 target detection model for training to obtain a target power equipment detection model in the visible light image; the target power equipment detection model can identify target power equipment in the visible light image, and acquire position information of the target power equipment in the visible light image, wherein the position information is used for an image and coordinate conversion registration module, and the position of the target power equipment in the infrared image is positioned in a subsequent matching mode;
small target re-recognition module: the device comprises a module for optimizing the problem that electric equipment with a target smaller than a set threshold parameter in an image shot by an unmanned aerial vehicle is difficult to detect and identify by a Yolov5 target detection model of a visible light image processing module, wherein the module further identifies small target electric equipment in a visible light image by utilizing position information obtained by the visible light image processing module and an original visible light image, improves the identification rate of the small target electric equipment, obtains the position of the target electric equipment, and the model used in the module is also the Yolov5 target detection model in the visible light image processing module;
an image preprocessing module: acquiring edge contour information of target power equipment in an infrared image and a visible light image by using an image edge extraction algorithm, removing non-target power equipment information, and using the edge contour information for a subsequent image and coordinate conversion registration module;
the image and coordinate conversion registration module: the method comprises the steps of performing image registration on a visible light image and a corresponding infrared image thereof, inputting the preprocessed infrared image and the corresponding visible light image into a Brisk algorithm to obtain matched characteristic points in two pictures, and then obtaining a transformation matrix for registering the infrared image to the position of the visible light image by using the characteristic points; applying the transformation matrix to the position information of the target power equipment obtained by the visible light image processing module, and finally obtaining the position information of the target power equipment in the infrared image;
the equipment temperature detection module: the infrared image is provided with temperature information, the module is used for detecting the position information of the target power equipment in the infrared image obtained by the image and coordinate conversion registration module, so that the temperature information of the target power equipment is obtained, and the temperature information is compared with the equipment early-warning temperature information to obtain a detection result.
2. The unmanned aerial vehicle acquisition multi-modal source fusion-based power equipment detection system according to claim 1, wherein the image edge extraction algorithm is implemented by
The Holistically-Nested Edge Detection model implementation.
3. The unmanned aerial vehicle acquisition multi-modal source fusion-based power equipment detection system of claim 1, wherein the BRISK algorithm in the image and coordinate conversion registration module uses a rotation distribution binary pattern RDB to generate feature descriptors and uses multi-layer classification to achieve scale invariance; extracting matched characteristic points in the infrared image and the visible light image through a BRISK algorithm; and (3) for the extracted characteristic points, carrying out characteristic point matching by adopting a FLANN algorithm to obtain a transformation matrix.
4. The unmanned aerial vehicle acquisition multi-mode source fusion-based power equipment detection system according to claim 3, wherein the transformation matrix is applied to the position information of the target power equipment in the visible light image, which is obtained in the visible light image processing module, namely, the position information of the target power equipment in the infrared image is obtained by dot multiplying the transformation matrix with the position information.
5. The power equipment detection method based on unmanned aerial vehicle acquisition multi-mode source fusion is characterized by comprising the following steps of:
step 1, unmanned aerial vehicle data acquisition, wherein an unmanned aerial vehicle carries infrared cameras and visible cameras, and images of shooting target power equipment are acquired, wherein the images comprise visible images and infrared images;
step 2, visible light image processing, which comprises training and predicting a target detection model, and marking the acquired visible light image of the target power equipment, wherein the marking method is to be close to the target contour; inputting the marked visible light image into the improved Yolov5 target detection model for training to obtain a target power equipment detection model in the visible light image; the model can identify the target power equipment in the visible light image, acquire the position information of the target power equipment in the visible light image, and the position information is used for the image and coordinate conversion registration module to subsequently coordinate and position the target power equipment in the infrared image;
step 3, image preprocessing, namely acquiring edge contour information of target power equipment in an infrared image and a visible light image by using an image edge extraction algorithm, and removing non-target power equipment information, wherein the edge contour information is used for a subsequent image and coordinate conversion registration module;
step 4, performing image registration on the visible light image and the corresponding infrared image, inputting the infrared image and the corresponding visible light image into a Brisk algorithm, and obtaining a transformation matrix for registering the infrared image to the position of the visible light image; applying the transformation matrix to the position information of the target power equipment obtained by the visible light image processing module, and finally obtaining the position information of the target power equipment in the infrared image;
and 5, detecting the temperature of the equipment, wherein the infrared image is provided with temperature information, and the module is used for detecting the position information of the target power equipment in the infrared image obtained by the image and coordinate conversion registration module, so as to obtain the temperature information of the target power equipment, and comparing the temperature information with the equipment early-warning temperature information to obtain a detection result.
6. The unmanned aerial vehicle acquisition multi-mode source fusion-based power equipment detection method according to claim 5, wherein the small target is re-identified before preprocessing in step 3: and the small target re-identifies a module for optimizing the problem that the power equipment with the target smaller than the set threshold in the image shot by the unmanned aerial vehicle is difficult to be detected and identified by a Yolov5 model in the visible light image processing process, the small target power equipment in the visible light image is further identified by utilizing the obtained position information of the visible light image processing and the original visible light image, the identification rate of the small target power equipment is improved, the position of the target power equipment is obtained, and the used model is the Yolov5 target detection model in the visible light image processing module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311501965.4A CN117423018A (en) | 2023-11-13 | 2023-11-13 | Power equipment detection system and method based on unmanned aerial vehicle acquisition multi-mode source fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311501965.4A CN117423018A (en) | 2023-11-13 | 2023-11-13 | Power equipment detection system and method based on unmanned aerial vehicle acquisition multi-mode source fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117423018A true CN117423018A (en) | 2024-01-19 |
Family
ID=89522835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311501965.4A Pending CN117423018A (en) | 2023-11-13 | 2023-11-13 | Power equipment detection system and method based on unmanned aerial vehicle acquisition multi-mode source fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117423018A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118097292A (en) * | 2024-03-18 | 2024-05-28 | 南京海汇装备科技有限公司 | AI model-based target multi-mode data fusion recognition system and method |
| CN118230196A (en) * | 2024-03-20 | 2024-06-21 | 广州中科云图智能科技有限公司 | Infrared temperature measurement method and device and electronic equipment |
-
2023
- 2023-11-13 CN CN202311501965.4A patent/CN117423018A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118097292A (en) * | 2024-03-18 | 2024-05-28 | 南京海汇装备科技有限公司 | AI model-based target multi-mode data fusion recognition system and method |
| CN118230196A (en) * | 2024-03-20 | 2024-06-21 | 广州中科云图智能科技有限公司 | Infrared temperature measurement method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110826538B (en) | Abnormal off-duty identification system for electric power business hall | |
| CN108537154B (en) | Power transmission line bird nest identification method based on HOG characteristics and machine learning | |
| US10269138B2 (en) | UAV inspection method for power line based on human visual system | |
| CN108416294B (en) | Fan blade fault intelligent identification method based on deep learning | |
| CN117423018A (en) | Power equipment detection system and method based on unmanned aerial vehicle acquisition multi-mode source fusion | |
| CN111091544B (en) | Method for detecting breakage fault of side integrated framework of railway wagon bogie | |
| CN112528979B (en) | Transformer substation inspection robot obstacle distinguishing method and system | |
| CN105373135A (en) | Method and system for guiding airplane docking and identifying airplane type based on machine vision | |
| CN115184726B (en) | Smart power grid fault real-time monitoring and positioning system and method | |
| CN109034272A (en) | A kind of transmission line of electricity heat generating components automatic identifying method | |
| CN111353487A (en) | Equipment information extraction method for transformer substation | |
| CN115294031A (en) | Photovoltaic module fault image identification method based on infrared thermal imaging analysis | |
| CN113688817A (en) | Instrument identification method and system for automatic inspection | |
| Wang et al. | Pointer meter recognition in UAV inspection of overhead transmission lines | |
| Lianqiao et al. | Recognition and application of infrared thermal image among power facilities based on yolo | |
| CN113485432A (en) | Photovoltaic power station electroluminescence intelligent diagnosis system and method based on unmanned aerial vehicle | |
| CN116310274A (en) | State evaluation method for power transmission and transformation equipment | |
| CN116681979A (en) | Power equipment target detection method under complex environment | |
| CN112288682A (en) | Electric power equipment defect positioning method based on image registration | |
| CN105447463B (en) | Across the camera to automatically track system that substation is identified based on characteristics of human body | |
| CN117252840B (en) | Photovoltaic array defect elimination evaluation method and device and computer equipment | |
| Zhou et al. | Wind turbine actual defects detection based on visible and infrared image fusion | |
| WO2024119983A1 (en) | Anomality detection method and apparatus for cable terminal of rail vehicle | |
| CN116524725B (en) | Intelligent driving traffic sign image data identification system | |
| CN111626104A (en) | Cable hidden danger point detection method and device based on unmanned aerial vehicle infrared thermal imagery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |