CN115841633A - Power tower and power line associated correction power tower and power line detection method - Google Patents
Power tower and power line associated correction power tower and power line detection method Download PDFInfo
- Publication number
- CN115841633A CN115841633A CN202211552072.8A CN202211552072A CN115841633A CN 115841633 A CN115841633 A CN 115841633A CN 202211552072 A CN202211552072 A CN 202211552072A CN 115841633 A CN115841633 A CN 115841633A
- Authority
- CN
- China
- Prior art keywords
- power
- detection
- power tower
- line
- tower
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
A power tower and power line detection method for power tower and power line correlation correction is characterized in that images of the power tower and the power line are shot from the upper part, and graying pretreatment is carried out on the images; carrying out edge detection on the preprocessed image to obtain a binary edge image; carrying out power line detection and power tower detection by using the edge map to obtain power lines represented by parallel line groups and a power tower area framed by a rectangular frame; mapping the rectangular frame to an edge map, excluding background areas on two sides of the rectangular frame of the power tower, and re-detecting the power lines in the internal area of the rectangular frame of the power tower; and after a new power line detection result is obtained, estimating the ratio of the power tower area in the edge graph according to the left and right widths of the power line distribution area, updating power tower detection parameters, and re-detecting the power tower until the requirements are met or the iteration times are reached. The method does not depend on a large-scale data set, and has higher detection speed and better effect.
Description
Technical Field
The invention belongs to the technical field of electric power facility safety detection, relates to inspection of an electric power tower and an electric power line, and particularly relates to an electric power tower and an electric power line detection method for correlation correction of the electric power tower and the electric power line.
Background
The traditional power line inspection mainly comprises manual inspection, helicopter inspection and other modes. The manual inspection is a mode of manually inspecting and checking the state of a power line after a worker climbs the power tower. The manual inspection has many problems, on one hand, the accuracy and efficiency are not high, and especially, a large amount of manpower and material resource cost needs to be consumed for a long-distance power transmission line; on the other hand, the safety is low, and especially, due to the influence of severe weather conditions on a line in a complex environment, manual inspection is very difficult and dangerous. Helicopter inspection is one of the other common inspection modes of power lines, and is mainly a mode of performing inspection by hovering and flying over a power line after a helicopter is provided with a professional and various sensing devices. Helicopter patrols and examines and has solved some problems of artifical patrolling and examining to a certain extent. For example, the helicopter can reach complex terrains such as canyons which are difficult for people to go to, and the inspection efficiency is greatly improved. However, the helicopter is inflexible to use, too large to enter many specific areas, and the inspection and maintenance costs are high, so that the helicopter inspection mode can be adopted only under certain special conditions. To sum up, the current traditional inspection mode cannot meet the inspection requirements of more and more huge power lines.
In recent years, with the development of artificial intelligence technology and robot technology, the technical scheme of relying on an unmanned aerial vehicle to arrange simple cameras to collect data and understand images through the artificial intelligence technology to inspect the power line has appeared. The unmanned aerial vehicle patrols and examines the mode and receives weather influence little to can work for a long time, not only improve greatly and detect the frequency and patrol and examine the quality, also guaranteed workman's safety when reducing the human cost, consequently embodied obvious advantage in the power line is patrolled and examined work. However, in the inspection mode of the unmanned aerial vehicle with the camera, the key is how to detect and identify the power line and the power tower by using the computer vision identification technology.
At present, the main methods for detecting power lines in images by researchers at home and abroad can be divided into two categories, namely a traditional image processing method and a data-driven machine learning method. The image processing method is to define the characteristic mode of the power line according to the physical and geometric characteristics of the power line and the like and artificial experience to identify the mode of the image to be detected. For example, according to the prior knowledge, the power line is regarded as a continuous straight line, and the detection of the power line is completed by a classical line segment detection method, such as Hough transformation, radon transformation, direction filtering and a straight line detection algorithm based on gradient and edge information. The traditional image processing method is easily interfered by noise, and the manual experience of the traditional image processing method is mostly specific to a specific scene, and the parameters need to be frequently adjusted in the face of different environments, even the characteristic mode is redefined, so that a better detection effect can be obtained. The data-driven machine learning method is characterized in that machine learning model learning and training are carried out on a large amount of marked data, and then a detection model capable of effectively describing power line characteristics is obtained. The best effect of the machine learning method is a method based on a deep learning model (usually a convolutional neural network). Data-driven machine learning methods typically require a large number of manually labeled datasets to train the model, and post-processing optimization of the extracted power line using power line structure information.
Meanwhile, the main technical route for detecting the power tower at present is to adopt a target detection method in the field of computer vision. In addition, the method with good effect is also a target detection method based on a deep learning model, such as a two-stage detection method based on fast R-CNN, mask R-CNN and the like and a one-stage target detection method based on YOLO. The two-stage detection algorithm has high precision, but has low detection speed, and is not suitable for real-time inspection of the unmanned aerial vehicle; a regression-based one-stage target detection algorithm, such as a YOLO algorithm and an SSD algorithm, is high in detection efficiency, but the detection accuracy rate cannot be guaranteed, and the false detection rate and the missing detection rate are high.
Even so, the overall detection accuracy of the conventional power line and power tower detection methods for power inspection and other applications is limited, and the requirements of high reliability and high availability are not completely met. The problem of insufficient machine learning model training caused by insufficient label training data also has the challenges of poor universality of the traditional image processing method caused by complexity and diversity of power lines, power towers and environments where the power lines and the power towers are located. In particular, power routing is a particular field problem in that less image data is available and less image data is marked. Thus, the data-driven machine learning approach, while theoretically more effective, lacks sufficient data to train to obtain a highly available detection model. In addition, the existing methods usually perform power line detection or power tower detection alone, and do not fully utilize prior knowledge in the power transmission field, such as the correlation structure relationship between the power line and the tower.
Disclosure of Invention
In order to overcome the above drawbacks of the prior art, an object of the present invention is to provide a power tower and a power line detection method for power tower and power line correlation correction, so as to solve at least one of the problems of insufficient training caused by less image data, poor detection accuracy of the power tower and the power line in a complex environment, and the like.
In order to achieve the purpose, the invention adopts the technical scheme that:
a power tower and power line detection method for power tower and power line correlation correction comprises the following steps:
step 1, shooting images of a power tower and a power line from the upper part, and carrying out gray-scale pretreatment on the images;
step 2, carrying out edge detection on the preprocessed image to obtain a binary edge image;
step 3, performing power line detection by using the edge map, wherein the power line detection sequentially comprises line segment detection, line segment combination and parallel line group clustering to obtain a power line represented by a parallel line group;
step 4, carrying out power tower detection by using the edge map to obtain a power tower area framed by a rectangular frame;
step 5, mapping the rectangular frame to the edge map, excluding background areas on two sides of the rectangular frame of the power tower, and re-detecting the power line in the internal area of the rectangular frame of the power tower; and after a new power line detection result is obtained, estimating the ratio of the power tower area in the edge graph according to the left and right widths of the power line distribution area, updating power tower detection parameters, and re-detecting the power tower until the requirements are met or the iteration times are reached.
In one embodiment, in step 1, each image taken contains only one power tower target and includes a power line; the preprocessing comprises the steps of adjusting image resolution, graying the image, equalizing a histogram and denoising; wherein, in the graying process, only the red and blue channels are reserved to generate the grayscale image.
In one embodiment, the step 3 detects two end points of a line segment in the edge map by using a probability Hough transformation; then calculating coordinates of the center points of the line segments, slope and intercept of the line segments, grouping according to the geometric distance of the line segments, fitting the line segments in each group into straight lines by using a least square method after grouping, extending the straight lines to a through image, and finally classifying the straight lines with approximate slopes into parallel line groups; and finally, clustering the parallel line groups according to the slope, reserving the straight line with the most counts in the clustering result, and filtering the disordered straight line with large difference with the slope of the power line.
In one embodiment, in the obtained power lines represented by the parallel line groups, the width of the power tower is estimated from the distance D between the two power lines at the edge, the area W where the power tower is located is estimated according to the aspect ratio of the power tower, and the edge map is rotated as a whole according to the power line detection result, so that the area of the power tower in the map is horizontal, wherein α is the included angle between the line group obtained after clustering and the vertical direction.
In an embodiment, in step 4, performing corner detection on the edge map by using a corner detection method based on a curvature scale space to obtain a corner distribution map; then, performing integral projection on the angular point distribution diagram in the horizontal direction and the vertical direction respectively through pixel statistical projection to obtain projection histograms in the horizontal direction and the vertical direction; replacing each projection value in the horizontal direction with a median value obtained by n adjacent projections on the left and right sides and replacing each projection value in the vertical direction with a median value obtained by m adjacent projections on the upper and lower sides in a median smoothing mode to obtain a smooth projection histogram; and respectively estimating the occupation ratio of the power tower in the edge map according to the width and the height of the power line region, dividing the projection histogram maps in the horizontal direction and the vertical direction into a plurality of small regions according to the occupation ratio, and calculating the fall of each small region and an adjacent region, wherein the region with the maximum fall has the left and right boundaries and the upper and lower boundaries of the power tower respectively.
In one embodiment, the updating the power tower detection parameter is updating the number of divisions of the small area.
In one embodiment, in the step 5, the result change degree in the iterative detection process of the power tower uses the IoU as an evaluation index, and the number of iterations is determined according to the IoU; the IoU is a result obtained by dividing a part where the two areas are overlapped by a set part of the two areas, and is used for measuring the overlapping degree of the two power tower detection results; when the IoU is larger than a set threshold value t, considering that the two detection results are consistent, outputting the current result as a final result, and ending the task; and when the IoU is less than or equal to the threshold t, continuing to perform the next round of iterative detection, if the iteration number exceeds the given maximum iteration number, forcibly ending the task, and outputting the detection result of the last round.
The invention also provides a power tower and power line detection system for power tower and power line correlation correction, which comprises:
the preprocessing module is used for carrying out gray level preprocessing on the images of the shooting power tower and the power line from the upper part;
the edge detection module is used for carrying out edge detection on the preprocessed image to obtain a binary edge image;
the power line detection module is used for carrying out power line detection by utilizing the edge graph and sequentially comprises line segment detection, line segment merging and parallel line group clustering to obtain power lines represented by parallel line groups;
the power tower detection module is used for detecting the power tower by utilizing the edge map to obtain a power tower area framed by a rectangular frame;
the correlation correction module is used for mapping the rectangular frame to the edge map, eliminating background areas on two sides of the rectangular frame of the power tower and detecting power lines in the inner area of the rectangular frame of the power tower again; and after a new power line detection result is obtained, estimating the ratio of the power tower area in the edge graph according to the left and right widths of the power line distribution area, updating power tower detection parameters, and re-detecting the power tower until the requirements are met or the iteration times are reached.
In one embodiment, the detection system is deployed on an embedded platform of the unmanned aerial vehicle, and provides detection and positioning functions for positions of a power tower and a power line for a flight control program of the unmanned aerial vehicle; the images of the power tower and the power line are shot and acquired by the unmanned aerial vehicle.
Compared with the prior art, the method for independently detecting the power line and the power tower does not depend on a large-scale data set, has higher detection speed, and has better detection effect by performing correlation correction on the detection results of the power line and the power tower.
Drawings
FIG. 1 is a flow chart of the detection algorithm of the present invention.
FIG. 2 is a schematic diagram of power towers and power lines in clustered images of the present invention.
FIG. 3 is a schematic diagram of the area of a power tower after rectification in accordance with the present invention.
Fig. 4 is a schematic view of the corner projection of the present invention.
Fig. 5 is a schematic of the power line of the present invention excluding area re-detection on both sides of the power tower area.
FIG. 6 is a diagram of the detection area obtained by the IOU (intersection ratio) calculation method of the present invention, wherein the left diagram is the intersection and the right diagram is the union.
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the drawings and examples.
On one hand, the traditional target detection method faces the problem of less data, and on the other hand, the detection accuracy is still not high under the complex environment. Therefore, the invention provides a power tower and a power line detection method, which adopt an image processing route to effectively deal with the challenge of less image data, and can better deal with the problem of detection accuracy of the power tower and the power line in a complex environment through the correlation detection of the power line and the tower.
The invention aims to detect a power tower and a power line in a picture, wherein the power tower is framed by a rectangular frame on the periphery of the power tower, and the power line is marked by a straight line. The main processing flow according to the invention, referring to fig. 1, mainly comprises the following steps:
step 1, shooting images of a power tower and a power line, and performing necessary preprocessing on the obtained images, wherein the preprocessing at least comprises graying. Generally, images should be taken from above the power tower and the power line.
And 2, carrying out edge detection on the preprocessed image to obtain a binary edge image.
And 3, performing power line detection by using the edge graph, wherein the power line detection sequentially comprises line segment detection, line segment combination and parallel line group clustering to obtain a power line represented by a parallel line group. That is, two ends of a line segment are detected, then fitted to be a straight line, and clustered according to the degree of similarity of slopes.
And 4, detecting the power tower by using the edge map to obtain a power tower area framed by a rectangular frame.
Step 5, mapping the rectangular frame to the edge map, excluding background areas on two sides of the rectangular frame of the power tower, and re-detecting the power line in the internal area of the rectangular frame of the power tower; and after a new power line detection result is obtained, estimating the ratio of the power tower area in the edge graph according to the left and right widths of the power line distribution area, updating power tower detection parameters, and re-detecting the power tower until the requirements are met or the iteration times are reached.
Wherein the power line detection of step 3, the power tower detection of step 4 and the associated correction of step 5 are performed iteratively, as shown in fig. 1.
The method of the invention is mainly based on the method, firstly, the invention realizes the initial detection of the power line and the power tower by utilizing a more efficient traditional image processing method; and then, by using knowledge in the power field such as the incidence relation between the power line and the power tower, the gradual iterative organic fusion and mutual correction of line detection and tower detection are realized. Finally, one of the two problems described above is overcome simultaneously or at least.
Correspondingly, the invention provides a power tower and power line detection system for power tower and power line correlation correction, which comprises: the device comprises a preprocessing module, an edge detection module, a power line detection module, a power tower detection module and an associated correction module. In the following description, the execution content of the preprocessing module is consistent with step 1, the execution content of the edge detection module is consistent with step 2, the execution content of the power line detection module is consistent with step 3, the execution content of the power tower detection module is consistent with step 4, and the execution content of the associated rectification module is consistent with step 5.
The method and the system can be particularly used for detecting the power tower and the power line in the image picture acquired by the unmanned aerial vehicle power patrol, namely, the detection method or the detection system is deployed on the unmanned aerial vehicle embedded platform to provide the functions of detecting and positioning the positions of the power tower and the power line for the unmanned aerial vehicle flight control program. The unmanned aerial vehicle power inspection scene is as follows: after flying above the power tower and the power line by the unmanned aerial vehicle carrying the camera, shooting the power tower and the power line below according to a depression angle; considering the very long distance between the power towers in the long-distance power transmission scenario, the unmanned aerial vehicle according to the present invention only includes one tower target, that is, only includes one power tower target in each image captured, and at the same time, includes the power line. The present invention will be described in detail below by taking the execution content of each module as an example.
1. Preprocessing module (executing step 1)
The functions of the preprocessing module include adjusting image resolution, image graying, histogram equalization, and denoising. The preprocessing module receives video data shot by the unmanned aerial vehicle, and decomposes the video into single-frame images for preprocessing. Generally, an image shot by an unmanned aerial vehicle lens is a high-resolution color RGB image, and the direct processing speed is low, so that the image is firstly zoomed according to a certain proportion and is adjusted to the resolution which is convenient to process.
In the process of image graying, three channels of the RGB image are firstly separated. The unmanned aerial vehicle aerial images are usually based on the ground, and the main contents include green vegetation, yellow land, a blue-green water area and silvery power lines under normal illumination conditions. The silver-white color of the power line is composed of RGB (192, 192, 192), and contains more blue (B) primary color information than the background color. In order to eliminate the influence of the ground complex environment on the power line extraction, the method only needs to reserve the red and blue channels to generate the gray level image.
Since the power line detection is performed outdoors, overexposure or underexposure may be caused due to environmental or light change, which may result in a dark image as a whole, or may result in different contrast of the target line in the image, which may affect the image segmentation and power line extraction in the later stage. Therefore, in the process of processing the digital image, the number of times of the occurrence of the pixels with different gray levels is counted by utilizing a gray level histogram, the abscissa of the histogram is 0-255 gray levels, the ordinate of the histogram is the number of the occurrence of the pixels with different gray levels in the image, and the gray level histogram of the image can be used for observing the gray level distribution state of the image. Histogram equalization is to perform nonlinear stretching on the image and redistribute the pixel values in the image so as to achieve the purpose that the number of the pixel values in a certain area is approximately equal. Because the amount of information contained in an image is larger when the gray value distribution is more balanced in an image, that is, the color gradation of the result obtained by histogram equalization is more distinct, the linear features in the image are more recognizable.
The image is then smoothed using gaussian filtering or the like. The gaussian filtering is a linear smooth filtering, is suitable for eliminating gaussian noise, and is widely applied to a noise reduction process of image processing. Generally speaking, gaussian filtering is a process of performing weighted average on the whole image, a template is used for performing convolution calculation on the value of each pixel point, and then the value calculated by weighted average is used for replacing the value of the center of the template.
2. Edge detection module (executing step 2)
Edge detection is a parallel boundary segmentation technology based on gray discontinuity, and is the first step of all boundary segmentation methods. The edge is the boundary between the object and the background, and the edge extraction is an important step for distinguishing the object from the background. In general, the edge detection method is implemented by using the difference between the background and the target in the characteristics of color, texture, gray scale, and the like. The detection edge calculation is generally performed by using a first or second derivative, but in an actual digital image, derivation is approximated by using a difference operation instead of a differential operation. The points on both sides of the edge in the image have abrupt changes in their gray values, so these points will have large differential values, which are the largest when the direction of the differential is perpendicular to the boundary. The image edge can be obtained according to the characteristics. The Canny operator is used in the module, and the method comprises the following steps:
1) Gradient and direction calculation: and calculating the gradient and the direction of each pixel point by using a Sobel operator.
2) Non-maxima suppression: and eliminating spurious responses caused by edge detection.
3) Double threshold value: true and potential edges are detected.
4) The hysteresis technology comprises the following steps: edge detection tracking boundaries is accomplished by suppressing weak edges.
Edge detection has two roles: the edge of the power line image is an important characteristic of the power line, and the edge detection is an essential step of the power line detection; in addition, after the edges of the image are extracted, filling, extending and smoothing are carried out on the edges, so that corner point detection can be carried out according to the change rate of pixel points on the edges, and further power tower detection can be carried out.
3. Power line detection module (executing step 3)
The input of the module is a binarized edge map obtained after edge detection. The power line detection steps mainly comprise line segment detection, line segment combination and parallel line clustering.
1) The line segment detection adopts probability Hough transformation. The Hough transform is a feature extraction technique in image processing, which detects an object having a specific shape by a voting algorithm. In the process, a set which is in accordance with the specific shape is obtained in a parameter space by calculating the local maximum value of the accumulated result and is used as a Hough transformation result, and the method is a common method for detecting line segments in an image. The standard Hough transform essentially maps an image onto its parameter space, which requires the computation of all M edge points, which results in a large amount of computation and memory space required. If only M (M < M) edge points are processed in the input image, the M edge points are selected with a certain probability, which is the probability Hough transform. The power line in the image shot by the actual unmanned aerial vehicle usually cannot be displayed as a clear and continuous long straight line but a plurality of short broken line segments, and the probability Hough transformation has the important characteristic of being capable of detecting two end points of the line segments in the image and accurately positioning the straight line in the image. Therefore, the invention uses probability Hough transformation to combine distance measurement to complete grouping and combination of straight lines on the basis of the traditional straight line detection algorithm, namely two end points of a line segment in an edge graph are detected, so that a broken line segment can be fitted into a straight line, and the integrity of power line detection is improved.
2) Merging line segments: and after obtaining the coordinates of the line segment end points through probability Hough transformation, calculating the coordinates of the line segment center points, the line segment slopes and the intercept, and grouping according to the geometric distance of the line segments. And after grouping, fitting the line segments in each group into straight lines by using a least square method, extending the fitted straight lines to penetrate through the image, and finally classifying the straight lines with the similar slopes into parallel line groups.
3) Parallel line group clustering: in images taken by drones, power lines generally exhibit the following characteristics: approximately straight, with a range of tilt angles, approximately parallel between power lines, throughout the entire image, vertically across the power tower area. Therefore, the parallel line groups are clustered according to the slope, the straight line with the most number in the clustering result is reserved, and the disordered straight line with larger difference with the slope of the power line can be filtered.
In the embodiment of the invention, the parallel line group clustering adopts a K-means + + algorithm. The K-means is a common clustering algorithm based on Euclidean distance, and considers that the closer the distance between two targets is, the greater the similarity is, and the K-means + + algorithm optimizes the selection of the initial clustering center. The clustering process is roughly: i) Firstly, randomly selecting K samples (K parallel line groups with different slopes) from a sample set (a flat line group set, wherein each sample is a flat line group) as a cluster center, and calculating the distance between all the samples and the K cluster centers; ii) for each sample, dividing it into clusters where the "cluster center" closest to it is located, calculating a new "cluster center" for each cluster for the new cluster; iii) The above process is repeated until the "cluster center" has not moved.
As shown in fig. 2, after clustering, an included angle α between the detected power line group and the vertical direction is calculated for image correction, and the position and size of the power tower region can be estimated according to the power line distribution region. Specifically, as shown in fig. 3, in the power lines represented by the obtained parallel line groups, the width of the power tower can be estimated from the distance D between the two power lines at the edge, and the area W where the power tower is located can be estimated according to the aspect ratio of the power tower. The reason why the image needs to be rectified before the power tower detection is that a horizontal bounding box is generally used in the target detection task to represent the approximate range of the target in the image, while the object in the unmanned aerial vehicle image is generally in any direction, and the use of the horizontal bounding box to detect the target causes that this type of object detection box contains many background areas and cannot reflect the size and aspect ratio of the target object. This not only increases the difficulty of the detection task, but also can lead to inaccurate target range representation.
According to the structure of the power tower and the power lines, a plurality of power lines vertically penetrate through one power tower area, the power towers are not densely distributed in the unmanned aerial image, and only one power tower area is arranged in the image. Therefore, the invention utilizes the characteristic to rotate the edge map by alpha integrally according to the power line detection result, so that the power tower area in the map can be horizontal, and the subsequent power tower detection is more accurate.
4. Power tower detection module (executing step 4)
The input of the power tower Detection module is an edge map obtained by the edge Detection module, and after the edge map is obtained by the module through Corner Detection (such as a Curvature Scale Space Corner detector, CSS Corner Detection method), the module performs threshold judgment on horizontal and vertical projection of corners to determine the upper, lower, left and right boundaries. The basic principle is that a complex steel frame structure which is staggered horizontally and vertically exists inside the power tower, and the density of the edges and the angular points inside the tower is obviously greater than that of a background area. Thus, this feature can be utilized to achieve differentiation between tower and background areas.
In fact, most existing methods directly use edge information for statistics of edge point density distribution for power tower detection. However, this method has some disadvantages, for example, the selection of the high and low thresholds of the most commonly used Canny edge detection operator is not set according to the image characteristics, but the threshold setting is performed by means of a priori experience. If the high and low thresholds are set to be larger in the edge detection process, more details of the image edge are lost, and thus, the result of detecting the edge discontinuity is obtained. If the high and low thresholds are set to be small, false detection of edges will occur.
Therefore, in the invention, for the internal structure of the power tower, corner Detection based on the Curvature Scale Space is further performed on the basis of edge Detection, namely, a Curvature Scale Space Corner Detection (CSS Corner Detection) algorithm. The algorithm has the following steps: firstly, filling gaps in a binary edge contour by using an image contour extracted by a Canny edge detection operator and other methods; secondly, calculating the curvature of each pixel point on the contour under a large scale after filling, and if the curvature exceeds a threshold value, judging as a candidate corner point; and finally, tracking each pixel point in the candidate corner set under a small scale, and accurately positioning the position of the corner.
After the corner distribution graph of the image is obtained, the corner distribution graph is projected in the horizontal direction through pixel statistical projection, and a projection histogram in the horizontal direction is obtained. In the projection histogram, the height of the projection indicates the edge density in the horizontal direction, and the projection height is higher at a position where the edge density is large. Meanwhile, each projection value in the horizontal direction is replaced by a median value of n adjacent projections on the left and right sides by a median smoothing mode to eliminate noise. The smoothing process is mainly to remove isolated noise points in the image while preserving the details of the image well. This results in a smoother projection histogram as shown in fig. 4. In the projection histogram, the height of the projection column inside the power tower is far larger than that of the background area, and the edge of the power tower has obvious wave peaks. The occupation ratio of the power tower in the edge map can be estimated according to the width of the power line area, the projection histogram is divided into a plurality of small areas according to the occupation ratio, the fall between each small area and the adjacent area is calculated, and the left and right boundaries of the power tower exist in the area with the maximum fall. Similarly, projecting the vertical direction may be used to determine the upper and lower boundaries of the power tower. Performing integral projection on the angular point distribution diagram in the vertical direction through pixel statistical projection to obtain a projection histogram in the vertical direction; replacing each projection value in the vertical direction with a median value obtained by m adjacent projections up and down in a median smoothing mode to obtain a smooth projection histogram; and estimating the occupation ratio of the power tower in the edge map according to the height of the power line region, dividing the projection histogram map in the vertical direction into a plurality of small regions according to the occupation ratio, and calculating the fall of each small region and an adjacent region, wherein the region with the maximum fall has the upper and lower boundaries of the power tower.
5. Correlation correction module (execution step 5)
After the modules finish the primary detection of the power lines and the power tower, the power lines represented by the parallel line groups and the power tower area framed by the rectangular frame can be obtained. And then, the association and verification module carries out iterative updating (namely, the step 3, the step 4 and the step 5 are iteratively executed) according to the spatial structure association relationship between the power tower and the power line, and corrects the detection results of the power line and the power tower.
Specifically, a rectangular frame representing the power tower detection result is first mapped to the edge map, background areas on both sides of the rectangular frame area of the power tower are excluded, and the power lines are re-detected in the internal area of the rectangular area of the power tower by the same method as the initial detection. Secondly, after a new power line detection result is obtained, the occupation ratio of the power tower area in the image is estimated according to the left width and the right width of the power line distribution area, the parameters of the power tower detection module (namely the number of small area partitions in the power tower detection module) are updated, and the histogram projection method is reused for detecting the power tower.
IoU (interaction over Unit) is used as an evaluation index according to the result change degree in the iterative detection process of the power tower, and the number of iterations is determined according to the IoU. The IoU is the result obtained by dividing the overlapped part of the two areas by the integrated part of the two areas, and is used for measuring the overlapping degree of the two power tower detection results. When the IoU is larger than a set threshold value t, considering that the two detection results are consistent, outputting the current result as a final result, and ending the task; and when the IoU is less than or equal to the threshold t, continuing to perform the next round of iterative detection, if the iteration number exceeds the given maximum iteration number, forcibly ending the task, and outputting the detection result of the last round.
The following describes a primary workflow of the present detection algorithm. The process is that a small unmanned aerial vehicle transmits pictures in real time, and an algorithm automatically detects the power tower and the power line.
Firstly, the unmanned aerial vehicle is started, the camera and the laser radar are started, the unmanned aerial vehicle hovers over the power tower, and images with the resolution of 1600 x 1200 pixels are shot. After receiving the image transmitted by the unmanned aerial vehicle, the preprocessing module scales the image to 800 × 600 resolution according to the proportion so as to facilitate subsequent processing. And after zooming, separating three channels of RGB of the image, and extracting a red channel and a blue channel to generate a gray image. And denoising the gray map by using Gaussian filtering. The two-dimensional gaussian function is formulated as follows:
two-dimensional gaussian distribution:
in the formula, δ is a gaussian distribution parameter, which can be calculated from the size of a filter kernel, and a 3 × 3 template is used to filter the image.
And inputting the preprocessed gray level image into an edge detection module, performing edge detection by using a Canny algorithm, and detecting and identifying a set formed by pixels with severe brightness change in the image. The Canny algorithm uses the Sobel operator.
Formula for calculating the convolution kernel:
a is the original image.
The horizontal and vertical gray values of each pixel of the image are combined by the following formula to calculate the size of the point gray:
G＝|G x |+|G y |
after the edge detection, the edge map is respectively subjected to power line detection and power tower detection. The power line detection uses probability Hough transformation to detect line segments, deletes and merges repeated line segments by calculating the slope and the mutual distance of the line segments, and classifies parallel line segments.
The algorithm flow of the line segment detection and combination is as follows:
1) And acquiring coordinates of line segment endpoints through probability Hough transformation.
2) And calculating the coordinates of the center point of the line segment, the slope and the intercept of the line segment, and marking the state as UNUSED.
3) Selecting the line segment where the minimum point of the ordinate in the end points is as the initial line segment L j From this point to the other line segment L i Geometric distance of center point is D i 。
4) Traversing the line segment set with the state of the UNUSED line segment, and setting the distance threshold value as W θ If D is i <W θ Then store the line segment into the line segmentGroup K j Setting the state of the line segment to be USED, and if all the line segments are traversed, re-performing the step 2) until the state of all the line segments is USED.
5) Obtain line segment group K j And fitting the line segments by using the coordinate of the end point of each line segment in the image through a least square method to obtain a fitted straight line and then extending the fitted straight line to penetrate through the whole image.
6) The slopes of the fitting straight lines are sequentially compared, and the slope difference value is smaller than a threshold value S θ The straight lines of (2) are classified into parallel line groups, and the parallel line groups are represented by average slopes.
After classification, the parallel line groups are clustered according to the slope, interference straight lines are eliminated, and finally the power lines are detected. Clustering uses the Kmeans + + algorithm, and the steps are as follows:
1) Selecting K parallel line groups as initial clustering center points { C 1 ,C 2 ,C 3 ,…,C k I.e. the initially aggregated cluster center, where C i Representing the slope of the ith parallel line group.
2) Respectively calculating the sample point X represented by each parallel line group i And (3) finding the cluster center closest to the point according to Euclidean distances to the K cluster centers, and attributing the cluster center to a corresponding cluster, wherein the Euclidean distance is calculated as follows:
3) After all points belong to a cluster, all M sample points (parallel line groups) are divided into K clusters { S } 1 ,S 2 ,S 3, …,S k }. The center of gravity (average distance center) of each cluster is then recalculated, and is designated as the new "cluster center", which is calculated as follows:
4) And iterating the steps 2-3 repeatedly until the cluster center is not changed any more.
Theoretically, in a picture taken by the drone, the number of line segments belonging to the power line is the largest, and the slopes of these line segments are almost the same. Therefore, through the clustering process, the cluster with the largest number of the obtained line segments should be the line segment set corresponding to the power line. Next, one cluster with the most parallel line segments is selected from all the clusters, and the average slope α of all the line segments in the cluster is calculated, where the slope is the inclination angle of the power line in the current picture. Subsequently, the image is rotated by an angle α according to the slope α of the power line, so that the power line is perpendicular to the horizontal direction. Meanwhile, since the power line should be approximately perpendicular to the peripheral rectangular frame of the power tower in practical situations, the power tower area should be a view parallel to the vertical direction on the left and right sides of the power tower as shown in fig. 3 after the above-mentioned rotating operation.
Meanwhile, the edge image of the original image is correspondingly rotated to obtain the edge image after rotation correction, and then CSS corner detection is carried out. The process of corner detection is as follows:
1) Aiming at the edge contour extracted by a Canny edge detection operator, when an edge point is an end point, if other end points exist in the neighborhood of the end point, replacing a non-edge point between the two end points with the edge point; if there are other edge lines in the end point's neighborhood, it is marked as a T-type node.
2) And calculating the curvature of each pixel point on the contour by using a large-scale Gaussian filter function aiming at each edge contour.
3) If the curvature of a certain pixel point on the edge line is larger than a preset threshold value, is a local unique maximum value, and simultaneously satisfies that the curvature value is larger than twice of the minimum value of the curvature in the neighborhood, the pixel point is marked as a candidate corner point, and the curvature is calculated as follows:
the edge curve is defined according to the arc length coefficient u:
Γ(u)＝(x(u),y(u))
performing smooth denoising on the extracted edge curve by using a one-dimensional Gaussian filter function to obtain a smooth curve:
Γ(u,σ)＝(X(u,σ),Y(u,σ))
wherein
Representing a convolution operation, g (u, σ) represents a one-dimensional gaussian filter function with standard deviation σ. The curvature function of the curve can be obtained according to the edge curve function:
where g '(u, σ), g' (u, σ) are the first and second partial derivatives of g (u, σ) with respect to u, respectively.
4) Because the large-scale filter used in 2) blurs the height of the curve to obtain only a candidate corner set which is roughly screened, each pixel point in the candidate corner set needs to be tracked by using small-scale Gaussian filter to accurately position the corner, so that the positioning accuracy of the corner is improved
5) And deleting one of the obtained T-shaped nodes and the detected candidate corner points if the two are adjacent.
After obtaining a binarized corner point image through corner point detection, respectively performing integral projection on the corner point image in the horizontal direction and the vertical direction to obtain projection histograms in the horizontal direction and the vertical direction. Taking the vertical direction as an example, the number of corner points of each column of the image is accumulated during projection, the number of the corner points is represented by the height of a column, and the distribution of the corner points in the vertical direction can be visually represented.
And after obtaining the integral projection histogram, respectively carrying out smooth denoising on the projection histogram by adopting the idea of median filtering. For example, traversing the projection graph by using a sliding window with the size of 9 pixels, storing the heights of all columns of projection columns to form a sequence, sorting the heights of the projection columns in the sequence from large to small, storing the height of the projection column at the middle position, namely a median value, and giving a pixel point at the middle position of the sliding window, so that a smoother corner point projection graph can be obtained.
The width of the power line distribution obtained by power line detection is W, W can be approximately regarded as the width of a power tower, and the image size X after the proportion is adjusted is combined 0 *Y 0 And dividing the projection image into R (x, y) small areas in the transverse direction and the longitudinal direction after denoising. The calculation method is as follows:
to be provided with
For example, the number of corner points per column (row) in the first cell { p } 00 ,p 01 ,…,p 0m Is accumulated, wherein>
The corner density of a region is represented by the total number of corners P in the region. The number of corner points P is calculated as follows: />
To obtain { P 0 ,P 1 ,P 2 ,…,P 19 Calculating the absolute value of the difference between each small area and the adjacent area:
D i ＝|P i+1 -P i |
maximum difference max { D 0 ,D 1 ,D 2 ,…,D i There is a side boundary of the power tower in between the two corresponding areas.
Similarly, projecting the vertical direction may determine the upper and lower boundaries of the power tower.
Meanwhile, correlation analysis is carried out on a plurality of parallel line groups obtained by detecting the angle point diagram and the line segments, and irrelevant parallel line groups are removed. The correlation analysis here mainly considers that the power line is generally connected to some support position of the power tower, and the support position is usually a corner point, so that the candidate line which is not possible to be the power line can be further eliminated by judging whether the detected line passes through the corner point in the corner point diagram. Specifically, after all the parallel line groups are extended into a straight line spanning the entire image by minimum binarization, it is sequentially determined whether the straight line passes through any corner point in the corner point map. Parallel lines that do not pass through any one corner point are removed and are no longer a possible choice for power lines.
After the primary detection of the power line and the power tower is finished, correcting the result through iterative detection, wherein the iterative process is as follows:
1) And mapping the power tower detection result to the edge detection map.
2) As shown in fig. 5, excluding regions W on both sides of the power tower region 1 、W 2 The power line is re-detected.
3) And estimating the occupation ratio of the power tower area in the image according to the left and right width of the power line distribution.
4) And reusing the projection method to detect the power tower according to the result of the step 3).
5) And after the power tower is detected again, calculating the IoU value of the previous power tower detection result. The IoU threshold is set to 0.7. After correction, comparing the result with the previous detection result, if the IoU is more than 0.7, indicating that the two detection results are consistent, outputting the result, and ending the task; if the value is less than or equal to 0.7, the deviation is generated from the previous detection result, and the iteration is continued until the result is consistent or exceeds the limit iteration number. B is 1 、B 2 Is as shown in FIG. 6The illustrated two detected regions, ioU, are calculated as follows:
Claims (10)
1. a power tower and power line detection method for power tower and power line correlation rectification is characterized by comprising the following steps:
step 1, shooting images of a power tower and a power line from the upper part, and carrying out gray-scale pretreatment on the images;
step 2, carrying out edge detection on the preprocessed image to obtain a binary edge image;
step 3, performing power line detection by using the edge map, wherein the power line detection sequentially comprises line segment detection, line segment combination and parallel line group clustering to obtain a power line represented by a parallel line group;
step 4, carrying out power tower detection by using the edge map to obtain a power tower area framed by a rectangular frame;
step 5, correcting the detection results of the power line and the power tower through iterative detection, wherein the method comprises the following steps: mapping the rectangular frame to the edge map, excluding background areas on two sides of the rectangular frame of the power tower, and re-detecting the power lines in the internal area of the rectangular frame of the power tower; and after a new power line detection result is obtained, estimating the ratio of the power tower area in the edge graph according to the left and right widths of the power line distribution area, updating power tower detection parameters, and re-detecting the power tower until the requirements are met or the iteration times are reached.
2. The method for detecting the power tower and the power line according to claim 1, wherein in the step 1, each image is taken, wherein the image only contains one power tower target and the power line; the preprocessing comprises the steps of adjusting image resolution, graying the image, equalizing a histogram and denoising; during the graying process, only the red and blue channels are reserved to generate a gray map.
3. The method for detecting the power tower and the power line associated with rectification according to claim 1, wherein the step 3 is to detect two end points of a line segment in the edge map by using probability Hough transformation; then calculating coordinates of the center points of the line segments, slope and intercept of the line segments, grouping according to the geometric distance of the line segments, fitting the line segments in each group into straight lines by using a least square method after grouping, extending the straight lines to a through image, and finally classifying the straight lines with approximate slopes into parallel line groups; and finally, clustering the parallel line groups according to the slope, reserving the straight line with the most counts in the clustering result, and filtering the disordered straight line with large difference with the slope of the power line.
4. The method as claimed in claim 3, wherein the power tower and power line detection method comprises estimating the width of the power tower from the distance D between the two power lines at the edge of the power tower among the power lines represented by the obtained parallel line groups, estimating the area W where the power tower is located according to the aspect ratio of the power tower, and rotating the edge map as a whole according to the power line detection result to make the power tower area in the map horizontal, wherein α is the included angle between the clustered line groups and the vertical direction.
5. The method for detecting the power tower and the power line associated with rectification according to claim 4, wherein in the step 4, an angular point detection method based on a curvature scale space is adopted to detect the angular point of the edge map, so as to obtain an angular point distribution map; then, performing integral projection on the angular point distribution diagram in the horizontal direction and the vertical direction respectively through pixel statistical projection to obtain projection histograms in the horizontal direction and the vertical direction; replacing each projection value in the horizontal direction with a median value obtained by n adjacent projections on the left and right sides and replacing each projection value in the vertical direction with a median value obtained by m adjacent projections on the upper and lower sides in a median smoothing mode to obtain a smooth projection histogram; and estimating the occupation ratio of the power tower in the edge map respectively according to the width and the height of the power line area, dividing the projection histogram maps in the horizontal direction and the vertical direction into a plurality of small areas respectively according to the occupation ratio, calculating the fall of each small area and an adjacent area, wherein the area with the maximum fall has the left and right boundaries and the upper and lower boundaries of the power tower respectively.
6. The method for detecting the power tower and the power line associated with rectification according to claim 5, wherein the method for detecting the corner based on the curvature scale space comprises: firstly, filling gaps in a binary edge profile by using an image profile extracted by edge detection; secondly, calculating the curvature of each pixel point on the contour under a large scale after filling, and if the curvature exceeds a threshold value, judging as a candidate corner point; and finally, tracking each pixel point in the candidate corner set under a small scale, and accurately positioning the position of the corner.
7. The method as claimed in claim 5, wherein the updating of the power tower detection parameter is updating of the number of divisions of the small area.
8. The method for detecting the power tower and the power line associated with rectification according to claim 1 or 6, wherein in the step 5, the degree of change of the result in the iterative detection process of the power tower uses the IoU as an evaluation index, and the number of iterations is determined according to the IoU; the IoU is a result obtained by dividing a part where the two areas are overlapped by a set part of the two areas, and is used for measuring the overlapping degree of the two power tower detection results; when the IoU is larger than a set threshold value t, considering that the two detection results are consistent, outputting the current result as a final result, and ending the task; and when the IoU is less than or equal to the threshold t, continuing to perform the next round of iterative detection, if the iteration number exceeds the given maximum iteration number, forcibly ending the task, and outputting the detection result of the last round.
9. A power tower and power line detection system for power tower and power line association rectification, comprising:
the preprocessing module is used for carrying out gray level preprocessing on the images of the shooting power tower and the power line from the upper part;
the edge detection module is used for carrying out edge detection on the preprocessed image to obtain a binary edge image;
the power line detection module is used for carrying out power line detection by utilizing the edge graph and sequentially comprises line segment detection, line segment combination and parallel line group clustering to obtain power lines represented by parallel lines;
the power tower detection module is used for detecting a power tower by using the edge map to obtain a power tower area framed by a rectangular frame;
the correlation correction module is used for mapping the rectangular frame to the edge map, eliminating background areas on two sides of the rectangular frame of the power tower and detecting power lines in the inner area of the rectangular frame of the power tower again; and after a new power line detection result is obtained, estimating the ratio of the power tower area in the edge graph according to the left and right widths of the power line distribution area, updating power tower detection parameters, and re-detecting the power tower until the requirements are met or the iteration times are reached.
10. The power tower and power line associated rectification power tower and power line detection system of claim 9, wherein the detection system is deployed on an unmanned aerial vehicle embedded platform and provides detection and positioning functions for the power tower and power line positions for unmanned aerial vehicle flight control programs; the images of the power tower and the power line are shot and acquired by the unmanned aerial vehicle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211552072.8A CN115841633A (en) | 2022-12-05 | 2022-12-05 | Power tower and power line associated correction power tower and power line detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211552072.8A CN115841633A (en) | 2022-12-05 | 2022-12-05 | Power tower and power line associated correction power tower and power line detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115841633A true CN115841633A (en) | 2023-03-24 |
Family
ID=85578062
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211552072.8A Pending CN115841633A (en) | 2022-12-05 | 2022-12-05 | Power tower and power line associated correction power tower and power line detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115841633A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116524004A (en) * | 2023-07-03 | 2023-08-01 | 中国铁路设计集团有限公司 | Method and system for detecting size of steel bar based on HoughLines algorithm |
| CN116843909A (en) * | 2023-05-12 | 2023-10-03 | 国家电网有限公司华东分部 | Power line extraction method and device, storage medium and computer equipment |
-
2022
- 2022-12-05 CN CN202211552072.8A patent/CN115841633A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116843909A (en) * | 2023-05-12 | 2023-10-03 | 国家电网有限公司华东分部 | Power line extraction method and device, storage medium and computer equipment |
| CN116843909B (en) * | 2023-05-12 | 2024-03-08 | 国家电网有限公司华东分部 | Power line extraction method and device, storage medium and computer equipment |
| CN116524004A (en) * | 2023-07-03 | 2023-08-01 | 中国铁路设计集团有限公司 | Method and system for detecting size of steel bar based on HoughLines algorithm |
| CN116524004B (en) * | 2023-07-03 | 2023-09-08 | 中国铁路设计集团有限公司 | Method and system for detecting size of steel bar based on HoughLines algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111428748B (en) | HOG feature and SVM-based infrared image insulator identification detection method | |
| CN110097536B (en) | Hexagonal bolt looseness detection method based on deep learning and Hough transform | |
| US10290219B2 (en) | Machine vision-based method and system for aircraft docking guidance and aircraft type identification | |
| CN107909081B (en) | Method for quickly acquiring and quickly calibrating image data set in deep learning | |
| CN107103320B (en) | Embedded medical data image identification and integration method | |
| CN115841633A (en) | Power tower and power line associated correction power tower and power line detection method | |
| CN110097596B (en) | Object detection system based on opencv | |
| CN108197604A (en) | Fast face positioning and tracing method based on embedded device | |
| CN111814686A (en) | Vision-based power transmission line identification and foreign matter invasion online detection method | |
| CN114549981A (en) | Intelligent inspection pointer type instrument recognition and reading method based on deep learning | |
| CN109918971B (en) | Method and device for detecting number of people in monitoring video | |
| CN109685045A (en) | A kind of Moving Targets Based on Video Streams tracking and system | |
| CN109376740A (en) | A kind of water gauge reading detection method based on video | |
| CN113139521A (en) | Pedestrian boundary crossing monitoring method for electric power monitoring | |
| CN110414308B (en) | Target identification method for dynamic foreign matters on power transmission line | |
| CN112417931B (en) | Method for detecting and classifying water surface objects based on visual saliency | |
| CN113034378B (en) | Method for distinguishing electric automobile from fuel automobile | |
| CN110751619A (en) | Insulator defect detection method | |
| CN111476804A (en) | Method, device and equipment for efficiently segmenting carrier roller image and storage medium | |
| CN114926407A (en) | Steel surface defect detection system based on deep learning | |
| CN111695373A (en) | Zebra crossing positioning method, system, medium and device | |
| CN113538503A (en) | Solar panel defect detection method based on infrared image | |
| CN113537211A (en) | Deep learning license plate frame positioning method based on asymmetric IOU | |
| CN113313107A (en) | Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge | |
| CN113435452A (en) | Electrical equipment nameplate text detection method based on improved CTPN algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |