CN113076899B - High-voltage transmission line foreign matter detection method based on target tracking algorithm - Google Patents
High-voltage transmission line foreign matter detection method based on target tracking algorithm Download PDFInfo
- Publication number
- CN113076899B CN113076899B CN202110387931.1A CN202110387931A CN113076899B CN 113076899 B CN113076899 B CN 113076899B CN 202110387931 A CN202110387931 A CN 202110387931A CN 113076899 B CN113076899 B CN 113076899B
- Authority
- CN
- China
- Prior art keywords
- target
- transmission line
- tracking
- detection
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a high-voltage transmission line foreign matter detection method based on a target tracking algorithm, which comprises the following steps: placing a camera device on the power transmission tower and transmitting data; collecting and marking foreign matters of the high-voltage transmission line; training a yolov5 model and optimizing; using Hough linear detection to the detection range to obtain the approximate range of the high-voltage transmission line; detecting the video stream, optimizing a detection result by using a soft-nms algorithm, and selecting a target to be tracked; predicting and tracking the target position by using Kalman filtering based on a Deepsort target tracking algorithm; the correlation degree of the predicted target and the detected target is calculated by using a Hungarian algorithm, and matching is carried out; obtaining whether the target track can reach the high-voltage transmission line area or not according to the existing tracking target track; and obtaining whether the tracking target is an abnormal target affecting the high-voltage transmission line or not according to the final position of the tracking target. The method can realize timely and accurate detection of the foreign matters approaching and adhering to the power transmission line and quickly give an alarm.
Description
Technical Field
The invention relates to the technical field of intelligent monitoring of an electric power system, in particular to a high-voltage transmission line foreign matter detection method based on a target tracking algorithm, which can be applied to monitoring foreign matters close to a high-voltage transmission line and giving an alarm in time.
Background
The electric power system occupies a more and more important position in the current social life and development, and the transmission line is responsible for connecting a power plant, a transformer substation and a user together and plays an irreplaceable role. The safety and stability of the device are very important. In order to reduce the energy loss during the transportation process, high-voltage power transmission is mostly adopted. However, since most of the high-voltage transmission lines are located in mountainous regions and hilly terrain, and the span is very large, the detection and maintenance are very difficult. It is more difficult to accurately find the occurrence of the unexpected situation in time. At present, one of the main hidden dangers of the high-voltage transmission line is as follows: kites, floats and the like can be wound on objects on lines, and foreign matters such as nests of birds and birds. Traditional manual inspection mode, not only the time of spending is long, and speed is slow, and the precision of patrolling and examining moreover, the promptness that the potential safety hazard was reported to police to and the safety problem of patrolling and examining personnel all is difficult to guarantee. Machine vision in recent years, or target detection schemes, are susceptible to weather or shelters, so that accuracy and timeliness of the machine vision or target detection schemes cannot be guaranteed.
The method aims to provide the high-voltage transmission line foreign matter detection method based on the target tracking algorithm, and the method has good performances in timeliness, accuracy and stability when the video stream is detected on the high-voltage transmission line. As the foreign object target is always gradually close to the high-voltage transmission line in principle, a target tracking algorithm based on the Deepsort algorithm is added and entered on the basis of the deep learning model at the present stage, and the function of target tracking is realized. The method can accurately and timely monitor the approach of the foreign object on the power transmission line, and timely alarm in the approach process so as to remind workers of paying attention to the safe operation of the power transmission line.
By combining the above discussion, the high-voltage transmission line detection method which has high precision and high early warning speed has high practical application value.
Disclosure of Invention
The invention aims to overcome the defects and shortcomings of the prior art and provides a foreign matter detection method for a high-voltage transmission line based on a target tracking algorithm. The method has simple requirements on the data set and has good performance in rapidity and accuracy. The video stream can be used to detect foreign objects approaching and adhering to the power transmission line and to quickly give an alarm.
In order to achieve the purpose, the technical scheme provided by the invention is as follows: a high-voltage transmission line foreign matter detection method based on a target tracking algorithm comprises the following steps:
1) The method comprises the following steps that a camera is installed on a power transmission tower of a high-voltage power transmission line, the angle of the camera is adjusted to enable the camera to be aligned to the high-voltage power transmission line, and the cameras are installed on the power transmission towers at two ends of the same high-voltage power transmission line, so that the same line can be captured by at least two cameras at the same time;
2) Shooting foreign matters on the high-voltage transmission line through the camera built in the step 1), and collecting pictures of the foreign matters on the Internet; arranging the obtained pictures, classifying the pictures, labeling the classified pictures in a one-to-one correspondence manner, and ensuring that the labeling modes of all the pictures are the same; randomly disorganizing the marked pictures, and dividing the pictures into a training set, a verification set and a test set;
3) Training the training set obtained in the step 2) by using a yolov5 model training method, iterating and updating the yolov5 model, selecting the yolov5 model with the minimum total loss index as a final obtained result, actually measuring the final result on a test set, and after a good target detection result is obtained, reserving the used yolov5 model for target tracking;
4) Predicting the video stream by using the yolov5 model obtained in the step 3) to obtain a detection frame at the current moment, and processing a detection result of the yolov5 model by using an improved non-maximum suppression (nms) algorithm, namely a soft-nms algorithm to obtain the yolov5 model with the soft-nms algorithm; the Soft-nms algorithm takes the overlapping degree into consideration on the basis of a non-maximum value inhibition (nms) algorithm to achieve the purpose of combining the repeatedly detected targets without omitting two close position relations;
5) Performing target detection by using the yolov5 model with soft-nms algorithm in the step 4), setting the picture of the first frame of the video stream captured by the camera as an initial state, and performing Hough direct processing on the picture in the initial statePerforming line detection to obtain the position relation of the high-voltage transmission line; extracting the minimum abscissa X of the detected set L of straight-line points min Minimum ordinate Y min And the maximum abscissa X max Maximum ordinate Y max Will be (X) min ,Y min ),(X min ,Y max ),(X max ,Y min ),(X max ,Y max ) As 4 angular points, establishing a rectangular area J where the high-voltage transmission line is located;
6) Continuously detecting the video stream in the step 5), and when a target is detected to appear, setting the current time as t 0 At the moment, the currently detected target is set to the target P i Determining whether the appearance position is located at the edge position; if so, retaining the current target P i Putting the tracking target set S into the tracking target set S, and if not, judging that the tracking target set S is not a new tracking object; as the targets needing alarming are all gradually close to the high-voltage power transmission line from a place far away from the high-voltage power transmission line, the targets are required to be located at the edge position of the whole monitoring range when being detected for the first time, the inherent attributes of the video streaming device are obtained, the numerical value of the resolution ratio is obtained, the boundary threshold value is set, the four first indexes of target output obtained by using the yolov5 model are (x, y, w, h), wherein (x, y) represents the abscissa and the ordinate of the central point of the detected object, and (w, h) represents the width and the height of a rectangular frame, so that as long as the central point (x, y) is located in the range of subtracting the boundary threshold value from the resolution ratio, the current target P is judged i The detection result is positioned at the edge position, otherwise, the detection result is not regarded as a new tracking object;
7) Will t 0 The next frame at a time is set to t 1 Time, the target detected at that time is set to P 1 Using a target tracking algorithm based on the Deepsort algorithm to track the current target P 1 Matching and tracking all targets in the target set S; if the current target P 1 And tracking the target S 1 If the matching is successful, the target S is tracked 1 Is updated to the current target P 1 And S is calculated from the position information of 2 Putting the target set S into a target set S; if the current target P 1 Does not match any existing targets in set S, but the current targetP 1 At the edge position described in step 6), the target P is determined 1 As a new tracking target S 2 Putting the target set S into a target set S;
8) Continuously and uninterruptedly detecting the video stream, and completing target tracking through the step 6) and the step 7); according to the tracked target S i Fitting a future motion equation of the motion trajectory, namely fitting a motion equation in a straight line; substituting points in the rectangular area J into a linear fitting motion equation, and if the equation is equal left and right, judging that the tracking target S is equal i The method comprises the following steps that (1) a trend of regional movement of the high-voltage transmission line is shown, and a prompt needs to be sent;
9) For the tracking target S in step 8) i Continuing to track the target if the target S is tracked i Finally, if the high-voltage transmission line stops moving in the rectangular area where the high-voltage transmission line is located, the foreign matter which can interfere the operation of the high-voltage transmission line is judged, and an alarm needs to be given; if the target S is tracked i And finally, the monitoring range is separated from the boundary, so that the monitoring range is judged to be a passing object, the normal operation of the high-voltage transmission line is not interfered, and the reminding needs to be sent again.
In the step 1), the camera installed on the power transmission tower is lower than the height of the high-voltage power transmission line, so that the camera can shoot the high-voltage power transmission line from bottom to top while ensuring the shot pictures to be clear; by adopting two cameras, namely the bidirectional camera system can ensure that the target tracking algorithm can normally run when a plurality of foreign matters and the foreign matters are overlapped and shielded, and the rapidity and the accuracy are not influenced.
In the step 2), data enhancement is carried out on the data acquired in the camera, and the data enhancement comprises random cutting, noise addition and mirror image turning.
In the step 2), the picture is marked by using a marking tool labelme, namely, a rectangle with proper size is used for framing the corresponding target object to obtain marking information x min 、y min 、x max 、y max Wherein x is min Is the left boundary abscissa, x, of the rectangular frame max Is the right boundary abscissa, y, of the rectangular frame min Is the lower boundary ordinate, y, of the rectangular frame max Is the vertical coordinate of the upper boundary of the rectangular frame; randomly disorganizing the marked pictures, and dividing the pictures into a training set, a verification set and a test set, wherein the proportion of the training set, the verification set and the test set is 8.
In the step 4), the confidence scores of the detection frames are sorted from high to low, and the detection frame with the highest confidence is placed in the set M; the improved non-maximum inhibition algorithm is modified by reducing the confidence score of a new detection frame according to the sizes of all elements IOU in M, wherein the larger the IOU is, the more the confidence is reduced, and the smaller the IOU is, the smaller the confidence is reduced, if the confidence after iterative update is still larger than a threshold value, the new detection frame is added into the set M until all pictures go through, and through the improvement, the detection accuracy can be greatly improved.
In step 7), firstly, the targets S in the tracking target set S are i Using Kalman filtering, object S i According to t 0 State of time, predicted target S i At t 1 Position coordinates (x) of time of day i ,y i ,w i ,h i ) Through t 0 The covariance of the moment yields t 1 The prediction error of the moment is calculated to obtain the Kalman gain, and the target S is predicted through the Kalman gain i The predicted position of (a) is corrected to obtain the final predicted position coordinate (x) i ,y i ,w i ,h i ) Then to t 1 Predicting by using the yolov5 model with soft-nms algorithm in the step 4) at any moment to obtain a detection target P i Position (x' i ,y' i ,w' i ,h' i )。
In step 7), the Hungarian algorithm is used to detect the target P i And predicting the target S i Matching: calculating a detection target P i And predicting the target S i If the distance between the two is smaller than the threshold value, the two are judged to be the same target and successfully matched, and the target S is selected i Is updated to (x' i ,y' i ,w' i ,h' i ) Completing target tracking; if all the predicted targets and detected targets P in the set S i In the Ma's system ofIf the distances are both larger than the threshold value or the distances between the two are smaller than the threshold value but the two are targets of different types, further judgment is carried out; if the target P is detected i If the position information center point (x, y) in the step 6) is within the range of subtracting the boundary threshold value from the resolution, the detection target P is judged i Putting the new tracking target into a target set S; if the target P is detected i If the position information center point (x, y) in the step 6) is outside the range of the resolution minus the boundary threshold, the detection target P is judged i Deleting the detection target P for error detection information i 。
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. foreign matters close to the high-voltage transmission line are found in the first time through a target tracking method based on a Deepsort algorithm, and continuous tracking detection is carried out on the foreign matters, so that the timeliness and the accuracy of detection are greatly improved.
2. The training samples are not limited to pictures shot from the power transmission tower camera, the number of the samples is greatly increased, and the training result is more accurate.
3. The mahalanobis distance is adopted to refer to the traditional calculation of the Euclidean distance, so that the difference between different attributes of the sample can be distinguished, the correlation interference between variables is eliminated, and the accuracy of target tracking is improved.
4. The soft-nms algorithm is used for replacing a non-maximum value suppression algorithm, so that the method is more applicable to the high-voltage transmission line foreign matters which are easy to overlap and shield, and the detection accuracy is improved.
5. The Kalman filtering algorithm is used for predicting the position of the tracking target, so that the matching degree of the same target is improved, and the false detection rate and the omission factor are reduced.
6. The method has the capability of processing the video stream, can continuously detect the high-voltage transmission line, reduces the interference of the external environment on the detection, and improves the timeliness and the accuracy of the detection.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The present invention will be further described with reference to the following specific examples.
As shown in fig. 1, the specific conditions of the method for detecting a foreign object in a high-voltage power transmission line based on a target tracking algorithm provided in this embodiment are as follows:
1) And cameras with the height slightly lower than that of the high-voltage transmission line are arranged on the power transmission towers at the two ends of the high-voltage transmission line. The angle of the camera is adjusted so that the camera can be aligned to the high-voltage power transmission line by using the overlooking angle. Meanwhile, the background in the camera is ensured to be as simple as possible, and no other object exists at the edge position. To same high tension transmission line, on the transmission tower at circuit both ends, all need install the camera, reach same high tension transmission line and can catch by two cameras simultaneously at least, can the multi-angle carry out the purpose that the target tracking monitored.
2) Collecting the pictures captured by the camera in the step 1). And simultaneously, searching a picture of a target to be tracked on the Internet. And sorting the obtained pictures to remove images which have low resolution and serious occlusion and are difficult to identify. The labeling tool labelme is used for labeling the pictures in a one-to-one correspondence mode, targets needing tracking monitoring are selected by using a rectangular frame with a proper size, different labels are inserted into different types of targets, and meanwhile the mode that the targets are selected in each picture needs to be guaranteed to be the same as far as possible. Automatically generating corresponding marking files and acquiring specific marking information x in the corresponding marking files min ,y min ,x max ,y max Wherein x is min Is the left boundary abscissa, x, of the rectangular frame max Is the right boundary abscissa of a rectangular box, wherein y min Is the lower boundary ordinate of the rectangular box, where y max Is the upper boundary ordinate of the rectangular frame. And reading the path information and the marking information of all the pictures by using the script file, and summarizing the path information and the marking information into a txt document. And randomly disorganizing the gathered picture information, and randomly selecting pictures according to the proportion of 8.
3) Training the training set obtained in the step 2) by using a yolov5 model training method, wherein the adopted specific parameters are as follows: yolov5s is selected for the structure of the yolov5 model, 8 is selected for the batch size, and 0.0001 is positioned for the initial learning rate by using an adam optimizer. After each round of training of the training set, the size of the map is tested in the test set using the obtained yolov5 model. If the current map value is greater than the previous maximum map value, the training weights for the current training batch are saved. If the current map value is less than the previous maximum map value, the next round of training continues. And stopping training when the total loss of the training set is continuously less than 0.1 or the training of 300 training batches is completed. The optimal yolov5 model and training weights are retained for later target detection.
4) And sending the video streams collected by the cameras on the power transmission towers at two ends of the high-voltage power transmission line to a background server, extracting a first frame of picture of the video stream, and carrying out Hough linear detection on the first frame of picture so as to extract the position of the high-voltage power transmission line. Extracting the minimum abscissa X of the detected set L of straight-line points min Minimum ordinate Y min And the maximum abscissa X max Maximum ordinate Y max Will be (X) min ,Y min ),(X min ,Y max ),(X max ,Y min ),(X max ,Y max ) And establishing a rectangular area J of the high-voltage transmission line as 4 corner points.
5) And detecting the video stream by using the optimal yolov5 model obtained in the step 3) to obtain a detection frame set F. And then, optimizing and screening the detection frame set F by using a soft-nms algorithm, wherein the specific process is as follows:
setting the detection confidence threshold value as W. Arranging all detection frames with the confidence degrees larger than W from high confidence degrees to low confidence degrees, and setting the detection frame with the i-th high confidence degree as a detection frame F i . Will detect the frame F 1 Put into the target set P. Sequentially obtaining detection frames F according to the sequence of i from small to large i And the intersection ratio IOU of all detection frames in the target set P i . The formula is as follows:
a cross-over ratio threshold V is set,if the detection frame F i And the intersection ratio IOU of all detection frames in the target set P i Are all less than the threshold value V, the detection frame F is set i Put into the target set P. If the detection frame K i The intersection ratio IOU of the detection frame in the target set P i If the value is larger than the threshold value V, the detection frame F is detected i Multiplying the confidence of (1-IOU) i If the frame F is detected after traversing the target set P i Is still greater than the detection confidence threshold W, the detection box F is put into i Put into the target set P, otherwise remove the detection frame F i . And (5) after the above operations are completed on all the detection frames, obtaining a final target set P.
6) Acquiring a newly added target P in a target set P i I.e. the first four outputs (x, y, w, h) of yolov5 model, wherein (x, y) represents the center point of the detected object, and (w, h) represents the width and height of the rectangular frame. The resolution of the video stream acquired by the camera is set to be M multiplied by N, and the threshold value of the boundary position is set to be a. If x is in the range of (0, a), (M-a, M), and y is in the range of (0, a), (N-a, N). The newly added target P is judged i Is at the edge of the whole detection range, is a foreign object that we want to detect and track, and is put into the tracking target set S. Otherwise, judging that the target is not a new tracking target.
7) All the tracking targets S in the tracking target set S at the t-1 th moment in the video are obtained through the output of the yolov5 model j Position information (x) of t-1 ,y t-1 ,w t-1 ,h t-1 ) The covariance of the prediction error at that time is G t-1 。
The rough predicted position at the t-th time is obtained by the formula (2)
Wherein A is the influence coefficient of the t-1 moment position to the t moment position, B is the influence coefficient of external input, u t-1 For external deliveryThe size of the inlet is larger than the size of the inlet,
is the preliminary predicted location.
Calculating the predicted prediction error at the time t by the formula (3)
Wherein A is the influence coefficient of the t-1 moment position to the t moment position, Q is the noise interference in the process,
the predicted prediction error at time t.
Kalman gain K is obtained through formula (4) t ：
Wherein
For the predicted prediction error at time t, H is the conversion of the measured value to the actual value, R is the measurement noise error, K t Is the kalman gain factor.
The corrected predicted position x is found by equation (5) t ：
Wherein
For the preliminarily predicted position, H is the conversion of the measured value into the actual value, and ` H `>
Is the predicted position at time t. y is t Is a measurement parameter, and the formula is as follows:
wherein
For the preliminary predicted position, H is the conversion of the measured value to the actual value, and R is the measurement noise error. />
The prediction error G at time t is calculated by equation (7) t ：
Wherein I is an identity matrix, K t For the kalman gain factor, H is the conversion of the measured value to the actual value,
predicted error for time t
Will y t-1 ,w t-1 ,h t-1 The corrected predicted position y is also obtained by substituting the calculated values into equations (2) to (7) t ,w t ,h t . And calculates the prediction error G at the time t t For position prediction at time t + 1.
The tracking target S is obtained by integrating the formula j Predicted tracking target position at time t
The coordinates of which are (x) t ,y t ,w t ,h t )。
8) And 5) acquiring the target detection frame set P at the time t from the video at the time t through the step 5). Wherein the detection frame P i The position information of (x) i ,y i ,w i ,h i )，
Calculating the detection frame P by the formula (8) i And predicting the tracking target position
Mahalanobis distance d of (1) (i,j)：
Wherein P is i The position information for the detection frame is (x) i ,y i ,w i ,h i )，
Tracking target location information (x) for prediction t ,y t ,w t ,h t )，E i Is a covariance matrix in the spatial domain of both.
Calculating the minimum cosine distance d between all Feature vectors tracked by the ith object and the jth object detection through the formula (9) (2) (i,j)：
Wherein r is j Is a detection frame P i A corresponding 128-dimensional feature vector is computed over the CNN network,
for the past k successful tracings, the detection box P i Corresponding k Feature vector sets.
Setting a distance threshold d 1 And d 2 When d is present (1) (i,j)＜d 1 Degree of correlation
When d is (1) (i,j)＞d 1 When it is, the degree of association>
When d is (2) (i,j)＜d 2 When, degree of association>
When d is (2) (i,j)＞d 2 When, degree of association>
Total degree of association
The total distance relationship C is obtained by the formula (10) i,j ：
C i,j ＝λd (1) (i,j)+(1-λ)d (2) (i,j) (10)
Where λ is a set correlation coefficient, d (1) (i, j) is a detection frame P i And predicting the tracking target position
Mahalanobis distance of d (2) (i, j) is the minimum cosine distance between all Feature vectors tracked for the ith object and the jth object detection.
At the total degree of association b i,j On the condition other than 0, the relation C to the total distance i,j Calculating and predicting the position of the tracking target by using a minimum cost algorithm
Associated detection Box P i . Use the detection frame P i Position information (x) of i ,y i ,w i ,h i ) To track target locations in lieu of predictions>
And the position information of (2) and the tracking target S j Matching and putting into a tracking target set S.
Repeating the iteration step 8) until no more successfully matched detection boxes P exist i And predicting the tracking target position
Location. And 6) carrying out edge detection on the rest targets, if the condition is met, taking the rest targets as new tracking targets to be placed in the tracking target set S, and otherwise, deleting the detection frame.
9) The whole video is processed by the step 7) and the step 8) to carry out target tracking detection and obtain a tracking target S j The fitted trajectory equation y = kx + b is obtained by the least square method. Randomly selecting points in a plurality of rectangular areas J to substitute into a linear fitting motion equation, and if a certain point is in accordance with a fitting trajectory equation y = kx + b, indicating that the target S is tracked j The high-voltage transmission line can move into the high-voltage transmission line area in the future, and the patrol personnel is prompted to have the possibility that foreign matters can approach the high-voltage transmission line and need to pay attention.
10 Continuously tracking the target S j And if the detection area finally leaves from the edge, the detection area is judged to be a foreign object target which does not interfere with the high-voltage transmission line and does not need to be processed. If the terminal stops in the monitoring range, the coordinate position of the stop is obtained as (x) 0 ,y 0 ,w 0 ,h 0 ) Wherein (x) 0 ,y 0 ) Abscissa and ordinate (w) representing the center point of the detected object 0 ,h 0 ) Representing the width and height of the rectangular box. If x 0 -w 0 /2 or x 0 +w 0 /2 in the high-voltage transmission line range (X) min ,X max ) Inner, y 0 -h 0 /2 or y 0 +h 0 /2 in the high-voltage transmission line range (Y) min ,Y max ) If the two are all true, the tracking target S is judged j Finally, the high-voltage transmission line is suspended on the high-voltage transmission line and needs to be overhauled and checked. And sending an alarm to the inspection personnel to remind the staff to process as soon as possible so as to prevent further damage and loss to the power transmission line and the power transmission tower. Meanwhile, the surrounding environment needs to be patrolled, and foreign objects which possibly become threats to the high-voltage transmission line are strived to be removed.
The above-mentioned embodiments are merely preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, so that the changes in the shape and principle of the present invention should be covered within the protection scope of the present invention.
Claims (6)
1. A high-voltage transmission line foreign matter detection method based on a target tracking algorithm is characterized by comprising the following steps:
1) The method comprises the following steps that a camera is installed on a power transmission tower of a high-voltage power transmission line, the angle of the camera is adjusted to enable the camera to be aligned to the high-voltage power transmission line, and the cameras are installed on the power transmission towers at two ends of the same high-voltage power transmission line, so that the same line can be captured by at least two cameras at the same time;
2) Shooting foreign matters on the high-voltage transmission line through the camera built in the step 1), and collecting pictures of the foreign matters on the Internet; sorting the obtained pictures, classifying the pictures, labeling the classified pictures in a one-to-one correspondence manner, and ensuring that the labeling modes of all the pictures are the same; randomly disordering the marked pictures, and dividing the pictures into a training set, a verification set and a test set;
3) Training the training set obtained in the step 2) by using a yolov5 model training method, iterating and updating a yolov5 model, selecting a model with the minimum total loss index as a final obtained result, actually measuring the final result on a test set, and after obtaining a good target detection result, reserving the used yolov5 model for target tracking;
4) Predicting the video stream by using the yolov5 model obtained in the step 3) to obtain a detection frame at the current moment, and processing a detection result of the yolov5 model by using an improved non-maximum suppression algorithm, namely a soft-nms algorithm to obtain the yolov5 model with the soft-nms algorithm; the Soft-nms algorithm takes the overlapping degree into consideration on the basis of non-maximum value inhibition to achieve the purpose of combining the repeatedly detected targets without omitting two close position relations;
5) Performing target detection by using the yolov5 model with soft-nms algorithm in the step 4), setting the picture of the first frame of the video stream captured by the camera to be in an initial state, and performing target detection on the picture in the initial stateDetecting a Hough straight line to obtain the position relation of the high-voltage transmission line; extracting the minimum abscissa X of the detected set L of straight-line points min Minimum ordinate Y min And the maximum abscissa X max Maximum ordinate Y max Will be (X) min ,Y min ),(X min ,Y max ),(X max ,Y min ),(X max ,Y max ) As 4 angular points, establishing a rectangular area J where the high-voltage transmission line is located;
6) Continuously detecting the video stream in the step 5), and when a target is detected to appear, setting the current time as t 0 At the moment, the currently detected target is set to the target P i Determining whether the appearance position is located at the edge position; if so, retaining the current target P i Putting the tracking target set S into the tracking target set S, and if not, judging that the tracking target set S is not a new tracking object; as the targets needing to be alarmed are all gradually close to the high-voltage power transmission line from a place far away from the high-voltage power transmission line, the targets are located at the edge position of the whole monitoring range when being detected for the first time, the inherent attributes of the video streaming device are obtained, the numerical value of the resolution ratio is obtained, a boundary threshold value is set, the four first indexes of target output obtained by using a yolov5 model are (x, y, w, h), wherein (x, y) represent the horizontal coordinate and the vertical coordinate of the central point of the detected object, and (w, h) represent the width and the height of a rectangular frame, so that as long as the central point (x, y) is located in the range of subtracting the boundary threshold value from the resolution ratio, the current target P is judged i The detection result is positioned at the edge position, otherwise, the detection result is not regarded as a new tracking object;
7) Will t 0 The next frame at a time is set to t 1 Time, the target detected at that time is set to P 1 Using a target tracking algorithm based on the Deepsort algorithm to track the current target P 1 Matching and tracking all targets in the target set S; if the current target P 1 And tracking the target S 1 If the matching is successful, the target S is tracked 1 Is updated to the current target P 1 And S is calculated from the position information of 2 Putting the target set S into a target set S; if the current target P 1 Does not match any existing targets in set S, but is currently presentMark P 1 At the edge position described in step 6), the target P is positioned 1 As a new tracking target S 2 Putting the target set S into a target set S;
using Hungarian algorithm to detect target P i And predicting the target S i Matching: calculating a detection target P i And predicting the target S i If the distance between the two is smaller than the threshold value, the two are judged to be the same target and successfully matched, and the target S is selected i Is updated to (x' i ,y' i ,w' i ,h' i ) Completing target tracking; if all the predicted targets and detected targets P in the set S i The Mahalanobis distances are all larger than the threshold value or the two are different types of targets although the distance between the two is smaller than the threshold value, further judgment is carried out; if the target P is detected i If the position information center point (x, y) in the step 6) is within the range of subtracting the boundary threshold value from the resolution, the detection target P is judged i Putting the new tracking target into a target set S; if the target P is detected i If the position information center point (x, y) in the step 6) is outside the range of the resolution minus the boundary threshold, the detection target P is judged i Deleting the detection target P for error detection information i ；
8) Continuously and uninterruptedly detecting the video stream, and completing target tracking through the step 6) and the step 7); according to the tracked target S i Fitting a future motion equation of the motion trajectory, namely fitting a motion equation in a straight line; substituting points in the rectangular area J into a linear fitting motion equation, and if the equation is equal left and right, judging that the tracking target S is equal i The method comprises the following steps that (1) a trend of regional movement of the high-voltage transmission line is shown, and a prompt needs to be sent;
9) For the tracking target S in step 8) i Continuing to track the target if the target S is tracked i Finally, if the high-voltage transmission line stops moving in the rectangular area where the high-voltage transmission line is located, the foreign matter which can interfere the operation of the high-voltage transmission line is judged, and an alarm needs to be given; if the target S is tracked i Finally, the monitoring range is separated from the boundary, which judges that the object is a passing object and does not interfere with the highAnd when the power transmission line normally runs, the reminding needs to be sent again.
2. The method for detecting the foreign matters on the high-voltage transmission line based on the target tracking algorithm according to claim 1, wherein the method comprises the following steps: in the step 1), the camera installed on the power transmission tower is lower than the height of the high-voltage power transmission line, so that the camera can shoot the high-voltage power transmission line from bottom to top while ensuring the shot pictures to be clear; by adopting two cameras, namely a bidirectional camera system can ensure that a target tracking algorithm can normally run when a plurality of foreign matters are overlapped and shielded, and rapidity and accuracy are not influenced.
3. The method for detecting the foreign matter on the high-voltage transmission line based on the target tracking algorithm according to claim 1, characterized in that: in the step 2), data enhancement is carried out on the data acquired in the camera, wherein the data enhancement comprises random cutting, noise addition and mirror image turning.
4. The method for detecting the foreign matter on the high-voltage transmission line based on the target tracking algorithm according to claim 1, characterized in that: in the step 2), the picture is marked by using a marking tool labelme, namely, a rectangle with a proper size is used for framing the corresponding target object, so as to obtain marking information x min 、y min 、x max 、y max Wherein x is min Is the left boundary abscissa, x, of the rectangular frame max Is the right boundary abscissa, y, of the rectangular frame min Is the lower boundary ordinate, y, of the rectangular frame max Is the vertical coordinate of the upper boundary of the rectangular frame; randomly disorganizing the marked pictures, and dividing the pictures into a training set, a verification set and a test set, wherein the proportion of the pictures is 8.
5. The method for detecting the foreign matters on the high-voltage transmission line based on the target tracking algorithm according to claim 1, wherein the method comprises the following steps: in step 4), the confidence scores of the detection frames are sorted from high to low, and the detection frame with the highest confidence is placed in a set M; the improved non-maximum inhibition algorithm is modified by reducing the confidence score of a new detection frame according to the size of all elements IOU in M, wherein the larger the IOU is, the more the reduced confidence is, and the smaller the IOU is, the smaller the reduced confidence is, and if the confidence after iterative update is still larger than a threshold value, the new detection frame is added into the set M until all pictures go through.
6. The method for detecting the foreign matter on the high-voltage transmission line based on the target tracking algorithm according to claim 1, characterized in that: in step 7), firstly, the targets S in the tracking target set S are i Using Kalman filtering, object S i According to t 0 State of time, predicted target S i At t 1 Position coordinates (x) of time of day i ,y i ,w i ,h i ) Through t 0 The covariance of the moment yields t 1 The prediction error of the moment is calculated to obtain the Kalman gain, and the target S is predicted through the Kalman gain i The predicted position of (a) is corrected to obtain the final predicted position coordinate (x) i ,y i ,w i ,h i ) Then for t 1 Predicting by using the yolov5 model with soft-nms algorithm in the step 4) at any moment to obtain a detection target P i Position (x' i ,y' i ,w' i ,h' i )。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110387931.1A CN113076899B (en) | 2021-04-12 | 2021-04-12 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110387931.1A CN113076899B (en) | 2021-04-12 | 2021-04-12 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113076899A CN113076899A (en) | 2021-07-06 |
| CN113076899B true CN113076899B (en) | 2023-04-07 |
Family
ID=76617310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110387931.1A Active CN113076899B (en) | 2021-04-12 | 2021-04-12 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113076899B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114724091A (en) * | 2022-06-07 | 2022-07-08 | 智洋创新科技股份有限公司 | Method and device for identifying foreign matters on transmission line wire |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109871763A (en) * | 2019-01-16 | 2019-06-11 | 清华大学 | A kind of specific objective tracking based on YOLO |
| CN112541424A (en) * | 2020-12-07 | 2021-03-23 | 南京工程学院 | Real-time detection method for pedestrian falling under complex environment |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2694025B2 (en) * | 1989-08-31 | 1997-12-24 | 株式会社フジクラ | Overhead transmission line obstacle approach detection device |
| CN101739694B (en) * | 2010-01-07 | 2011-10-05 | 北京智安邦科技有限公司 | Image analysis-based method and device for ultrahigh detection of high voltage transmission line |
| CN104463904A (en) * | 2014-09-22 | 2015-03-25 | 国家电网公司 | High-voltage line foreign matter invasion target detection method |
| CN106056821B (en) * | 2016-08-13 | 2018-04-24 | 哈尔滨理工大学 | Electric line foreign matter intrusion intelligent video monitors assessment system on-line |
| CN110516556B (en) * | 2019-07-31 | 2023-10-31 | 平安科技（深圳）有限公司 | Multi-target tracking detection method and device based on Darkflow-deep Sort and storage medium |
| CN111444809B (en) * | 2020-03-23 | 2023-02-14 | 华南理工大学 | Power transmission line abnormal target detection method based on improved YOLOv3 |
-
2021
- 2021-04-12 CN CN202110387931.1A patent/CN113076899B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109871763A (en) * | 2019-01-16 | 2019-06-11 | 清华大学 | A kind of specific objective tracking based on YOLO |
| CN112541424A (en) * | 2020-12-07 | 2021-03-23 | 南京工程学院 | Real-time detection method for pedestrian falling under complex environment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113076899A (en) | 2021-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102129893B1 (en) | Ship tracking method and system based on deep learning network and average movement | |
| US20240062518A1 (en) | Method, apparatus, computer device and storage medium for detecting objects thrown from height | |
| CN115691026B (en) | Intelligent early warning monitoring management method for forest fire prevention | |
| CN110084165B (en) | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation | |
| CN109344753A (en) | A kind of tiny fitting recognition methods of Aerial Images transmission line of electricity based on deep learning | |
| CN112485781B (en) | Anti-unmanned aerial vehicle unmanned system and method based on deep learning | |
| CN113076899B (en) | High-voltage transmission line foreign matter detection method based on target tracking algorithm | |
| CN114841920A (en) | Flame identification method and device based on image processing and electronic equipment | |
| CN115995058A (en) | Power transmission channel safety on-line monitoring method based on artificial intelligence | |
| CN115880231A (en) | Power transmission line hidden danger detection method and system based on deep learning | |
| CN116740833A (en) | Line inspection and card punching method based on unmanned aerial vehicle | |
| CN117058541B (en) | Insulator hyperspectral data acquisition system and method thereof | |
| CN115171006B (en) | Detection method for automatically identifying person entering electric power dangerous area based on deep learning | |
| CN115100249B (en) | Intelligent factory monitoring system based on target tracking algorithm | |
| CN113554682B (en) | Target tracking-based safety helmet detection method | |
| CN115187880A (en) | Communication optical cable defect detection method and system based on image recognition and storage medium | |
| CN111025288B (en) | Security radar monitoring device and system | |
| CN114912536A (en) | Target identification method based on radar and double photoelectricity | |
| CN112149579A (en) | Improved yolo _ v 3-based bird nest hidden danger detection method | |
| CN113158800A (en) | Enclosure intrusion hybrid detection method and enclosure intrusion hybrid detection system | |
| CN116681955B (en) | Method and computing device for identifying traffic guardrail anomalies | |
| AU2021103901A4 (en) | Method for identifying abnormity of overhead lines with unbalanced samples | |
| CN116912721B (en) | Power distribution network equipment body identification method and system based on monocular stereoscopic vision | |
| CN115861366B (en) | Multi-source perception information fusion method and system for target detection | |
| CN113963502B (en) | All-weather illegal behavior automatic inspection method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |