CN113076899A - High-voltage transmission line foreign matter detection method based on target tracking algorithm - Google Patents

High-voltage transmission line foreign matter detection method based on target tracking algorithm Download PDF

Info

Publication number
CN113076899A
CN113076899A CN202110387931.1A CN202110387931A CN113076899A CN 113076899 A CN113076899 A CN 113076899A CN 202110387931 A CN202110387931 A CN 202110387931A CN 113076899 A CN113076899 A CN 113076899A
Authority
CN
China
Prior art keywords
target
transmission line
tracking
detection
voltage transmission
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110387931.1A
Other languages
Chinese (zh)
Other versions
CN113076899B (en
Inventor
杜启亮
张楠
田联房
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Zhuhai Institute of Modern Industrial Innovation of South China University of Technology
Original Assignee
South China University of Technology SCUT
Zhuhai Institute of Modern Industrial Innovation of South China University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT, Zhuhai Institute of Modern Industrial Innovation of South China University of Technology filed Critical South China University of Technology SCUT
Priority to CN202110387931.1A priority Critical patent/CN113076899B/en
Publication of CN113076899A publication Critical patent/CN113076899A/en
Application granted granted Critical
Publication of CN113076899B publication Critical patent/CN113076899B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a high-voltage transmission line foreign matter detection method based on a target tracking algorithm, which comprises the following steps: placing a camera device on the power transmission tower and transmitting data; collecting and marking foreign matters of the high-voltage transmission line; training a yolov5 model and optimizing; using Hough linear detection to the detection range to obtain the approximate range of the high-voltage transmission line; detecting the video stream, optimizing a detection result by using a soft-nms algorithm, and selecting a target to be tracked; predicting and tracking the target position by using Kalman filtering based on a Deepsort target tracking algorithm; the Hungarian algorithm is used for calculating the degree of association between the predicted target and the detected target and matching; obtaining whether the target track can reach the high-voltage transmission line area or not according to the existing tracking target track; and obtaining whether the tracking target is an abnormal target affecting the high-voltage transmission line or not according to the final position of the tracking target. The method can realize timely and accurate detection of the foreign matters approaching and adhering to the power transmission line and quickly give an alarm.

Description

High-voltage transmission line foreign matter detection method based on target tracking algorithm
Technical Field
The invention relates to the technical field of intelligent monitoring of an electric power system, in particular to a high-voltage transmission line foreign matter detection method based on a target tracking algorithm, which can be applied to monitoring foreign matters close to a high-voltage transmission line and giving an alarm in time.
Background
The electric power system occupies a more and more important position in the current social life and development, and the transmission line is responsible for connecting a power plant, a transformer substation and a user together and plays an irreplaceable role. The safety and stability of the device are very important. In order to reduce the energy loss in the conveying process, high-voltage power transmission is often adopted. However, since most of the high-voltage transmission lines are located in mountainous regions and hilly terrains and have very large span, the detection and maintenance are very difficult. It is more difficult to accurately find the occurrence of the unexpected situation in time. At present, one of the main hidden dangers of the high-voltage transmission line is as follows: kites, floats and the like can be wound on objects on lines, and foreign matters such as nests of birds and birds. Traditional manual inspection mode, not only the time of spending is long, and is slow, and the precision of patrolling and examining moreover, the promptness that the potential safety hazard was reported to the police to and the personnel's of patrolling and examining safety problem all is difficult to guarantee. Machine vision in recent years, or target detection schemes, are susceptible to weather or shelters, so that accuracy and timeliness of the machine vision or target detection schemes cannot be guaranteed.
The method aims to provide the high-voltage transmission line foreign matter detection method based on the target tracking algorithm, and the method has good performances in timeliness, accuracy and stability when the video stream is detected on the high-voltage transmission line. As the foreign object target is always gradually close to the high-voltage transmission line in principle, a target tracking algorithm based on the Deepsort algorithm is added on the basis of the deep learning model at the present stage, and the function of target tracking is realized. The method can accurately and timely monitor the approach of the foreign object on the power transmission line, and timely alarm in the approach process so as to remind workers of paying attention to the safe operation of the power transmission line.
By combining the above discussion, the high-voltage transmission line detection method which has high precision and high early warning speed has high practical application value.
Disclosure of Invention
The invention aims to overcome the defects and shortcomings of the prior art and provides a foreign matter detection method for a high-voltage transmission line based on a target tracking algorithm. The method has simple requirements on the data set and has good performance in rapidity and accuracy. The video stream can be used to detect foreign objects approaching and adhering to the power transmission line and to rapidly send out an alarm.
In order to achieve the purpose, the technical scheme provided by the invention is as follows: a high-voltage transmission line foreign matter detection method based on a target tracking algorithm comprises the following steps:
1) the method comprises the following steps that a camera is installed on a power transmission tower of a high-voltage power transmission line, the angle of the camera is adjusted to enable the camera to be aligned to the high-voltage power transmission line, and the cameras are installed on the power transmission towers at two ends of the same high-voltage power transmission line, so that the same line can be captured by at least two cameras at the same time;
2) shooting foreign matters on the high-voltage transmission line through the camera built in the step 1), and collecting pictures of the foreign matters on the Internet; arranging the obtained pictures, classifying the pictures, labeling the classified pictures in a one-to-one correspondence manner, and ensuring that the labeling modes of all the pictures are the same; randomly disorganizing the marked pictures, and dividing the pictures into a training set, a verification set and a test set;
3) training the training set obtained in the step 2) by using a yolov5 model training method, iterating and updating a yolov5 model, selecting a yolov5 model with the minimum total loss index as a final obtained result, carrying out actual measurement on the final obtained result on a test set, and after a good target detection result is obtained, reserving the used yolov5 model for target tracking;
4) predicting the video stream by using the yolov5 model obtained in the step 3) to obtain a detection frame at the current moment, and processing a detection result of the yolov5 model by using an improved non-maximum suppression (nms) algorithm, namely a soft-nms algorithm to obtain a yolov5 model with a soft-nms algorithm; the Soft-nms algorithm takes the overlapping degree into consideration on the basis of a non-maximum value inhibition (nms) algorithm to achieve the purpose of combining the repeatedly detected targets without omitting two close position relations;
5) carrying out target detection by using the yolov5 model with soft-nms algorithm in the step 4), setting the picture of the first frame of the video stream captured by the camera as an initial state, and carrying out Hough linear detection on the picture in the initial state to obtain the position relation of the high-voltage transmission line; extracting the minimum abscissa X of the detected set L of straight-line pointsminMinimum ordinate YminAnd the maximum abscissa XmaxMaximum ordinate YmaxWill be (X)min,Ymin),(Xmin,Ymax),(Xmax,Ymin),(Xmax,Ymax) As 4 angular points, establishing a rectangular area J where the high-voltage transmission line is located;
6) continuously detecting the video stream in the step 5), and when a target is detected to appear, setting the current time as t0At the moment, the currently detected target is set to the target PiDetermining whether the appearance position is located at the edge position; if so, retaining the current target PiPutting the tracking target set S into the tracking target set S, and if not, judging that the tracking target set S is not a new tracking object; as the targets needing alarming are all gradually close to the high-voltage power transmission line from a place far away from the high-voltage power transmission line, the targets are located at the edge position of the whole monitoring range when being detected for the first time, the inherent properties of the video streaming device are obtained, the numerical value of the resolution is obtained, the boundary threshold value is set, the first four indexes of the target output obtained by using the yolov5 model are (x, y, w, h), wherein (x, y) represent the abscissa and the ordinate of the central point of the detected object,(w, h) represents the width and height of the rectangular box, so that the current target P is determined as long as the center point (x, y) is within the range of the resolution minus the boundary thresholdiThe detection result is positioned at the edge position, otherwise, the detection result is not regarded as a new tracking object;
7) will t0The next frame at a time is set to t1Time, the target detected at that time is set to P1Using a target tracking algorithm based on the Deepsort algorithm to track the current target P1Matching and tracking all targets in the target set S; if the current target P1And tracking the target S1If the matching is successful, the target S is tracked1Is updated to the current target P1And S is calculated from the position information of2Putting the target set S into a target set S; if the current target P1Does not match any existing targets in set S, but the current target P1At the edge position described in step 6), the target P is positioned1As a new tracking target S2Putting the target set S into a target set S;
8) continuously and uninterruptedly detecting the video stream, and completing target tracking through the step 6) and the step 7); according to the tracked target SiFitting a future motion equation of the future motion trajectory, namely fitting a motion equation linearly; substituting points in the rectangular area J into a linear fitting motion equation, and if the equation is equal left and right, judging that the tracking target S is equaliThe method comprises the following steps that (1) a trend of regional movement of the high-voltage transmission line is shown, and a prompt needs to be sent;
9) for the tracking target S in step 8)iContinuing to track the target if the target S is trackediFinally, if the high-voltage transmission line stops moving in the rectangular area where the high-voltage transmission line is located, the foreign matter which can interfere the operation of the high-voltage transmission line is judged, and an alarm needs to be given; if the target S is trackediAnd finally, the monitoring range is separated from the boundary, so that the monitoring range is judged to be a passing object, the normal operation of the high-voltage transmission line is not interfered, and the reminding needs to be sent again.
In the step 1), the camera installed on the power transmission tower is lower than the height of the high-voltage power transmission line, so that the camera can shoot the high-voltage power transmission line from bottom to top while ensuring the shot pictures to be clear; by adopting two cameras, namely a bidirectional camera system can ensure that a target tracking algorithm can normally run when a plurality of foreign matters are overlapped and shielded, and rapidity and accuracy are not influenced.
In the step 2), data enhancement is carried out on the data acquired in the camera, and the data enhancement comprises random cutting, noise addition and mirror image turning.
In the step 2), the picture is marked by using a marking tool labelme, namely, a rectangle with proper size is used for framing the corresponding target object to obtain marking information xmin、ymin、xmax、ymaxWherein x isminIs the left boundary abscissa, x, of the rectangular framemaxIs the right boundary abscissa, y, of the rectangular frameminIs the lower boundary ordinate, y, of the rectangular framemaxIs the vertical coordinate of the upper boundary of the rectangular frame; randomly disorganizing the marked pictures, and dividing the pictures into a training set, a verification set and a test set, wherein the ratio of the training set to the verification set to the test set is 8:1: 1.
In the step 4), the confidence scores of the detection frames are sorted from high to low, and the detection frame with the highest confidence is placed in the set M; the improved non-maximum inhibition algorithm is modified by reducing the confidence score of a new detection frame according to the sizes of all elements IOU in M, wherein the larger the IOU is, the more the confidence is reduced, and the smaller the IOU is, the smaller the confidence is reduced, if the confidence after iterative update is still larger than a threshold value, the new detection frame is added into the set M until all pictures go through, and through the improvement, the detection accuracy can be greatly improved.
In step 7), firstly, the targets S in the tracking target set S areiUsing Kalman filtering, object SiAccording to t0State of time, predicted target SiAt t1Position coordinates (x) of time of dayi,yi,wi,hi) Through t0The covariance of the moment is found to be t1The prediction error of the moment is calculated to obtain the Kalman gain, and the target S is predicted through the Kalman gainiIs predicted to be at a locationCorrecting to obtain final predicted position coordinate (x)i,yi,wi,hi) Then to t1Using the yolov5 model with soft-nms algorithm in the step 4) to predict at any moment to obtain the detection target PiPosition (x'i,y'i,w'i,h'i)。
In step 7), the Hungarian algorithm is used to detect the target PiAnd predicting the target SiMatching: calculating a detection target PiAnd predicting the target SiIf the distance between the two is smaller than the threshold value, the two are judged to be the same target and successfully matched, and the target S is selectediIs updated to (x'i,y'i,w'i,h'i) Completing target tracking; if all the predicted targets and detected targets P in the set SiThe Mahalanobis distances are all larger than the threshold value or the two are different types of targets although the distance between the two is smaller than the threshold value, further judgment is carried out; if the target P is detectediIf the position information center point (x, y) in the step 6) is within the range of subtracting the boundary threshold value from the resolution, the detection target P is judgediPutting the new tracking target into a target set S; if the target P is detectediIf the position information center point (x, y) in the step 6) is outside the range of the resolution minus the boundary threshold, the detection target P is judgediDeleting the detection target P for error detection informationi。
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the target tracking method based on the Deepsort algorithm is used for finding the foreign matters close to the high-voltage transmission line at the first time and carrying out continuous tracking detection on the foreign matters, so that the timeliness and the accuracy of detection are greatly improved.
2. The training samples are not limited to pictures shot from the power transmission tower camera, the number of the samples is greatly increased, and the training result is more accurate.
3. The mahalanobis distance is adopted to refer to the traditional calculation of the Euclidean distance, so that the difference between different attributes of the sample can be distinguished, the correlation interference between variables is eliminated, and the accuracy of target tracking is improved.
4. The soft-nms algorithm is used for replacing a non-maximum value suppression algorithm, so that the method is more applicable to the high-voltage transmission line foreign matters which are easy to overlap and shield, and the detection accuracy is improved.
5. The Kalman filtering algorithm is used for predicting the position of the tracking target, so that the matching degree of the same target is improved, and the false detection rate and the omission factor are reduced.
6. The method has the capability of processing the video stream, can continuously detect the high-voltage transmission line, reduces the interference of the external environment on the detection, and improves the timeliness and the accuracy of the detection.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The present invention will be further described with reference to the following specific examples.
As shown in fig. 1, the specific conditions of the method for detecting a foreign object in a high-voltage power transmission line based on a target tracking algorithm provided in this embodiment are as follows:
1) and cameras with the height slightly lower than that of the high-voltage transmission line are arranged on the power transmission towers at the two ends of the high-voltage transmission line. The angle of the camera is adjusted so that the camera can be aligned to the high-voltage power transmission line by using the overlooking angle. Meanwhile, the background in the camera is ensured to be as simple as possible, and no other object exists at the edge position. To same high tension transmission line, on the transmission tower at circuit both ends, all need install the camera, reach same high tension transmission line and can catch by two cameras simultaneously at least, can the multi-angle carry out the purpose that the target tracking monitored.
2) Collecting the pictures captured by the camera in the step 1). And simultaneously, searching a picture of a target to be tracked on the Internet. And sorting the obtained pictures to remove images which are low in resolution, serious in occlusion and difficult to identify. Using a labeling tool labelme to label the pictures in a one-to-one correspondence manner, selecting the targets to be tracked and monitored by using a rectangular frame with a proper size, inserting different labels into the targets of different types, and ensuring the condition as much as possibleThe target is selected in the same way for each picture. Automatically generating corresponding marking files and acquiring specific marking information x in the corresponding marking filesmin,ymin,xmax,ymaxWherein x isminIs the left boundary abscissa, x, of the rectangular framemaxIs the right boundary abscissa of the rectangular box, where yminIs the lower boundary ordinate of the rectangular box, where ymaxIs the upper boundary ordinate of the rectangular frame. And reading the path information and the marking information of all the pictures by using the script file, and summarizing the path information and the marking information into a txt document. And randomly disorganizing the gathered picture information, and randomly selecting pictures according to the ratio of 8:1:1 to form a training set, a verification set and a test set of the yolov5 model.
3) Training the training set obtained in the step 2) by using a yolov5 model training method, wherein the adopted specific parameters are as follows: the yolov5 model structure is selected to be yolov5s, the batch size is selected to be 8, and the adam optimizer is used for positioning the initial learning rate to be 0.0001. After each training round of the training set, the size of the map was tested using the obtained yolov5 model in the test set. If the current map value is greater than the previous maximum map value, the training weights for the current training batch are saved. If the current map value is less than the previous maximum map value, the next round of training continues. And stopping training when the total loss of the training set is continuously less than 0.1 or after the training of 300 training batches is completed. The optimal yolov5 model and training weights are retained for later target detection.
4) And sending the video streams collected by the cameras on the power transmission towers at two ends of the high-voltage power transmission line to a background server, extracting a first frame of picture of the video stream, and carrying out Hough linear detection on the first frame of picture so as to extract the position of the high-voltage power transmission line. Extracting the minimum abscissa X of the detected set L of straight-line pointsminMinimum ordinate YminAnd the maximum abscissa XmaxMaximum ordinate YmaxWill be (X)min,Ymin),(Xmin,Ymax),(Xmax,Ymin),(Xmax,Ymax) And establishing a rectangular area J of the high-voltage transmission line as 4 corner points.
5) And detecting the video stream by using the optimal yolov5 model obtained in the step 3) to obtain a detection frame set F. And then carrying out optimization screening on the detection frame set F by using a soft-nms algorithm, wherein the specific flow is as follows:
setting the detection confidence threshold value as W. Arranging all detection frames with the confidence degrees larger than W from high confidence degrees to low confidence degrees, and setting the detection frame with the i-th high confidence degree as a detection frame Fi. Will detect the frame F1Put into the target set P. Sequentially obtaining detection frames F according to the sequence of i from small to largeiAnd the intersection ratio IOU of all detection frames in the target set Pi. The formula is as follows:
Figure BDA0003015639580000071
setting a threshold V of the cross-over ratio, if detecting the frame FiAnd the intersection ratio IOU of all detection frames in the target set PiAre all less than the threshold value V, the detection frame F is setiPut into the target set P. If the detection frame KiThe intersection ratio IOU of the detection frame in the target set PiIf the value is larger than the threshold value V, the detection frame F is detectediMultiplying the confidence of (1-IOU)iIf the frame F is detected after traversing the target set PiIs still greater than the detection confidence threshold W, the detection box F is put intoiPut into the target set P, otherwise remove the detection frame Fi. And (5) after the above operations are completed on all the detection frames, obtaining a final target set P.
6) Acquiring a newly added target P in a target set PiI.e., the first four outputs (x, y, w, h) of yolov5 model, wherein (x, y) represents the center point of the detected object, and (w, h) represents the width and height of the rectangular frame. The resolution of the video stream acquired by the camera is set to be M multiplied by N, and the threshold value of the boundary position is set to be a. If x is in the range of (0, a), (M-a, M), and y is in the range of (0, a), (N-a, N). The newly added target P is judgediIs at the edge of the whole detection range, is a foreign object that we want to detect and track, and is put into the tracking object set S. Otherwise, judging that the target is not a new tracking target.
7) By passingOutput of yolov5 model acquires all tracked targets S in tracked target set S at t-1 moment in videojPosition information (x) oft-1,yt-1,wt-1,ht-1) The covariance of the prediction error at that time is Gt-1。
The rough predicted position at the t-th time is obtained through the formula (2)
Figure BDA0003015639580000072
Figure BDA0003015639580000073
Wherein A is the influence coefficient of the t-1 moment position to the t moment position, B is the influence coefficient of external input, ut-1Is the size of the external input and,
Figure BDA0003015639580000081
is the preliminary predicted location.
Calculating the predicted prediction error at the time t by the formula (3)
Figure BDA0003015639580000082
Figure BDA0003015639580000083
Wherein A is the influence coefficient of the t-1 moment position to the t moment position, Q is the noise interference in the process,
Figure BDA0003015639580000084
the predicted prediction error at time t.
Kalman gain K is obtained through formula (4)t：
Figure BDA0003015639580000085
Wherein
Figure BDA0003015639580000086
For the predicted prediction error at time t, H is the conversion of the measured value to the actual value, R is the measurement noise error, KtIs the kalman gain factor.
The corrected predicted position x is found by equation (5)t：
Figure BDA0003015639580000087
Wherein
Figure BDA0003015639580000088
For the preliminary predicted position, H is the conversion of the measured value to the actual value,
Figure BDA0003015639580000089
is the predicted position at time t. y istIs a measurement parameter, and the formula is as follows:
Figure BDA00030156395800000810
wherein
Figure BDA00030156395800000811
For the preliminary predicted position, H is the conversion of the measured value to the actual value, and R is the measurement noise error.
The prediction error G at time t is calculated by equation (7)t：
Figure BDA00030156395800000812
Wherein I is an identity matrix, KtFor the kalman gain factor, H is the conversion of the measured value to the actual value,
Figure BDA00030156395800000813
predicted error for time t
Will yt-1,wt-1,ht-1The corrected predicted position y is also obtained by substituting the equations (2) to (7)t,wt,ht. And calculates the prediction error G at the time ttFor position prediction at time t + 1.
The tracking target S is obtained by combining the formulasjPredicted tracking target position at time t
Figure BDA00030156395800000814
The coordinates of which are (x)t,yt,wt,ht)。
8) And 5) acquiring the target detection frame set P at the time t from the video at the time t through the step 5). Wherein the detection frame PiThe position information of (x)i,yi,wi,hi)，
Calculating the detection frame P by the formula (8)iAnd predicting the tracking target position
Figure BDA0003015639580000091
Mahalanobis distance d of(1)(i,j)：
Figure BDA0003015639580000092
Wherein P isiThe position information for the detection frame is (x)i,yi,wi,hi)，
Figure BDA0003015639580000093
Tracking target location information (x) for predictiont,yt,wt,ht)，EiIs a covariance matrix in both spatial domains.
Calculating the minimum cosine distance d between all Feature vectors of the ith object tracking and the jth object detection through the formula (9)(2)(i,j)：
Figure BDA0003015639580000094
Wherein r isjFor detecting frame PiA corresponding 128-dimensional feature vector is computed over the CNN network,
Figure BDA0003015639580000095
for the past k successful tracings, the detection box PiCorresponding k Feature vector sets.
Setting a distance threshold d1And d2When d is(1)(i,j)＜d1Degree of correlation
Figure BDA0003015639580000096
When d is(1)(i,j)＞d1Degree of correlation
Figure BDA0003015639580000097
When d is(2)(i,j)＜d2Degree of correlation
Figure BDA0003015639580000098
When d is(2)(i,j)＞d2Degree of correlation
Figure BDA0003015639580000099
Total degree of association
Figure BDA00030156395800000910
The total distance relationship C is obtained by the formula (10)i,j：
Ci,j＝λd(1)(i,j)+(1-λ)d(2)(i,j) (10)
Where λ is a set correlation coefficient, d(1)(i, j) is a detection frame PiAnd predicting the tracking target position
Figure BDA00030156395800000911
Mahalanobis distance of d(2)(i, j) is the minimum cosine distance between all Feature vectors tracked by the ith object and the jth object detection.
At the total degree of association bi,jOn the condition other than 0, the relation C to the total distancei,jCalculating and predicting the position of the tracking target by using a minimum cost algorithm
Figure BDA00030156395800000912
Associated detection frame Pi. Use the detection frame PiPosition information (x) ofi,yi,wi,hi) Instead of predicting the tracking target position
Figure BDA00030156395800000913
And the position information of (2) and the tracking target SjMatching and putting into a tracking target set S.
Repeating the iteration step 8) until no more successfully matched detection boxes P existiAnd predicting the tracking target position
Figure BDA00030156395800000914
Location. And 6) carrying out edge detection on the rest targets, if the condition is met, taking the rest targets as new tracking targets to be placed in the tracking target set S, and otherwise, deleting the detection frame.
9) The whole video is processed by the step 7) and the step 8) to carry out target tracking detection and obtain a tracking target SjThe fitted trajectory equation y is kx + b by the least square method. Randomly selecting points in a plurality of rectangular areas J to substitute into a linear fitting motion equation, and if a certain point fitting trajectory equation y is kx + b, indicating that the target S is trackedjThe high-voltage transmission line can move into the high-voltage transmission line area in the future, and the patrol personnel is prompted to have the possibility that foreign matters can approach the high-voltage transmission line and need to pay attention.
10) Continuously tracking target SjAnd if the detection area is finally departed from the edge, the detection area is judged to be a foreign object target which does not interfere the high-voltage transmission line and does not need to be processed. If the terminal stop is in the monitoring range, the coordinate position of the stop is obtained as (x)0,y0,w0,h0) Wherein (x)0,y0) Abscissa and ordinate (w) representing the center point of the detected object0,h0) Representing the width and height of the rectangular box. If x0-w0/2 or x0+w0/2 in the high-voltage transmission line range (X)min,Xmax) Inner, y0-h0/2 or y0+h0/2 in the high-voltage transmission line range (Y)min,Ymax) If the two are all true, the tracking target S is judgedjFinally, the high-voltage transmission line is suspended on the high-voltage transmission line and needs to be overhauled and checked. And sending an alarm to the inspection personnel to remind the staff to process as soon as possible so as to prevent further damage and loss to the power transmission line and the power transmission tower. Meanwhile, the surrounding environment needs to be inspected, and foreign objects which are possibly threatened by the high-voltage transmission line are removed.
The above-mentioned embodiments are merely preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, so that the changes in the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims (7)

1. A high-voltage transmission line foreign matter detection method based on a target tracking algorithm is characterized by comprising the following steps:
1) the method comprises the following steps that a camera is installed on a power transmission tower of a high-voltage power transmission line, the angle of the camera is adjusted to enable the camera to be aligned to the high-voltage power transmission line, and the cameras are installed on the power transmission towers at two ends of the same high-voltage power transmission line, so that the same line can be captured by at least two cameras at the same time;
2) shooting foreign matters on the high-voltage transmission line through the camera built in the step 1), and collecting pictures of the foreign matters on the Internet; arranging the obtained pictures, classifying the pictures, labeling the classified pictures in a one-to-one correspondence manner, and ensuring that the labeling modes of all the pictures are the same; randomly disorganizing the marked pictures, and dividing the pictures into a training set, a verification set and a test set;
3) training the training set obtained in the step 2) by using a yolov5 model training method, iterating and updating a yolov5 model, selecting a model with the minimum total loss index as a finally obtained result, actually measuring the model on a test set, and after obtaining a good target detection result, reserving the used yolov5 model for target tracking;
4) predicting the video stream by using the yolov5 model obtained in the step 3) to obtain a detection frame at the current moment, and processing a detection result of the yolov5 model by using an improved non-maximum suppression algorithm, namely a soft-nms algorithm to obtain a yolov5 model with a soft-nms algorithm; the Soft-nms algorithm takes the overlapping degree into consideration on the basis of non-maximum value inhibition to achieve the purpose of combining the repeatedly detected targets without omitting two close position relations;
5) carrying out target detection by using the yolov5 model with soft-nms algorithm in the step 4), setting the picture of the first frame of the video stream captured by the camera as an initial state, and carrying out Hough linear detection on the picture in the initial state to obtain the position relation of the high-voltage transmission line; extracting the minimum abscissa X of the detected set L of straight-line pointsminMinimum ordinate YminAnd the maximum abscissa XmaxMaximum ordinate YmaxWill be (X)min,Ymin),(Xmin,Ymax),(Xmax,Ymin),(Xmax,Ymax) As 4 angular points, establishing a rectangular area J where the high-voltage transmission line is located;
6) continuously detecting the video stream in the step 5), and when a target is detected to appear, setting the current time as t0At the moment, the currently detected target is set to the target PiDetermining whether the appearance position is located at the edge position; if so, retaining the current target PiPutting the tracking target set S into the tracking target set S, and if not, judging that the tracking target set S is not a new tracking object; as the targets needing alarming are all gradually close to the high-voltage power transmission line from a place far away from the high-voltage power transmission line, the targets are located at the edge position of the whole monitoring range when being detected for the first time, the inherent properties of a video streaming device are obtained, the numerical value of resolution is obtained, the boundary threshold value is set, and the four first indexes of target output obtained by using a yolov5 model are (x, y, w, h), wherein (x, y) represents the abscissa and the ordinate of the central point of the detected object, and (w, h) represents a rectangular frameSo that the current target P is determined as long as the center point (x, y) is within the resolution minus the boundary thresholdiThe detection result is positioned at the edge position, otherwise, the detection result is not regarded as a new tracking object;
7) will t0The next frame at a time is set to t1Time, the target detected at that time is set to P1Using a target tracking algorithm based on the Deepsort algorithm to track the current target P1Matching and tracking all targets in the target set S; if the current target P1And tracking the target S1If the matching is successful, the target S is tracked1Is updated to the current target P1And S is calculated from the position information of2Putting the target set S into a target set S; if the current target P1Does not match any existing targets in set S, but the current target P1At the edge position described in step 6), the target P is positioned1As a new tracking target S2Putting the target set S into a target set S;
8) continuously and uninterruptedly detecting the video stream, and completing target tracking through the step 6) and the step 7); according to the tracked target SiFitting a future motion equation of the future motion trajectory, namely fitting a motion equation linearly; substituting points in the rectangular area J into a linear fitting motion equation, and if the equation is equal left and right, judging that the tracking target S is equaliThe method comprises the following steps that (1) a trend of regional movement of the high-voltage transmission line is shown, and a prompt needs to be sent;
9) for the tracking target S in step 8)iContinuing to track the target if the target S is trackediFinally, if the high-voltage transmission line stops moving in the rectangular area where the high-voltage transmission line is located, the foreign matter which can interfere the operation of the high-voltage transmission line is judged, and an alarm needs to be given; if the target S is trackediAnd finally, the monitoring range is separated from the boundary, so that the monitoring range is judged to be a passing object, the normal operation of the high-voltage transmission line is not interfered, and the reminding needs to be sent again.
2. The method for detecting the foreign matters on the high-voltage transmission line based on the target tracking algorithm according to claim 1, wherein the method comprises the following steps: in the step 1), the camera installed on the power transmission tower is lower than the height of the high-voltage power transmission line, so that the camera can shoot the high-voltage power transmission line from bottom to top while ensuring the shot pictures to be clear; by adopting two cameras, namely a bidirectional camera system can ensure that a target tracking algorithm can normally run when a plurality of foreign matters are overlapped and shielded, and rapidity and accuracy are not influenced.
3. The method for detecting the foreign matters on the high-voltage transmission line based on the target tracking algorithm according to claim 1, wherein the method comprises the following steps: in the step 2), data enhancement is carried out on the data acquired in the camera, and the data enhancement comprises random cutting, noise addition and mirror image turning.
4. The method for detecting the foreign matters on the high-voltage transmission line based on the target tracking algorithm according to claim 1, wherein the method comprises the following steps: in the step 2), the picture is marked by using a marking tool labelme, namely, a rectangle with proper size is used for framing the corresponding target object to obtain marking information xmin、ymin、xmax、ymaxWherein x isminIs the left boundary abscissa, x, of the rectangular framemaxIs the right boundary abscissa, y, of the rectangular frameminIs the lower boundary ordinate, y, of the rectangular framemaxIs the vertical coordinate of the upper boundary of the rectangular frame; randomly disorganizing the marked pictures, and dividing the pictures into a training set, a verification set and a test set, wherein the ratio of the training set to the verification set to the test set is 8:1: 1.
5. The method for detecting the foreign matters on the high-voltage transmission line based on the target tracking algorithm according to claim 1, wherein the method comprises the following steps: in the step 4), the confidence scores of the detection frames are sorted from high to low, and the detection frame with the highest confidence is placed in the set M; the improved non-maximum inhibition algorithm is modified by reducing the confidence score of a new detection frame according to the sizes of all elements IOU in M, wherein the larger the IOU is, the more the confidence is reduced, and the smaller the IOU is, the smaller the confidence is reduced, if the confidence after iterative update is still larger than a threshold value, the new detection frame is added into the set M until all pictures go through, and through the improvement, the detection accuracy can be greatly improved.
6. The method for detecting the foreign matters on the high-voltage transmission line based on the target tracking algorithm according to claim 1, wherein the method comprises the following steps: in step 7), firstly, the targets S in the tracking target set S areiUsing Kalman filtering, object SiAccording to t0State of time, predicted target SiAt t1Position coordinates (x) of time of dayi,yi,wi,hi) Through t0The covariance of the moment is found to be t1The prediction error of the moment is calculated to obtain the Kalman gain, and the target S is predicted through the Kalman gainiThe predicted position of (a) is corrected to obtain the final predicted position coordinate (x)i,yi,wi,hi) Then to t1Using the yolov5 model with soft-nms algorithm in the step 4) to predict at any moment to obtain the detection target PiPosition (x'i,y'i,w'i,h'i)。
7. The method for detecting the foreign matters on the high-voltage transmission line based on the target tracking algorithm according to claim 1, wherein the method comprises the following steps: in step 7), the Hungarian algorithm is used to detect the target PiAnd predicting the target SiMatching: calculating a detection target PiAnd predicting the target SiIf the distance between the two is smaller than the threshold value, the two are judged to be the same target and successfully matched, and the target S is selectediIs updated to (x'i,y'i,w'i,h'i) Completing target tracking; if all the predicted targets and detected targets P in the set SiThe Mahalanobis distances are all larger than the threshold value or the two are different types of targets although the distance between the two is smaller than the threshold value, further judgment is carried out; if the target P is detectediThe center point (x, y) of the position information in step 6) is located at the resolution minus the boundary thresholdWithin the range, the detection target P is determinediPutting the new tracking target into a target set S; if the target P is detectediIf the position information center point (x, y) in the step 6) is outside the range of the resolution minus the boundary threshold, the detection target P is judgediDeleting the detection target P for error detection informationi。
CN202110387931.1A 2021-04-12 2021-04-12 High-voltage transmission line foreign matter detection method based on target tracking algorithm Active CN113076899B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110387931.1A CN113076899B (en) 2021-04-12 2021-04-12 High-voltage transmission line foreign matter detection method based on target tracking algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110387931.1A CN113076899B (en) 2021-04-12 2021-04-12 High-voltage transmission line foreign matter detection method based on target tracking algorithm

Publications (2)

Publication Number Publication Date
CN113076899A true CN113076899A (en) 2021-07-06
CN113076899B CN113076899B (en) 2023-04-07

Family

ID=76617310

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110387931.1A Active CN113076899B (en) 2021-04-12 2021-04-12 High-voltage transmission line foreign matter detection method based on target tracking algorithm

Country Status (1)

Country Link
CN (1) CN113076899B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114724091A (en) * 2022-06-07 2022-07-08 智洋创新科技股份有限公司 Method and device for identifying foreign matters on transmission line wire
CN115761537A (en) * 2022-11-14 2023-03-07 西安科技大学 Power transmission line foreign matter intrusion identification method oriented to dynamic characteristic supplement mechanism
CN117742431A (en) * 2023-12-25 2024-03-22 中科酷原量子科技(武汉)有限公司 High-voltage stabilization method and control equipment thereof

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0389103A (en) * 1989-08-31 1991-04-15 Fujikura Ltd Apparatus for detecting approach of obstacle to overhead power transmission line
CN101739694A (en) * 2010-01-07 2010-06-16 北京智安邦科技有限公司 Image analysis-based method and device for ultrahigh detection of high voltage transmission line
CN104463904A (en) * 2014-09-22 2015-03-25 国家电网公司 High-voltage line foreign matter invasion target detection method
CN106056821A (en) * 2016-08-13 2016-10-26 哈尔滨理工大学 Power-transmission-line foreign-matter-invasion intelligent-video on-line monitoring evaluation system
CN109871763A (en) * 2019-01-16 2019-06-11 清华大学 A kind of specific objective tracking based on YOLO
CN110516556A (en) * 2019-07-31 2019-11-29 平安科技（深圳）有限公司 Multi-target tracking detection method, device and storage medium based on Darkflow-DeepSort
CN111444809A (en) * 2020-03-23 2020-07-24 华南理工大学 Power transmission line abnormal target detection method based on improved YO L Ov3
CN112541424A (en) * 2020-12-07 2021-03-23 南京工程学院 Real-time detection method for pedestrian falling under complex environment

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0389103A (en) * 1989-08-31 1991-04-15 Fujikura Ltd Apparatus for detecting approach of obstacle to overhead power transmission line
CN101739694A (en) * 2010-01-07 2010-06-16 北京智安邦科技有限公司 Image analysis-based method and device for ultrahigh detection of high voltage transmission line
CN104463904A (en) * 2014-09-22 2015-03-25 国家电网公司 High-voltage line foreign matter invasion target detection method
CN106056821A (en) * 2016-08-13 2016-10-26 哈尔滨理工大学 Power-transmission-line foreign-matter-invasion intelligent-video on-line monitoring evaluation system
CN109871763A (en) * 2019-01-16 2019-06-11 清华大学 A kind of specific objective tracking based on YOLO
CN110516556A (en) * 2019-07-31 2019-11-29 平安科技（深圳）有限公司 Multi-target tracking detection method, device and storage medium based on Darkflow-DeepSort
CN111444809A (en) * 2020-03-23 2020-07-24 华南理工大学 Power transmission line abnormal target detection method based on improved YO L Ov3
CN112541424A (en) * 2020-12-07 2021-03-23 南京工程学院 Real-time detection method for pedestrian falling under complex environment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
贾宇辉 等: "基于S3C2440的电力线巡检机器人系统设计", 《电测与仪表》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114724091A (en) * 2022-06-07 2022-07-08 智洋创新科技股份有限公司 Method and device for identifying foreign matters on transmission line wire
CN115761537A (en) * 2022-11-14 2023-03-07 西安科技大学 Power transmission line foreign matter intrusion identification method oriented to dynamic characteristic supplement mechanism
CN117742431A (en) * 2023-12-25 2024-03-22 中科酷原量子科技(武汉)有限公司 High-voltage stabilization method and control equipment thereof

Also Published As

Publication number Publication date
CN113076899B (en) 2023-04-07

Similar Documents

Publication Publication Date Title
CN113076899B (en) High-voltage transmission line foreign matter detection method based on target tracking algorithm
KR102129893B1 (en) Ship tracking method and system based on deep learning network and average movement
US20240062518A1 (en) Method, apparatus, computer device and storage medium for detecting objects thrown from height
CN108009515B (en) Power transmission line positioning and identifying method of unmanned aerial vehicle aerial image based on FCN
CN110084165B (en) Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation
CN109344753A (en) A kind of tiny fitting recognition methods of Aerial Images transmission line of electricity based on deep learning
CN104966304A (en) Kalman filtering and nonparametric background model-based multi-target detection tracking method
Wang et al. Pointer meter recognition in UAV inspection of overhead transmission lines
Song et al. Deformable YOLOX: detection and rust warning method of transmission line connection fittings based on image processing technology
CN115880231A (en) Power transmission line hidden danger detection method and system based on deep learning
CN115995058A (en) Power transmission channel safety on-line monitoring method based on artificial intelligence
CN113095160B (en) Power system personnel safety behavior identification method and system based on artificial intelligence and 5G
CN117058541B (en) Insulator hyperspectral data acquisition system and method thereof
CN116503957B (en) Gas household operation behavior identification method
CN111967323B (en) Electric power live working safety detection method based on deep learning algorithm
CN116740833A (en) Line inspection and card punching method based on unmanned aerial vehicle
CN115100249B (en) Intelligent factory monitoring system based on target tracking algorithm
CN116311034A (en) Robot inspection system based on contrast detection
CN115187880A (en) Communication optical cable defect detection method and system based on image recognition and storage medium
CN113554682B (en) Target tracking-based safety helmet detection method
CN116055521A (en) Inspection system and image recognition method for electric inspection robot
CN111025288B (en) Security radar monitoring device and system
CN112541403B (en) Indoor personnel falling detection method by utilizing infrared camera
CN112149579A (en) Improved yolo _ v 3-based bird nest hidden danger detection method
CN116912721B (en) Power distribution network equipment body identification method and system based on monocular stereoscopic vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant