CN115761537A - Power transmission line foreign matter intrusion identification method oriented to dynamic characteristic supplement mechanism - Google Patents
Power transmission line foreign matter intrusion identification method oriented to dynamic characteristic supplement mechanism Download PDFInfo
- Publication number
- CN115761537A CN115761537A CN202211418023.5A CN202211418023A CN115761537A CN 115761537 A CN115761537 A CN 115761537A CN 202211418023 A CN202211418023 A CN 202211418023A CN 115761537 A CN115761537 A CN 115761537A
- Authority
- CN
- China
- Prior art keywords
- foreign
- foreign matter
- transmission line
- power transmission
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 84
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000007246 mechanism Effects 0.000 title claims abstract description 44
- 239000013589 supplement Substances 0.000 title claims abstract description 32
- 238000001514 detection method Methods 0.000 claims abstract description 70
- 238000012549 training Methods 0.000 claims abstract description 38
- 230000004927 fusion Effects 0.000 claims abstract description 24
- 239000002775 capsule Substances 0.000 claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims abstract description 15
- 238000001914 filtration Methods 0.000 claims abstract description 10
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 230000003042 antagnostic effect Effects 0.000 claims abstract description 4
- 238000012545 processing Methods 0.000 claims description 9
- 238000012360 testing method Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 5
- 238000012795 verification Methods 0.000 claims description 5
- 230000009545 invasion Effects 0.000 claims description 3
- 230000009469 supplementation Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000001419 dependent effect Effects 0.000 claims 2
- 238000012935 Averaging Methods 0.000 claims 1
- 238000012544 monitoring process Methods 0.000 abstract description 18
- 230000006870 function Effects 0.000 description 12
- 230000015654 memory Effects 0.000 description 12
- 239000013598 vector Substances 0.000 description 10
- 230000002457 bidirectional effect Effects 0.000 description 9
- 230000006399 behavior Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 238000007689 inspection Methods 0.000 description 7
- 238000012423 maintenance Methods 0.000 description 7
- 230000002159 abnormal effect Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 5
- 238000009499 grossing Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 230000003068 static effect Effects 0.000 description 5
- 238000013459 approach Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000012212 insulator Substances 0.000 description 3
- 230000007787 long-term memory Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000006403 short-term memory Effects 0.000 description 3
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 101100441251 Arabidopsis thaliana CSP2 gene Proteins 0.000 description 1
- 208000025274 Lightning injury Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 235000005770 birds nest Nutrition 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000010924 continuous production Methods 0.000 description 1
- 238000013434 data augmentation Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000005684 electric field Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 239000002985 plastic film Substances 0.000 description 1
- 229920006255 plastic film Polymers 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000036319 strand breaking Effects 0.000 description 1
- 235000005765 wild carrot Nutrition 0.000 description 1
Images
Abstract
The invention relates to the technical field of foreign matter identification of a power transmission line, and discloses a power transmission line foreign matter intrusion identification method oriented to a dynamic feature supplement mechanism, which comprises the following steps: acquiring a real-time video image around the power transmission line; preprocessing by adopting a Gaussian filtering algorithm; inputting a video frame sequence into an image detection model with multi-layer feature fusion, and identifying and positioning foreign matters; extracting foreign body type information by a matrix capsule network classifier; expanding the training data set by a method for generating an antagonistic network; judging the foreign matter type information by using a pre-trained foreign matter detection model facing a dynamic characteristic supplement mechanism to determine the foreign matter type and determine an early warning grade; after the early warning level is output, the influence of foreign matters on the power transmission line on the line is prompted by the system, and the foreign matter intrusion identification and early warning method can better avoid the problem of missing identification, reduce the labor cost and improve the foreign matter monitoring efficiency.
Description
Technical Field
The invention relates to the technical field of power transmission line foreign matter identification, in particular to a power transmission line foreign matter intrusion identification method oriented to a dynamic characteristic supplement mechanism.
Background
The power transmission line is a main carrier of power transmission, is also an important component of a power system, and plays an important role in safe and stable operation of the system. In recent years, with the increasing electricity consumption, the voltage grade of the transmission lines in China is continuously improved, the number of the transmission lines is greatly expanded, and the transmission lines are widely distributed in various areas such as towns and rural areas in China. In addition, the position of the power transmission line is complex and variable, and foreign matters are easy to adhere to the power transmission line in places such as plateau areas, hills, basins and mountainous areas. And in places such as densely populated residential areas and commercial areas, articles of daily use such as knitwear, balloons, kites, plastic films and the like are easy to attach. The transmission line is also vulnerable to bird damage in natural environment, such as bird nests in the transmission line. If the foreign matters are not discovered and cleaned in time, the normal operation of the power transmission line can be influenced, the personal safety can be harmed under severe conditions, and accidental electric shock accidents and the like can be caused. When dense forests are arranged near the power transmission line, branches can be attached to the power transmission line under the condition of severe weather environment, such as storm, lightning stroke and the like, and the branches can weaken the electric field of the power transmission line at the moment and can cause power failure accidents in severe cases. If these foreign matters can not discover in time and clear up, will influence transmission line's normal work, even cause the power failure or incident.
At present, the unmanned aerial vehicle inspection technology is used for daily power transmission line inspection, so that inspection time can be effectively saved, and meanwhile, the safety of inspection personnel can be guaranteed. Generally adopt unmanned aerial vehicle to monitor transmission line, shoot the video by unmanned aerial vehicle's airborne camera, look over the video image that unmanned aerial vehicle shot by operating personnel again, and then investigate the foreign matter condition on the transmission cable. The method relies on manual identification of foreign matters, and can not meet the requirement of monitoring the foreign matter condition on the power transmission line in real time under dangerous conditions. Therefore, a more accurate and convenient algorithm identification method is urgently needed to liberate manual labor force and accurately identify the type of the foreign matter and the influence of the foreign matter on the power transmission line.
Disclosure of Invention
The invention provides a power transmission line foreign matter intrusion identification method facing a dynamic characteristic supplement mechanism, which trains data facing the dynamic characteristic supplement mechanism, further improves the accuracy of system monitoring data and the processing capacity of the data, and solves the problems of low accuracy of traditional samples and inaccurate identifiable characteristics.
The invention provides a transmission line foreign matter intrusion identification method facing a dynamic characteristic supplement mechanism, which comprises the following steps:
transmitting a video image around the power transmission line acquired by the unmanned aerial vehicle to a power supply management platform through a wireless communication interface, and preprocessing the received video image by the power supply management platform by adopting a Gaussian filter algorithm to obtain a processed video image;
inputting the processed video image into an image detection model with multi-layer feature fusion, and identifying and positioning the foreign matters to obtain a video frame sequence with a foreign matter mark frame;
establishing different foreign matter type data sets of the power transmission line, wherein the data sets comprise a training data set, a testing data set and a verification data set;
extracting foreign body type information by matrix capsule network classifier
Carrying out foreign matter classification on the video frame sequence with the foreign matter mark frame through a matrix capsule network classifier to obtain a classified foreign matter image set, and carrying out transformation processing to generate a new foreign matter image;
expanding a training data set by using a method for generating an antagonistic network, and training a foreign body detection model facing a dynamic characteristic supplementation mechanism by using the expanded training data set;
judging the foreign matter type information by using a pre-trained foreign matter detection model facing a dynamic feature supplement mechanism to determine the foreign matter type, and judging to determine an early warning grade according to the foreign matter type;
after the early warning level is output, the system prompts the influence of foreign matters on the power transmission line on the line, and meanwhile, the state of the foreign matters in the video is marked.
The preprocessing is performed through a gaussian filtering algorithm, a weighted average process is performed on the whole image, the value of each pixel point is obtained by performing weighted average on the value of each pixel point and other pixel values in the neighborhood, and the specific method is as follows:
scanning each pixel in the acquired video frame image by using a Gaussian mask template with the size of 3 multiplied by 3, and replacing the value of the central pixel point of the Gaussian mask template by using the weighted average gray value of the pixels in the neighborhood determined by the Gaussian mask template.
The specific method for identifying and positioning the foreign object by inputting the video frame sequence into the image detection model with multi-layer feature fusion comprises the following steps:
performing behavior time sequence feature extraction on foreign matters in a complex scene by using an image detection model by adopting a feature extraction method based on target detection and tracking;
performing targeted training on the image detection model with multi-layer feature fusion by using a self-built data set;
processing and analyzing the extracted foreign body behavior time sequence characteristics to obtain a video frame sequence with a foreign body mark frame;
the multi-layer feature fusion image detection model is used for dynamically optimizing the expression capability of a feature graph by fusing different levels of feature graphs to aggregate context information and generating output weights of the feature graphs of each level in a self-adaptive mode according to the size of a training target sample in consideration of different contributions of information contained in the feature graphs of different levels to a small target detection task.
The specific method for establishing the data sets of different foreign matter types of the power transmission line comprises the following steps:
the different foreign matter type data sets comprise data sets of a plurality of different types of foreign matter samples, and the data sets are divided into a training data set, a testing data set and a verification data set according to a certain proportion; the training data set is used for training a dynamic characteristic-oriented supplementary model, the testing data set is used for testing the generalization ability of the model after the model training is completed, and the verifying data set is used for testing whether the trained model has an overfitting phenomenon.
The specific method for expanding the training data set by the method for generating the countermeasure network comprises the following steps:
generating a new image by converting the image with the foreign matter mark frame through horizontal overturning, scaling and translation by the countermeasure network;
and splicing the foreign body part into the foreign body-free image in an image splicing mode.
The method for determining the type of the foreign body comprises the following steps of judging the type information of the foreign body by using a pre-trained foreign body detection model facing a dynamic characteristic supplement mechanism, wherein the specific method for determining the type of the foreign body comprises the following steps:
because the power transmission line belongs to a small-probability event under daily conditions, if the tracking data of each foreign object in a complex scene is detected, a large amount of resources are consumed, and on the basis of ensuring the precision, the video frame sequence with the foreign object marking frame is firstly analyzed;
then, a plurality of detection threshold values are set, and all foreign matter types in the video are classified according to the detection threshold values, wherein the types include foreign matters, foreign matter-like matters and no foreign matters.
The foreign object detection model for the dynamic feature supplement mechanism is input at the current moment not only by the previous video frame but also by the next video frame.
Compared with the prior art, the invention has the beneficial effects that:
(1) At present, foreign body samples of the power transmission line are difficult to collect, so that the number of the samples is small, and the problems of insufficient sample diversity and few samples can be well solved by adopting a generation countermeasure network technology.
(2) The existing algorithm network only singly adopts the traditional picture as analyzed data, the definition of the traditional visible light picture is limited, the network trained by the method has no universality and cannot completely include all recognizable foreign matter conditions, and the problem can be well optimized by utilizing the preprocessed video image.
(3) Foreign matters on the power transmission line are a continuous process in time, specific foreign matter categories are difficult to judge only by independent picture information, and the situations of misjudgment and poor detection effect are easy to occur in a complex scene. For monitoring foreign matters, the most important is to extract robust behavior characteristics to adapt to the change of the environment in a complex scene, and the second is to detect the foreign matters on the power transmission line by utilizing space-time characteristics in continuous video frames through context information and various judgment methods. On the basis of the features extracted by the image detection network, an improved foreign matter identification algorithm based on fusion attention mechanism pre-judgment facing to a dynamic feature supplement mechanism is used for integrating the time sequence information of the video so as to monitor the foreign matters. In order to reduce model parameters and improve detection efficiency, a pre-judgment method based on attention mechanism behavior characteristics is used, and a foreign matter monitoring algorithm is improved.
(4) The dynamic characteristic supplement mechanism oriented network well solves the problem that the traditional long and short term memory network can only rely on the previous information to predict and the neglected result also has response influence on the current information.
Drawings
Fig. 1 is a schematic flow chart of a transmission line foreign matter intrusion identification method oriented to a dynamic feature supplement mechanism.
Fig. 2 is a block diagram of foreign matter monitoring feature extraction of a power transmission line in a complex environment.
FIG. 3 is a schematic diagram of a matrix capsule classifier.
Fig. 4 is a general framework of a foreign matter monitoring model of a power transmission line under a complex environment.
Fig. 5 is a network architecture diagram for a dynamic feature supplementation mechanism.
Fig. 6 is a flow chart of an early warning system.
Detailed Description
An embodiment of the present invention will be described in detail below with reference to fig. 1-6, but it should be understood that the scope of the present invention is not limited to the embodiment.
As shown in fig. 1, a method for identifying intrusion of a foreign object into a power transmission line facing a dynamic feature supplement mechanism according to an embodiment of the present invention includes the following steps:
1. the monitoring platform is arranged on an unmanned aerial vehicle, flies near the power transmission line through the unmanned aerial vehicle, and uses a system camera to carry out real-time video recording and monitoring to obtain a video image on the real-time power transmission line; preprocessing the collected video image data to obtain corresponding processing data; inputting the preprocessed video stream data into a multi-layer feature fusion image detection model, identifying and positioning foreign matters in a complex environment by using the multi-layer feature fusion image detection model, marking the foreign matters in the video, and identifying the foreign matters; establishing different foreign matter type data sets of the power transmission line, wherein the data sets comprise sampling sequences of different power transmission lines and corresponding foreign matter type label values, extracting foreign matter type information from a matrix capsule network classifier, sorting the foreign matter type information into time sequence data, expanding a training data set by using a method for generating a countermeasure network, and sending the training data set to a foreign matter detection model of the next step; judging the foreign body by using a pre-trained network foreign body detection model facing a dynamic feature supplement mechanism according to the time sequence data and the foreign body type information, judging according to output, and determining an early warning level; after the early warning level is output, the system prompts the influence of foreign matters on the power transmission line, and meanwhile, the state of the foreign matters in the video is marked.
2. The specific method for identifying and early warning the foreign matter invasion comprises the following steps:
(1) The wireless communication interface for real-time video recording and transmission:
the real-time video recording and transmission of the unmanned aerial vehicle need use a wireless communication interface, a bidirectional wireless communication link is established with a remote power supply management platform, the power supply management platform sends out a control command, the remote video recording and transmission system is also used for forwarding foreign body position information, foreign body types and foreign body early warning information sent by a digital signal processor to the power supply management platform, and the control command comprises power transmission line measurement height and power transmission line measurement positioning data.
(2) Preprocessing the acquired image data by adopting a Gaussian filtering algorithm:
and (3) carrying out weighted average on the whole video frame image by adopting a Gaussian filtering algorithm, wherein the value of each pixel point is obtained by carrying out weighted average on the value of each pixel point and other pixel values in the neighborhood. The specific operation is to scan each pixel in the video frame image obtained by a 3 × 3 gaussian mask template (or convolution or mask), and to replace the value of the central pixel point of the template with the weighted average gray value of the pixels in the neighborhood determined by the template.
(3) Identifying and positioning foreign matters in a complex environment by using a multi-layer feature fusion image detection model structure:
the multilayer feature fusion image detection model is obtained by adding a real-time target tracking method module in the existing YOLOv5s algorithm model, and a foreign object marking frame detected in the detection method of the multilayer feature fusion image detection model can well track a foreign object to be marked, so that the multilayer feature fusion image detection model has strong robustness in a complex environment.
The image detection model structure with the multi-layer feature fusion is used for identifying and positioning the foreign matters in the complex environment, the feature extraction method based on target detection and tracking is adopted, and the image detection model is used for extracting the behavior time sequence features of the foreign matters in the complex scene. The self-built data set is used for carrying out targeted training on the image detection model with the multi-layer feature fusion so as to improve the foreign matter detection rate on the power transmission line, and then extracted foreign matter features are processed and analyzed. Considering that information contained in different levels of feature maps contributes differently to a small target detection task, a plurality of levels of feature fusion layers are designed to integrate different feeling information, context information is aggregated by fusing different levels of feature maps, and the output weight of each level of feature map is generated in a self-adaptive mode according to the size of a training target sample to dynamically optimize the expression capacity of the feature map.
The network structure of YOLOv5s is mainly divided into four parts of an input end, a Backbone, a Neck and a Prediction.
Focus structure and CSP structure were used in the Backbone section of the body. Two CSP structures are designed in YOLOv5, wherein a CSP1_ X structure is in a Backbone network of a Backbone, and a CSP2_ X structure is in a Neck. In the Neck part, YOLOv5s uses the same FPN + PAN structure as that of the conventional YOLOv4, but the Neck structure of YOLOv5s adopts a CSP2 structure designed by referring to CSPnet to enhance the capability of network feature fusion. The GIOU Loss is used as the Loss of the Bounding Box in the output section. The loss of class probability and target score is calculated using binary cross entropy and logs loss functions.
<xnotran> , 8 (u, v, γ, </xnotran>
And (v) as a direct observation model of the dynamic foreign body state, predicting an updated track state (u, v, gamma,) wherein (u, v) represents the position of a central point of a boundary frame, gamma represents the width-to-height ratio of the boundary frame, h represents the height, and the last four parameters represent the relative speeds of the first four parameters on corresponding image coordinates. And calculating the difference value between the current frame and the last matching successful frame in each track. And judging the change of the life cycle of the dynamic foreign matters on the track by using the max _ age threshold, deleting the track if no dynamic change exists, and initializing the track to be a new track if the tracking monitoring target cannot be matched with the dynamic track of the existing foreign matters. The state when the new path is initialized is an undetermined state, and the undetermined state can be converted into a determined state only if three continuous frames meet the preset running track of the dynamic object. Such asIf the tracking monitoring target in the undetermined state is not matched with foreign object detection in the initial track set, the tracking monitoring target is changed into a deleted state, namely deleted from the track set.
The specific implementation flow of the image detection model with the multi-layer feature fusion is that the video image preprocessed by the Gaussian filter algorithm is added with the minimum filling to the original image by using a self-adaptive method, is uniformly scaled to a standard size, and is input into a detection network. The method comprises the steps of detecting and positioning foreign matters around a power transmission line, which are objects needing to be detected, in a video image by using a YOLOv5s algorithm, carrying out target tracking on the dynamic foreign matters by using a real-time target tracking method in order to grasp continuous behavior characteristics of the dynamic foreign matters after the foreign matters are positioned, so as to obtain the motion trail of the dynamic foreign matters of the target, and using the motion trail to carry out feature extraction and foreign matter type judgment later.
(4) Extracting foreign body type information by using a matrix capsule network classifier:
the classifier using the matrix capsule network as a target detection framework is used for judging the relationship among the constituent features of different types of foreign matters and aims to obtain a more accurate foreign matter type classification result. . In order to accurately recognize the same type of foreign object images under different conditions such as different postures, angles, illumination and the like, a large amount of training data covering all the differences is required. (5) Augmenting training data set with method of generating countermeasure network
For the faults of the power transmission line, data collection is very difficult, the diversity of samples is stronger, the number is less, and therefore in order to prevent the fault recognition model from being over-fitted and enhance the generalization capability of the fault recognition model, a method for generating a countermeasure network is needed to be used for expanding a training data set.
The generated countermeasure network transforms the image with the foreign matter mark frame through methods of horizontal turning, scaling, translation and the like to generate a new image, and then splices the foreign matter part into the image without the foreign matter part in an image splicing mode, aiming at obtaining a more accurate recognition result under the training of a small amount of samples. Because the mode that actual circuit erect and the angle that unmanned aerial vehicle closed on transmission line flight are various, therefore these above image operation all are meaningful to the data expansion of transmission parts such as insulator, wire.
The generation of a countermeasure network consists of two important parts: a Generator (Generator, abbreviated G) and a Discriminator (Discriminator, abbreviated D). In the specific training process, the generated model simulates generated data based on a hidden random vector z by inputting image data with a foreign body marking frame and marks the data as G (z), wherein x is real foreign body image data acquired by an unmanned aerial vehicle, and the relation between the two is written into a formula, namely x = G (z). Judging the input parameter of the model to be x, outputting the probability that D (x) represents that x is the real foreign object image data, respectively inputting the image data of x and G (z) into the judgment model, performing two-classification prediction, and finally updating the judgment model parameter by using the two-classification cross entropy loss; and then, fixing a judgment model to optimize a generation model, and regarding the generation model, in order to deceive the judgment model as much as possible, namely to make the judgment model judge the generated false foreign object image as real foreign object image data as much as possible, generally considering the optimization with the aim of maximizing the judgment probability of the generation model. If the output D (x) probability is 1, it represents that 100% of the output foreign-matter image data is true data, and if the output is 0, it represents that it is impossible to be true data.
In the training optimization stage, on one hand, the objective function is maximized, so that the prediction probability D (x) of a real data sampling sample x is close to 1 as much as possible, and the prediction probability D (G (z)) of a generating sample G (z) is close to 0; on the other hand, the generative model is to minimize the objective function, and logD (x) is independent of the generative model, so the latter term is mainly minimized in this case, so that the generative model generation sample allows the judgment model prediction probability D (G (z)) to approach 1.
(6) Using a foreign object detection model oriented to a dynamic feature replenishment mechanism:
the improved foreign matter identification algorithm for dynamic characteristic supplement mechanism prejudgment is used for identifying and monitoring foreign matters on the power transmission line by processing time sequence information between continuous frames through a bidirectional long-time and short-time memory algorithm. Analyzing the behavior characteristic time sequence information obtained by the image detection network model structure with the multi-layer characteristic fusion in the step (3), setting a rough detection threshold value, classifying all foreign body types in the video, directly distinguishing easily distinguishable non-foreign body images, and sending the foreign body images and the foreign body-like images which are difficult to distinguish into a foreign body detection model of a bidirectional long-time and short-time memory network based on an attention mechanism for detection and judgment, thereby reducing the calculation complexity and improving the detection efficiency.
The dynamic characteristic-oriented supplementary mechanism well solves the problems that the traditional long and short term memory network can only depend on the previous information to predict the subsequent result and the neglected result has response influence on the current information, and the bidirectional long and short term memory network is characterized in that the input at the current moment not only depends on the previous video frame but also depends on the next video frame.
The network facing the dynamic characteristic supplement mechanism comprises an input layer, a forward propagation layer, a backward propagation layer and an output layer, wherein the forward propagation layer and the backward propagation layer are connected with the output layer together, forward calculation is carried out on the forward propagation layer from 1 moment to t moment, and the output of the forward propagation layer at each moment is obtained and stored. And calculating in the backward propagation layer in the backward direction from the moment t to the moment 1 in the backward direction, obtaining and storing the output of the backward propagation layer at each moment, and obtaining the final output by combining the corresponding output results at each moment of the forward propagation layer and the backward propagation layer. And taking the average value of the two vectors corresponding to the time as an output characteristic vector, and inputting the vector into an attention mechanism to learn the network weight.
(7) The early warning system is mainly used for dispatching the abnormal early warning information judged by the expert analysis system and recycling the operation and maintenance feedback. The abnormity early warning makes different prompts according to different danger grades, a popup prompt and an alarm sound appear in dangerous situations, and only a flashing prompt is given in other situations; the click-in interface can generate abnormal early warning information, which comprises abnormal information (geographical region and tower pole information, terminal equipment data, abnormal terminal equipment number, abnormal time and the like), foreign matter information (foreign matter occurrence probability and foreign matter type), early warning grade (classified into danger, general and temporary influence), dispatching maintenance information (dispatching operation and maintenance personnel number, maintenance condition and the like) and operation and maintenance feedback (whether foreign matter exists, foreign matter type, whether maintenance exists or not, maintenance description and the like).
The system comprises a static memory, a reference image template and a foreign matter type alarm comparison table, wherein the reference image template is used for storing various foreign matters in advance, the foreign matter type alarm comparison table takes the foreign matter types as indexes and stores the alarm grade of each foreign matter type, the higher the alarm grade is, the greater the damage of the corresponding foreign matters to the power transmission line is, the static memory is also used for storing alarm grade threshold values in advance, and the various foreign matter types in the foreign matter type alarm comparison table stored in the static memory in advance comprise plastic bags, balloons, kites, branches and the like;
s100: flying by an unmanned aerial vehicle close to the power transmission line, and carrying out real-time video recording and monitoring by using a system camera to obtain a video image on the real-time power transmission line;
s200: inputting the preprocessed video stream data into a multi-layer feature fusion image detection algorithm, identifying and positioning foreign matters in a complex environment by using a target detection algorithm, marking the foreign matters in the video, and identifying the foreign matters;
s300: establishing different foreign matter type data sets of the power transmission line, wherein the data sets comprise sampling sequences of different power transmission lines and corresponding foreign matter type label values, extracting foreign matter type information from a matrix capsule network classifier, and sorting the foreign matter type information into time sequence data;
s400: expanding a training data set by a method for generating an antagonistic network, and sending the training data set to a foreign body detection model of the next step;
s500: and judging the foreign matter detection model by using a pre-trained bidirectional long-short time memory network foreign matter detection model based on an attention mechanism according to the time sequence data and the foreign matter type information, judging according to the output, and determining the early warning level.
S600: after the early warning level is output, the system prompts the influence of foreign matters on the power transmission line on the line, and meanwhile, the state of the foreign matters in the video is marked.
2. The most important parameter of the gaussian filter template is the standard deviation δ of the gaussian distribution. It represents the discrete degree of data, if delta is smaller, the larger the center coefficient of the generated template is, and the smaller the surrounding coefficients are, the less obvious the smoothing effect of the image is; when delta is larger, the difference of each coefficient of the generated template is not large, the coefficient is close to the average template, and the smoothing effect on the image is obvious. Gaussian filtering all that is first done is to find the gaussian kernel of the image: assuming that the center point coordinate is (0,0), the 8 points closest to it are taken, and δ =1.5 is set, a gaussian kernel with a blur radius of 1 can be obtained. It is also ensured that these nine points add up to 1 (characteristic of the gaussian template), so that the final gaussian template is obtained by dividing all 9 values by the weight of these 9 points.
The form of the two-dimensional Gaussian function is formula (1), the two-dimensional Gaussian function is a discrete point on an image, the function is discretized to obtain formula (2), the smoothing effect of Gaussian filtering depends on delta, the larger the delta, the closer the generated template center coefficient is to the surrounding neighborhood coefficients, the better the smoothing effect is, the smaller the delta, the larger the center coefficient is and the smaller the surrounding neighborhood coefficients are, and the worse the smoothing effect is.
Gaussian convolution kernel H (2k + 1) x (2k + 1), k is the filter kernel, and δ is the variance.
For a 3 × 3 image, let δ =1.5, and the blur radius be 1, a weight matrix may be obtained, which multiplies the image points by step sizes of 1, respectively, to traverse the entire image, thereby obtaining an image processed by gaussian filtering.
3. The multilayer feature fusion image detection method firstly inputs an original 608 × 3 image into a Focus structure, and adopts a slicing operation to firstly change the original 608 × 3 image into a 304 × 12 feature map, and then a convolution operation of 32 convolution kernels is performed to finally change the original 608 × 3 image into a 304 × 32 feature map. And detecting and positioning foreign matters around the power transmission line in the YOLOv5s network, and finally performing target tracking detection on the dynamic foreign matters by using a real-time target tracking method.
In which the main part of the real-time target tracking methodThe process comprises the following steps: 1) Reading the position of a foreign matter detection frame of the current frame and the depth characteristics of image blocks of each detection frame; 2) Filtering the foreign matter detection frames according to the confidence coefficient, and deleting the foreign matter detection frames and the features with low confidence coefficient; 3) Carrying out non-maximum inhibition on the detection frame, and eliminating a plurality of frames of a foreign object image; 4) And (3) prediction: and predicting the position of the foreign matter in the current frame by using Kalman filtering. 4. In the matrix capsule network, the attitude matrix output by the capsule i on the l-th layer is set as V i The output of the capsule j of the l +1 th layer is required, and V is firstly output i And a transformation matrix W with view angle invariance ij Multiplying to obtain vote (vote) V ^ j|i ＝W ij V i (ii) a Wherein W ij Training is performed by Back-Propagation (BP), which represents the local global relationship. The voting needs to be weighted by the agreement coefficients, which are obtained by Expectation-Maximization (EM) clustering based on Gaussian Mixture Model (GMM). Finally, if the Routing agent of the capsule i of the l-th layer converges to the capsule j of the l + 1-th layer, it means that the extracted feature of the capsule i is one of the components of the capsule j, and is clustered into the same cluster together with the other capsules belonging to the feature component of the capsule j of the l-th layer. The matrix capsule classifier is schematically illustrated in fig. 3.
5. In the specific training process of generating the confrontation network, a generator and a discriminator are alternately trained: firstly, fixing a generator, simulating G (z) as a negative sample by using the generator based on a hidden random vector z, sampling from real data to obtain a positive sample x, inputting the positive sample and the negative sample into a discriminator, performing two-class prediction, and finally updating parameters of the discriminator by using two-class cross entropy loss of the positive sample and the negative sample; the arbiter is then fixed and the generator is optimized, generally considering the goal of maximizing the probability of discrimination of the generated samples, in order to fool the arbiter as much as possible, i.e. to make the arbiter judge the generated "false" samples as positive samples as much as possible.
Generating a cost function in the countermeasure network, for discriminator D, with half of the input samples from real data and half from the generator, the cost function being written as J (D) J { (D) } J (D), then the two-class cross-entropy loss for D can be expressed as:
wherein E represents an expected probability, x to Pdaax represent x satisfying P \{ data } distribution,
respectively corresponding to the sum of cross entropy losses of the real data and the generated data, so as to obtain an optimized objective function V:
then, in a training optimization stage, on one hand, the discriminator maximizes an objective function, so that the prediction probability D (x) of a real data sampling sample x approaches to 1 as much as possible, and the prediction probability D (G (z)) of a generating sample G (z) approaches to 0; on the other hand, the generator is to minimize the objective function, and the logD (x) term is generator independent, so that the latter term is mainly minimized, so that the generator generates samples to make the predicted probability D (G (z)) of the discriminator approach to 1.
6. The algorithm facing to the dynamic characteristic supplement mechanism well solves the problems that the traditional long and short time memory network can only depend on the previous information to predict the subsequent result and the neglected result has response influence on the current information, and is realized by three structures: the device comprises an input gate, a forgetting gate and an output gate, wherein the forgetting gate comprises a forward propagation layer and a backward propagation layer. The forward propagation layer and the backward propagation layer are connected with the output layer together, and 6 sharing weights g are included 1 -g 6 . And performing forward calculation from the 1 moment to the t moment in the forward propagation layer to obtain and store the output of the forward hidden layer at each moment. And reversely calculating once along the time t to the time 1 in the backward propagation layer, and obtaining and storing the output of the backward hidden layer at each time. And combining the results of the corresponding outputs of the forward propagation layer and the backward propagation layer at each moment to obtain a final output, wherein the mathematical expression is as follows:
in the above formula, the first and second light sources are,
is a bias of a bidirectional long and short time memory network o' t And o' t The result is that the two memory units process the feature vectors output by the VGG layer at the corresponding time, and tanh is an activation function. The average of the two vectors at the corresponding time is taken as the output feature vector o as shown in equation (9) t And inputting the vector into an attention mechanism to learn the network weight.
The attention mechanism facing the dynamic feature supplement is similar to a brain signal processing mechanism specific to human vision, some important features are highlighted, and the whole network model can show better performance by calculating the weight of output feature vectors of the bidirectional long-time and short-time memory network at different time steps. The calculation formula (10) is as follows:
wherein α in the formula (10) t The parameter is x t The softmax value of the position is calculated by the following formula (11)
In order to verify the effect of a foreign matter detection model with different parameter inputs facing a dynamic feature supplement mechanism, two methods are used for verification.
The first method is to send the video of the static foreign matters (kites or industrial garbage and the like which are easy to distinguish the foreign matters) which are easy to identify into an improved bidirectional long-time memory network for detecting the type of the foreign matters, and directly judge the remaining foreign matters which are difficult to identify (the foreign matters which have small volume and are difficult to judge or dynamic) and images without the foreign matters into non-foreign matters.
The second method is to send foreign matters of various forms (static foreign matters easy to identify and foreign matters difficult to identify) into an improved bidirectional long-time and short-time memory network for foreign matter type detection, and only images without foreign matters are directly judged as non-foreign matters. By comparing the two methods, a method which is more suitable in precision and speed is selected as a final result.
The invention provides a transmission line foreign matter invasion identification method facing a dynamic characteristic supplement mechanism, which belongs to the technical field of transmission line identification and comprises the steps of utilizing a monitoring model to monitor a video image shot and processed by an unmanned aerial vehicle for foreign matters, extracting a characteristic model of a transmission line key part and a foreign matter phenomenon thereof and distinguishing the characteristic model by a classifier, thereby realizing intelligent identification and diagnosis and early warning of phenomena of irregular icing of an insulator or a lead of a transmission line, external force damage, growth contact of trees under the line, galloping adhesion of the lead, insulator string defect, strand breaking of the lead, surface defect of the lead and the like under a complex environment background. The main research contents comprise the design of a regional candidate network combined with the prior scale information of the power transmission component, the design of a classifier aiming at multi-attitude small samples, the design of a fine-grained foreign matter identification network, data augmentation and data set construction. The image inspection of the power transmission line can effectively detect foreign matters, timely eliminate hidden dangers, avoid the foreign matters from damaging the normal operation of the power transmission line, and better overcome the problems of high cost, easy missed inspection, low efficiency and the like of the traditional manual inspection mode.
According to the invention, the monitoring model is adopted to replace manual identification of the foreign matters, so that the problem of missing identification can be better avoided, and the labor cost is also reduced. In addition, the processing speed of the computer is benefited, a large number of power transmission line images can be rapidly and accurately identified by using the monitoring model, and the foreign matter monitoring efficiency is improved.
The described embodiments of the invention are only some, but not all embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the examples of the present invention without any inventive step, are within the scope of the present invention.
Claims (7)
1. A transmission line foreign matter intrusion identification method facing a dynamic characteristic supplement mechanism is characterized by comprising the following steps:
transmitting a video image around the power transmission line acquired by the unmanned aerial vehicle to a power supply management platform through a wireless communication interface, and preprocessing the received video image by the power supply management platform by adopting a Gaussian filter algorithm to obtain a processed video image;
inputting the processed video image into an image detection model with multi-layer feature fusion, and identifying and positioning the foreign matters to obtain a video frame sequence with a foreign matter mark frame;
establishing different foreign matter type data sets of the power transmission line, wherein the data sets comprise a training data set, a test data set and a verification data set;
extracting foreign body type information by matrix capsule network classifier
Carrying out foreign matter classification on the video frame sequence with the foreign matter mark frame through a matrix capsule network classifier to obtain a classified foreign matter image set, and carrying out transformation processing to generate a new foreign matter image;
expanding a training data set by using a method for generating an antagonistic network, and training a foreign body detection model facing a dynamic characteristic supplementation mechanism by using the expanded training data set;
judging the foreign matter type information by using a pre-trained foreign matter detection model facing a dynamic feature supplement mechanism to determine the foreign matter type, and judging to determine an early warning grade according to the foreign matter type;
after the early warning level is output, the system prompts the influence of foreign matters on the power transmission line on the line, and meanwhile, the state of the foreign matters in the video is marked.
2. The method for identifying the invasion of the foreign matters into the power transmission line facing the dynamic feature supplement mechanism according to claim 1, wherein the preprocessing is performed by a gaussian filtering algorithm, the whole image is weighted and averaged, and the value of each pixel point is obtained by weighting and averaging the value of each pixel point and other pixel values in the neighborhood, and the specific method is as follows:
scanning each pixel in the obtained video frame image by using a Gaussian mask template with the size of 3 multiplied by 3, and replacing the value of the central pixel point of the Gaussian mask template by using the weighted average gray value of the pixels in the neighborhood determined by the Gaussian mask template.
3. The method for identifying the intrusion of the foreign object into the power transmission line facing the dynamic feature supplement mechanism according to claim 1, wherein the step of inputting the video frame sequence into the image detection model with multi-layer feature fusion comprises the specific steps of:
performing behavior time sequence feature extraction on foreign matters in a complex scene by using an image detection model by adopting a feature extraction method based on target detection and tracking;
performing targeted training on the image detection model with multi-layer feature fusion by using a self-built data set;
processing and analyzing the extracted foreign body behavior time sequence characteristics to obtain a video frame sequence with a foreign body mark frame;
the multi-layer feature fusion image detection model is used for dynamically optimizing the expression capability of a feature graph by fusing different levels of feature graphs to aggregate context information and generating output weights of the feature graphs of each level in a self-adaptive mode according to the size of a training target sample in consideration of different contributions of information contained in the feature graphs of different levels to a small target detection task.
4. The method for identifying the intrusion of the foreign object into the power transmission line facing the dynamic feature supplement mechanism according to claim 1, wherein the specific method for establishing the data sets of different foreign object types of the power transmission line is as follows:
the data sets of different foreign body types comprise data sets of a plurality of foreign body samples of different types, and the data sets are divided into a training data set, a testing data set and a verification data set according to a certain proportion; the training data set is used for training for the dynamic characteristic supplementary model, the testing data set is used for verifying the generalization ability of the model after the model training is completed, and the verifying data set is used for verifying whether the trained model has the overfitting phenomenon.
5. The method for identifying the intrusion of the foreign object into the power transmission line facing the dynamic feature supplement mechanism according to claim 1, wherein the specific method for expanding the training data set by the method for generating the countermeasure network comprises:
generating a new image by converting the image with the foreign matter mark frame through horizontal overturning, scaling and translation by the countermeasure network;
and splicing the foreign body part into the foreign body-free image in an image splicing mode.
6. The method for identifying the intrusion of the foreign object into the power transmission line facing the dynamic feature supplement mechanism according to claim 1, wherein a pre-trained foreign object detection model facing the dynamic feature supplement mechanism is used to judge the type information of the foreign object, and the specific method for determining the type of the foreign object is as follows:
because the power transmission line belongs to a small-probability event under daily conditions, if the tracking data of each foreign object in a complex scene is detected, a large amount of resources are consumed, and on the basis of ensuring the precision, the video frame sequence with the foreign object marking frame is firstly analyzed;
then, a plurality of detection threshold values are set, and all foreign matter types in the video are classified according to the detection threshold values, wherein the types include foreign matters, foreign matter-like matters and no foreign matters.
7. The method for identifying the intrusion of the foreign object into the power transmission line facing the dynamic feature supplement mechanism according to claim 6, wherein the input of the foreign object detection model facing the dynamic feature supplement mechanism at the current moment is not only dependent on the previous video frame but also dependent on the next video frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211418023.5A CN115761537A (en) | 2022-11-14 | 2022-11-14 | Power transmission line foreign matter intrusion identification method oriented to dynamic characteristic supplement mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211418023.5A CN115761537A (en) | 2022-11-14 | 2022-11-14 | Power transmission line foreign matter intrusion identification method oriented to dynamic characteristic supplement mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115761537A true CN115761537A (en) | 2023-03-07 |
Family
ID=85370680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211418023.5A Pending CN115761537A (en) | 2022-11-14 | 2022-11-14 | Power transmission line foreign matter intrusion identification method oriented to dynamic characteristic supplement mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115761537A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116486313A (en) * | 2023-06-25 | 2023-07-25 | 安元科技股份有限公司 | Video analysis system and method suitable for scenes |
| CN116563680A (en) * | 2023-05-05 | 2023-08-08 | 北京卫星信息工程研究所 | Remote sensing image feature fusion method based on Gaussian mixture model and electronic equipment |
| CN117275156A (en) * | 2023-09-13 | 2023-12-22 | 武汉卓讯互动信息科技有限公司 | Unattended chess and card room reservation sharing system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112464851A (en) * | 2020-12-08 | 2021-03-09 | 国网陕西省电力公司电力科学研究院 | Smart power grid foreign matter intrusion detection method and system based on visual perception |
| CN112906620A (en) * | 2021-03-09 | 2021-06-04 | 唐山职业技术学院 | Unmanned aerial vehicle-assisted insulator fault detection method and device and electronic equipment |
| CN113076899A (en) * | 2021-04-12 | 2021-07-06 | 华南理工大学 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
| CN114332744A (en) * | 2022-03-10 | 2022-04-12 | 成都诺比侃科技有限公司 | Transformer substation self-adaptive security method and system based on machine vision |
-
2022
- 2022-11-14 CN CN202211418023.5A patent/CN115761537A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112464851A (en) * | 2020-12-08 | 2021-03-09 | 国网陕西省电力公司电力科学研究院 | Smart power grid foreign matter intrusion detection method and system based on visual perception |
| CN112906620A (en) * | 2021-03-09 | 2021-06-04 | 唐山职业技术学院 | Unmanned aerial vehicle-assisted insulator fault detection method and device and electronic equipment |
| CN113076899A (en) * | 2021-04-12 | 2021-07-06 | 华南理工大学 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
| CN114332744A (en) * | 2022-03-10 | 2022-04-12 | 成都诺比侃科技有限公司 | Transformer substation self-adaptive security method and system based on machine vision |
Non-Patent Citations (2)
| Title |
|---|
| 周仿荣;方明;马御棠;潘浩;: "基于YOLO v3的输电线路缺陷快速检测方法", 云南电力技术, no. 04, 15 August 2020 (2020-08-15), pages 116 - 120 * |
| 赵天宇: "基于YOLO-V4的输电线路多故障检测与跟踪", 中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑, 15 August 2022 (2022-08-15), pages 1 - 76 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116563680A (en) * | 2023-05-05 | 2023-08-08 | 北京卫星信息工程研究所 | Remote sensing image feature fusion method based on Gaussian mixture model and electronic equipment |
| CN116563680B (en) * | 2023-05-05 | 2024-02-06 | 北京卫星信息工程研究所 | Remote sensing image feature fusion method based on Gaussian mixture model and electronic equipment |
| CN116486313A (en) * | 2023-06-25 | 2023-07-25 | 安元科技股份有限公司 | Video analysis system and method suitable for scenes |
| CN116486313B (en) * | 2023-06-25 | 2023-08-29 | 安元科技股份有限公司 | Video analysis system and method suitable for scenes |
| CN117275156A (en) * | 2023-09-13 | 2023-12-22 | 武汉卓讯互动信息科技有限公司 | Unattended chess and card room reservation sharing system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115761537A (en) | Power transmission line foreign matter intrusion identification method oriented to dynamic characteristic supplement mechanism | |
| CN106816038A (en) | A kind of inland waters abnormal behaviour ship automatic identification system and method | |
| CN109344753A (en) | A kind of tiny fitting recognition methods of Aerial Images transmission line of electricity based on deep learning | |
| CN112101088B (en) | Unmanned aerial vehicle electric power automatic inspection method, device and system | |
| CN110569843B (en) | Intelligent detection and identification method for mine target | |
| CN110084165A (en) | The intelligent recognition and method for early warning of anomalous event under the open scene of power domain based on edge calculations | |
| CN112686833A (en) | Industrial product surface defect detecting and classifying device based on convolutional neural network | |
| CN112950634A (en) | Method, equipment and system for identifying damage of wind turbine blade based on unmanned aerial vehicle routing inspection | |
| CN111539325A (en) | Forest fire detection method based on deep learning | |
| CN110929646A (en) | Power distribution tower reverse-off information rapid identification method based on unmanned aerial vehicle aerial image | |
| CN114399719B (en) | Transformer substation fire video monitoring method | |
| CN108664875A (en) | Underground belt-conveying monitoring method based on image recognition | |
| CN113469050A (en) | Flame detection method based on image subdivision classification | |
| CN111582069B (en) | Track obstacle zero sample classification method and device for air-based monitoring platform | |
| CN114359695A (en) | Insulator breakage identification method based on uncertainty estimation | |
| CN116846059A (en) | Edge detection system for power grid inspection and monitoring | |
| Yandouzi et al. | Investigation of combining deep learning object recognition with drones for forest fire detection and monitoring | |
| CN111860117A (en) | Human behavior recognition method based on deep learning | |
| CN115620178A (en) | Real-time detection method for abnormal and dangerous behaviors of power grid of unmanned aerial vehicle | |
| CN111667655A (en) | Infrared image-based high-speed railway safety area intrusion alarm device and method | |
| CN112907138A (en) | Power grid scene early warning classification method and system from local perception to overall perception | |
| CN117423157A (en) | Mine abnormal video action understanding method combining migration learning and regional invasion | |
| CN116704688A (en) | Ocean buoy passive defense method and system | |
| CN115083229B (en) | Intelligent recognition and warning system of flight training equipment based on AI visual recognition | |
| CN114998783A (en) | Front-end equipment for video analysis of smoke, fire and personnel behaviors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |