CN106595551B - Ice covering thickness detection method in powerline ice-covering image based on deep learning - Google Patents

Abstract

Ice covering thickness detection method in the invention discloses a kind of powerline ice-covering image based on deep learning, belong to digital image understanding field, it aims to overcome that and existing answers pulling force monitoring medium sensitivity and the not high problem of reliability, improve the accuracy and the degree of automation of electric power line ice-covering thickness monitoring, ice covering thickness monitoring and transfinite alarm of the present invention for power system transmission line, comprising: (1) collect icing image；(2) pretreatment image and data set is established；(3) convolutional neural networks are established；(4) training and test model；(5) ice covering thickness information is extracted and five steps such as be transmitted back to control centre；Digital picture characteristic recognition method is introduced into the ice covering thickness detection of power transmission line and shaft tower by the present invention, thickness information is automatically extracted using the morphological feature of icing in image, deicing plan is formulated for operation maintenance personnel, to guarantee that safe and stable operation of power system provides a kind of new intuitive and intelligentized means.

Description

Detection method of ice coating thickness in transmission line icing images based on deep learning

technical field

The invention belongs to the technical field of digital image recognition, and in particular relates to a method for detecting the thickness of ice coating in an image based on a deep learning algorithm, which can be used for ice coating monitoring and over-limit warning of power transmission network equipment in a power system.

Background technique

The importance of the safe and stable operation of the power grid to the development of the national economy is self-evident. With the continuous deepening of the interconnection of the power grid and the gradual implementation of the power market, the operating environment of the power grid has become more complex, which puts forward higher requirements for the stability and reliability of the power grid. Require. my country has a vast territory, diverse climates, and complex terrain. The power grids across the country are often damaged by various natural disasters. Most of the country often causes large-scale power outages due to extreme low temperature and icing. Ice-covered disasters can lead to mechanical and electrical failures in the power grid, such as substation shutdowns, tower collapses, ice flash trips, line galloping, and substation equipment damage.

Ice covering causes major equipment losses to my country's power grid and leads to large-scale power outages. In addition, ice-covered areas often suffer from harsh climatic conditions, interruption of traffic and communication, and difficulty in emergency repairs, resulting in large-scale power outages and seriously affecting the reliability of power supply. The icing of transmission lines and power equipment exists objectively and cannot be fundamentally eliminated. In order to reduce the disaster caused by icing, it is necessary to protect the electrical equipment in the power grid from icing, and eliminate the hidden danger of icing in time.

At present, there are two main methods of icing protection: monitoring and suppression. The monitoring method is to install sensors and cameras on electrical equipment to monitor icing status online, or to inspect key lines manually to find hidden faults. The icing monitoring system has been installed in the disaster-affected areas to monitor the icing status of key lines and nodes of the power grid. However, the frequent occurrence of ice disasters in recent years has proved that the current monitoring methods cannot meet the requirements of safe and stable operation of the power system. , Taking 2014 as an example, there were 597 trips due to ice coating, the trip rate was 0.103 times/100 km·year, and the combined power was 46.4%, an increase of 376 times compared with 221 times in 2013, an increase of 170.1% . In 2014, the number of non-stop faults caused by icing was 320 times, and the non-stop rate of faults was 0.055 times/100 km·year. The reasons why the current icing monitoring is difficult to meet the power grid operation requirements can be summarized as follows:

(1) The sensor monitoring ice coating is greatly affected by the working environment, and the harsh weather conditions or electromagnetic field interference will reduce the measurement accuracy of the sensor;

(2) The camera monitoring judges the current icing status by shooting the icing images of the towers or lines at the monitoring points, but due to the lack of effective processing and utilization methods for icing images, reliable icing information cannot be obtained from it;

(3) Manual inspection methods or helicopter inspections are costly and inefficient, making it difficult to monitor the entire network;

(4) The system cannot detect the hidden danger of icing in time, resulting in the inability to issue icing warning in time. In addition to the relative lag of deicing operation, icing cannot be eliminated in time.

SUMMARY OF THE INVENTION

In order to solve the above technical problems, the present invention provides a method for identifying ice-covered images of transmission lines based on a deep learning algorithm, which can automatically identify the ice-covered thickness according to the inputted ice-covered images to ensure timely line ice-covered conditions in the power sector.

The technical solution adopted in the present invention is: a method for detecting ice thickness in a transmission line icing image based on deep learning, which is characterized by comprising the following steps:

Step 1: Obtain icing image data and corresponding stress and tension monitoring data;

Step 2: Preprocess the original ice-covered image, process the size of the original ice-covered image into an image with the same size, and use the ice-covered thickness measured by the tension sensor as an image label;

Step 3: Establish a deep learning convolutional neural network model, establish corresponding model parameters for the number and size of images, and set the number of units and activation functions of each layer of the network;

Step 4: Adjust the weights and train the model, extract and combine the features of the images, judge and output the thickness of the ice coating;

Step 5: Analyze the model training classification results, and extract the ice thickness information of the ice-covered image.

There are deficiencies in the real-time and effectiveness of power icing monitoring in my country, as long as the reason is to improve the monitoring effect of the sensor, it is mainly improved from the hardware aspect; the present invention starts from the icing image data, and studies a more rapid and accurate method for monitoring the icing state , eliminate hidden troubles in time, improve the reliability of the power grid icing monitoring system, and ensure the safe and stable operation of the power system.

The invention introduces the digital image feature recognition method into the ice thickness detection of power transmission lines and towers, uses the morphological features of the ice in the image to automatically extract the thickness information, formulates a deicing plan for the operation and maintenance personnel, and ensures the safe and stable operation of the power system. Provides a new intuitive and intelligent means.

Instruction drawings

1 is a schematic flowchart of an embodiment of the present invention;

Fig. 2 is the original icing image of the embodiment of the present invention;

Fig. 3 is the image segmentation result of the iterative method of the embodiment of the present invention;

Fig. 4 is the LoG operator edge detection result of the automatic threshold of the embodiment of the present invention;

Fig. 5 is the Prewitt operator edge detection result of the automatic threshold of the embodiment of the present invention;

Fig. 6 is the Sobel operator edge detection result of the automatic threshold of the embodiment of the present invention;

7 is a schematic diagram of feature extraction of a convolutional neural network according to an embodiment of the present invention;

8 is a schematic diagram of a convolution process according to an embodiment of the present invention;

9 is a schematic diagram of a pooling process according to an embodiment of the present invention;

10 is a comparison of the 1-time recognition accuracy of the traversal data of the convolutional neural network of each structure according to the embodiment of the present invention;

11 is a comparison of the recognition accuracy of the convolutional neural network traversing data 5 times of each structure according to the embodiment of the present invention;

12 is a comparison of 10 recognition accuracy of traversing data by a convolutional neural network of each structure according to an embodiment of the present invention;

13 is a comparison of 15 recognition accuracy of traversing data by a convolutional neural network of each structure according to an embodiment of the present invention;

14 is a comparison of 20 recognition accuracy of traversing data by a convolutional neural network of each structure according to an embodiment of the present invention;

FIG. 15 is a comparison of recognition errors of different models of a convolutional neural network according to an embodiment of the present invention.

Detailed ways

In order to facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention, but not to limit it. this invention.

Referring to Fig. 1 , a method for detecting the thickness of ice coating in a transmission line icing image based on deep learning provided by the present invention includes the following steps:

Step 1: Collect icing image data and corresponding strain monitoring data from the icing monitoring system of the power grid department;

The collected icing images include icing images of power lines, towers, fittings, electrical equipment such as transformers, etc. The collected icing images should be as clear as possible; the collected stress and tension data include the tower model, location, and equivalent values measured by the stress and tension sensor. Ice thickness.

Step 2: Preprocess the original image, adjust the image size to the same, and establish an ice-covered image data set;

Preprocess the original image, including image segmentation and edge extraction;

Image segmentation is the technology and process of dividing an image into regions with different characteristics and extracting objects of interest;

Threshold segmentation is a commonly used method in image segmentation. All pixels with grayscale greater than or equal to the threshold value are determined to belong to the object, and the grayscale value is "255" to indicate the foreground, otherwise these pixels are excluded from the object area. Threshold segmentation includes bimodal and iterative methods;

The bimodal method divides the image into two parts: foreground and background. The gray distribution curve of the image is approximately considered to be composed of two normal distribution functions. and When superimposed, the histogram of the image will have two separate peaks, and the trough between the two peaks is the threshold of the image;

The iterative method is an improvement to the bimodal method. First, an approximate threshold T is selected, the image is divided into parts R ₁ and R ₂ , the mean values μ ₁ and μ ₂ of the regions R ₁ and R ₂ are calculated, and a new segmentation threshold is selected. T=(μ ₁ +μ ₂ )/2, repeat the above steps until μ ₁ and μ ₂ no longer change, the present invention uses an iterative method to perform image segmentation processing on the image;

Edge extraction first detects the edge points in the image, and then connects the edge points into a contour to form a segmentation area. Since the edge is the dividing line between the target and the background to be extracted, the target and the background can be separated only by extracting the edge. The gradient modulus operator has The properties of displacement invariance and isotropy are suitable for edge detection, and the direction of grayscale change, that is, the direction of the boundary, can be determined by θ _g = arctan(f _y /f _x ), where f _x and f _y respectively is the direction modulo of x and y, and θ _g is the direction of the maximum gradient value of continuous image edge detection.

Step 3: Establish a deep learning convolutional neural network model, establish appropriate model parameters for the number and size of pictures, and set the number of units and activation functions of each layer of the network;

The establishment of a deep learning convolutional neural network is a deep learning model based on BP neural network transformation. Its weight sharing network structure can reduce the complexity of the network model and reduce the number of weights. The advantage is that the input of the network is The multi-dimensional image is more obvious, so that the image can be directly used as the input of the network, avoiding the complex feature extraction and data reconstruction process in the traditional recognition algorithm;

Deconvolve the input of this layer with a convolution kernel in a convolutional layer. First, convolve the data at the same position of each output feature map of the previous layer with the convolution kernel of this layer, and then add all the results of the convolution at the same position to obtain the output of the corresponding position of the output feature map of this layer. In order to reduce the number of parameters and reduce the difficulty of model training, deep learning adopts a weight sharing mechanism. The same output feature map uses the same convolution kernel, and each convolution kernel has its corresponding filter corresponding to it. One convolution kernel only extracts one feature to ensure that feature extraction does not occur aliasing;

After the features are obtained by convolution, all the extracted features are used to train the classifier. The present invention uses the Softmax classifier. Each feature and image convolution will obtain an n-dimensional convolution feature. Since there are n features, Learning a classifier with more than 20 million feature inputs is inconvenient and prone to overfitting;

Step 4: Adjust the weights and train the model, perform feature extraction and extraction on the image, judge and output the thickness of the ice coating, and compare the thickness information of the image label;

The established convolutional neural network is trained and tested, and a model that meets the accuracy requirements is obtained to detect the thickness of ice coating in the picture. The training process includes the following steps:

(1) Sensitivity and error correction;

For the C classification problem, there are N training samples in total, and the mean square error of the model can be expressed as:

in represents the expected output of the kth class of the nth sample, represents the actual output of the kth class for the nth sample.

For the nth sample, the mean square error between the actual output and the ideal output can be expressed as:

Suppose L is the output layer, l is the hidden layer, and 1 is the input layer; the activation output of the lth layer is: x ^l =f(u ^l ), where u ^l =W ^l x ^l-1 +b ^l , f( ) is the activation function, W ^l is the weight of the l-th layer, b ^l is the bias of the l-th layer; the sensitivity is defined as:

in Therefore, the sensitivities of the lth layer and the output layer are respectively expressed as:

The resulting error correction (η is the learning rate) can be expressed as:

(2) Forward propagation

1) Convolutional layer

Assuming that the lth layer is a convolutional layer, the feature map and feature map size output by this layer are respectively expressed as:

output.size=input.size-kernel.size+1

in, is the ith output of layer l-1, is the jth convolution kernel of the lth layer for the ith input, is the jth bias of the lth layer, f( ) is the activation function, is the jth output of the lth layer.

The feature extraction of convolutional neural network has two characteristics:

I. Through convolution, a pixel of the output feature map is used to represent the pixel feature of the local area of the input feature map, which is the feature extraction of the convolutional neural network, and also reduces the data dimension;

II. Weight sharing, the same feature map uses the same convolution kernel to extract a feature, which can reduce the number of parameters and reduce the time complexity.

2) Subsampling layer

The subsampling layer can be regarded as a pooling process and a dimensionality reduction process, so that the input feature map is re-represented on the output feature map without overlapping, that is, the combination of features; in addition, pooling can also reduce the data dimension and speed up the calculation. The expression of the output graph can be expressed as:

where down(x) is the sampling operation on the n×n pixel area of the input image, is the control factor, which controls the value of the sampling result within the range of the color pixel value while reducing noise interference.

(3) Backpropagation

1) Convolutional layer

The sensitivity of the convolutional layer can be expressed as the following formula. Considering the subsampling layers before and after the convolutional layer, it can be rewritten as:

where β is the weight,

instead of ∑δ ^l+1 .

The gradient of the basis and the convolution kernel can be expressed as:

Among them, u and v are the parts corresponding to the image of the previous layer to be deconvolved with the convolution kernel. in, is the sensitivity of the jth convolution kernel of the lth layer, is the momentum factor of layer l-1;

2) Subsampling layer

In the forward pass has been saved: So the weight gradient is:

in, is the downsampling operation function of the lth layer.

(4) Feature combination

An important feature of the deep learning network is the automatic learning of features. When the representation in the training process is the combination of learning feature maps, weights are assigned to each extracted feature, and forward propagation and back propagation are repeated to correct errors and adjust. Weights, to achieve the purpose of feature optimization combination. The feature weight is represented by α _ij , which represents the weight or contribution of the i-th input feature map in the j-th output feature map, which is usually expressed by the following formula:

Among them, c _ij represents the weight of the i-th input feature map in the j-th output feature map, and Σ _k exp(c _kj ) represents the sum of the weights of the j-th output feature map.

Then the jth feature output can be rewritten as:

where f( ) is the activation function, α _ij represents the weight of the output feature map, is the layer input, is the convolution kernel, is the layer bias.

The above formula satisfies: _0≤αij≤1 ;

For a single output unit, the index j is ignored, since:

and

After obtaining the base gradient, convolution kernel gradient, connection weight gradient and feature weight gradient of the hidden layer, the error correction is shown in Table 1 (η is the learning rate).

Table 1 Convolutional Neural Network Correction Values

where η is the learning rate, is the gradient of the basis, is the gradient of the convolution kernel, is the connection gradient, and α _i is the feature weight.

Step 5: Analyze the model training classification results, extract the ice thickness information of the icing image, and send it back to the icing monitoring center.

Step 1 of this embodiment collects the original icing image data and stress sensor data of the power grid. Figure 2 shows the icing image of the transmission line. The icing image requires that the part of the icing shot is clear and not covered by turbidity. The tensile force data includes the type and location of the tower and the ice thickness measured by the tensile force sensor, so as to find the fault location in time;

Step 2 of this embodiment preprocesses the ice-covered image through image segmentation and edge extraction, sets the size of the original image to be the same size, and uses the ice-covered thickness measured by the tension sensor as the image label; image segmentation and edge processing are performed on the image, In order to remove noise and improve the efficiency of feature extraction, image segmentation and edge extraction are to eliminate the interference of the environment and irrelevant objects on the extraction of ice-covered features. Due to the complex surrounding environment of high-voltage transmission lines, there are often interference from environmental factors such as trees and insects, which need to be used. Image segmentation technology eliminates irrelevant factors in the ice-covered image to avoid interference with feature extraction; Figure 3 is the effect of iterative image segmentation, Figure 4-6 is the automatic threshold LoG operator edge detection, Prewitt operator edge detection and Sobel operator edge detection results;

Step 3 of the present embodiment establishes a convolutional neural network, and the convolutional neural network utilizes the spatial relationship to reduce the number of parameters to be learned to improve the training performance of the forward and backward propagation algorithms, and accompanying drawing 7 is a brief description of the model structure;

Convolutional neural network includes input layer, hidden layer and output layer. The internal hidden layer of convolutional neural network is iteratively constructed by convolutional layer and pooling layer. The convolutional layer can extract data features, also called feature extraction layer. is a convolution, and FIG. 8 is a schematic diagram of the convolution process. The sub-sampling layer is also called a feature mapping layer. The feature map is obtained by the weighted sum of the pixels and the activation function. FIG. 9 is a schematic diagram of the pooling process;

In the convolutional neural network, a part of the image (local receptive area) is used as the input of the lowest layer of the hierarchical structure, and the information is transmitted to different layers in turn. Each layer passes a digital filter to obtain the most significant features of the observation data. Therefore, it is possible to obtain salient features of observation data that are invariant to translation, scaling and rotation,

In step 3, the model parameters are defined and each parameter is initialized, and the activation function is set to "sigmoid";

Step 4 of this embodiment is to train the established convolutional neural network, and relevant parameters such as the model structure, the number of training steps, the batch size, the activation function, the number of neurons in each layer, and the number of neurons can be adjusted according to the training effect;

Due to the large amount of model parameters, the present invention uses 15235 icing pictures for training. According to the monitoring mechanism of the icing state by the power grid, the model monitoring thickness is judged according to the model into no icing (0cm), slight icing (0-5cm), Moderate icing (5-10cm), severe icing (10-15cm), dangerous icing (15-20cm) and fault warning icing (above 20cm), the output is compared with the previous image to judge and adjust error.

Step 5 of the present embodiment analyzes the model training classification results, extracts the ice thickness information of the icing image and sends it back to the icing monitoring center;

The present invention builds four models of the convolutional neural network, which are (1) 6-4-12-2, (2) 12-4-24-2, (3) 4-2-8-2-16- 2-32-2, (4) 16-2-8-2-4-2-2-2. The odd-numbered layers are convolutional layers, and the even-numbered layers are pooling layers. When the four models traverse the data for different times, the mean square error of each model changes. The mean square error of the model measures the training effect of the model. When the mean square error of the model is smaller, it means that the training effect is better, the parameter setting is more reasonable, and the obtained output is closer to the ideal result. The number of steps represents the number of times to adjust the weights. In the deep learning algorithm, the training samples are usually trained in batches, and the weights are adjusted once after training a batch of samples to improve the training speed and training effect;

Figure 10 shows four model structures: (1) 6-4-12-2, (2) 12-4-24-2, (3) 4-2-8-2-16-2-32-2, (4) 16-2-8-2-4-2-2-2, traverse the data once respectively, the change of the mean square error of the training process, the abscissa indicates the number of adjustment parameters, and the ordinate indicates the mean square error; the curve trend can be Reflect the training effect of the model, and adjust the model parameters according to the training effect to achieve the best training effect. Similarly, Figures 11-14 show the changes of the mean square error when the four model structures traverse the data 5, 10, 15 and 20 times respectively. Generally, the more traversal times, the better the classification performance of the model.

Figure 15 is for 4 model structures: (1) 6-4-12-2, (2) 12-4-24-2, (3) 4-2-8-2-16-2-32-2 , (4) Comparative analysis of the classification results of 16-2-8-2-4-2-2-2, the abscissa is the number of times of traversing the data, and the ordinate is the classification error.

It should be understood that the parts not described in detail in this specification belong to the prior art.

It should be understood that the above description of the preferred embodiments is relatively detailed, and therefore should not be considered as a limitation on the protection scope of the patent of the present invention. In the case of the protection scope, substitutions or deformations can also be made, which all fall within the protection scope of the present invention, and the claimed protection scope of the present invention shall be subject to the appended claims.

Claims (8)

1. a method for detecting the thickness of ice coating in a transmission line icing image based on deep learning, is characterized in that, comprises the following steps: Step 1: Obtain icing image data and corresponding stress and tension monitoring data; Step 2: Preprocess the original ice-covered image, process the size of the original ice-covered image into an image with the same size, and use the ice-covered thickness measured by the tension sensor as an image label; Step 3: Establish a deep learning convolutional neural network model, establish corresponding model parameters for the number and size of images, and set the number of units and activation functions of each layer of the network; Step 4: Adjust the weights and train the model, extract and combine the features of the images, judge and output the thickness of the ice coating; The training and testing of the established convolutional neural network includes the following sub-steps: Step 4.1: Calculate sensitivity and error correction; In step 4.1, for the C classification problem, there are a total of N training samples, represents the expected output of the kth class of the nth sample, represents the actual output of the kth class of the nth sample, then the output layer error of the model is expressed as the mean square error between the ideal output and the actual output: For the nth sample, the mean square error between the actual output and the ideal output can be expressed as: Suppose L is the output layer, l is the hidden layer, and 1 is the input layer; the activation output of the lth layer is: x ^l =f(u ^l ), where u ^l =W ^l x ^l-1 +b ^l , f( ) is the activation function, W ^l is the weight of the l-th layer, b ^l is the bias of the l-th layer; the sensitivity is defined as: in Therefore, the sensitivities of the lth layer and the output layer are respectively expressed as: This leads to the error correction as: where η is the learning rate; Step 4.2: Forward propagation; Step 4.3: Backpropagation; Step 4.4: Feature combination; assign a weight to each extracted feature, repeat forward propagation and back propagation to correct the error and adjust the weight to achieve the purpose of optimizing the combination of features; Step 5: Analyze the model training classification results, and extract the ice thickness information of the ice-covered image. 2 . The method for detecting the thickness of ice coating in the deep learning transmission line ice coating image according to claim 1 , wherein the ice coating image in step 1 includes a power transmission line ice coating image, a tower ice coating image, The icing image of metal fittings and the icing image of electrical equipment; the stress and tension monitoring data includes the type, location, and thickness of the icing. 3. The method for detecting icing thickness in a deep learning transmission line icing image according to claim 1, characterized in that: said in step 2, the original icing image is preprocessed, including image segmentation and edge extraction; The image segmentation is to perform image segmentation processing on the image using an iterative method, and the gradient modulo operator in the edge extraction is expressed in the form of a differential operator and implemented by a fast convolution function. 4. The ice coating thickness detection method in the transmission line icing image of deep learning according to claim 1, is characterized in that: the establishment of deep learning convolutional neural network described in step 3 is based on a BP neural network transformation. A deep learning model, in the convolution layer, the input of the layer is deconvolved with a convolution kernel. First, the data at the same position of each output feature map of the previous layer is convolved with the convolution kernel of this layer, and then the convolution kernel of the layer is convolved. Add all the results of convolution at the same position to obtain the output of the corresponding position of the output feature map of this layer. 5. The method for detecting icing thickness in deep learning transmission line icing images according to claim 4, characterized in that: in order to reduce the number of parameters and reduce the difficulty of model training, a weight sharing mechanism is adopted, and the same output feature map is used. Using the same convolution kernel, each convolution kernel has its own corresponding filter, and a convolution kernel only extracts one kind of feature to ensure that no aliasing occurs in feature extraction. 6. The method for detecting the thickness of ice coating in the transmission line ice coating image of deep learning according to claim 1, it is characterized in that: in step 4.2, for the convolutional layer, assuming that the first layer is a convolutional layer, then this layer The output feature map and feature map size are: output.size=input.size-kernel.size+1 in, is the ith output of layer l-1, is the jth convolution kernel of the lth layer for the ith input, is the jth bias of the lth layer, f( ) is the activation function, is the jth output of the lth layer; For the subsampling layer, the output graph is: Among them, down(x) is the sampling operation for the n×n pixel area of the input image, is the control factor, which controls the value of the sampling result within the range of the color pixel value while reducing noise interference. 7. The method for detecting ice thickness in deep learning transmission line ice images according to claim 6, wherein in step 4.3, the sensitivity of the convolution layer can be expressed as: where β is the weight, instead of ∑δ ^l+1 ; The gradient of the basis and the convolution kernel can be expressed as: Among them, u and v are the parts corresponding to the image of the previous layer to be deconvolved with the convolution kernel; For subsampling layers, the forward pass has saved So the weight gradient is: 8 . The method for detecting the thickness of ice coating in a transmission line icing image based on deep learning according to claim 7 , wherein: in step 4.4, the feature weight is α _ij , indicating that the jth output feature map of the i input feature map weights or contributions; Then the jth feature output can be rewritten as: The above formula satisfies: For a single output unit, the index j is ignored, since: and After obtaining the base gradient, convolution kernel gradient, connection weight gradient and feature weight gradient of the hidden layer, the errors are respectively corrected as: the corrected value of the base is The correction value of the convolution kernel is The corrected value of the connection weight is The corrected value of the feature weight is where η is the learning rate, is the gradient of the basis, is the gradient of the convolution kernel, is the connection gradient, and α _i is the feature weight.