CN110020598B

CN110020598B - Method and device for detecting foreign matters on telegraph pole based on deep learning

Info

Publication number: CN110020598B
Application number: CN201910150713.9A
Authority: CN
Inventors: 林兴萍; 舒元昊; 刘庆杰; 叶晶晶
Original assignee: CETHIK Group Ltd
Current assignee: CETHIK Group Ltd
Priority date: 2019-02-28
Filing date: 2019-02-28
Publication date: 2022-04-15
Anticipated expiration: 2039-02-28
Also published as: CN110020598A

Abstract

The invention discloses a method and a device for detecting foreign matters on a telegraph pole based on deep learning, wherein a preset number of pictures containing the telegraph pole and the foreign matters are selected, the pictures are expanded to obtain a picture library with preset multiples, and the position information of the telegraph pole and the foreign matters in each picture is marked to serve as a sample library; taking a feature map extracted from the 6 th convolutional layer of the SSD object detection algorithm as a 7 th grid, combining the 7 th grid with the original 6 th grid, and taking the combined grid and the original 6 th grid as input of a final loss function to train to obtain a detection model of the telegraph pole and the foreign matter; finally, detecting the telegraph pole and the foreign matters in the input picture by using the detection model obtained by training; and calculating a first ratio of the overlapping area of the telegraph pole and the foreign matter to the area of the foreign matter, and eliminating the foreign matter with the first ratio lower than a preset threshold value, so that the detection fault tolerance rate is improved. The method has the advantages of simplifying the process and improving the accuracy of the model to the foreign matters of small objects.

Description

Method and device for detecting foreign matters on telegraph pole based on deep learning

Technical Field

The invention belongs to the field of machine vision, and particularly relates to a method and a device for detecting foreign matters on a telegraph pole based on deep learning.

Background

Foreign objects (such as bird nests, balloons, kites, etc.) on utility poles can have a great influence on the transmission lines, for example, causing short-circuit of the lines, resulting in a large area of power supply being damaged. Therefore, foreign matters on the telegraph pole need to be detected and removed. The traditional detection method needs the field operation of workers, consumes a large amount of manpower and material resources, and has a very limited monitoring range.

Detecting the position of the target object in the image by using the deep learning method is a popular technical means. Through the image data of outdoor collection, combine the ability of machine degree of depth study and discernment, foreign matter on the wire pole in the automatic discernment detection image can help the staff to reduce the work load, reduces manpower and materials cost.

Although the prior art provides some technical schemes for detecting the track foreign matter based on deep learning, the accuracy of the existing deep learning for detecting objects with smaller scales is lower, and the accuracy of the foreign matter detection is not high.

Disclosure of Invention

The invention aims to provide a method and a device for detecting foreign matters on a telegraph pole based on deep learning, which can improve the detection accuracy and the fault tolerance rate.

In order to achieve the purpose, the technical scheme of the invention is as follows:

the invention discloses a method for detecting foreign matters on a telegraph pole based on deep learning, which comprises the following steps:

selecting a preset number of pictures containing telegraph poles and foreign matters, expanding the pictures to obtain a picture library with preset multiples, and marking position information of the telegraph poles and the foreign matters in each picture as a sample library;

taking a feature map extracted from the 6 th convolutional layer of the SSD object detection algorithm as a 7 th grid, combining the 7 th grid with the original 6 th grid, and taking the combined grid and the original 6 th grid as input of a final loss function to train to obtain a detection model of the telegraph pole and the foreign matter;

detecting the telegraph pole and the foreign matters in the input picture by using the detection model obtained by training;

and calculating a first ratio of the overlapping area of the telegraph pole and the foreign matter to the area of the foreign matter, and rejecting the foreign matter with the first ratio lower than a preset threshold value, so that the detection fault tolerance rate is improved.

Further, the picture is subjected to expansion processing, including:

turning the picture left and right;

or/and turning the picture up and down;

or/and, carrying out size scaling processing on the picture;

or/and carrying out random salt and pepper noise processing on the picture.

Further, the training obtains a detection model of the utility pole and the foreign object, and the training further comprises:

and training the model according to a preset learning rate, and sequentially attenuating the learning rate when the model is trained to a preset number of times.

The invention also provides a device for detecting foreign matters on the telegraph pole based on deep learning, which comprises a processor and a memory, wherein the memory stores a plurality of computer instructions, and the computer instructions are executed by the processor to realize the steps of the method.

According to the method and the device for detecting the foreign matters on the telegraph pole based on the deep learning, manual intervention is reduced by using the unsupervised classification mode of the deep learning, and compared with the traditional supervised classification mode, the process is simplified. The default boxes are designed by utilizing the minimum width and height ratio of the foreign matters in the statistical image, so that the accuracy of the model to the foreign matters of small objects is improved. By calculating the overlap ratio of the telegraph pole and the foreign matter, the detection fault tolerance rate is improved. The invention only needs one-time detection, and has much higher efficiency compared with the common method of extracting coarse grain and then fine grain.

Drawings

FIG. 1 is a flow chart of a method for detecting foreign objects on a utility pole based on deep learning according to the present invention;

FIG. 2 is a schematic diagram of merging detection model meshes according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

In one embodiment, as shown in fig. 1, there is provided a method for detecting foreign objects on a utility pole based on deep learning, comprising:

and S1, selecting a preset number of pictures containing the telegraph poles and the foreign matters, expanding the pictures to obtain a picture library with preset multiples, and marking the position information of the telegraph poles and the foreign matters in each picture as a sample library.

For example, about 500 pictures containing the telegraph pole and the foreign matter are selected, the size of each picture is fixed (512 ), and then an XML file of the telegraph pole and the foreign matter position information cluster is marked for each picture.

And (3) respectively carrying out image processing such as turning, scaling and random salt and pepper noise on each picture, and expanding the picture quantity by 20 times.

The method comprises the steps of reversely reading an XML label file corresponding to an original image, obtaining position information of a telegraph pole and foreign matters in the image, automatically calculating position information clusters of the telegraph pole and the foreign matters of an extended image, and storing the position information clusters as XML files, wherein part of calculation strategies are as follows:

the position information of any object obtained by the original image is assumed as follows: sx (start point X coordinate), Ex (end point X coordinate), Sy (start point Y coordinate), and Ey (end point Y coordinate), and the width and height of the image are W and H, respectively. TSx, TEx, TSy, and TEy below represent the starting point X coordinate, ending point X coordinate, starting point Y coordinate, and ending point Y coordinate, respectively, of the enhanced image.

Turning the image left and right: TSx-W-Ex, TEx-W-Sx, TSy-Sy, and TEy-Ey.

Turning the image up and down: TSx, TEx, Ex, TSy, TEy, H-Sy.

Image size scaling: TSx ═ α × Sx, TEx ═ α × Ex, TSy ═ α × Sy, TEy ═ α × Ey, and α is a scaling ratio.

Random salt and pepper noise: TSx, TEx, Ex, TSy, ty, Ey.

It should be understood that the preset number of the present embodiment is about 500 pictures, and each picture is respectively subjected to image processing such as flipping, scaling, random salt and pepper noise, and the like, so as to expand the picture amount by 20 times. The number of the sample pictures is related to the effect of the model training, and a person skilled in the art can deduce the required picture amount according to the experimental effect, which is not described herein again.

Step S2, taking the feature map extracted by the 6 th layer convolution layer of the SSD object detection algorithm as the 7 th type grid, then merging the grid and the original 6 types of grids, and taking the merged grid and the original 6 types of grids as the input of the final loss function to train and obtain the detection model of the telegraph pole and the foreign body.

Common target detection algorithms, such as Faster R-CNN, suffer from slow speed. The SSD object detection algorithm (Single Shot multi box Detector) not only improves speed, but also improves accuracy. In the SSD algorithm, a picture is fixedly divided into 6 grids (default boxes) with different specifications, and then position and classification regression is carried out on each default box, so that the algorithm has good adaptability to multi-scale objects.

By counting the height ratio and the width ratio of the foreign matter in the sample picture, taking the minimum value of the height ratio and the width ratio, and finding that the minimum value is far smaller than the minimum height ratio and the minimum width ratio of 6 fixed default boxes in the SSD, the algorithm structure of the SSD is adjusted, and one layer of additional default boxes is added, so that the height ratio and the width ratio of the default boxes are slightly smaller than the minimum height ratio and the minimum width ratio in the sample.

Specifically, the present embodiment first reads the XML tag file in the reverse direction to obtain the position coordinate cluster of the foreign object. And then calculating a minimum proportion value of the size of the foreign matter and the size of the picture according to the position of the foreign matter, and rounding up the reciprocal of the minimum proportion value to obtain the grid number of the SSD object detection algorithm.

Specifically, the following calculation is performed:

calculating the width of the foreign matter: w1 ═ Ex-Sx;

calculating the height of the foreign matter: h1 ═ Ey-Sy;

substituting the formula P into Min (W1/W, H1/H), the P value is recorded.

The above calculation is repeated until Pmin Min (P1, P2, P3,.., Pn) is found, (n ∈ total amount of foreign matter).

Then, the reciprocal of Pmin is obtained, and the reciprocal is rounded up to obtain: q is Ceil (1/Pmin).

The minimum power exponent N of 2, which is not less than Pmin, is found to be f (N).

The original SSD algorithm uses 6 default boxes for classification and regression of location boxes, so that a picture is divided into 6 grids of different sizes, and each grid can sense different object sizes.

Using the above calculated N value (N ═ 6 calculated in the implementation), the default boxes are designed in a manner of N powers of 2 (N ∈ 0,1,2,. N), so it can be known that the final default boxes actually have 7 specifications of (1,1), (2,2), (4,4), (8,8), (16,16), (32,32) and (64,64), i.e. the number of grids of the SSD object detection algorithm obtained by calculation is 7, however, the loss input of the original SSD algorithm is fixed to 6 default boxes, and exactly (64,64) default boxes are lacked.

As shown in fig. 2, in this embodiment, a feature map (feature map) extracted from the 6 th convolutional layer is used as the 7 th default blocks, and then the default blocks and the original 6 default blocks are merged and used as the input of the final loss function, so as to improve the accuracy of detecting the object with a smaller scale. When the calculated grid number of the SSD object detection algorithm is larger than 6, the feature map (feature map) extracted from the 6 th convolutional layer is used as the 7 th default blocks, and then the default blocks and the original 6 default blocks are merged to be used as the input of the final loss function. And if the grid number of the calculated SSD object detection algorithm is less than or equal to 6, training a detection model according to the original SSD object detection algorithm.

The SSD algorithm extracts a network of features consisting of 18 convolutional layers, each containing one (3,3) convolutional kernel. The original SSD algorithm would obtain feature vectors (feature maps) of the image from convolutional layers of 6 different dimensions, each feature vector may look like a mesh shape partition (default boxes) of the image, and the final algorithm would also classify and coordinate-regress the presentations in each mesh. The sizes of the target objects to be detected in one image are always different, and the image is divided into grids, so that the target objects with any size can be theoretically covered as long as the division scale is enough. Based on such assumption, the aspect ratio of the foreign object in the sample is then calculated, and the smallest foreign object shape is found to be much smaller than the smallest original mesh shape, so that the original mesh partitioning strategy is not applicable to the current sample, and smaller mesh partitioning is required. In practice, the original 6 mesh partitions are (1,1), (2,2), (4,4), (8,8), (16,16), (32,32), respectively, and then the mesh shape needs to be expanded in the form of a common ratio of 2, and the (64,64) -dimensional vector extracted by the sixth convolutional layer just satisfies the current sample, so this embodiment takes the feature vector extracted by the 6 th convolutional layer as the 7 th mesh shape, and combines the feature vector and the original 6 as the input of the final loss function.

In one embodiment, when training to obtain a detection model of the utility pole and the foreign object, the method further comprises:

For example, the learning rate is adjusted to 0.001, the model is trained, the iteration is performed 120000 times, the learning rate is attenuated at 80000 and 100000 times, the attenuation rate is 0.9 each time, and a detection model for the utility pole and the foreign object is obtained.

And step S3, detecting the telegraph pole and the foreign matters in the input picture simultaneously by using the trained detection model.

This embodiment just can adopt camera equipment to obtain the picture that contains the wire pole after training and obtaining the detection model to wire pole and foreign matter to the picture that will acquire is input the detection model, detects wire pole and foreign matter, realizes the detection to the foreign matter on the wire pole.

Step S4, calculating a first ratio of the overlapping area of the telegraph pole and the foreign matter to the area of the foreign matter, eliminating the foreign matter with the first ratio lower than a preset threshold value, and improving the detection fault tolerance rate.

In the embodiment, after the utility pole and the foreign matters in the image are detected by using the trained model, the situation that false positive foreign matters exist is found, and most of the false positive foreign matters actually do not intersect with the position area of the utility pole or the intersection is very small.

Therefore, in the embodiment, the first ratio overlap ratio of the overlapping area of the utility pole and the foreign matter and the area of the foreign matter is calculated two by two, (S utility pole n S foreign matter)/S foreign matter, and the foreign matter with the overlap ratio lower than 0.9 is selected to be filtered out to eliminate a large number of false positive detection results, so that the fault tolerance rate is improved.

The method for calculating the overlap ratio is as follows:

assuming that the location area of a certain utility pole is (Px1, Py1, Px2, Py2) and the location area of a foreign object is (Tx1, Ty1, Tx2, Ty2), the area St of the foreign object is (Tx2-Tx1) × (Ty2-Ty1), the area So of the overlap is (Min (Px2, Tx2) -Max (Px1, Tx1)) (Min (Py2, Ty2) -Max (Py1, Ty1)), and the overlap ratio is So/St. The overlap ratios of the foreign matters in all samples are comprehensively counted, and a good effect can be obtained when the overlap ratio is greater than 0.9.

It should be understood that, although the steps in the flowchart of fig. 1 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

In one embodiment, the present application further provides an apparatus for detecting a foreign object on a utility pole based on deep learning, comprising a processor and a memory storing computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the method for detecting a foreign object on a utility pole based on deep learning.

For specific limitations of the device for detecting foreign matter on a utility pole based on deep learning, reference may be made to the above limitations of the method for detecting foreign matter on a utility pole based on deep learning, and details thereof are not repeated herein. The modules in the device for detecting foreign matters on the telegraph pole based on deep learning can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

The memory and the processor are electrically connected, directly or indirectly, to enable transmission or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can be executed on the processor, and the processor executes the computer program stored in the memory, thereby implementing the network topology layout method in the embodiment of the present invention.

The Memory may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory is used for storing programs, and the processor executes the programs after receiving the execution instructions.

The processor may be an integrated circuit chip having data processing capabilities. The Processor may be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), and the like. The various methods, steps and logic blocks disclosed in embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. The method for detecting the foreign matter on the telegraph pole based on the deep learning is characterized by comprising the following steps of:

calculating a minimum proportion value of the size of the foreign matter and the size of the picture according to the position of the foreign matter, rounding up the reciprocal of the minimum proportion value to obtain the grid number of an SSD object detection algorithm, taking a feature map extracted from a 6 th layer convolution layer of the SSD object detection algorithm as a 7 th grid when the grid number of the SSD object detection algorithm obtained by calculation is larger than 6, then combining the grid and the original 6 grids as the input of a final loss function, and training to obtain a detection model of the telegraph pole and the foreign matter;

2. The method for detecting foreign matter on a telegraph pole based on deep learning of claim 1, wherein the picture is subjected to expansion processing, and the method comprises the following steps:

turning the picture left and right;

or/and turning the picture up and down;

or/and, carrying out size scaling processing on the picture;

or/and carrying out random salt and pepper noise processing on the picture.

3. The method for detecting foreign objects on utility poles based on deep learning of claim 1, wherein the training results in a model for detecting utility poles and foreign objects, further comprising:

4. An apparatus for detecting foreign objects on a utility pole based on deep learning, comprising a processor and a memory storing computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the method of any one of claims 1 to 3.