CN113487541A - Insulator detection method and device - Google Patents

Insulator detection method and device Download PDF

Info

Publication number
CN113487541A
CN113487541A CN202110662619.9A CN202110662619A CN113487541A CN 113487541 A CN113487541 A CN 113487541A CN 202110662619 A CN202110662619 A CN 202110662619A CN 113487541 A CN113487541 A CN 113487541A
Authority
CN
China
Prior art keywords
insulator
anchor point
network
anchor points
connection information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110662619.9A
Other languages
Chinese (zh)
Other versions
CN113487541B (en
Inventor
魏宁
陈冠华
李向阳
陈鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Three Gorges University CTGU
Original Assignee
China Three Gorges University CTGU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Three Gorges University CTGU filed Critical China Three Gorges University CTGU
Priority to CN202110662619.9A priority Critical patent/CN113487541B/en
Publication of CN113487541A publication Critical patent/CN113487541A/en
Application granted granted Critical
Publication of CN113487541B publication Critical patent/CN113487541B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides an insulator detection method and device, comprising the following steps: inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution map of each anchor point and a connection information distribution map between two adjacent anchor points; acquiring a candidate position of each anchor point according to the probability distribution map corresponding to each anchor point, and calculating the confidence of connection information corresponding to each combination according to pairwise combinations of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points; and determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator. The embodiment avoids the interference of other useless information in the detection result, and enables the detection result to be more accurate.

Description

Insulator detection method and device
Technical Field
The invention relates to the technical field of image detection, in particular to an insulator detection method and device.
Background
The insulator is one of important components of the power transmission line, is a part which can not be lost in the safe operation of a power system, and has important functions of supporting the line, electrically insulating, increasing creepage distance and the like. The insulator is defective, which causes the transmission line to stop working and causes huge loss. In order to reduce power transmission accidents caused by insulator faults, preventive measures need to be taken before the accidents happen, and the accidents are avoided. The first step in repairing the insulator is to determine the position of the insulator.
In the past, the insulator inspection mode is mainly manual inspection. This kind of mode of patrolling and examining is inefficient to need the workman to climb the electric tower, intensity of labour is too big, and factor of safety is low. This approach has been gradually phased out. With the rise of computers, the detection and positioning of insulators by using computer image processing technology become the main content of power grid inspection. The current main detection algorithms can be divided into two methods based on the traditional method and the deep learning method.
The insulator is segmented mainly through the characteristics of contours, textures, colors and the like based on a traditional method. The traditional method does not need a large amount of data training, so the advantages are obvious in an ideal background, but the method has high false detection and missing detection rate in the face of a complex background, and is difficult to put into practical application.
The deep learning-based method takes a convolutional neural network as a main feature description means of an image, and learns in all regions where insulators can appear. Currently, the method is generally based on frameworks such as fast RCNN, YOLO v3, YOLO v4 and SSD. Although these methods have significant improvements in robustness and detection accuracy over conventional methods, the description of the location of the insulator is not rigorous. The main reason is that such methods are usually based on axial rectangles, i.e. the sides of a rectangular box are perpendicular or parallel to the X and Y coordinate axes, like the white rectangles in fig. 1. When the insulator inclines, a large amount of useless backgrounds are contained in the detection result of the axial rectangle, so that the existing deep learning method is not suitable for marking and detecting the strip-shaped object with the directional attitude, such as the insulator.
In summary, although the deep learning method overcomes the problems of the conventional method, the use of the axial rectangle results in poor describing capability of the spatial position and the attitude of the insulator, and thus results in poor detection effect of the insulator.
Disclosure of Invention
The invention provides an insulator detection method and device, which are used for solving the defect of poor insulator detection effect of a deep learning method in the prior art and achieving the purpose of improving insulator detection precision.
The invention provides an insulator detection method, which comprises the following steps:
inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution map of pixels in the insulator image to be detected belonging to each anchor point on an insulator and a connection information distribution map between any two adjacent anchor points when the pixels are positioned on a connection line between any two adjacent anchor points;
acquiring one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point, and calculating the connection information confidence coefficient of each combination between two adjacent anchor points according to the pairwise combination of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points;
determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator;
the multitask deep learning network is obtained by training by taking an insulator image sample as a sample and taking the actual position of each anchor point on an insulator in the insulator image sample as a label.
According to the insulator detection method provided by the invention, the multitask deep learning network comprises a backbone network and a plurality of stage networks;
each phase network comprises a first convolution network and a second convolution network;
correspondingly, the inputting the insulator image to be detected into the multitask deep learning network, outputting a probability distribution graph that the pixels in the insulator image to be detected belong to each anchor point on the insulator, and outputting a connection information distribution graph between two adjacent anchor points when the pixels are located on a connection line between any two adjacent anchor points, including:
inputting the insulator image to be detected into the trunk network, outputting a characteristic diagram of the insulator image to be detected, and simultaneously inputting the characteristic diagram into a first convolution network and a second convolution network in a stage network connected with the trunk network;
obtaining the probability distribution map of each anchor point through the first convolutional network, splicing the probability distribution maps of all anchor points, obtaining the connection information distribution map between all two adjacent anchor points through the second convolutional network, and splicing the connection information distribution maps between all two adjacent anchor points;
splicing the splicing result of the probability distribution map, the splicing result of the connection information distribution map and the feature map into a new feature map, and simultaneously inputting the new feature map into a next-stage network of the current-stage network until the next-stage network is the last-stage network;
and outputting the probability distribution map of each anchor point through a first convolution network of the network in the last stage, and outputting the connection information distribution map between two adjacent anchor points through a second convolution network of the network in the last stage.
According to the insulator detection method provided by the invention, the first convolution network comprises a plurality of continuous first convolution layers and a plurality of continuous second convolution layers;
wherein a convolution kernel size of the first convolution layer is larger than a convolution kernel size of the second convolution layer;
the first convolutional layer is positioned in front of the second convolutional layer;
the structure of the second convolutional network is the same as that of the first convolutional network.
According to the insulator detection method provided by the invention, the step of inputting the insulator image to be detected into the multitask deep learning network, outputting the probability distribution graph that the pixel in the insulator image to be detected belongs to each anchor point on the insulator, and training the multitask deep learning network through the following loss functions before the connection information distribution graph between any two adjacent anchor points when the pixel is positioned on the connection line between any two adjacent anchor points:
Figure BDA0003115961550000041
Figure BDA0003115961550000042
Figure BDA0003115961550000043
Figure BDA0003115961550000044
Figure BDA0003115961550000045
wherein f represents the value of the loss function, N is the total number of the stage networks,
Figure BDA0003115961550000046
a loss function representing the probability distribution map S of anchors in the network of the nth stage,
Figure BDA0003115961550000047
representing a loss function of a connection information distribution diagram R between two adjacent anchor points in the nth stage network, D representing the number of anchor points on each insulator, P representing any position in the insulator image sample, W (P) representing the weight at the position P, and W (P) being 0 when P is not on the insulator; when P is presentW (P) is not 0 when it is on the insulator,
Figure BDA0003115961550000048
and
Figure BDA0003115961550000049
respectively representing the actual probability that the P point belongs to the d anchor point and the actual connection information between the d anchor point and the d +1 anchor point when the P point is positioned between the d anchor point and the d +1 anchor point, Sd(P) and Rd(P) respectively representing the probability of the P point in the probability distribution map corresponding to the d anchor point in the n node network and the connection information in the connection information distribution map,
Figure BDA00031159615500000410
and
Figure BDA00031159615500000411
respectively representing the actual coordinates of the d-th and d + 1-th anchor points closest to the point P in the insulator image sample,
Figure BDA0003115961550000051
representing the actual coordinates, σ, of the d-th anchor point on the insulator2Representing the gaussian variance.
According to the insulator detection method provided by the invention, the step of acquiring one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point comprises the following steps:
and performing Gaussian filtering on the probability distribution map corresponding to each anchor point, and taking the peak point of which the probability in the probability distribution map corresponding to each anchor point is greater than a second preset threshold value as a candidate position of each anchor point.
According to the insulator detection method provided by the invention, the confidence coefficient of the connection information of each combination between two adjacent anchor points is calculated according to the connection information distribution diagram between the two adjacent anchor points and the pairwise combination of any candidate positions between the two adjacent anchor points by the following formula:
Figure BDA0003115961550000052
wherein E isjIndicating that for the j-th pair of two adjacent anchor points numbered i and i-1, the candidate position at anchor point numbered i is XiAnd the candidate position of the anchor point with the number of i-1 is Xi-1The confidence of the connection information in the combination of (1), (y) ═ Xi+yXi-1Is represented by XiAnd Xi-1At any position in between, y is between 0 and 1, D represents the number of anchor points on each insulator, Rj(P (y)) represents the connection information at the position P (y) in the connection information distribution diagram between the adjacent two anchor points of the jth pair.
According to the insulator detection method provided by the invention, the step of determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator comprises the following steps:
connecting the final positions of the anchor points on each insulator according to a preset connection sequence of the anchor points to obtain a main axis of the insulator, and calculating the slope of the main axis;
acquiring a maximum region with the final position of each anchor point on each insulator as the center from the insulator image to be detected, wherein the probability of each position in the region in the probability distribution map corresponding to each anchor point is greater than a third preset threshold;
calculating the width of each insulator according to the area taking the final position of each anchor point of each insulator as the center;
calculating the length of each insulator according to the final positions of anchor points at two ends of each insulator;
and determining the inclined rectangular frame of each insulator according to the length and the width of each insulator and the slope of the main axis.
The present invention also provides an insulator detection apparatus, comprising:
the device comprises a characteristic extraction module, a data processing module and a data processing module, wherein the characteristic extraction module is used for inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution diagram that a pixel in the insulator image to be detected belongs to each anchor point on an insulator and a connection information distribution diagram between any two adjacent anchor points when the pixel is positioned on a connection line between any two adjacent anchor points;
the calculation module is used for acquiring one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point, and calculating the connection information confidence coefficient of each combination between two adjacent anchor points according to pairwise combinations of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points;
the determining module is used for determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator;
the multitask deep learning network is obtained by training by taking an insulator image sample as a sample and taking the actual position of each anchor point on an insulator in the insulator image sample as a label.
The present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above insulator detection methods when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the insulator detection method according to any one of the above-mentioned claims.
According to the insulator detection method and device provided by the invention, the insulator is abstracted into two parts of anchor points and connection information between the anchor points by utilizing the linear regular arrangement characteristics of the insulator, the repeated characteristics of the surface of the insulator are learned through the anchor points, the characteristics between the anchor points of the insulator are learned through the connection information between two adjacent anchor points, the position and the direction of the insulator can be obtained only by predicting the position of the anchor point and the connection information of the anchor point of the insulator by using a multitask deep learning network during detection, the interference of other useless information in a detection result is avoided, and the detection result is more accurate.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic diagram illustrating an axial rectangular detection result of an insulator in an insulator detection method provided by the prior art;
fig. 2 is a schematic flow chart of an insulator detection method provided by the present invention;
fig. 3 is a schematic structural diagram illustrating the probability distribution and connection information visualization of the anchor points of the insulator in the insulator detection method according to the present invention;
FIG. 4 is a schematic diagram illustrating the insulator detection effect in the insulator detection method according to the present invention;
FIG. 5 is a schematic structural diagram of a multitask deep learning network in the insulator detection method provided by the present invention;
fig. 6 is a second schematic flow chart of the insulator detection method provided by the present invention;
fig. 7 is a schematic structural diagram of an insulator testing apparatus provided by the present invention;
fig. 8 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The insulator detection method of the present invention is described below with reference to fig. 2, and includes: step 201, inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution graph that a pixel in the insulator image to be detected belongs to each anchor point on an insulator and a connection information distribution graph between any two adjacent anchor points when the pixel is located on a connection line between any two adjacent anchor points;
the insulator image to be detected is an image which needs to be detected and contains an insulator. The number of the insulators contained in the insulator image to be detected is one or more. The number and connection sequence of anchor points on each insulator are predetermined. And two adjacent anchor points on each insulator are determined according to the connection sequence of the anchor points.
A multitasking deep learning network generally includes a shared layer and a plurality of branches, each branch for learning a different task. The multitask deep learning network in the embodiment includes two branches, one branch is used for learning the probability distribution map of each anchor point on the insulator, and the other branch is used for learning the connection information distribution map between two adjacent anchor points on the insulator. The present embodiment is not limited to a specific structure of the multitask deep learning network.
Suppose that the insulator image to be detected I is in the range of RwxhWherein, M insulators are arranged, D anchor points with equal intervals are arranged on each insulator, and then the coordinate of the D anchor point of the M insulator is recorded as
Figure BDA0003115961550000081
Forming a probability distribution map corresponding to the d anchor point according to the probability that each pixel in the insulator image to be detected belongs to the d anchor point, and recording the probability distribution map as Sd∈RwxhIf S is equal to (S)1,S2,…,SD) Representing the set of probability distribution maps corresponding to all anchor points, the visualization structure is shown in fig. 3. Wherein the black dots in the white circles in the first row of images on the right side of fig. 3 represent the points with the highest probability in the probability distribution map corresponding to each anchor point, the white areas in the white circles represent the points with the higher probability in the probability distribution map corresponding to each anchor point, and the white areas represent the points with the higher probability in the probability distribution map corresponding to each anchor pointThe gray areas around the circles represent the less probable points in the probability distribution map for each anchor point.
Vector for connection information between any two adjacent anchor points on each insulator
Figure BDA0003115961550000091
And (4) showing. Under the condition that two ends of a certain position in the insulator image to be detected are anchor points, the coordinate difference between the two adjacent anchor points is used as the connection information between the two adjacent anchor points, and the connection information of other areas which are not positioned between the two adjacent anchor points is set to be 0. With the set R ═ (R)1,R2,…,RD-1) And showing a set formed by a connecting information distribution graph between any two adjacent anchor points, and the visualization result is shown in figure 3. Wherein the vector indicated by the arrow in the second line of images on the right side of fig. 3 is the connection information between two adjacent anchor points. Every two adjacent anchor points correspond to a connection information distribution map, so that the number of the connection information distribution maps is D-1.
202, acquiring one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point, and calculating the connection information confidence of each combination between two adjacent anchor points according to the pairwise combination of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points;
optionally, the probability distribution map corresponding to each anchor point is classified, and the coordinates of the point with the maximum probability in each category with the average probability greater than the preset value are used as the candidate position of each anchor point. The present embodiment is not limited to a specific determination method of the candidate position of each anchor point.
In the case of multiple insulators, there are multiple candidate positions for each anchor point. The pairwise combination of any candidate positions between two adjacent anchor points refers to a combination formed by randomly selecting one from the candidate positions of one anchor point and randomly selecting one from the candidate positions of the other anchor point. For example, if there are 3 candidate positions of one anchor point and 2 candidate positions of the other anchor point in two adjacent anchor points, there are 3 × 2 — 6 combinations.
Optionally, for candidate anchor positions X in each combinationiAnd Xi-1Along the candidate anchor point position X by the connecting information distribution diagram corresponding to two adjacent anchor pointsiAnd Xi-1The line segment between calculates the line integral between two positions to obtain the connection information confidence degree between two adjacent anchor points for each combination.
Step 203, determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator;
optionally, if there is any anchor point with the same candidate position between any two combinations of any two adjacent anchor points, the combination with the lower confidence of the connection information in the two combinations is deleted. After deletion, if the candidate position of any anchor point in any two combinations is the same between the combinations corresponding to any two pairs of adjacent anchor points, the two combinations are divided into a group. The combinations having the same combination are combined into one large group. And if the number of the combinations in any large group is the sum of the anchor points D minus 1, taking the candidate position of each anchor point corresponding to the combination in the group as the final position of each anchor point on each insulator. The number of the insulators is the same as the number of the large groups of which the combination number is equal to the total number D of the anchor points minus 1.
For example, there are 4 anchor points on each insulator. The corresponding pairwise combination numbers of the three pairs of adjacent anchor points are respectively 4, 3 and 2. The candidate positions of the anchor point A in the first two adjacent anchor points are the coordinate 1 and the coordinate 2, and the candidate position of the other anchor point B is the coordinate 3 and the coordinate 4. The first pair of adjacent anchor points are correspondingly combined with (coordinate 1, coordinate 3), (coordinate 1, coordinate 4), (coordinate 2, coordinate 3) and (coordinate 2, coordinate 4). The candidate position of the anchor point a between (coordinate 1, coordinate 3) and (coordinate 1, coordinate 4) is the same, and if the confidence of the connection information of the combination (coordinate 1, coordinate 3) is smaller than that of the combination (coordinate 1, coordinate 4), the combination (coordinate 1, coordinate 3) is deleted. Then, for the same candidate position of anchor point B in the combinations (coordinate 1, coordinate 4) and (coordinate 2, coordinate 4), the combination with the smaller confidence of the connection information is deleted therefrom.
After the deletion process, the first pair is (coordinate 2, coordinate 4), (coordinate 2, coordinate 3), the second pair is (coordinate 3, coordinate 5), and the third pair is (coordinate 5, coordinate 7). What is obtained through one round of grouping is { (coordinate 2, coordinate 3), (coordinate 3, coordinate 5) }, { (coordinate 5, coordinate 7), (coordinate 3, coordinate 5) } and { (coordinate 2, coordinate 4) }. The large groups obtained after the combination were { (coordinate 2, coordinate 3), (coordinate 3, coordinate 5), (coordinate 5, coordinate 7) } and ((coordinate 2, coordinate 4)). Each insulator is provided with 4 anchor points, and the large group with the combination number of 4-1-3 is { (coordinate 2, coordinate 3), (coordinate 3, coordinate 5), (coordinate 5, coordinate 7) }. And taking the coordinate 2, the coordinate 3, the coordinate 5 and the coordinate 7 as final positions of 4 anchor points on a certain insulator. Optionally, the connection information confidences corresponding to all combinations in the large group with the combination number being-1 of the total number of anchor points are added. The large group with the lower addition result was deleted by Non-Maximum Suppression (NMS). The detection result obtained according to the final position of each anchor point on each insulator is shown in fig. 4. Wherein, the circle represents the anchor point, the ellipse represents the connection information between the anchor points, and the inclined rectangular frame represents the detection result of the insulator.
The multitask deep learning network is obtained by training by taking an insulator image sample as a sample and taking the actual position of each anchor point on an insulator in the insulator image sample as a label.
In the process of training the multitask deep learning network, firstly, the anchor point position of each insulator in an acquired insulator image sample is marked, then the marked insulator image sample is input into the multitask deep learning network for training, and parameters of the multitask deep learning network are adjusted to obtain optimal parameters.
In the embodiment, the insulator is abstracted into two parts, namely the anchor point and the connection information between the anchor points, by utilizing the linear rule arrangement characteristics of the insulator, the repeated characteristics of the surface of the insulator are learned through the anchor points, the characteristics between the anchor points of the insulator are learned through the connection information between two adjacent anchor points, the position and the direction of the insulator can be obtained only by predicting the anchor point position and the anchor point connection information of the insulator through a multitask deep learning network when detection is carried out, the interference of other useless information in a detection result is avoided, and the detection result is more accurate.
On the basis of the above embodiment, as shown in fig. 5, the multitask deep learning network in this embodiment includes a backbone network and a plurality of stage networks; each phase network comprises a first convolution network and a second convolution network;
optionally, the backbone network is a ResNet network. The number of the stage networks is N, and the number of the stage networks is not limited in this embodiment. The stage networks are connected in series, and the backbone network is connected with the first stage network. The network structure of the network is the same for all phases.
Each stage network comprises a first convolution network and a second convolution network, and the first convolution network and the second convolution network in each stage are parallel. The present embodiment does not limit the specific structure of the two convolutional networks.
Correspondingly, the inputting the insulator image to be detected into the multitask deep learning network, outputting a probability distribution graph that the pixels in the insulator image to be detected belong to each anchor point on the insulator, and outputting a connection information distribution graph between two adjacent anchor points when the pixels are located on a connection line between any two adjacent anchor points, including: inputting the insulator image to be detected into the trunk network, outputting a characteristic diagram of the insulator image to be detected, and simultaneously inputting the characteristic diagram into a first convolution network and a second convolution network in a stage network connected with the trunk network;
and extracting the coded characteristic diagram F from the insulator image I to be detected through a backbone network, and then repeating the N stage networks. The feature map F is simultaneously input to two convolutional networks in parallel in the first stage network.
Obtaining the probability distribution map of each anchor point through the first convolutional network, splicing the probability distribution maps of all anchor points, obtaining the connection information distribution map between all two adjacent anchor points through the second convolutional network, and splicing the connection information distribution maps between all two adjacent anchor points;
the first convolution network is used for carrying out anchor point estimation on the insulator image to be detected to obtain a probability distribution map set S of anchor points. The second convolution network is used for carrying out anchor point connection information estimation on the insulator image to be detected to obtain a connection information distribution map set R between two adjacent anchor points.
Optionally, the multiple distribution maps are accumulated by using convolution kernels of 1 × 1 for the S and R sets respectively to obtain MSAnd MR。
Splicing the splicing result of the probability distribution map, the splicing result of the connection information distribution map and the feature map into a new feature map, and simultaneously inputting the new feature map into a next-stage network of the current-stage network until the next-stage network is the last-stage network;
then, M is addedS、MRAnd F, splicing into a new feature graph F 'and simultaneously inputting the new feature graph F' into the next-stage network until the next-stage network is the last node network, namely repeating the N-stage networks.
And outputting the probability distribution map of each anchor point through a first convolution network of the network in the last stage, and outputting the connection information distribution map between two adjacent anchor points through a second convolution network of the network in the last stage.
And taking S and R output by the last network as the output of the multitask deep learning network.
In the embodiment, the anchor point estimation characteristics and the anchor point connection information estimation characteristics are conducted in different convolutional networks synchronously, so that the detection and training speed is higher compared with the two modes of step-by-step detection and training.
On the basis of the foregoing embodiments, in this embodiment, the first convolutional network includes a plurality of continuous first convolutional layers and a plurality of continuous second convolutional layers; wherein a convolution kernel size of the first convolution layer is larger than a convolution kernel size of the second convolution layer; the first convolutional layer is positioned in front of the second convolutional layer; the structure of the second convolutional network is the same as that of the first convolutional network.
Optionally, the first convolution layer in the first convolution network has three layers, and the size of the convolution kernel is 3 × 3; the second convolution layer has two layers, the convolution kernel size being 1x 1. The structure of the second convolutional network is the same as that of the first convolutional network.
The present embodiment expands the perception range of the anchor point and the anchor point connection information by a plurality of consecutive first convolution layers and a plurality of consecutive second convolution layers. However, the increasing number of convolutional layers also causes the problem of gradient disappearance, some important features are lost in forward propagation, and the more convolutional layers, the greater the risk of feature loss. To solve this problem, the network uses a staged architecture, with the original feature F and the previous stage network feature M being combined before entering the network at each stageSAnd MRAnd splicing and combining.
On the basis of the foregoing embodiment, in this embodiment, before the inputting the image of the insulator to be detected into the multitask deep learning network and outputting the probability distribution graph that the pixel in the image of the insulator to be detected belongs to each anchor point on the insulator and the connection information distribution graph between two adjacent anchor points when the pixel is located on the connection line between any two adjacent anchor points, the multitask deep learning network is trained by using the following loss function:
Figure BDA0003115961550000131
Figure BDA0003115961550000141
Figure BDA0003115961550000142
Figure BDA0003115961550000143
Figure BDA0003115961550000144
wherein f represents the value of the loss function, N is the total number of the stage networks,
Figure BDA0003115961550000145
a loss function representing the probability distribution map S of anchors in the network of the nth stage,
Figure BDA0003115961550000146
representing a loss function of a connection information distribution diagram R between two adjacent anchor points in the nth stage network, D representing the number of anchor points on each insulator, P representing any position in the insulator image sample, W (P) representing the weight at the position P, and W (P) being 0 when P is not on the insulator; when P is on the insulator, W (P) is not 0,
Figure BDA0003115961550000147
and
Figure BDA0003115961550000148
respectively representing the actual probability that the P point belongs to the d anchor point and the actual connection information between the d anchor point and the d +1 anchor point when the P point is positioned between the d anchor point and the d +1 anchor point, Sd(P) and Rd(P) respectively representing the probability of the P point in the probability distribution map corresponding to the d anchor point in the n node network and the connection information in the connection information distribution map,
Figure BDA0003115961550000149
and
Figure BDA00031159615500001410
respectively representing the actual coordinates of the d-th and d + 1-th anchor points closest to the point P in the insulator image sample,
Figure BDA00031159615500001411
representing the actual coordinates, σ, of the d-th anchor point on the insulator2Representing the gaussian variance.
In particular, in the case where two convolutional networks use multiple convolutional layers to extend the perceptual range of the network for S and R features, the problem of gradient vanishing is caused and some important features are lost in forward propagation. To solve this problem, the present embodiment calculates the loss function f for each stage of the networkSAnd fRThe loss functions of the N stage networks are summed. W (p) is a mask superimposed on the actual value, i.e. non-zero, typically set to 1, only in the regions where there are real anchor points or between adjacent real anchor points, thus only the loss of w (p) non-zero regions is calculated.
Alternatively, if the P point satisfies
Figure BDA00031159615500001412
And is
Figure BDA0003115961550000151
Point P is said to be on the insulator. Theta denotes a vector
Figure BDA0003115961550000152
And
Figure BDA0003115961550000153
the width of the insulator is shown as the included angle between the two insulators.
On the basis of the foregoing embodiments, in this embodiment, the obtaining one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point includes: and performing Gaussian filtering on the probability distribution map corresponding to each anchor point, and taking the peak point of which the probability in the probability distribution map corresponding to each anchor point is greater than a second preset threshold value as a candidate position of each anchor point.
Specifically, each point on the probability distribution map S output by the multitask deep learning network represents the probability that the point is an anchor point. And performing Gaussian filtering on the probability distribution map corresponding to each anchor point to obtain the position of a peak point of the Gaussian filtering. And searching the probability of each peak point from the probability distribution map corresponding to the anchor point, and taking the peak point with the probability greater than a second preset threshold value as the candidate position of each anchor point predicted by the multitask deep learning network.
On the basis of the above embodiments, in this embodiment, the confidence of the connection information between two adjacent anchor points for each combination is calculated according to the following formula according to the connection information distribution diagram between two adjacent anchor points and the pairwise combination of any candidate positions between two adjacent anchor points:
Figure BDA0003115961550000154
wherein E isjIndicating that for the j-th pair of two adjacent anchor points numbered i and i-1, the candidate position at anchor point numbered i is XiAnd the candidate position of the anchor point with the number of i-1 is Xi-1The confidence of the connection information in the combination of (1), (y) ═ Xi+yXi-1Is represented by XiAnd Xi-1At any position in between, y is between 0 and 1, D represents the number of anchor points on each insulator, Rj(P (y)) represents the connection information at the position P (y) in the connection information distribution diagram between the adjacent two anchor points of the jth pair.
Specifically, integrating a connection information distribution graph R between two adjacent anchor points output by the multitask deep learning network to obtain a connection information confidence coefficient between every two adjacent anchor points.
Figure BDA0003115961550000161
Representing candidate anchor point position X corresponding to two adjacent anchor pointsiAnd Xi-1Unit vector between combinations. I Xi-1-Xi||2Representing the distance between the two coordinates. P (y) ═ X (1-y)i+yXi-1Represents XiAnd Xi-1At any position in between. And calculating the confidence coefficient of the connection information between any two adjacent anchor points for any combination by adopting the formula.
On the basis of the foregoing embodiments, in this embodiment, the determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator includes: connecting the final positions of the anchor points on each insulator according to a preset connection sequence of the anchor points to obtain a main axis of the insulator, and calculating the slope of the main axis;
optionally, the preset connection order of the anchor points is an order from small to large of the numbers of the anchor points. And connecting the anchor points on each insulator in sequence from small to large according to the serial numbers of the anchor points to obtain the main axis of each insulator, thereby further obtaining the slope of the main axis.
Acquiring a maximum region with the final position of each anchor point on each insulator as the center from the insulator image to be detected, wherein the probability of each position in the region in the probability distribution map corresponding to each anchor point is greater than a third preset threshold; calculating the width of each insulator according to the maximum area corresponding to each anchor point of each insulator;
take the first anchor point on each insulator as an example. And finding a maximum region taking the first anchor point as a center in the insulator image to be detected, wherein the region needs to meet the condition that the probability of each position in the region in the probability distribution map corresponding to the first anchor point is greater than a third preset threshold value.
Because the anchor point is located on the main axis of the insulator, the pixels with higher probability of belonging to the anchor point are also located on the insulator. The diameter of the maximum area defined to satisfy the condition is the width of the insulator.
Calculating the length of each insulator according to the final positions of anchor points at two ends of each insulator; and determining the inclined rectangular frame of each insulator according to the length and the width of each insulator and the slope of the main axis.
For example, if each insulator has D anchor points, the length of each insulator is obtained according to the distance between the final position of the first anchor point and the final position of the D-th anchor point on each insulator. And obtaining the inclined rectangular frame of the insulator according to the length and the width of each insulator and the slope of the main axis. The insulator testing process is shown in fig. 6.
To illustrate the detection effect of the present embodiment, the detection effect was evaluated and compared. The commonly used algorithms fast RCNN, YOLO and SSD use the conventional Intersection over Union (IoU) as a measure for the quality of the prediction result when detecting the target. Wherein IoU is defined as follows:
Figure BDA0003115961550000171
wherein B ispreRepresenting the model prediction result, BGIndicating the actual test result. And n and u denote intersection and union, respectively.
The above formula represents the ratio of the intersection and union between the predicted result and the actual detection result. It can be found that when the insulator postures are arranged in a non-axial direction, the insulator usually only occupies a smaller area in the axial labeling rectangle, so that the accuracy of the insulator prediction by the model cannot be well measured only by using the intersection and the union of the axial labeling areas.
The present embodiment proposes a more reasonable directional Intersection of Union (DIoU) to calculate the error of the tilted insulator. When the insulator postures are arranged in a non-axial direction, the insulators occupy almost all the non-axial direction labeling rectangular areas detected in the embodiment, the actual detection area is a non-axial direction minimum inclined rectangular frame surrounding the insulators, and the value of the DIoU is 1 only when the prediction result is completely overlapped with the real insulator area, so that useless information interference is avoided. The evaluation result is more accurate.
Compared with the directional intersection in the embodiment, the directional intersection considers the influence of the direction information of the insulator, and can more accurately describe the position and the direction information of the insulator, so that the evaluation result of the model is more accurate and reasonable.
The following describes the insulator detection apparatus provided by the present invention, and the insulator detection apparatus described below and the insulator detection method described above may be referred to in correspondence with each other.
As shown in fig. 7, the apparatus includes a feature extraction module 701, a calculation module 702, and a determination module 703, wherein:
the feature extraction module 701 is configured to input an insulator image to be detected into a multitask deep learning network, and output a probability distribution map that a pixel in the insulator image to be detected belongs to each anchor point on an insulator, and a connection information distribution map between any two adjacent anchor points when the pixel is located on a connection line between the two adjacent anchor points;
the calculation module 702 is configured to obtain one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point, and calculate a connection information confidence for each combination between two adjacent anchor points according to pairwise combinations of the connection information distribution map between the two adjacent anchor points and any candidate position between the two adjacent anchor points;
the determining module 703 is configured to determine a final position of each anchor point on each insulator according to the confidence of the connection information for each combination between all two adjacent anchor points, and determine an inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator;
the multitask deep learning network is obtained by training by taking an insulator image sample as a sample and taking the actual position of each anchor point on an insulator in the insulator image sample as a label.
In the embodiment, the insulator is abstracted into two parts, namely the anchor point and the connection information between the anchor points, by utilizing the linear rule arrangement characteristics of the insulator, the repeated characteristics of the surface of the insulator are learned through the anchor points, the characteristics between the anchor points of the insulator are learned through the connection information between two adjacent anchor points, the position and the direction of the insulator can be obtained only by predicting the anchor point position and the anchor point connection information of the insulator through a multitask deep learning network when detection is carried out, the interference of other useless information in a detection result is avoided, and the detection result is more accurate.
Fig. 8 illustrates a physical structure diagram of an electronic device, and as shown in fig. 8, the electronic device may include: a processor (processor)810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. Processor 810 may invoke logic instructions in memory 830 to perform a method of insulator detection, the method comprising: inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution map of each anchor point and a connection information distribution map between two adjacent anchor points; acquiring a candidate position of each anchor point according to the probability distribution map corresponding to each anchor point, and calculating the confidence of connection information corresponding to each combination according to pairwise combinations of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points; and determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator.
In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the insulator detection method provided by the above methods, the method comprising: inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution map of each anchor point and a connection information distribution map between two adjacent anchor points; acquiring a candidate position of each anchor point according to the probability distribution map corresponding to each anchor point, and calculating the confidence of connection information corresponding to each combination according to pairwise combinations of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points; and determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, is implemented to perform the insulator detection method provided above, the method including: inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution map of each anchor point and a connection information distribution map between two adjacent anchor points; acquiring a candidate position of each anchor point according to the probability distribution map corresponding to each anchor point, and calculating the confidence of connection information corresponding to each combination according to pairwise combinations of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points; and determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. An insulator detection method, comprising:
inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution map of pixels in the insulator image to be detected belonging to each anchor point on an insulator and a connection information distribution map between any two adjacent anchor points when the pixels are positioned on a connection line between any two adjacent anchor points;
acquiring one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point, and calculating the connection information confidence coefficient of each combination between two adjacent anchor points according to the pairwise combination of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points;
determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator;
the multitask deep learning network is obtained by training by taking an insulator image sample as a sample and taking the actual position of each anchor point on an insulator in the insulator image sample as a label.
2. The insulator detection method according to claim 1, wherein the multitask deep learning network comprises a backbone network and a plurality of stage networks;
each phase network comprises a first convolution network and a second convolution network;
correspondingly, the inputting the insulator image to be detected into the multitask deep learning network, outputting a probability distribution graph that the pixels in the insulator image to be detected belong to each anchor point on the insulator, and outputting a connection information distribution graph between two adjacent anchor points when the pixels are located on a connection line between any two adjacent anchor points, including:
inputting the insulator image to be detected into the trunk network, outputting a characteristic diagram of the insulator image to be detected, and simultaneously inputting the characteristic diagram into a first convolution network and a second convolution network in a stage network connected with the trunk network;
obtaining the probability distribution map of each anchor point through the first convolutional network, splicing the probability distribution maps of all anchor points, obtaining the connection information distribution map between all two adjacent anchor points through the second convolutional network, and splicing the connection information distribution maps between all two adjacent anchor points;
splicing the splicing result of the probability distribution map, the splicing result of the connection information distribution map and the feature map into a new feature map, and simultaneously inputting the new feature map into a next-stage network of the current-stage network until the next-stage network is the last-stage network;
and outputting the probability distribution map of each anchor point through a first convolution network of the network in the last stage, and outputting the connection information distribution map between two adjacent anchor points through a second convolution network of the network in the last stage.
3. The insulator detection method of claim 2, wherein the first convolution network includes a plurality of consecutive first convolution layers and a plurality of consecutive second convolution layers;
wherein a convolution kernel size of the first convolution layer is larger than a convolution kernel size of the second convolution layer;
the first convolutional layer is positioned in front of the second convolutional layer;
the structure of the second convolutional network is the same as that of the first convolutional network.
4. The insulator detection method according to claim 2, wherein the inputting the insulator image to be rate-resolved into a multitask deep learning network, outputting a profile that a pixel in the insulator image to be detected belongs to each anchor point on the insulator, and before the profile of connection information between any two adjacent anchor points when the pixel is located on a connection line between any two adjacent anchor points, further comprises training the multitask deep learning network by using the following loss function:
Figure FDA0003115961540000021
Figure FDA0003115961540000022
Figure FDA0003115961540000023
Figure FDA0003115961540000024
Figure FDA0003115961540000031
wherein f represents the value of the loss function, N is the total number of the stage networks,
Figure FDA0003115961540000032
a loss function representing the probability distribution map S of anchors in the network of the nth stage,
Figure FDA0003115961540000033
representing a loss function of a connection information distribution diagram R between two adjacent anchor points in the nth stage network, D representing the number of anchor points on each insulator, P representing any position in the insulator image sample, W (P) representing the weight at the position P, and W (P) being 0 when P is not on the insulator; when P is on the insulator, W (P) is not 0,
Figure FDA0003115961540000034
and
Figure FDA0003115961540000035
respectively representing the actual probability that the P point belongs to the d anchor point and the actual connection information between the d anchor point and the d +1 anchor point when the P point is positioned between the d anchor point and the d +1 anchor point, Sd(P) and Rd(P) respectively representing the probability of the P point in the probability distribution map corresponding to the d anchor point in the n node network and the connection information in the connection information distribution map,
Figure FDA0003115961540000036
and
Figure FDA0003115961540000037
respectively representing the actual coordinates of the d-th and d + 1-th anchor points closest to the point P in the insulator image sample,
Figure FDA0003115961540000038
representing the actual coordinates, σ, of the d-th anchor point on the insulator2Representing the gaussian variance.
5. The insulator detection method according to any one of claims 1 to 4, wherein the obtaining one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point comprises:
and performing Gaussian filtering on the probability distribution map corresponding to each anchor point, and taking the peak point of which the probability in the probability distribution map corresponding to each anchor point is greater than a second preset threshold value as a candidate position of each anchor point.
6. The insulator detection method according to any one of claims 1 to 4, wherein the confidence of the connection information between the two adjacent anchor points for each combination is calculated according to the following formula based on the connection information distribution map between the two adjacent anchor points and the pairwise combination of any candidate positions between the two adjacent anchor points:
Figure FDA0003115961540000039
wherein E isjIndicating that for the j-th pair of two adjacent anchor points numbered i and i-1, the candidate position at anchor point numbered i is XiAnd the candidate position of the anchor point with the number of i-1 is Xi-1The confidence of the connection information in the combination of (1), (y) ═ Xi+yXi-1Is represented by XiAnd Xi-1At any position in between, y is between 0 and 1, D represents the number of anchor points on each insulator, Rj(P (y)) represents the connection information at the position P (y) in the connection information distribution diagram between the adjacent two anchor points of the jth pair.
7. The insulator detection method according to any one of claims 1 to 4, wherein the determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator comprises:
connecting the final positions of the anchor points on each insulator according to a preset connection sequence of the anchor points to obtain a main axis of the insulator, and calculating the slope of the main axis;
acquiring a maximum region with the final position of each anchor point on each insulator as the center from the insulator image to be detected, wherein the probability of each position in the region in the probability distribution map corresponding to each anchor point is greater than a third preset threshold;
calculating the width of each insulator according to the area taking the final position of each anchor point of each insulator as the center;
calculating the length of each insulator according to the final positions of anchor points at two ends of each insulator;
and determining the inclined rectangular frame of each insulator according to the length and the width of each insulator and the slope of the main axis.
8. An insulator testing apparatus, comprising:
the device comprises a characteristic extraction module, a data processing module and a data processing module, wherein the characteristic extraction module is used for inputting an insulator image to be detected into a multitask deep learning network, and outputting a probability distribution diagram that a pixel in the insulator image to be detected belongs to each anchor point on an insulator and a connection information distribution diagram between any two adjacent anchor points when the pixel is positioned on a connection line between any two adjacent anchor points;
the calculation module is used for acquiring one or more candidate positions of each anchor point in the insulator image to be detected according to the probability distribution map corresponding to each anchor point, and calculating the connection information confidence coefficient of each combination between two adjacent anchor points according to pairwise combinations of the connection information distribution map between two adjacent anchor points and any candidate position between two adjacent anchor points;
the determining module is used for determining the final position of each anchor point on each insulator according to the confidence coefficient of the connection information of each combination between every two adjacent anchor points, and determining the inclined rectangular frame of each insulator according to the final position of each anchor point on each insulator;
the multitask deep learning network is obtained by training by taking an insulator image sample as a sample and taking the actual position of each anchor point on an insulator in the insulator image sample as a label.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the insulator detection method according to any one of claims 1 to 7 are implemented when the processor executes the program.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the insulator detection method according to any one of claims 1 to 7.
CN202110662619.9A 2021-06-15 2021-06-15 Insulator detection method and device Active CN113487541B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110662619.9A CN113487541B (en) 2021-06-15 2021-06-15 Insulator detection method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110662619.9A CN113487541B (en) 2021-06-15 2021-06-15 Insulator detection method and device

Publications (2)

Publication Number Publication Date
CN113487541A true CN113487541A (en) 2021-10-08
CN113487541B CN113487541B (en) 2022-05-03

Family

ID=77935106

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110662619.9A Active CN113487541B (en) 2021-06-15 2021-06-15 Insulator detection method and device

Country Status (1)

Country Link
CN (1) CN113487541B (en)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101487866A (en) * 2008-11-06 2009-07-22 姚建刚 On-line detection method for AC power transmission line porcelain zero resistance insulator based on infrared thermal imaging
US20100202040A1 (en) * 2009-02-06 2010-08-12 U.S. Government As Represented By The Secretary Of The Army Dynamic imaging and/or identification apparatus and method thereof
CN102053046A (en) * 2010-11-02 2011-05-11 武汉大学 System and method for detecting hydrophilic angle on surface of composite insulator
CN102508110A (en) * 2011-10-10 2012-06-20 上海大学 Texture-based insulator fault diagnostic method
CN104978580A (en) * 2015-06-15 2015-10-14 国网山东省电力公司电力科学研究院 Insulator identification method for unmanned aerial vehicle polling electric transmission line
CN105678760A (en) * 2016-01-04 2016-06-15 国家电网公司 Method for recognizing insulator image on the basis of Canny edge detection algorithm
CN107464233A (en) * 2017-07-19 2017-12-12 国家电网公司 The image detecting method and system of composite insulator based on Lab color modes
US20200074176A1 (en) * 2017-03-15 2020-03-05 Siemens Aktiengesellschaft Method and arrangement for condition monitoring of an installation with operating means
CN110992307A (en) * 2019-11-04 2020-04-10 华北电力大学（保定） Insulator positioning and identifying method and device based on YOLO
US20200131632A1 (en) * 2018-10-25 2020-04-30 Greene, Tweed Technologies, Inc. Plasma Resistant Multi-Layer Coatings and Related Methods of Preparing Same
CN111899216A (en) * 2020-06-16 2020-11-06 西安交通大学 Abnormity detection method for insulator fastener of high-speed rail contact network
CN112183667A (en) * 2020-10-31 2021-01-05 哈尔滨理工大学 Insulator fault detection method in cooperation with deep learning

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101487866A (en) * 2008-11-06 2009-07-22 姚建刚 On-line detection method for AC power transmission line porcelain zero resistance insulator based on infrared thermal imaging
US20100202040A1 (en) * 2009-02-06 2010-08-12 U.S. Government As Represented By The Secretary Of The Army Dynamic imaging and/or identification apparatus and method thereof
CN102053046A (en) * 2010-11-02 2011-05-11 武汉大学 System and method for detecting hydrophilic angle on surface of composite insulator
CN102508110A (en) * 2011-10-10 2012-06-20 上海大学 Texture-based insulator fault diagnostic method
CN104978580A (en) * 2015-06-15 2015-10-14 国网山东省电力公司电力科学研究院 Insulator identification method for unmanned aerial vehicle polling electric transmission line
CN105678760A (en) * 2016-01-04 2016-06-15 国家电网公司 Method for recognizing insulator image on the basis of Canny edge detection algorithm
US20200074176A1 (en) * 2017-03-15 2020-03-05 Siemens Aktiengesellschaft Method and arrangement for condition monitoring of an installation with operating means
CN107464233A (en) * 2017-07-19 2017-12-12 国家电网公司 The image detecting method and system of composite insulator based on Lab color modes
US20200131632A1 (en) * 2018-10-25 2020-04-30 Greene, Tweed Technologies, Inc. Plasma Resistant Multi-Layer Coatings and Related Methods of Preparing Same
CN110992307A (en) * 2019-11-04 2020-04-10 华北电力大学（保定） Insulator positioning and identifying method and device based on YOLO
CN111899216A (en) * 2020-06-16 2020-11-06 西安交通大学 Abnormity detection method for insulator fastener of high-speed rail contact network
CN112183667A (en) * 2020-10-31 2021-01-05 哈尔滨理工大学 Insulator fault detection method in cooperation with deep learning

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
EHAB UR RAHMAN 等: "Autonomous Vision-Based Primary Distribution Systems Porcelain Insulators Inspection Using UAVs", 《SENSORS》 *
李向阳 等: "基于深度学习的绝缘子串检测算法研究", 《长江信息通信》 *
郭威 等: "输电线路中缺失绝缘子的检测与定位", 《太原科技大学学报》 *

Also Published As

Publication number Publication date
CN113487541B (en) 2022-05-03

Similar Documents

Publication Publication Date Title
KR20210002104A (en) Target detection and training of target detection networks
Akagic et al. Pothole detection: An efficient vision based method using rgb color space image segmentation
CN111681273B (en) Image segmentation method and device, electronic equipment and readable storage medium
CN110264444B (en) Damage detection method and device based on weak segmentation
CN113920107A (en) Insulator damage detection method based on improved yolov5 algorithm
CN113095333B (en) Unsupervised feature point detection method and unsupervised feature point detection device
CN113240623B (en) Pavement disease detection method and device
CN113205041A (en) Structured information extraction method, device, equipment and storage medium
CN116883887A (en) Leakage gas detection method for infrared video
CN115239646A (en) Defect detection method and device for power transmission line, electronic equipment and storage medium
CN114581419A (en) Transformer insulating sleeve defect detection method, related equipment and readable storage medium
CN113592839A (en) Distribution network line typical defect diagnosis method and system based on improved fast RCNN
CN113487541B (en) Insulator detection method and device
CN116229419B (en) Pedestrian detection method and device
CN115546476A (en) Multi-object detection method and data platform based on multi-scale features
CN115937492A (en) Transformer equipment infrared image identification method based on feature identification
CN113256607B (en) Defect detection method and device
CN112614094B (en) Insulator string abnormity positioning and identifying method based on sequence state coding
CN111429411B (en) X-ray defect image sample generation method for carbon fiber composite core wire
US10699153B2 (en) Providing diverse and representative swatches extracted from an input image
CN113554024A (en) Method and device for determining cleanliness of vehicle and computer equipment
CN112862002A (en) Training method of multi-scale target detection model, target detection method and device
CN113947567B (en) Defect detection method based on multitask learning
CN113255668B (en) Text recognition method and device, electronic equipment and storage medium
CN117541842A (en) Reflective strip detection method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant