CN111650204B - Power transmission line hardware defect detection method and system based on cascade target detection - Google Patents
Power transmission line hardware defect detection method and system based on cascade target detection Download PDFInfo
- Publication number
- CN111650204B CN111650204B CN202010393940.7A CN202010393940A CN111650204B CN 111650204 B CN111650204 B CN 111650204B CN 202010393940 A CN202010393940 A CN 202010393940A CN 111650204 B CN111650204 B CN 111650204B
- Authority
- CN
- China
- Prior art keywords
- target detection
- network
- detection model
- image
- transmission line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8861—Determining coordinates of flaws
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a transmission line hardware defect detection method and system based on cascade target detection, comprising the following steps: using a trained first target detection model to detect a connection area of the power transmission line image, and cutting the detected connection area; taking n connecting areas with the area size meeting preset conditions as images to be identified; performing fine hardware defect detection on the image to be identified by using a trained second target detection model, and obtaining coordinates of the fine hardware defect on the image to be identified; according to the method, the defect of the fine hardware fitting is displayed in an original image according to the mapping relation between the coordinates of the image to be identified and the coordinates of the original image, and the method adopts a cascade target detection algorithm to deeply convolve the neural network for identifying the small target of the fine hardware fitting of the power transmission line, and then identifies the connection area in the power transmission line image, and then identifies the defect condition of the fine hardware fitting of the connection area, so that the defect detection precision of the fine hardware fitting is remarkably improved.
Description
Technical Field
The invention relates to the field of transmission line defect detection, in particular to a transmission line hardware defect detection method and system based on cascade target detection.
Background
The tiny hardware such as the electric bolts is applied to the common power transmission and distribution lines, needs to withstand long-time field operation corrosion and strong collision friction, has huge quantity in the power grid, and plays a role in stabilizing base, line equipment and the like. However, the thin and fine hardware is a component which is extremely easy to damage due to the complex and severe environment of the thin and fine hardware. Once broken, a power interruption is caused, which affects the safe operation of the whole power grid. At present, image recognition processing is carried out on defect pictures such as bolt missing pins, bolt missing nuts and the like through a deep learning image recognition technology, so that defect diagnosis is formed.
When the deep learning network model is used for image recognition processing, a large number of training samples are needed, but for defect detection of fine hardware, defect categories such as pin missing, pin falling and the like mainly comprise, but no matter which category, too few samples can be collected, the samples are easy to be subjected to fitting during deep learning, so that the effect is not ideal during testing, and the traditional image augmentation modes such as random cutting, random translation, random rotation and the like cannot provide more low-level characteristics of the defects of the fine hardware.
And because the small hardware defects have too small targets and account for less than 5% of the original image, the traditional deep learning image recognition method has insufficient feature extraction capability, so that the detection effect of the small hardware defects cannot meet the actual service requirements.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a power transmission line hardware defect detection method based on cascade target detection, which adopts a generated countermeasure network to amplify a shot image, increases the number of training samples of a deep learning network, and provides a two-stage detection strategy which can remarkably improve the detection precision of small hardware defects and has popularization and application values, and the method specifically comprises the following steps:
(11) Acquiring a patrol shooting image and preprocessing;
(12) Using a trained first target detection model to detect a connection area of the inspection shooting image, and cutting out the detected rectangular connection area;
(13) Acquiring the area size of the connecting areas, and taking n connecting areas with the area size meeting preset conditions as images to be identified;
(14) Performing fine hardware defect detection on the image to be identified by using a trained second target detection model, and obtaining coordinates of the fine hardware defect on the image to be identified;
(15) And displaying the fine hardware defects in the original image according to the mapping relation between the coordinates of the image to be identified and the coordinates of the original image.
As a further optimization of the scheme, before the first target detection model and the second target detection model are trained, data augmentation processing is carried out on the acquired power transmission line images, and the image generated by the augmentation processing and the acquired original power transmission line images are used as a data set for training the first target detection model and the second target detection model.
As a further optimization of the above scheme, the data augmentation process employs a trained generation countermeasure network model.
As a further optimization of the above solution, the step of generating the training of the countermeasure network model is:
(31) Establishing a generator and a discriminator, setting a loss function of the discriminator and generating an objective function of an countermeasure network;
(32) Inputting a random noise signal z into a generator to obtain a generated sample, marking the acquired power transmission line image and the generated sample, inputting the marked power transmission line image and the generated sample into a discriminator, discriminating a real sample and the generated sample, regulating network parameters of a discrimination network by using a back propagation algorithm, and maximizing a target function of a generated countermeasure network to obtain an optimized discrimination network;
(33) Substituting the obtained network parameters of the optimized discrimination network into a generated countermeasure network objective function, and adjusting the network parameters of the generated network by using a back propagation algorithm to minimize the generated countermeasure network objective function, so as to obtain the network parameters of the optimized discrimination network, and further obtain the optimized generated network;
(34) And (3) judging whether the iteration times reach the preset maximum iteration times, if so, repeating the steps (32) - (34), otherwise, stopping training, and storing the generated countermeasure model after training is completed.
As a further optimization of the above solution, the first target detection model and the second target detection model use a deep neural network based on a target detection algorithm.
As a further optimization of the above solution, the first target detection model and the second target detection model adopt a deep neural network based on a FasterRCNN algorithm, and the training process of the FasterRCNN network model includes:
(61) Nesting and labeling a data set for training a model, firstly labeling a power transmission line connection region in an image, and then labeling whether a tiny hardware fitting has a defect or not based on the connection region;
(62) Dividing data: dividing a training set and a verification set in proportion;
(63) Establishing a first target detection model and a second target detection model, wherein a forward network adopts a basic FasterRCNN network, and a corresponding loss function and a weight updating method are set;
(64) Respectively inputting a first target detection model and a second target detection model by taking a training set carrying nested marking data as input data, calculating the loss function values of the output data of the forward network and the marking data of the connecting area by using a loss function preset by the first target detection model, and calculating the loss function values of the output data of the forward network and the marking data of the small hardware defects by using a loss function preset by the second target detection model;
(65) Stopping the corresponding network model training when the loss function value converges or the iteration number reaches the preset maximum iteration number, and performing step (66), otherwise, performing weight update on the FasterRCNN network by using a corresponding weight update method, and repeating steps (64) and (65);
(66) After training is stopped, inputting verification set data into a FaterRCNN network to obtain an output result, counting the recall ratio and the precision ratio, if the recall ratio and the precision ratio meet preset conditions, finishing training, storing a trained target detection model, and otherwise, retraining.
As a further optimization of the above, the loss function of the second object detection model comprises a second class loss and a second location loss,
the second class loss employs a modified binary cross entropy loss function as follows:
wherein N represents the number of predicted frames output by the network, and P 0i To predict the probability value of the frame network output as a normal pin, P 1i To predict the probability value of the frame network output as a defect pin, y 0i =1 indicates that the prediction box is marked as a normal pin, y 0i =0 as abnormal pin, y 1i =1 indicates that the prediction box is marked as a defect pin, y 1i =0, denoted as non-defective pin, g 0 And g 1 Loss weight distribution of the normal pin and the defect pin is respectively shown;
the second position loss formula is as follows:
where M represents the number of positive samples of the network output, i.e. the tag value y 0i Or y 1i Number of prediction frames of 1, (x) i ,y i ) Representing the predicted box center position, (w) i ,h i ) Representing prediction block size, (x) i gt ,y i gt ) Representing the position of the center point of the marking frame, (w) i gt ,h i gt ) Representing the size of the annotation frame; l (L) 1 (x) The SmoothL1 function is used, with the following formula:
g i the weight assignment in the representation location loss is consistent with the weight assignment in the category loss as follows:
as further optimization of the scheme, the weight updating methods of the first target detection model and the second target detection model both adopt a random gradient descent method to optimize the loss function value.
The invention also provides a transmission line hardware defect detection system based on cascade target detection based on the transmission line hardware defect detection method based on cascade target detection, comprising the following steps:
the data acquisition module is used for acquiring the inspection shooting image and preprocessing the inspection shooting image;
the connection area detection module is used for detecting the connection area of the inspection shooting image by using the trained first target detection model, and cutting the detected rectangular connection area;
the connection region screening module is used for acquiring the area size of the connection regions and taking n connection regions with the area size meeting preset conditions as images to be identified;
the fine hardware defect detection module is used for detecting the fine hardware defect of the image to be identified by using the trained second target detection model, and acquiring coordinates of the fine hardware defect on the image to be identified;
and the fine hardware defects original image display module is used for displaying the fine hardware defects on the original image according to the mapping relation between the coordinates of the image to be identified and the original image coordinates.
The method is further optimized, and further comprises a data augmentation processing module, wherein the data augmentation processing module is used for carrying out data augmentation processing on the acquired power transmission line image before training the first target detection model and the second target detection model, and the image generated by the augmentation processing and the acquired original power transmission line image are used as a data set for training the first target detection model and the second target detection model.
As a further optimization of the above solution, the first target detection model and the second target detection model use a deep neural network based on a target detection algorithm.
The transmission line hardware defect detection method based on cascade target detection has the following beneficial effects:
1. according to the transmission line hardware defect detection method based on cascade target detection, for identification detection of small targets of fine and small hardware of the transmission line, as few samples can be collected in defect detection of the fine hardware, the situation that the effect is not ideal in test because the number of samples is increased by generating an countermeasure network for shooting images is avoided when deep learning is directly used for identification.
2. According to the transmission line hardware defect detection method based on cascade target detection, for the identification detection of small targets of fine hardware of a transmission line, a cascade target detection algorithm deep convolutional neural network is adopted, a first target detection model is adopted for identifying a larger target, namely a connection region, in a transmission line image shot by inspection, and the defect condition of the fine hardware is identified for the connection region through a second target detection model based on the identification result of the first target detection model, so that the characteristic extraction work of a large number of invalid image information is reduced, the network is focused on a key region, the loss of effective information of the image is greatly reduced, the quality of key pixel information is ensured, and the fine hardware defect detection precision is remarkably improved.
Drawings
Fig. 1 is a block diagram of an overall flow of fine hardware defect detection for a inspection image to be detected in a power transmission line hardware defect detection method based on cascade target detection;
fig. 2 is an overall flow chart of model parameter training for a countermeasure generation network model, a first target detection model and a second target detection model on sample data in the transmission line hardware defect detection method based on cascading target detection of the present invention;
FIG. 3 is a block flow diagram of training parameters of a challenge-generating network model for sample data in the cascaded object detection-based power transmission line hardware defect detection method of the present invention;
fig. 4 is an overall flow chart of model parameter training for a first target detection model and a second target detection model of sample data in the power transmission line hardware defect detection method based on cascading target detection of the present invention;
fig. 5 is a block diagram of an internal structure of the transmission line hardware defect detection system based on cascade target detection according to the present invention;
fig. 6 is a diagram of nested labeling of images in the method for detecting the defects of the power transmission line hardware based on cascading target detection;
fig. 7 is a schematic diagram of a fine hardware defect detected by using a second target detection model in the method for detecting a hardware defect of a power transmission line based on cascaded target detection according to the present invention;
fig. 8 is a diagram showing the fine hardware defects identified in fig. 7 in an original drawing in the method for detecting the hardware defects of the power transmission line based on the cascade target detection of the present invention.
Description of the embodiments
The invention will now be described in detail with reference to the drawings and specific examples.
Referring to fig. 1-8, the invention provides a method for detecting the defects of the hardware fitting of a power transmission line based on cascade target detection, which considers that the defect targets of the faults of the small hardware fitting in the power transmission line are too small and account for less than 5% of the original image proportion, and the traditional deep learning image recognition method has insufficient feature extraction capability, so that the defect detection effect of the small hardware fitting can not meet the actual service requirement.
In addition, in consideration of a large number of training samples required for image recognition processing by using a deep learning network model, defect detection of fine hardware mainly comprises defect categories such as pin missing and pin falling-out, but no matter which category, too few samples can be collected, fitting is easy to occur during deep learning, so that the effect is not ideal during testing.
The data augmentation processing in the scheme adopts a trained generation countermeasure network model, and the step of training the generation countermeasure network model is as follows:
(31) Establishing a generator and a discriminator, setting a loss function of the discriminator and generating an objective function of an countermeasure network; the loss function of the arbiter is:
l (P) = -y×ln (P) + (y-1) ×ln (1-P), where P is a probability value for determining network output; y represents a tag value, which takes on a value of 0 or 1;
the generation of the antagonism network objective function is:
V(D,G)＝E X～pd(x) lgD(x)+E X～pz(x) lg (1-D (x)), wherein,
x represents the input of the discrimination network, G represents the generation network, D represents the discrimination network, X-Pd (X) represents the distribution Pd (X) of the transmission line image data that X obeys the acquisition, namely, represents that X is from the transmission line image that is actually acquired, X-Pz (X) represents the distribution Pz (X) that X obeys the random noise signal z, namely, X is a generation sample, and E [. Cndot ] represents mathematical expectation.
(32) Inputting a random noise signal z into a generator to obtain a generated sample, marking the acquired power transmission line image and the generated sample, inputting the marked power transmission line image and the generated sample into a discriminator, discriminating a real sample and the generated sample, regulating network parameters of a discrimination network by using a back propagation algorithm, and maximizing a target function of a generated countermeasure network to obtain an optimized discrimination network;
(33) Substituting the obtained network parameters of the optimized discrimination network into a generated countermeasure network objective function, and adjusting the network parameters of the generated network by using a back propagation algorithm to minimize the generated countermeasure network objective function, so as to obtain the network parameters of the optimized discrimination network, and further obtain the optimized generated network;
(34) And (3) judging whether the iteration times reach the preset maximum iteration times, if so, repeating the steps (32) - (34), otherwise, stopping training, and storing the generated countermeasure model after training is completed.
And then calling a trained generation countermeasure network, performing data augmentation processing on the acquired transmission line image by using a generator in the generated countermeasure network to obtain a generated sample, and then using the generated sample generated by the augmentation processing and the acquired original transmission line image together as a data set for training a first target detection model and a second target detection model.
In this embodiment, the first target detection model and the second target detection model adopt a deep neural network based on a target detection algorithm, where the target detection algorithm includes but is not limited to FasterRCNN, FPN, yoloV3, and in this scheme, the training process for the first target detection model and the second target detection model includes:
(61) Nested labeling is carried out on a data set for training a model, referring to fig. 6, firstly, labeling is carried out on a power transmission line connection area in an image, and then labeling is carried out on whether a tiny hardware fitting has defects or not based on the connection area;
(62) Dividing data: dividing a training set and a verification set in proportion;
(63) Establishing a first target detection model and a second target detection model, wherein a forward network adopts a basic FasterRCNN network, and a corresponding loss function and a weight updating method are set;
(64) Respectively inputting a first target detection model and a second target detection model by taking a training set carrying nested marking data as input data, calculating the loss function values of the output data of the forward network and the marking data of the connecting area by using a loss function preset by the first target detection model, and calculating the loss function values of the output data of the forward network and the marking data of the small hardware defects by using a loss function preset by the second target detection model;
(65) Stopping the corresponding network model training when the loss function value converges or the iteration number reaches the preset maximum iteration number, and performing step (66), otherwise, performing weight update on the FasterRCNN network by using a corresponding weight update method, and repeating steps (64) and (65);
(66) After training is stopped, inputting verification set data into a FaterRCNN network to obtain an output result, counting the recall ratio and the precision ratio, if the recall ratio and the precision ratio meet preset conditions, finishing training, storing a trained target detection model, and otherwise, retraining.
Wherein the loss function of the first object detection model comprises a first class loss and a first position loss,
the first class loss employs a cross entropy loss function as follows:
wherein N represents the number of predicted frames output by the network, and defaults to 2000, p i Outputting probability value, y for network of connection area i A label value is indicated, 1 indicates that the region is marked as a connected region, and 0 indicates a background region.
The first position loss formula is as follows:
where M represents the number of positive samples of the network output, i.e. the tag value y i Number of prediction frames of 1, (x) i ,y i ) Representing the predicted box center position, (w) i ,h i ) Representing prediction block size, (x) i gt ,y i gt ) Representing the position of the center point of the marking frame, (w) i gt ,h i gt ) Representing the size of the annotation frame; l (L) 1 (x) The SmoothL1 function is used, with the following formula:
the loss function of the second object detection model includes a second class loss and a second location loss,
the second class loss employs a modified binary cross entropy loss function as follows:
wherein N represents the number of predicted frames output by the network, and defaults to 2000 and P 0i To predict the probability value of the frame network output as a normal pin, P 1i To predict the probability value of the frame network output as a defect pin, y 0i =1 indicates that the prediction box is marked as a normal pin, y 0i =0 as abnormal pin, y 1i =1 indicates that the prediction box is marked as a defect pin, y 1i =0, denoted as non-defective pin, g 0 And g 1 Loss weight assignment for normal and defective pins, respectively, g because the model focuses on defective pins 1 Greater than g 0 Default g 0 ＝0.7，g 1 ＝1.4。
The second position loss formula is as follows:
where M represents the number of positive samples of the network output, i.e. the tag value y 0i Or y 1i Number of prediction frames of 1, (x) i ,y i ) Representing the predicted box center position, (w) i ,h i ) Representing prediction block size, (x) i gt ,y i gt ) Representing the position of the center point of the marking frame, (w) i gt ,h i gt ) Representing the size of the annotation frame; l (L) 1 (x) The SmoothL1 function is used, with the following formula:
g i the weight assignment in the representation location loss is consistent with the weight assignment in the category loss as follows:
the weight updating method of the first target detection model and the weight updating method of the second target detection model are used for optimizing the loss function value by adopting a random gradient descent method.
When the transmission line fine hardware defect identification detection method based on cascading target detection is adopted, the following steps are adopted:
(11) Firstly, acquiring a patrol shooting image and preprocessing, wherein the patrol shooting image is normalized and denoised in the embodiment in consideration of the fact that the shooting image is input into a neural network model for recognition;
(12) Then using a trained first target detection model to detect a connection area of the inspection shooting image, and cutting out the detected rectangular connection area;
(13) Acquiring the area size of the connecting areas, and taking n connecting areas with the area size meeting preset conditions as images to be identified;
specifically, considering the study of the distribution characteristics of the defects of the inspection image, the connection area with the defects usually appears in a part with a larger foreground area, and other areas can be used as a background for neglecting treatment, so that the area size of the cut rectangular connection area is calculated through the acquired coordinates, the detected connection areas are ordered from large to small according to the area size, the first n connection areas after the ordering are selected as images to be identified, and in the embodiment, n=3, namely, the 3 connection areas with the largest area are adopted for fine hardware defect identification.
(14) Performing fine hardware defect detection on the image to be identified by using the trained second target detection model, and acquiring coordinates of the fine hardware defect on the image to be identified, see fig. 7;
(15) And displaying the fine hardware defects in the original image according to the mapping relation between the coordinates of the image to be identified and the coordinates of the original image, see fig. 8.
Referring to fig. 5, the invention further provides a transmission line hardware defect detection system based on cascade target detection based on the transmission line hardware defect detection method, which comprises the following steps:
the data acquisition module is used for acquiring the inspection shooting image and preprocessing the inspection shooting image;
the connection area detection module is used for detecting the connection area of the inspection shooting image by using the trained first target detection model, and cutting the detected rectangular connection area; the training data sets of the first target detection model and the second target detection model are based on the data sets generated after data augmentation processing is performed on the countermeasure network, the number of training samples of the network model is increased, the occurrence of the over-fitting phenomenon during deep learning of the target detection model is reduced, the accuracy of target detection of the network model is improved, and the first target detection model and the second target detection model adopt a deep neural network based on a target detection algorithm.
The connection region screening module is used for acquiring the area size of the connection regions and taking n connection regions with the area size meeting preset conditions as images to be identified;
the fine hardware defect detection module is used for detecting the fine hardware defect of the image to be identified by using the trained second target detection model, and acquiring coordinates of the fine hardware defect on the image to be identified;
and the fine hardware defects original image display module is used for displaying the fine hardware defects on the original image according to the mapping relation between the coordinates of the image to be identified and the original image coordinates.
And the data augmentation processing module is used for carrying out data augmentation processing on the acquired power transmission line image before training the first target detection model and the second target detection model by adopting the generation countermeasure network, and taking the image generated by the augmentation processing and the acquired original power transmission line image together as a data set for training the first target detection model and the second target detection model.
The present invention is not limited to the above-described specific embodiments, and various modifications may be made by those skilled in the art without inventive effort from the above-described concepts, and are within the scope of the present invention.
Claims (10)
1. The power transmission line hardware defect detection method based on cascade target detection is characterized by comprising the following steps of: comprising the following steps:
(11) Acquiring a patrol shooting image and preprocessing;
(12) Using a trained first target detection model to detect a connection area of the inspection shooting image, and cutting out the detected rectangular connection area;
(13) Acquiring the area size of the connecting areas, and taking n connecting areas with the area size meeting preset conditions as images to be identified;
(14) Performing fine hardware defect detection on the image to be identified by using a trained second target detection model, and obtaining coordinates of the fine hardware defect on the image to be identified;
(15) And displaying the fine hardware defects in the original image according to the mapping relation between the coordinates of the image to be identified and the coordinates of the original image.
2. The method for detecting the defects of the power transmission line hardware based on the cascading target detection according to claim 1, wherein the method comprises the following steps: before the first target detection model and the second target detection model are trained, data augmentation processing is carried out on the acquired power transmission line images, and the image generated by the augmentation processing and the acquired original power transmission line images are used as a data set for training the first target detection model and the second target detection model.
3. The method for detecting the defects of the power transmission line hardware based on the cascading target detection according to claim 2, wherein the method comprises the following steps: the data augmentation processing adopts a trained generation countermeasure network model, and the step of generating the countermeasure network model training is as follows:
(31) Establishing a generator and a discriminator, setting a loss function of the discriminator and generating an objective function of an countermeasure network;
(32) Inputting a random noise signal z into a generator to obtain a generated sample, marking the acquired power transmission line image and the generated sample, inputting the marked power transmission line image and the generated sample into a discriminator, discriminating a real sample and the generated sample, regulating network parameters of a discrimination network by using a back propagation algorithm, and maximizing a target function of a generated countermeasure network to obtain an optimized discrimination network;
(33) Substituting the obtained network parameters of the optimized discrimination network into a generated countermeasure network objective function, and adjusting the network parameters of the generated network by using a back propagation algorithm to minimize the generated countermeasure network objective function, so as to obtain the network parameters of the optimized discrimination network, and further obtain the optimized generated network;
(34) And (3) judging whether the iteration times reach the preset maximum iteration times, if so, repeating the steps (32) - (34), otherwise, stopping training, and storing the generated countermeasure model after training is completed.
4. The method for detecting the defects of the power transmission line hardware based on the cascading target detection according to claim 2, wherein the method comprises the following steps: the first target detection model and the second target detection model adopt a deep neural network based on a target detection algorithm.
5. The method for detecting the defects of the power transmission line hardware based on the cascading target detection according to claim 4, wherein the method comprises the following steps: the first target detection model and the second target detection model adopt a deep neural network based on a FaterRCNN algorithm, and the training process of the FaterRCNN network model comprises the following steps:
(51) Nesting and labeling a data set for training a model, firstly labeling a power transmission line connection region in an image, and then labeling whether a tiny hardware fitting has a defect or not based on the connection region;
(52) Dividing data: dividing a training set and a verification set in proportion;
(53) Establishing a first target detection model and a second target detection model, wherein a forward network adopts a basic FasterRCNN network, and a corresponding loss function and a weight updating method are set;
(54) Respectively inputting a first target detection model and a second target detection model by taking a training set carrying nested marking data as input data, calculating the loss function values of the output data of the forward network and the marking data of the connecting area by using a loss function preset by the first target detection model, and calculating the loss function values of the output data of the forward network and the marking data of the small hardware defects by using a loss function preset by the second target detection model;
(55) Stopping the corresponding network model training when the loss function value converges or the iteration number reaches the preset maximum iteration number, and performing step (56), otherwise, performing weight update on the FasterRCNN network by using a corresponding weight update method, and repeating steps (54) and (55);
(56) After training is stopped, inputting verification set data into a FaterRCNN network to obtain an output result, counting the recall ratio and the precision ratio, if the recall ratio and the precision ratio meet preset conditions, finishing training, storing a trained target detection model, and otherwise, retraining.
6. The method for detecting the defects of the power transmission line hardware based on the cascading target detection according to claim 5, wherein the method comprises the following steps: the loss function of the second object detection model includes a second class loss and a second location loss,
the second class loss employs a modified binary cross entropy loss function as follows:
wherein N represents the number of predicted frames output by the network, and P 0i To predict the probability value of the frame network output as a normal pin, P 1i To predict the probability value of the frame network output as a defect pin, y 0i =1 indicates that the prediction box is marked as a normal pin, y 0i =0 as abnormal pin, y 1i =1 indicates that the prediction box is marked as a defect pin, y 1i =0, denoted as non-defective pin, g 0 And g 1 Loss weight distribution of the normal pin and the defect pin is respectively shown;
the second position loss formula is as follows:
where M represents the number of positive samples of the network output, i.e. the tag value y 0i Or y 1i Number of prediction frames of 1, (x) i ,y i ) Representing the predicted box center position, (w) i ,h i ) Representing prediction block size, (x) i gt ,y i gt ) Representing the position of the center point of the marking frame, (w) i gt ,h i gt ) Representing the size of the annotation frame; l (L) 1 (x) The SmoothL1 function is used, with the following formula:
g i the weight assignment in the representation location loss is consistent with the weight assignment in the category loss as follows:
7. the method for detecting the defects of the power transmission line hardware based on the cascading target detection according to claim 5, wherein the method comprises the following steps: the weight updating methods of the first target detection model and the second target detection model both adopt a random gradient descent method to optimize the loss function value.
8. The utility model provides a transmission line gold utensil defect detection system based on cascade target detection which characterized in that: comprising the following steps:
the data acquisition module is used for acquiring the inspection shooting image and preprocessing the inspection shooting image;
the connection area detection module is used for detecting the connection area of the inspection shooting image by using the trained first target detection model, and cutting the detected rectangular connection area;
the connection region screening module is used for acquiring the area size of the connection regions and taking n connection regions with the area size meeting preset conditions as images to be identified;
the fine hardware defect detection module is used for detecting the fine hardware defect of the image to be identified by using the trained second target detection model, and acquiring coordinates of the fine hardware defect on the image to be identified;
and the fine hardware defects original image display module is used for displaying the fine hardware defects on the original image according to the mapping relation between the coordinates of the image to be identified and the original image coordinates.
9. The cascading object detection-based transmission line hardware defect detection system as claimed in claim 8, wherein: the system further comprises a data augmentation processing module, wherein the data augmentation processing module is used for carrying out data augmentation processing on the acquired power transmission line image before the first target detection model and the second target detection model are trained, and the image generated by the augmentation processing and the acquired original power transmission line image are used as a data set for training the first target detection model and the second target detection model together.
10. The cascading object detection-based transmission line hardware defect detection system as claimed in claim 7, wherein: the first target detection model and the second target detection model adopt a deep neural network based on a target detection algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010393940.7A CN111650204B (en) | 2020-05-11 | 2020-05-11 | Power transmission line hardware defect detection method and system based on cascade target detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010393940.7A CN111650204B (en) | 2020-05-11 | 2020-05-11 | Power transmission line hardware defect detection method and system based on cascade target detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111650204A CN111650204A (en) | 2020-09-11 |
| CN111650204B true CN111650204B (en) | 2023-05-12 |
Family
ID=72346054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010393940.7A Active CN111650204B (en) | 2020-05-11 | 2020-05-11 | Power transmission line hardware defect detection method and system based on cascade target detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111650204B (en) |
Families Citing this family (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112419680B (en) * | 2020-11-19 | 2022-09-27 | 中国南方电网有限责任公司超高压输电公司检修试验中心 | Power transmission line potential safety hazard classification and identification method and system |
| CN112508905A (en) * | 2020-12-01 | 2021-03-16 | 广东电网有限责任公司 | Hardware rust image detection method and computer readable storage medium |
| CN112581445A (en) * | 2020-12-15 | 2021-03-30 | 中国电力科学研究院有限公司 | Detection method and device for bolt of power transmission line, storage medium and electronic equipment |
| CN112683916A (en) * | 2020-12-17 | 2021-04-20 | 华能新能源股份有限公司云南分公司 | Method and device for identifying missing or mounting error of small hardware fittings of current collecting line tower |
| CN112669264B (en) * | 2020-12-17 | 2024-07-30 | 国网山西省电力公司运城供电公司 | Artificial intelligent defect identification method and system for unmanned aerial vehicle inspection of distribution network line |
| CN112712060A (en) * | 2021-01-15 | 2021-04-27 | 国网浙江省电力有限公司电力科学研究院 | Automatic high-voltage post porcelain insulator acoustic signal identification method |
| CN112837281B (en) * | 2021-01-27 | 2022-10-28 | 湘潭大学 | Pin defect identification method, device and equipment based on cascade convolution neural network |
| CN112975953B (en) * | 2021-01-28 | 2022-06-03 | 珠海迪沃航空工程有限公司 | Manipulator motion trajectory planning method and bolt grabbing detection system |
| CN112926401A (en) * | 2021-01-29 | 2021-06-08 | 广州中科智巡科技有限公司 | Transmission line hardware corrosion detection method and system |
| CN114943675A (en) * | 2021-02-09 | 2022-08-26 | 富泰华工业（深圳）有限公司 | Defect sample data augmentation method, electronic device, and storage medium |
| CN113012107B (en) * | 2021-02-22 | 2022-07-08 | 江苏方天电力技术有限公司 | Power grid defect detection method and system |
| CN113052809B (en) * | 2021-03-18 | 2021-12-10 | 中科海拓(无锡)科技有限公司 | EfficientNet-based nut surface defect classification method |
| CN113052125B (en) * | 2021-04-09 | 2022-10-28 | 内蒙古科电数据服务有限公司 | Construction site violation image recognition and alarm method |
| CN113327255A (en) * | 2021-05-28 | 2021-08-31 | 宁波新胜中压电器有限公司 | Power transmission line inspection image processing method based on YOLOv3 detection, positioning and cutting and fine-tune |
| CN113469951B (en) * | 2021-06-08 | 2022-08-16 | 燕山大学 | Hub defect detection method based on cascade region convolutional neural network |
| CN113537341A (en) * | 2021-07-14 | 2021-10-22 | 安徽炬视科技有限公司 | Online monitoring device and identification method for line hidden danger based on big data and self-learning |
| CN114049305B (en) * | 2021-10-15 | 2024-07-16 | 三峡大学 | Distribution line pin defect detection method based on improved ALI and fast-RCNN |
| CN114202526A (en) * | 2021-12-10 | 2022-03-18 | 北京百度网讯科技有限公司 | Quality detection method, system, apparatus, electronic device, and medium |
| CN113903009B (en) * | 2021-12-10 | 2022-07-05 | 华东交通大学 | Railway foreign matter detection method and system based on improved YOLOv3 network |
| CN114155246B (en) * | 2022-02-10 | 2022-06-14 | 国网江西省电力有限公司电力科学研究院 | Deformable convolution-based power transmission tower pin defect detection method |
| CN114397306B (en) * | 2022-03-25 | 2022-07-29 | 南方电网数字电网研究院有限公司 | Power grid grading ring hypercomplex category defect multi-stage model joint detection method |
| CN114782679B (en) * | 2022-05-05 | 2024-08-06 | 国家电网有限公司 | Cascade network-based method and device for detecting hardware defects in power transmission line |
| CN114882010A (en) * | 2022-06-09 | 2022-08-09 | 苏州大学 | Surface defect detection method based on picture recognition |
| CN115619763A (en) * | 2022-10-31 | 2023-01-17 | 国网湖北省电力有限公司超高压公司 | Power transmission line small part image target detection method based on multistage reasoning detection |
| CN116228778B (en) * | 2023-05-10 | 2023-09-08 | 国网山东省电力公司菏泽供电公司 | Insulator rupture detection method and system based on multi-mode information fusion |
| CN116843686B (en) * | 2023-08-31 | 2023-11-10 | 成都考拉悠然科技有限公司 | Method and device for detecting defects of wire clamps and nuts of contact net locator |
| CN116883391B (en) * | 2023-09-05 | 2023-12-19 | 中国科学技术大学 | Two-stage distribution line defect detection method based on multi-scale sliding window |
| CN117974566B (en) * | 2023-12-29 | 2024-09-10 | 北京建筑大学 | Transmission line key component defect detection method and system and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016078081A1 (en) * | 2014-11-21 | 2016-05-26 | 河南送变电工程公司 | Multi-rotor-wing inspection aircraft and power transmission line inspection system |
| CN109344753A (en) * | 2018-09-21 | 2019-02-15 | 福州大学 | A kind of tiny fitting recognition methods of Aerial Images transmission line of electricity based on deep learning |
-
2020
- 2020-05-11 CN CN202010393940.7A patent/CN111650204B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016078081A1 (en) * | 2014-11-21 | 2016-05-26 | 河南送变电工程公司 | Multi-rotor-wing inspection aircraft and power transmission line inspection system |
| CN109344753A (en) * | 2018-09-21 | 2019-02-15 | 福州大学 | A kind of tiny fitting recognition methods of Aerial Images transmission line of electricity based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 邹捷 ; 许瑞庆 ; 习雨同 ; .深度学习在输电线路工程验收智能缺陷识别中的应用.江西电力.2020,(第02期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111650204A (en) | 2020-09-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111650204B (en) | Power transmission line hardware defect detection method and system based on cascade target detection | |
| US11488294B2 (en) | Method for detecting display screen quality, apparatus, electronic device and storage medium | |
| CN111339882B (en) | Power transmission line hidden danger detection method based on example segmentation | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN110569841B (en) | Contact gateway key component target detection method based on convolutional neural network | |
| CN113095438B (en) | Wafer defect classification method, device and system thereof, electronic equipment and storage medium | |
| CN110992349A (en) | Underground pipeline abnormity automatic positioning and identification method based on deep learning | |
| WO2022262054A1 (en) | Image processing method, apparatus and device, and storage medium | |
| CN117635565B (en) | Semiconductor surface defect detection system based on image recognition | |
| CN112967255A (en) | Shield segment defect type identification and positioning system and method based on deep learning | |
| CN112528979A (en) | Transformer substation inspection robot obstacle distinguishing method and system | |
| CN111695493B (en) | Method and system for detecting hidden danger of power transmission line | |
| CN111199213A (en) | Equipment defect detection method and device for transformer substation | |
| CN115439458A (en) | Industrial image defect target detection algorithm based on depth map attention | |
| CN116978834B (en) | Intelligent monitoring and early warning system for wafer production | |
| CN113487570A (en) | High-temperature continuous casting billet surface defect detection method based on improved yolov5x network model | |
| CN114331997A (en) | Wire defect monitoring method based on image recognition | |
| CN113888462A (en) | Crack identification method, system, readable medium and storage medium | |
| Su et al. | A defect detection method of gear end-face based on modified YOLO-V3 | |
| Lyu et al. | Artificial intelligence‐assisted fatigue fracture recognition based on morphing and fully convolutional networks | |
| CN112381794B (en) | Printing defect detection method based on deep convolution generation network | |
| CN113506290A (en) | Method and device for detecting defects of line insulator | |
| CN113421223A (en) | Industrial product surface defect detection method based on deep learning and Gaussian mixture | |
| CN117114420A (en) | Image recognition-based industrial and trade safety accident risk management and control system and method | |
| CN111626104A (en) | Cable hidden danger point detection method and device based on unmanned aerial vehicle infrared thermal imagery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |