CN111563893A - Grading ring defect detection method, device, medium and equipment based on aerial image - Google Patents
Grading ring defect detection method, device, medium and equipment based on aerial image Download PDFInfo
- Publication number
- CN111563893A CN111563893A CN202010395305.2A CN202010395305A CN111563893A CN 111563893 A CN111563893 A CN 111563893A CN 202010395305 A CN202010395305 A CN 202010395305A CN 111563893 A CN111563893 A CN 111563893A
- Authority
- CN
- China
- Prior art keywords
- training
- grading ring
- image
- data
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Abstract
The invention provides a method, a device, a medium and equipment for detecting defects of a grading ring based on aerial images, wherein the method comprises the following steps: marking the aerial images, classifying and storing the marked image data, and establishing a defect data set of the grading ring, wherein the defect data set comprises a training set and a testing set; performing neural network training by using the image data in the training set to generate a training model; and evaluating the training model by using the test set, and detecting the grading ring defects of the aerial image of the inspection power transmission line by using the training model after the evaluation precision result meets the requirement. The method of the invention improves the Faster R-CNN network by applying methods of data enhancement, online difficult sample mining, mitigation non-maximum inhibition algorithm, AdaGrad adaptive gradient algorithm and the like. The invention also selects ResNet-152 to replace the VGG16 feature extraction network of the original Faster R-CNN to improve the defect identification rate of the grading ring. The method can be suitable for detecting the faults of the grading rings of various types of power transmission lines.
Description
Technical Field
The invention relates to the technical field of image detection and identification, in particular to a method, a device, a medium and equipment for detecting defects of an equalizing ring based on aerial images.
Background
The grading ring is used as an important component of the power transmission line, and regular inspection of the grading ring is an important measure for guaranteeing safe operation of the power system. At present, the equalizing ring inspection in China is mainly carried out in a manual mode, and the unmanned aerial vehicle aerial image analysis is assisted. The traditional digital image processing is adopted for detecting the edge of the grading ring and matching the local shape contour, the characteristics are manually designed, algorithm parameters are set according to the environment, the accuracy is low, the algorithm applicability is not strong, and the grading ring cannot be positioned.
For detecting the grading ring, a common method at present is to identify through local contour features. The shape profile can be decomposed into simple profiles such as straight lines, curved lines or randomly broken segments, and the local shape information can be described by a single segment or a combination of adjacent segments.
The existing detection method utilizes a splitting and combining method to construct a curve outline of an expandable end point, describes the self outline property of the curve and the geometric relationship between the curves through the unitary descriptor and the binary descriptor of the circular arc, and uses the attribute relational graph of the model to identify the target.
In the prior art, the curvature of a target contour is obtained through angular point detection, the target is split into main contour segments at a high curvature point, a target model is established according to the main contour segments, and then all contour segments of the target are detected.
The prior art manually segments the object into overlapping contour segments, and then detects and combines the contour segments in the image using a particle filtering method. In a complex natural environment, a local contour may be shielded and deformed, so that a single contour segment is too large in change, and local information of a shape cannot be well represented.
The method is a relatively mature algorithm applied in the process of detecting the grading ring at present, but the detection result shows that the detection contour of the grading ring by adopting the method is incomplete, and the grading ring cannot be accurately positioned. The requirements on the background environment and the shooting angle are high, the shape of the equalizing ring with definite curve relation characteristics is aimed at, and the aim that the circular equalizing ring commonly adopted in the current electric power system in China cannot achieve detection is aimed at.
Disclosure of Invention
The method aims to solve the technical problems that in the prior art, the prior digital image processing is adopted to carry out edge detection and local shape contour matching of the grading ring, characteristics need to be designed manually, algorithm parameters are set aiming at the environment, so that the accuracy is low, the algorithm applicability is not strong, and the grading ring cannot be positioned. The invention provides a method, a device, a medium and equipment for detecting defects of a grading ring based on aerial images.
In a first aspect, the invention provides an aerial image-based grading ring defect detection method, which comprises the following steps:
marking the aerial images, classifying and storing the marked image data, and establishing a defect data set of the grading ring, wherein the defect data set comprises a training set and a testing set;
performing neural network training by using the image data in the training set to generate a training model;
and evaluating the training model by using the test set, and detecting the grading ring defects of the aerial image of the inspection power transmission line by using the training model after the evaluation precision result meets the requirement.
The invention has the beneficial effects that:
the method for detecting the defects of the grading ring of the power transmission line based on the aerial images solves the technical problems that the inspection difficulty is high, the period is long and the inspection result is greatly influenced by the skills of inspection personnel and objective factors such as weather, terrain and the like mainly depending on manual inspection and visual inspection in the prior art. The method for detecting the defects of the grading ring of the power transmission line based on the aerial images improves the Faster R-CNN network by applying Data Augmentation (Data Augmentation), Online Hard sample Mining (OHEM), soft-Non-Maximum Suppression (soft-NMS) algorithm, AdaGrad adaptive gradient algorithm and other methods. The invention also selects ResNet-152 to replace the VGG16 feature extraction network of the original FasterR-CNN to improve the defect identification rate of the grading ring. The method provided by the invention is more convenient, safer and more efficient than manual inspection, and can be suitable for detecting the faults of the grading rings of aerial images of power transmission lines with various types and scales.
Further, the defective result specifically includes one of the following three types of defective results:
damage to the grading ring, inclination of the grading ring and falling off of the grading ring.
Further, the performing neural network training using the image data in the training set to generate a training model specifically includes:
extracting the characteristics of the grading rings in the aerial images in the test set to obtain a plurality of characteristic blocks;
inputting the extracted feature block into a regional candidate network RPN; to obtain a region suggestion box;
each region suggestion frame comprises identified semantic features containing the grading rings;
mapping the semantic features in the region suggestion box through a region pooling layer to output feature information with the same size, and inputting the feature information into a full connection layer;
and the output of the full connection layer obtains accurate category prediction and target positioning through a classification function softmax and a function of frame regression bbox, and eliminates prediction targets at similar positions and in the same category through a relaxation non-maximum suppression algorithm to obtain a grading ring prediction frame.
Further, before obtaining the plurality of feature blocks, the method further includes:
and increasing the number of the shot image samples by data enhancement, and taking the images with increased number as the aerial images.
The beneficial effect of adopting the further scheme is that:
because the number of model training samples is small, the overfitting phenomenon is easy to occur, and the problem of the overfitting phenomenon caused by the small number of model training samples is effectively avoided after data enhancement is introduced.
Further, the increasing the number of captured image samples with data enhancement is achieved by at least one of:
gaussian noise data enhancement, salt-and-pepper noise data enhancement, speckle noise data enhancement, mean filtered data enhancement, gaussian filtered data enhancement, contrast enhanced filtered data enhancement, luminance transform data enhancement, and/or scale scaling data enhancement.
Further, the generated training model is an ROI network model;
before the pooling operation, the method further comprises the following steps: the region suggestion box is used for carrying out model training on the ROI network model by an online difficult sample Mining Onlinehard sample Mining method.
The beneficial effect of adopting the further scheme is that:
the method solves the technical problems that tens of thousands of candidate areas are generated in the model training process, and the large part of the candidate areas are backgrounds, so that the ratio of the number of negative samples to the number of target areas is overlarge, the data is unbalanced, and the learning iteration times of the positive sample category in the model are less, and the classification and the positioning of the network training are not facilitated. The Online Hard sample Mining (OHEM) method provides that most background areas and areas easy to detect and identify have relatively high performance in prediction type precision, and the loss is relatively small, and the Online Hard sample Mining (OHEM) method sets the weight with small loss to 0 in the model training process, so that the positive sample area with large loss can be further trained, and the model training effect is improved.
Further, the obtaining of the semantic features and the obtaining of the recognition result through the convolutional neural network further include:
and removing other homogeneous target probability scores near the target frame with higher prediction probability by using a soft-Non-Maximum Suppression (soft-NMS) algorithm.
In the fast R-CNN algorithm framework, the non-maximum suppression algorithm removes other similar objects near the object frame with higher prediction probability, and thus may cause missed detection of similar objects with close distance or overlapping. A soft-Non-Maximum Suppression (soft-NMS) algorithm is improved, only the probability scores of other similar targets near a target frame with higher prediction probability are reduced, and the reduction range is determined by the proximity degree of the targets with high probability, so that the prediction accuracy of the model when the dense targets are predicted is improved. The degree of proximity between the similar target and the high-probability target is defined by an Intersection-Over-Union (IOU).
The beneficial effect of adopting the further scheme is that:
and only reducing the probability scores of other similar targets near a target frame with higher prediction probability by adopting a soft-Non-Maximum-Suppression (soft-NMS) algorithm, wherein the reduction amplitude is determined by the proximity of the high-probability target, so that the prediction accuracy of the model when a denser target is predicted can be improved.
In a second aspect, the present invention provides an apparatus for detecting defects of a grading ring based on an aerial image, including:
the marking module is used for marking the aerial image, classifying and storing the marked image data and establishing a defect data set of the grading ring, wherein the defect data set comprises a training set and a testing set;
the training module is used for carrying out neural network training by utilizing the image data in the training set to generate a training model;
and the output module is used for evaluating the training model by using the test set, and detecting the grading ring defects of the aerial images of the inspection power transmission line by using the training model after the evaluation precision result meets the requirement.
Further, still include:
and the enhancement module is used for increasing the number of the shot image samples by utilizing data enhancement.
Further, the enhancement module is implemented by at least one of the following methods:
gaussian noise data enhancement, salt-and-pepper noise data enhancement, speckle noise data enhancement, mean filtered data enhancement, gaussian filtered data enhancement, contrast enhanced filtered data enhancement, luminance transform data enhancement, and/or scale scaling data enhancement.
Further, the training module performs model training on the RPN network model through an Online Hard sample Mining (OHEM) method.
Further, still include:
and the removing module is used for removing other similar target probability scores near the target frame with higher prediction probability by using a soft-Non-Maximum Suppression (soft-NMS) algorithm.
In a third aspect, the invention provides an electronic device, which includes a processor, and the processor is configured to execute any one of the above methods for detecting defects in an aerial image-based grading ring.
In a fourth aspect, a computer-readable storage medium stores a computer program, and after the program runs, the program controls the processor to execute any one of the above methods for detecting defects in a grading ring based on an aerial image.
Drawings
FIG. 1 is a schematic flow chart of a power transmission line grading ring defect detection method based on aerial images;
FIG. 2 is a schematic flow chart of a model training process of the power transmission line grading ring defect detection method based on aerial images;
FIG. 3 is a schematic flow chart of the method for detecting the grading ring defect of the power transmission line based on the aerial image;
FIG. 4 is a schematic diagram of two configurations of a ResNet network used in the present invention;
fig. 5 is a schematic structural diagram of the power transmission line grading ring defect detection device based on aerial images.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular equipment structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
As shown in fig. 1, the invention provides an aerial image-based grading ring defect detection method, which includes:
s1: marking the aerial images, classifying and storing the marked image data, and establishing a defect data set of the grading ring, wherein the defect data set comprises a training set and a testing set;
s2: performing neural network training by using the image data in the training set to generate a training model;
s3: and evaluating the training model by using the test set, and detecting the grading ring defects of the aerial image of the inspection power transmission line by using the training model after the evaluation precision result meets the requirement.
The invention has the beneficial effects that:
the defect detection method for the grading ring of the power transmission line based on the aerial image solves the technical problems that in the prior art, the edge detection and the local shape contour matching of the grading ring are carried out by adopting the traditional digital image processing, the characteristics need to be manually designed, the algorithm parameters are set aiming at the environment, the accuracy is low, the algorithm applicability is not strong, and the positioning of the grading ring cannot be realized. The invention also selects ResNet-152 to replace the VGG16 feature extraction network of the original Faster R-CNN to improve the defect identification rate of the grading ring. The method provided by the invention is more convenient, safer and more efficient than manual inspection, and can be suitable for detecting the faults of the grading rings of aerial images of power transmission lines with various types and scales.
In the method, firstly, a fault data set of the grading ring is established by adopting an artificial standard, and the fault types of the grading ring mainly comprise three types: damage to the grading ring, inclination of the grading ring and falling off of the grading ring. The method comprises the steps of increasing the data quantity of an original data set by using a data enhancement method, dividing the data set into a training set and a testing set, inputting the training set into a fast R-CNN network for model training, improving the training effect by adopting an Online Hard sample Mining (OHEM) method during model training, improving the precision of model prediction by using soft-Non-Maximum Suppression (soft-NMS) at the stage of obtaining a specific prediction frame, and finally evaluating the trained model by using the testing set.
The aerial images used by the invention are collected by the unmanned aerial vehicle, the pictures with defects are manually screened from the unmanned aerial vehicle polling images, 7500 pictures with defects are screened from the unmanned aerial vehicle polling images, the voltages of the screened polling pictures comprise four voltages of 35kV, 110kV, 220kV and 500kV, the resolution ratio of the images is 4288 x 2848, and the season covers four seasons.
As shown in fig. 2, in some demonstrative embodiments, S2 may include:
s2.1: extracting the characteristics of the grading rings in the aerial images in the test set to obtain a plurality of characteristic blocks;
s2.2: inputting the extracted feature block into a regional candidate network RPN; to obtain a region suggestion box;
s2.3: each region suggestion frame comprises identified semantic features containing the grading rings;
s2.4: mapping the semantic features in the region suggestion box through a region pooling layer to output feature information with the same size, and inputting the feature information into a full connection layer;
s2.5: and the output of the full connection layer obtains accurate category prediction and target positioning through a classification function softmax and a function of frame regression bbox, and eliminates prediction targets at similar positions and in the same category through a relaxation non-maximum suppression algorithm to obtain a grading ring prediction frame.
As shown in FIG. 3, the principle of the Faster R-CNN detection used in the present invention is explained in detail below:
firstly, a series of Feature blocks (Feature Maps) are obtained from aerial pictures through a Feature extraction Network, and the Feature Maps and a subsequently used Region suggestion Network (RPN) and a Faster R-CNN Network share parameter.
Inputting the features obtained in the last step into the RPN network to obtain a series of region suggestion boxes. The RPN network enables the network to predict the target suggestion box through training of the training set, namely, a neural network is used for predicting the generation of the region suggestion box instead of a selective search method (selective search) of traditional image processing.
Then, after a series of region suggestion boxes are obtained, semantic features corresponding to the region suggestion boxes are mapped through a region pooling layer (ROI posing) to output feature information of the same size, and then the feature information is input to a final full-link layer.
And finally, obtaining accurate class prediction and target positioning through the output of the full connection layer through classification functions softmax and a frame regression function of bbox, and finally, proposing prediction targets with similar positions and the same class through a Non-Maximum Suppression (NMS) algorithm to obtain a final target prediction frame.
The method for detecting the defects of the grading ring based on the aerial image also adopts ResNet to carry out model training, and ResNet has the advantages that the idea of residual error network learning is provided, the problems of gradient explosion and disappearance of the network are avoided to a certain extent by directly splicing input data and output data together, and the learning process and difficulty are simplified. As shown in FIG. 4 for both structures of ResNet, ResNet-152 uses a stack of three layers instead of the previous two layers. These three layers use convolutions of 1 × 1,3 × 3, and 1 × 1, respectively. In which 1 × 1 convolution is used to reduce and then raise dimensions, i.e. the problem of different dimensions is solved by using 1 × 1 convolution, and this structure can indeed improve its performance. On the general data set, the identification accuracy of ResNet-152 is higher than that of VGG16, and the bit sequence is first, so that ResNet-152 is selected to replace a VGG16 feature extraction network of original Faster R-CNN to improve the defect identification rate of the grading ring.
The method for detecting the defects of the grading ring based on the aerial image also adopts an AdaGrad (adaptive gradient) adaptive gradient algorithm, and the AdaGrad adaptive gradient algorithm is an improved random gradient descent algorithm. In the prior art algorithm, each parameter uses the same learning rate α. The AdaGrad algorithm can automatically adjust the learning rate in training, wherein parameters with low occurrence frequency are updated by using a larger alpha, and parameters with high occurrence frequency are updated by using a smaller alpha.
For the classic SGD optimization method, the update process of the parameter θ is as follows:
sampling from training set containing m samples { x(1)，...，x(m)Small batch with the corresponding target y(i)Oa is the learning rate, α is the momentum parameter, theta is the initial parameter, v is the initial velocity,
representing the gradient of the ith parameter.
Calculating a gradient estimate:
and (3) updating the calculation speed:
v←αv-òg
application updating:
θ←θ+v
the updating process of the AdaGrad algorithm is as follows:
is small constant, is set to about 10 for numerical stability-7R is a gradient accumulation variable, the initial value of r is 0, ⊙ represents element-by-element multiplication
Calculating the gradient:
cumulative squared gradient:
r←r+g⊙g
and (3) calculating and updating:
application updating:
θ←θ+Δθ
in the SGD algorithm, as the gradient increases, the learning step size increases. In the AdaGrad algorithm, r is larger and smaller as the algorithm is iterated, and the overall learning rate is smaller and smaller. The AdaGrad algorithm starts with incentive convergence and becomes slow and slow towards penalty convergence.
In the actual training process, the learning rate is expected to be slower and slower as the updating times are increased. Since the optimal solution is far from the loss function in the initial stage of the learning rate and gets closer to the optimal solution as the number of updates increases, the learning rate should also be slow. Therefore, the defect identification rate of the grading ring is improved by applying an SGD optimization strategy of replacing the original Faster R-CNN with AdaGrad.
In some illustrative embodiments, the defective result specifically includes one of three types of defective results:
damage to the grading ring, inclination of the grading ring and falling off of the grading ring.
The damage of the equalizing ring refers to the damage of hardware of the equalizing ring, such as cracks, on the equalizing ring.
The inclination of the grading ring refers to the defect generated by sound production offset of the mounting position of the grading ring compared with a standard position.
The dropping of the grading ring refers to the defect that the grading ring is lost due to the dropping of the grading ring.
In some demonstrative embodiments, obtaining the plurality of feature blocks may further include:
and increasing the number of the shot image samples by data enhancement, and taking the images with increased number as the aerial images.
The beneficial effect who adopts above-mentioned scheme is:
because the number of model training samples is small, the overfitting phenomenon is easy to occur, and the problem of the overfitting phenomenon caused by the small number of model training samples is effectively avoided after data enhancement is introduced.
In some illustrative embodiments, the increasing the number of captured image samples with data enhancement is accomplished by at least one of:
gaussian noise data enhancement, salt-and-pepper noise data enhancement, speckle noise data enhancement, mean filtered data enhancement, gaussian filtered data enhancement, contrast enhanced filtered data enhancement, luminance transform data enhancement, and/or scale scaling data enhancement. The salt and pepper noise data enhancement is also referred to as impulse noise data enhancement.
In some demonstrative embodiments, the generating of the training model is an ROI network model;
before the pooling operation, the method further comprises the following steps: the region suggestion box is used for carrying out model training on the ROI network model by an online difficult sample Mining Onlinehard sample Mining method.
The beneficial effect who adopts above-mentioned scheme is:
the method solves the technical problems that tens of thousands of candidate areas are generated in the model training process, and the large part of the candidate areas are backgrounds, so that the ratio of the number of negative samples to the number of target areas is overlarge, the data is unbalanced, and the learning iteration times of the positive sample category in the model are less, and the classification and the positioning of the network training are not facilitated. The Online Hard sample Mining (OHEM) method provides that most background areas and areas easy to detect and identify have relatively high performance in prediction type precision, and the loss is relatively small, and the Online Hard sample Mining (OHEM) method sets the weight with small loss to 0 in the model training process, so that the positive sample area with large loss can be further trained, and the model training effect is improved.
In some illustrative embodiments, before the mapping, by the region pooling layer, the semantic features corresponding to the obtained region suggestion boxes to output feature information of the same size, and outputting a final recognition result via the full connection layer, the method further includes:
and removing other homogeneous target probability scores near the target frame with higher prediction probability by using a soft-Non-Maximum Suppression (soft-NMS) algorithm.
The beneficial effect who adopts above-mentioned scheme is:
and only reducing the probability scores of other similar targets near a target frame with higher prediction probability by adopting a soft-Non-Maximum-Suppression (soft-NMS) algorithm, wherein the reduction amplitude is determined by the proximity of the high-probability target, so that the prediction accuracy of the model when a denser target is predicted can be improved.
As shown in fig. 5, the present invention further provides an apparatus for detecting defects of an equalizing ring based on an aerial image, including:
the marking module 100 is used for marking the aerial image, classifying and storing the marked image data, and establishing a defect data set of the grading ring, wherein the defect data set comprises a training set and a testing set;
a training module 200, configured to perform neural network training using the image data in the training set to generate a training model;
and the output module 300 is used for evaluating the training model by using the test set, and detecting the grading ring defects of the aerial images of the inspection power transmission line by using the training model after the evaluation precision result meets the requirement.
The invention has the beneficial effects that:
the method for detecting the defects of the grading ring of the power transmission line based on the aerial images solves the technical problems that the inspection difficulty is high, the period is long and the inspection result is greatly influenced by the skills of inspection personnel and objective factors such as weather, terrain and the like mainly depending on manual inspection and visual inspection in the prior art. The method for detecting the defects of the grading ring of the power transmission line based on the aerial images improves the Faster R-CNN network by applying Data Augmentation (Data Augmentation), Online Hard sample Mining (OHEM), soft-Non-Maximum Suppression (soft-NMS) algorithm and other methods. The method provided by the invention is more convenient, safer and more efficient than manual inspection, and can be suitable for detecting the faults of the grading rings of aerial images of power transmission lines with various types and scales.
In some demonstrative embodiments, the method further includes:
and the enhancement module is used for increasing the number of the shot image samples by utilizing data enhancement.
In some illustrative embodiments, the enhancement module is implemented by at least one of:
gaussian noise data enhancement, salt-and-pepper noise data enhancement, speckle noise data enhancement, mean filtered data enhancement, gaussian filtered data enhancement, contrast enhanced filtered data enhancement, luminance transform data enhancement, and/or scale scaling data enhancement.
In some demonstrative embodiments, the training module may model train the RPN network model via an Online hard sample Mining (OHEM) method.
In some demonstrative embodiments, the method further includes:
and the removing module is used for removing other similar target probability scores near the target frame with higher prediction probability by using a soft-Non-Maximum Suppression (soft-NMS) algorithm.
The embodiment of the invention also discloses electronic equipment which comprises a processor, wherein the processor is used for executing any one of the grading ring defect detection methods based on the aerial image.
Preferably, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program, and after the program is executed, the computer program is configured to control the processor to execute any one of the above methods for detecting defects of a grading ring based on an aerial image.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a logistics management server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. An aerial image-based grading ring defect detection method is characterized by comprising the following steps:
marking the aerial images, classifying and storing the marked image data, and establishing a defect data set of the grading ring, wherein the defect data set comprises a training set and a testing set;
performing neural network training by using the image data in the training set to generate a training model;
and evaluating the training model by using the test set, and detecting the grading ring defects of the aerial image of the inspection power transmission line by using the training model after the evaluation precision result meets the requirement.
2. The method of claim 1, wherein the annotated image and the output image from the detection of the grading ring defect comprise images having defects of one of three defect types:
damage to the grading ring, inclination of the grading ring and falling off of the grading ring.
3. The method according to claim 1, wherein the performing neural network training using the image data in the training set to generate a training model specifically comprises:
extracting the characteristics of the grading rings in the aerial images in the test set to obtain a plurality of characteristic blocks;
inputting the extracted feature block into a regional candidate network RPN; to obtain a region suggestion box;
each region suggestion frame comprises identified semantic features containing the grading rings;
mapping the semantic features in the region suggestion box through a region pooling layer to output feature information with the same size, and inputting the feature information into a full connection layer;
and the output of the full connection layer obtains accurate category prediction and target positioning through a classification function softmax and a function of frame regression bbox, and eliminates prediction targets at similar positions and in the same category through a relaxation non-maximum suppression algorithm to obtain a grading ring prediction frame.
4. The method according to claim 3, wherein before obtaining the plurality of feature blocks, the method further comprises:
and increasing the number of the shot image samples by data enhancement, and taking the images with increased number as the aerial images.
5. The method according to claim 4, wherein the increasing of the number of captured image samples by data enhancement is achieved by at least one of:
gaussian noise data enhancement, salt-and-pepper noise data enhancement, speckle noise data enhancement, mean filtered data enhancement, gaussian filtered data enhancement, contrast enhanced filtered data enhancement, luminance transform data enhancement, and/or scale scaling data enhancement.
6. The method according to claim 3, wherein the generated training model is an ROI network model;
before the pooling operation, the method further comprises the following steps: the region suggestion box is used for carrying out model training on the ROI network model by an Online difficult sample Mining Online HardExample Mining method.
7. The utility model provides an equalizer ring defect detecting device based on image of taking photo by plane which characterized in that includes:
the marking module is used for marking the aerial image, classifying and storing the marked image data and establishing a defect data set of the grading ring, wherein the defect data set comprises a training set and a testing set;
the training module is used for carrying out neural network training by utilizing the image data in the training set to generate a training model;
and the output module is used for evaluating the training model by using the test set, and detecting the grading ring defects of the aerial images of the inspection power transmission line by using the training model after the evaluation precision result meets the requirement.
8. The image-based grading ring defect detection device of claim 7, further comprising:
and the enhancement module is used for increasing the number of the shot image samples by utilizing data enhancement.
9. An electronic device comprising a processor configured to perform the method for detecting defects in an aerial image-based grading ring according to any of claims 1-6.
10. A computer-readable storage medium storing a computer program, wherein the program is configured to control a processor according to claim 9 to perform any one of the methods for detecting defects in an aerial image-based equalizer ring according to claims 1-6.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911154105.1A CN111008959A (en) | 2019-11-22 | 2019-11-22 | Grading ring defect detection method, device, medium and equipment based on aerial image |
| CN2019111541051 | 2019-11-22 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111563893A true CN111563893A (en) | 2020-08-21 |
| CN111563893B CN111563893B (en) | 2022-07-08 |
Family
ID=70113339
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911154105.1A Withdrawn CN111008959A (en) | 2019-11-22 | 2019-11-22 | Grading ring defect detection method, device, medium and equipment based on aerial image |
| CN202010395305.2A Active CN111563893B (en) | 2019-11-22 | 2020-05-12 | Grading ring defect detection method, device, medium and equipment based on aerial image |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911154105.1A Withdrawn CN111008959A (en) | 2019-11-22 | 2019-11-22 | Grading ring defect detection method, device, medium and equipment based on aerial image |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN111008959A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113034446A (en) * | 2021-03-08 | 2021-06-25 | 国网山东省电力公司平邑县供电公司 | Automatic transformer substation equipment defect identification method and system |
| CN113642486A (en) * | 2021-08-18 | 2021-11-12 | 国网江苏省电力有限公司泰州供电分公司 | Unmanned aerial vehicle distribution network inspection method with airborne front-end identification model |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582234B (en) * | 2020-05-22 | 2023-07-07 | 莫登奎 | Large-scale oil tea tree forest fruit intelligent detection and counting method based on UAV and deep learning |
| CN111735815A (en) * | 2020-06-18 | 2020-10-02 | 江苏方天电力技术有限公司 | Method and device for detecting defects of small hardware fittings of power transmission line and storage medium |
| CN112215809A (en) * | 2020-09-25 | 2021-01-12 | 北京林业大学 | Prawn processing shelling rate detection method and device, electronic equipment and storage medium |
| CN112101333A (en) * | 2020-11-23 | 2020-12-18 | 四川圣点世纪科技有限公司 | Smart cattle farm monitoring and identifying method and device based on deep learning |
| CN112633186A (en) * | 2020-12-26 | 2021-04-09 | 上海有个机器人有限公司 | Method, device, medium and robot for dividing drivable road surface in indoor environment |
| CN112884715A (en) * | 2021-01-28 | 2021-06-01 | 华南理工大学 | Composite insulator grading ring inclination fault detection method based on deep learning |
| CN114782398A (en) * | 2021-05-13 | 2022-07-22 | 深圳科亚医疗科技有限公司 | Training method and training system for learning network for medical image analysis |
| CN114397306B (en) * | 2022-03-25 | 2022-07-29 | 南方电网数字电网研究院有限公司 | Power grid grading ring hypercomplex category defect multi-stage model joint detection method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106877237A (en) * | 2017-03-16 | 2017-06-20 | 天津大学 | A kind of method of insulator missing in detection transmission line of electricity based on Aerial Images |
| CN109711474A (en) * | 2018-12-24 | 2019-05-03 | 中山大学 | A kind of aluminium material surface defects detection algorithm based on deep learning |
| CN109767427A (en) * | 2018-12-25 | 2019-05-17 | 北京交通大学 | The detection method of train rail fastener defect |
-
2019
- 2019-11-22 CN CN201911154105.1A patent/CN111008959A/en not_active Withdrawn
-
2020
- 2020-05-12 CN CN202010395305.2A patent/CN111563893B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106877237A (en) * | 2017-03-16 | 2017-06-20 | 天津大学 | A kind of method of insulator missing in detection transmission line of electricity based on Aerial Images |
| CN109711474A (en) * | 2018-12-24 | 2019-05-03 | 中山大学 | A kind of aluminium material surface defects detection algorithm based on deep learning |
| CN109767427A (en) * | 2018-12-25 | 2019-05-17 | 北京交通大学 | The detection method of train rail fastener defect |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113034446A (en) * | 2021-03-08 | 2021-06-25 | 国网山东省电力公司平邑县供电公司 | Automatic transformer substation equipment defect identification method and system |
| CN113642486A (en) * | 2021-08-18 | 2021-11-12 | 国网江苏省电力有限公司泰州供电分公司 | Unmanned aerial vehicle distribution network inspection method with airborne front-end identification model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111008959A (en) | 2020-04-14 |
| CN111563893B (en) | 2022-07-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111563893B (en) | Grading ring defect detection method, device, medium and equipment based on aerial image | |
| CN110827251B (en) | Power transmission line locking pin defect detection method based on aerial image | |
| CN109767422B (en) | Pipeline detection and identification method based on deep learning, storage medium and robot | |
| CN108038846A (en) | Transmission line equipment image defect detection method and system based on multilayer convolutional neural networks | |
| CN110264448B (en) | Insulator fault detection method based on machine vision | |
| CN107563433B (en) | Infrared small target detection method based on convolutional neural network | |
| CN110992349A (en) | Underground pipeline abnormity automatic positioning and identification method based on deep learning | |
| CN113469953B (en) | Transmission line insulator defect detection method based on improved YOLOv4 algorithm | |
| CN111044570A (en) | Defect identification and early warning method and device for power equipment and computer equipment | |
| CN109003275B (en) | Segmentation method of weld defect image | |
| CN110766058A (en) | Battlefield target detection method based on optimized RPN (resilient packet network) | |
| CN114387207A (en) | Tire flaw detection method and model based on self-attention mechanism and dual-field self-adaptation | |
| Li et al. | Cracked insulator detection based on R-FCN | |
| CN112907561A (en) | Notebook appearance flaw detection method based on deep learning | |
| CN112288700A (en) | Rail defect detection method | |
| CN108345898A (en) | A kind of novel line insulator Condition assessment of insulation method | |
| Wu et al. | Hot-rolled steel strip surface inspection based on transfer learning model | |
| CN114049305A (en) | Distribution line pin defect detection method based on improved ALI and fast-RCNN | |
| CN115578326A (en) | Road disease identification method, system, equipment and storage medium | |
| CN109740672B (en) | Multi-stream feature distance fusion system and fusion method | |
| CN110852318A (en) | Drainage pipeline defect accurate positioning method and system | |
| CN114511521A (en) | Tire defect detection method based on multiple representations and multiple sub-field self-adaption | |
| CN116542911A (en) | End-to-end semi-supervised steel surface defect detection method and system | |
| CN115761606A (en) | Box electric energy meter identification method and device based on image processing | |
| CN115797314A (en) | Part surface defect detection method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |