CN112070135A - Power equipment image detection method and device, power equipment and storage medium - Google Patents
Power equipment image detection method and device, power equipment and storage medium Download PDFInfo
- Publication number
- CN112070135A CN112070135A CN202010888398.2A CN202010888398A CN112070135A CN 112070135 A CN112070135 A CN 112070135A CN 202010888398 A CN202010888398 A CN 202010888398A CN 112070135 A CN112070135 A CN 112070135A
- Authority
- CN
- China
- Prior art keywords
- defect
- picture
- model
- pictures
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 125
- 230000007547 defect Effects 0.000 claims abstract description 358
- 238000012549 training Methods 0.000 claims abstract description 118
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000003062 neural network model Methods 0.000 claims abstract description 18
- 238000013145 classification model Methods 0.000 claims description 19
- 238000010586 diagram Methods 0.000 claims description 16
- 230000002950 deficient Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 7
- 238000007689 inspection Methods 0.000 abstract description 11
- 238000011176 pooling Methods 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 6
- 230000002093 peripheral effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 238000005260 corrosion Methods 0.000 description 3
- 230000007797 corrosion Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 239000003973 paint Substances 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011109 contamination Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2431—Multiple classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses an image detection method and device for power equipment, the power equipment and a storage medium. The power equipment image detection method comprises the following steps: constructing a first training set based on the sample defect picture; training a preset neural network model by using a first training set to obtain a defect detection model; classifying the pictures to be detected according to a preset method to obtain defect pictures; and identifying the defect picture through a defect detection model so as to determine the defect type and the defect position of the defect picture. The automatic detection is carried out on a large number of electric equipment pictures generated by the inspection equipment in the electric power system, the pictures are automatically classified based on the automatic detection result so as to pick out the defect pictures and automatically mark the defect positions of the defect pictures, so that the recognition efficiency and the recognition accuracy of the pictures are improved, and the problems of low efficiency and low recognition accuracy caused by manual recognition and picture defect marking in the prior art are solved.
Description
Technical Field
The embodiment of the invention relates to the technical field of operation and maintenance of electrical equipment, in particular to an image detection method and device for electrical equipment, the electrical equipment and a storage medium.
Background
In recent years, digital camera shooting and robot technology are continuously developed, newly-built intelligent substations and some reconstructed unattended substations realize efficient substation inspection through means such as video monitoring or a robot carrying a camera, and the like, and image data of a site is transmitted to a monitoring center for manual analysis and identification.
The existing image and video data is large in storage amount and high in growth speed, specific defects of substation equipment in patrol are various, the number of images generated in patrol is large, the images are mainly identified manually at present to find out specific defect positions of defect pictures, and the problems of low efficiency and low identification accuracy exist.
Disclosure of Invention
The embodiment of the invention provides an image detection method and device for electrical equipment, the electrical equipment and a storage medium, which are used for automatically identifying defects of batch inspection pictures and improving the identification efficiency and the identification accuracy of a large number of pictures.
In a first aspect, an embodiment of the present invention provides an image detection method for an electrical device, including:
constructing a first training set based on the sample defect picture;
training a preset neural network model by using the first training set to obtain a defect detection model;
classifying the pictures to be detected according to a preset method to obtain defect pictures;
and identifying the defect picture through the defect detection model so as to determine the defect type and the defect position of the defect picture.
Optionally, before the training of the preset neural network model by using the first training set, the method further includes:
and marking a defect area of the sample defect picture in the first training set, and allocating a defect type to the defect area.
Optionally, the training a preset neural network model by using the first training set to obtain a defect detection model, including:
generating a training sample parameter set based on a defect region of the sample defect picture, wherein the training sample parameter set comprises a defect category of the sample defect picture and position information of the defect region;
constructing a YOLOv3 model based on a Darknet-53 framework;
inputting the training sample parameter set into the YOLOv3 model, and training the YOLOv3 model to obtain the defect detection model.
Optionally, the identifying the defect picture through the defect detection model to determine the defect type and the defect position of the defect picture includes:
extracting a target feature unit of the defect picture by a feature extraction network of the defect detection model;
the defect detection model allocates the boundary frames with different sizes to the target feature unit, and the boundary frames with different sizes are applied to feature maps with different resolutions;
the defect detection model detects the boundary frame areas with different sizes on the characteristic diagram and outputs the defect position and the defect type of the defect picture through the detection layer.
Optionally, the defect category includes at least one of: component breakage, respirator defects, status indication defects, foreign matter, oil leakage, and unsightly appearance.
Optionally, the classifying the picture to be detected according to a preset method to obtain a defect picture includes:
training a VGG16 model by using a second training set to obtain a defect classification model, wherein the second training set comprises non-defective pictures and defective pictures;
and identifying the picture to be detected based on the defect classification model to obtain a defect picture.
In a second aspect, an embodiment of the present invention further provides an image detection apparatus for an electrical device, including:
the first training set constructing module is used for constructing a first training set based on the sample defect picture;
the model training module is used for training a preset neural network model by using the first training set to obtain a defect detection model;
the defect picture acquisition module is used for classifying the pictures to be detected according to a preset method to obtain defect pictures;
and the identification module is used for identifying the defect picture through the defect detection model so as to determine the defect type and the defect position of the defect picture.
Optionally, the image detection apparatus for power equipment further includes:
and the defect area marking module is used for marking the defect area of the sample defect picture in the first training set and distributing defect types to the defect area.
In a third aspect, an embodiment of the present invention further provides an electrical device, including:
one or more processors;
storage means for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors implement the power device image detection method according to any embodiment of the present invention.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the power device image detection method according to any embodiment of the present invention.
The image detection method for the power equipment provided by the embodiment of the invention comprises the steps of selecting pictures containing various defects to construct a first training set, and training a preset neural network model by the first training set to obtain a defect detection model representing the corresponding relation between picture characteristics and picture defect positions; classifying the pictures to be detected according to a certain method to obtain defective pictures; and then, identifying the obtained defect picture by using the trained defect detection model to obtain the defect picture with the specific defect position marked and the defect category associated, thereby realizing the defect detection of the picture to be detected. The embodiment solves the problems of low efficiency and low identification accuracy rate caused by manual identification and marking of the picture defects in the prior art, automatically detects a large number of electric power equipment pictures generated by inspection equipment in an electric power system, automatically classifies the pictures based on an automatic detection result to pick out the defect pictures and automatically mark defect positions of the defect pictures, thereby improving the identification efficiency and identification accuracy rate of the pictures, fully utilizing the large number of pictures generated by the inspection equipment, identifying the defect positions of the pictures from the generation of the pictures, and automatically executing by related electric power equipment, and further improving the automatic management level of the electric power system.
Drawings
Fig. 1 is a flowchart of an image detection method for an electrical device according to an embodiment of the present invention;
FIG. 2 is an overall architecture diagram of a VGG16 model provided by an embodiment of the invention;
fig. 3 is a flowchart of another power device image detection method according to an embodiment of the present invention;
FIG. 4 is an overall architecture diagram of the YOLOv3 model provided by an embodiment of the invention;
fig. 5 is a block diagram of an image detection apparatus for an electrical device according to an embodiment of the present invention;
fig. 6 is a block diagram of a power device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Fig. 1 is a flowchart of an image detection method for an electrical device according to an embodiment of the present invention, where the method is applicable to a situation where a huge number of pictures generated by an automatic inspection device in an electrical power system are automatically detected, a defect picture is identified, and a specific defect position of the defect picture is automatically labeled. By automatically marking the picture, operation and maintenance personnel can perform targeted equipment maintenance based on the defect position of the picture. The method may be performed by an electrical device, which may be, for example, a computer device configured with an image detection algorithm, or the like. Referring to fig. 1, the power equipment image detection method includes the steps of:
s110, constructing a first training set based on the sample defect picture.
The sample defect picture is a set of pictures containing various types of defect pictures.
The sample defect picture can be obtained by carrying out picture acquisition on the power equipment with typical defects. In one embodiment, a picture of the sample defect picture includes the following defect types: the picture comprises a picture of part breakage, a picture of respirator defect, a picture of state indication defect, a picture of foreign matter, a picture of leaked oil and a picture of untidy appearance.
For example, the sample defect picture may include 8000 pictures and an xml file corresponding to 8000 pictures, wherein 2000 pictures of the current transformer secondary junction box corrosion picture, 2000 pictures of the transformer oil leakage picture, 2000 pictures of the transformer bushing contamination picture, and 2000 pictures of the transformer paint removal defect picture. Note that no normal pictures are included in the sample defect picture.
And S120, training a preset neural network model by using the first training set to obtain a defect detection model.
The first training set contains pictures of various defect types, so that after the first training set trains the neural network model, the neural network model establishes the corresponding relation between the picture characteristic units and the image defect positions, namely the defect detection model.
Optionally, the defect category includes at least one of: component breakage, respirator defects, status indication defects, foreign matter, oil leakage, and unsightly appearance.
In one embodiment, the neural network model is a YOLO model, and the building process of the defect detection model can be referred to as the following description of the embodiments.
S130, classifying the pictures to be detected according to a preset method to obtain defect pictures.
Since the pictures to be detected may include normal pictures, i.e., non-defective pictures, and these normal pictures cannot be assigned with defect types, before the pictures to be detected are subjected to defect detection, the pictures to be detected need to be classified to screen out the pictures with defects.
In an embodiment, a VGG16 model is used to construct a classification model, and the classification model classifies a picture to be detected to obtain a defect picture, where the process specifically includes:
training the VGG16 model by using a second training set to obtain a defect classification model, wherein the second training set comprises non-defective pictures and defective pictures;
and identifying the picture to be detected based on the defect classification model to obtain a defect picture.
The second training set includes a certain number of normal pictures and a certain number of defect pictures, and the defect pictures cover common defect categories.
Fig. 2 is an overall architecture diagram of the VGG16 model according to an embodiment of the present invention, and as can be seen from fig. 2, the VGG16 model includes a convolutional layer, a pooling layer, a full-link layer, and an output layer. The convolutional layers and the input layers are regularly and alternately distributed, the convolutional layers perform feature extraction, the pooling layers perform feature selection, usually, a plurality of windows of the convolutional layers perform convolution operation on the input layers, and the pooling layers perform pooling operation on the convolutional layers through one window. The full connection layer can integrate the local information with category distinctiveness and integrate all local features into a global feature. And the output layer outputs the classification result.
In the embodiment, the defect classification model and the VGG16 model have the same network structure because the defect identification model is trained from the VGG16 model. Specifically, the convolution layer in the defect classification model is composed of a plurality of feature planes, each feature plane is composed of a plurality of feature units, and each feature unit is connected with a local area of a feature of a previous layer through a convolution kernel. Each feature unit in the convolution layer corresponds to a small block of 3 x 3 or 5 x 5 in the previous layer of input network, and the feature of the local small block in the previous layer is acted to obtain more abstract features. The convolution kernel is a weight matrix, each convolution kernel corresponds to a group of fixed weights, and different convolution kernels extract different features such as contour, texture, color depth and the like, so that a plurality of different convolution kernels are required for extracting different features of an image.
It can be seen that each feature block extracted by the convolutional layer is a partial feature of the power device, and these features are dispersed, so that these features need to be integrated to obtain a complete image feature of the power device.
Before each feature block is integrated, the pooling layer in the defect classification model processes the feature blocks so as to reduce data dimensionality and reduce calculated amount. The depth of the defect classification model cannot be changed by a pooling layer in the defect classification model, but the size of an input matrix can be reduced, so that the number of parameters of the defect classification model is reduced.
After several rounds of convolution and pooling, shallow layer feature information of the picture is abstracted into features with higher information content, convolution and pooling operation are image feature extraction processes, after the convolution and pooling operation of a plurality of layers, obtained feature maps are sequentially expanded in rows and are connected into vectors, the vectors are input into a full connection layer, the full connection layer can integrate local information with category distinctiveness, all local features are integrated into global features, and finally, classification results of the picture to be detected are output through an output layer. And summarizing the defect pictures of various defect types to obtain all the defect pictures.
S140, identifying the defect picture through the defect detection model to determine the defect type and the defect position of the defect picture.
The analysis shows that the defect detection model establishes the corresponding relationship between the defect position of the picture and the picture feature unit, so that the defect type and the specific defect position of each defect picture can be determined after the defect picture is identified by the defect detection model.
The image detection method for the power equipment provided by the embodiment of the invention comprises the steps of selecting pictures containing various defects to construct a first training set, and training a preset neural network model by the first training set to obtain a defect detection model representing the corresponding relation between picture characteristics and picture defect positions; classifying the pictures to be detected according to a certain method to obtain defective pictures; and then, identifying the obtained defect picture by using the trained defect detection model to obtain the defect picture with the specific defect position marked and the defect category associated, thereby realizing the defect detection of the picture to be detected. The embodiment solves the problems of low efficiency and low identification accuracy rate caused by manual identification and marking of the picture defects in the prior art, automatically detects a large number of electric power equipment pictures generated by inspection equipment in an electric power system, automatically classifies the pictures based on an automatic detection result to pick out the defect pictures and automatically mark defect positions of the defect pictures, thereby improving the identification efficiency and identification accuracy rate of the pictures, fully utilizing the large number of pictures generated by the inspection equipment, identifying the defect positions of the pictures from the generation of the pictures, and automatically executing by related electric power equipment, and further improving the automatic management level of the electric power system.
Optionally, fig. 3 is a flowchart of another power device image detection method provided in an embodiment of the present invention, on the basis of the above embodiment, the embodiment is optimized on the basis of the above embodiment, and with reference to fig. 3, the method specifically includes the following steps:
s310, constructing a first training set based on the sample defect picture.
S320, marking the defect area of the sample defect picture in the first training set, and distributing defect types for the defect area.
Wherein, the defect pictures in the first training set can be marked by a marking tool. In one embodiment, the defect picture is labeled using a LabelImg tool. After marking the defect pictures with LabelImg, an xml file conforming to the PASCAL VOC format was generated. There are 5 important parameters in the generated xml file, including: name of target tag, and horizontal and vertical coordinate values x of upper left corner of tag frameminAnd yminThe abscissa and ordinate values of the lower right cornermaxAnd ymax. Each marked mark frame in the picture has 5 parameters, and the parameters corresponding to each mark frame in the same picture are stored in the same xml file.
For example, table 1 provides a defect classification dividing method for the embodiments of the present invention, when the defect classification is performed on a defect region, a defect label corresponding to the defect classification can be allocated to a corresponding mark frame, because the defect label has a unique correspondence with the defect classification, the defect classification of each mark frame can be determined by allocating the defect label to the corresponding mark frame, and the association between the defect region and the defect classification is realized.
TABLE 1 Defect Categories and Label assignments
And S330, generating a training sample parameter set based on the defect area of the sample defect picture.
The training sample parameter set comprises the defect type of the sample defect picture and the position information of the defect area.
As can be seen from the above analysis, after the defect area of the picture is marked, each mark frame has 5 key parameters, so that the corresponding mark frame is uniquely characterized by the 5 key parameters.
The training sample parameter set is a parameter set formed by recording 5 key parameters for representing the defect area according to a file format recognizable by a YOLOv3 model. For example, for a certain sample defect picture, its image size is 3456 × 4608, name is wuzi, and the pixel coordinates of the upper left corner and the lower right corner of the mark box are (1035, 2097) and (1907, 2982). The parameter information of the marking frame is converted into a txt marking file which can be identified by YOLOv3 before the parameter information participates in training, 5 key parameters of the marking frame are correspondingly stored in the txt file and comprise the defect type of a target detection area and 4 coordinate values after the horizontal and vertical coordinates of the upper left corner and the lower right corner of the marking frame are normalized, the key parameters of the marking frame of the same sample defect picture are stored in the same txt file, and the txt files of all sample defect pictures form a training sample parameter set.
Illustratively, 3 sub-folders are created under a VOCdekit training data folder for training a defect detection model, an indications folder is used for storing 8000 xml files, a Main folder is created under an images folder and is used for storing a txt file recorded with 8000 picture names, JPEGImages is stored with 8000 training pictures, the 8000 xml files under the indications folder are also required to be converted into 8000 txt mark files recognizable by YOLOv3, and each mark frame in the training pictures corresponds to 5 key data (including the category of a detection target, and 4 coordinate values after the horizontal and vertical coordinates of the left upper corner and the right lower corner of the mark frame are normalized) in the txt file. 8000 txt files are put under a JPEGImaps folder together, and a VOCdekit data set used for training a defect detection model, namely a training sample parameter set, is constructed.
S340, constructing a YOLOv3 model based on the Darknet-53 framework.
The constructed YOLOv3 model has a good recognition effect on small targets by taking Darknet-53 as a backbone network of the YOLOv3 model.
In one embodiment, after a training sample parameter set participating in model training is prepared, an original model needs to be modified, wherein the original model includes a category of target detection, the number of convolution kernels at 3 different scale detection positions, an actual defect name corresponding to each defect category, and an interface of a training data set in a code.
And S350, inputting the training sample parameter set into a YOLOv3 model, and training the YOLOv3 model to obtain a defect detection model.
For example, the sample of the substation equipment defect detection is shown in table 2. The training sample is used for target detection training and comprises 8000 pictures and an xml file corresponding to the 8000 pictures, and the training sample does not contain normal pictures.
The size of the input picture is 416 x 416, the learning rate is 0.001, 8 pictures are processed in each batch, the number of training batches is 10000, and after training is finished, the trained model weight is stored in a binary file. And (3) detecting a corrosion picture of the secondary junction box of the current transformer, an oil leakage picture of the transformer, a dirt picture of a sleeve of the transformer and a paint removal picture of the transformer by using the trained defect detection model respectively. And evaluating the trained defect detection model through several indexes of success detection rate, recall rate, false detection rate and omission factor. The successful detection rate is the number of pictures with defects detected divided by the number of detection samples, the recall rate is the number of detected peripheral frames divided by the number of actually existing peripheral frames, the false detection rate is the total number of detected false defect frames divided by the total number of detected false defect frames containing correct and false defects, and the false detection rate is the number of undetected peripheral frames divided by the number of actually existing peripheral frames.
TABLE 2 Transformer substation equipment Defect detection sample
After the model training is finished, 200 current transformer secondary junction box corrosion pictures, 200 transformer oil leakage pictures, 200 transformer bushing pollution pictures and 200 transformer paint removal pictures are respectively used for verifying the defect detection model, and the verification results are shown in table 3. As can be seen from Table 3, the successful detection rate of the trained defect detection model on the four types of defects is over 80%, and the defect position recognition rate is very high.
TABLE 3 Defect Picture inspection case
And S360, classifying the pictures to be detected according to a preset method to obtain defect pictures.
And S370, identifying the defect picture through the defect detection model to determine the defect type and the defect position of the defect picture.
Under the condition that a defect detection model is obtained by using a YOLOv3 model training, the process of identifying the defect picture by the defect detection model can be specifically optimized as follows:
extracting a target characteristic unit of a defect picture by a characteristic extraction network of the defect detection model;
the defect detection model allocates the boundary frames with different sizes to the target feature unit, and applies the boundary frames with different sizes to the feature maps with different resolutions;
and the defect detection model detects the boundary frame areas with different sizes on the characteristic diagram and outputs the defect position and the defect type of the defect picture through the detection layer.
Specifically, fig. 4 is an overall architecture diagram of the YOLOv3 model provided in the embodiment of the present invention, the YOLOv3 is fused with YOLOv2, Darknet-19, and other novel residual error networks, the overall size is larger, the Darknet-53 is mounted on the first 53 layers, there are 52 convolution layers and 1 full connection layer, the first 52 layers of the YOLOv3 are completely sleeved on the first 52 layers of the Darknet-53, and the ultra-strong feature extraction capability of the Darknet-53 can be fully utilized to initialize the subsequent detection model. The Darknet-53 improves the standard residual basic unit, 1 × 1 convolution greatly reduces the number of channels of each convolution, reduces the calculated amount and parameters, increases the depth of the network, strengthens the feature expression and improves the detection effect. YOLOv3 also uses the skip layer connection in the residual error network for reference, and combines the characteristics of the deep layer and the shallow layer together through route operation, so that the network has better comprehensive learning expression effect.
For each output substation equipment feature diagram, the YOLOv3 firstly divides the substation equipment feature diagram into grids of S × S, and the YOLOv3 performs 3 detections with different scales when performing 32-fold, 16-fold and 8-fold down-sampling, so that the detection is beneficial to the detection of targets with different sizes, and the detection layer is used for outputting coordinate and category results. YOLOv3 predicts 3 bounding boxes per feature unit per scale, and when the input image is 416 × 416, YOLOv3 predicts (13 × 13+26 × 26+52 × 52) × 3 bounding boxes. Each feature cell in the feature map has three bounding boxes, each bounding box predicts 5 regression values (tx, ty, tw, th, to), the first 4 parameters characterize the position, and the probability of the 5 th specific class bounding box, so the number of predicted parameters in one scale is S (3 (4+1+ N)), and N is the total number of classes contained in the data set. The output position center point coordinates bx and by are positions relative to cell boundaries, and the width tw and height th are predicted relative to the entire image.
In the embodiment, a sample defect picture is selected to form a first training set, a defect region of the sample defect picture is marked, a corresponding relation between the defect characteristics of the sample defect picture and the defect region is established, a training sample parameter set is generated based on a marking frame marked on the sample defect picture, and the training sample parameter set is input into a built YOLOv3 model to train the YOLOv3 model to obtain a defect detection model, so that the defect detection model obtained by training has the corresponding relation between the picture defect region and the picture defect characteristics. The method comprises the steps of automatically screening massive pictures generated by inspection equipment through a preset classification model to obtain pictures with defects, automatically marking the defect positions of the classified defect pictures through the established defect detection model, and outputting the defect pictures with the marked defect positions and defect types.
Optionally, fig. 5 is a block diagram of a structure of an image detection apparatus for an electrical device according to an embodiment of the present invention, where the image detection apparatus for an electrical device includes: a first training set construction module 510, a model training module 520, a defect picture acquisition module 530, and a recognition module 540, wherein,
a first training set constructing module 510, configured to construct a first training set based on the sample defect picture;
a model training module 520, configured to train a preset neural network model using the first training set to obtain a defect detection model;
a defect picture obtaining module 530, configured to classify the picture to be detected according to a preset method to obtain a defect picture;
and the identifying module 540 is configured to identify the defect picture through the defect detection model to determine a defect type and a defect position of the defect picture.
Optionally, on the basis of the foregoing embodiment, the power equipment image detection apparatus further includes:
and the defect area marking module is used for marking the defect area of the sample defect picture in the first training set and distributing defect types for the defect area.
Optionally, on the basis of the foregoing embodiment, the model training module 520 includes:
the training sample parameter set generating unit is used for generating a training sample parameter set based on a defect area of the sample defect picture, wherein the training sample parameter set comprises the defect type of the sample defect picture and the position information of the defect area;
a YOLOv3 model construction unit used for constructing a YOLOv3 model based on Darknet-53 framework;
and the training unit is used for inputting the training sample parameter set into a YOLOv3 model, and training the YOLOv3 model to obtain a defect detection model.
Alternatively, on the basis of the above-described embodiments,
extracting a target characteristic unit of a defect picture by a characteristic extraction network of the defect detection model;
the defect detection model allocates the boundary frames with different sizes to the target feature unit, and applies the boundary frames with different sizes to the feature maps with different resolutions;
and the defect detection model detects the boundary frame areas with different sizes on the characteristic diagram and outputs the defect position and the defect type of the defect picture through the detection layer.
Optionally, on the basis of the foregoing embodiment, the defect category includes at least one of: component breakage, respirator defects, status indication defects, foreign matter, oil leakage, and unsightly appearance.
Optionally, on the basis of the foregoing embodiment, the defect picture acquiring module 530 includes:
the defect classification model training unit is used for training the VGG16 model by using a second training set to obtain a defect classification model, wherein the second training set comprises non-defective pictures and defective pictures;
and the identification unit is used for identifying the picture to be detected based on the defect classification model to obtain a defect picture.
The image detection device for the power equipment, provided by the embodiment of the invention, comprises a first training set formed by selecting a sample defect picture, a corresponding relation between the defect characteristics of the sample defect picture and the defect area is established by marking the defect area of the sample defect picture, a training sample parameter set is generated based on a marking frame marked on the sample defect picture, and the training sample parameter set is input into a built YOLOv3 model to train the YOLOv3 model to obtain a defect detection model, so that the defect detection model obtained by training has the corresponding relation between the picture defect area and the picture defect characteristics. The method comprises the steps of automatically screening massive pictures generated by inspection equipment through a preset classification model to obtain pictures with defects, automatically marking the defect positions of the classified defect pictures through the established defect detection model, and outputting the defect pictures with the marked defect positions and defect types.
Optionally, fig. 6 is a block diagram of a power device according to an embodiment of the present invention. Fig. 6 is a schematic structural diagram of an electrical device according to an embodiment of the present invention. FIG. 6 illustrates a block diagram of an exemplary power device 612 suitable for use in implementing embodiments of the present invention. The power device 612 shown in fig. 6 is only an example, and should not bring any limitation to the function and the scope of the application of the embodiment of the present invention.
As shown in fig. 6, the power device 612 is in the form of a general purpose computing device. The components of the power device 612 may include, but are not limited to: one or more processors or processing units 616, a system memory 628, and a bus 618 that couples various system components including the system memory 628 and the processing unit 616.
The power device 612 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by power device 612 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 628 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)630 and/or cache memory 632. The power device 612 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 634 may be used to read from or write to non-removable, nonvolatile magnetic media (not shown in FIG. 6, commonly referred to as a "hard disk drive"). Although not shown in FIG. 6, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be connected to bus 618 by one or more data media interfaces. Memory 628 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 640 having a set (at least one) of program modules 642 may be stored, for example, in memory 628, such program modules 642 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. The program modules 642 generally perform the functions and/or methods of the described embodiments of the present invention.
The power device 612 may also communicate with one or more external devices 614 (e.g., keyboard, pointing device, display 624, etc.), with one or more devices that enable a user to interact with the power device 612, and/or with any devices (e.g., network card, modem, etc.) that enable the power device 612 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 622. Also, the power device 612 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) through the network adapter 620. As shown, the network adapter 620 communicates with the other modules of the power device 612 over the bus 618. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the power device 612, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 616 executes various functional applications and data processing by running programs stored in the system memory 628, for example, implementing an image detection method for a power device provided by an embodiment of the present invention, the method including:
constructing a first training set based on the sample defect picture;
training a preset neural network model by using a first training set to obtain a defect detection model;
classifying the pictures to be detected according to a preset method to obtain defect pictures;
and identifying the defect picture through a defect detection model so as to determine the defect type and the defect position of the defect picture.
Optionally, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for detecting an image of an electrical device, where the method includes:
constructing a first training set based on the sample defect picture;
training a preset neural network model by using a first training set to obtain a defect detection model;
classifying the pictures to be detected according to a preset method to obtain defect pictures;
and identifying the defect picture through a defect detection model so as to determine the defect type and the defect position of the defect picture.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. An image detection method for power equipment is characterized by comprising the following steps:
constructing a first training set based on the sample defect picture;
training a preset neural network model by using the first training set to obtain a defect detection model;
classifying the pictures to be detected according to a preset method to obtain defect pictures;
and identifying the defect picture through the defect detection model so as to determine the defect type and the defect position of the defect picture.
2. The power device image detection method according to claim 1, wherein before the training of a preset neural network model using the first training set, the method further comprises:
and marking a defect area of the sample defect picture in the first training set, and allocating a defect type to the defect area.
3. The method for detecting the image of the power equipment according to claim 1, wherein the training a preset neural network model by using the first training set to obtain a defect detection model comprises:
generating a training sample parameter set based on a defect region of the sample defect picture, wherein the training sample parameter set comprises a defect category of the sample defect picture and position information of the defect region;
constructing a YOLOv3 model based on a Darknet-53 framework;
inputting the training sample parameter set into the YOLOv3 model, and training the YOLOv3 model to obtain the defect detection model.
4. The power equipment image detection method according to claim 1, wherein the identifying the defect picture through the defect detection model to determine the defect type and the defect position of the defect picture comprises:
extracting a target feature unit of the defect picture by a feature extraction network of the defect detection model;
the defect detection model allocates the boundary frames with different sizes to the target feature unit, and the boundary frames with different sizes are applied to feature maps with different resolutions;
the defect detection model detects the boundary frame areas with different sizes on the characteristic diagram and outputs the defect position and the defect type of the defect picture through the detection layer.
5. The power equipment image detection method according to claim 1, wherein the defect category includes at least one of: component breakage, respirator defects, status indication defects, foreign matter, oil leakage, and unsightly appearance.
6. The method for detecting the image of the power equipment according to claim 1, wherein the classifying the picture to be detected according to a preset method to obtain a defect picture comprises:
training a VGG16 model by using a second training set to obtain a defect classification model, wherein the second training set comprises non-defective pictures and defective pictures;
and identifying the picture to be detected based on the defect classification model to obtain a defect picture.
7. An image detection device for power equipment, comprising:
the first training set constructing module is used for constructing a first training set based on the sample defect picture;
the model training module is used for training a preset neural network model by using the first training set to obtain a defect detection model;
the defect picture acquisition module is used for classifying the pictures to be detected according to a preset method to obtain defect pictures;
and the identification module is used for identifying the defect picture through the defect detection model so as to determine the defect type and the defect position of the defect picture.
8. The power equipment image detection device according to claim 7, further comprising:
and the defect area marking module is used for marking the defect area of the sample defect picture in the first training set and distributing defect types to the defect area.
9. An electrical device, comprising:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the power device image detection method of any of claims 1-6.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the power device image detection method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010888398.2A CN112070135A (en) | 2020-08-28 | 2020-08-28 | Power equipment image detection method and device, power equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010888398.2A CN112070135A (en) | 2020-08-28 | 2020-08-28 | Power equipment image detection method and device, power equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112070135A true CN112070135A (en) | 2020-12-11 |
Family
ID=73659823
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010888398.2A Pending CN112070135A (en) | 2020-08-28 | 2020-08-28 | Power equipment image detection method and device, power equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112070135A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112508137A (en) * | 2021-02-09 | 2021-03-16 | 南方电网数字电网研究院有限公司 | Transformer abnormality detection method and device, computer equipment and storage medium |
| CN113077556A (en) * | 2021-03-29 | 2021-07-06 | 深圳大学 | Ticket checking system and method based on pedestrian re-identification |
| CN113313706A (en) * | 2021-06-28 | 2021-08-27 | 安徽南瑞继远电网技术有限公司 | Power equipment defect image detection method based on detection reference point offset analysis |
| CN113379728A (en) * | 2021-07-02 | 2021-09-10 | 上海电气集团股份有限公司 | Method, system, equipment and readable storage medium for detecting defects on surface of rail |
| CN113610749A (en) * | 2021-04-21 | 2021-11-05 | 北京智慧空间科技有限责任公司 | Fan blade defect detection method based on neural network |
| CN114202725A (en) * | 2021-12-07 | 2022-03-18 | 牧原食品股份有限公司 | Biogas slurry overflow monitoring method and system based on intelligent vision |
| CN114444550A (en) * | 2022-04-02 | 2022-05-06 | 中国人民解放军火箭军工程大学 | Structured data analysis method and device, electronic equipment and storage medium |
| CN115457297A (en) * | 2022-08-23 | 2022-12-09 | 中国航空油料集团有限公司 | Method and device for detecting oil leakage of aviation oil depot and aviation oil safety operation and maintenance system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106556781A (en) * | 2016-11-10 | 2017-04-05 | 华乘电气科技（上海）股份有限公司 | Shelf depreciation defect image diagnostic method and system based on deep learning |
| CN106803254A (en) * | 2017-01-17 | 2017-06-06 | 广东容祺智能科技有限公司 | The picture automatic testing method of insulator defect analysis in a kind of transmission line of electricity |
| CN108597053A (en) * | 2018-04-25 | 2018-09-28 | 北京御航智能科技有限公司 | Shaft tower and channel targets identification based on image data and neural network and defect diagnostic method |
| CN109711437A (en) * | 2018-12-06 | 2019-05-03 | 武汉三江中电科技有限责任公司 | A kind of transformer part recognition methods based on YOLO network model |
| CN110689531A (en) * | 2019-09-23 | 2020-01-14 | 云南电网有限责任公司电力科学研究院 | Automatic power transmission line machine inspection image defect identification method based on yolo |
| US20200151448A1 (en) * | 2018-11-13 | 2020-05-14 | Adobe Inc. | Object Detection In Images |
-
2020
- 2020-08-28 CN CN202010888398.2A patent/CN112070135A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106556781A (en) * | 2016-11-10 | 2017-04-05 | 华乘电气科技（上海）股份有限公司 | Shelf depreciation defect image diagnostic method and system based on deep learning |
| CN106803254A (en) * | 2017-01-17 | 2017-06-06 | 广东容祺智能科技有限公司 | The picture automatic testing method of insulator defect analysis in a kind of transmission line of electricity |
| CN108597053A (en) * | 2018-04-25 | 2018-09-28 | 北京御航智能科技有限公司 | Shaft tower and channel targets identification based on image data and neural network and defect diagnostic method |
| US20200151448A1 (en) * | 2018-11-13 | 2020-05-14 | Adobe Inc. | Object Detection In Images |
| CN109711437A (en) * | 2018-12-06 | 2019-05-03 | 武汉三江中电科技有限责任公司 | A kind of transformer part recognition methods based on YOLO network model |
| CN110689531A (en) * | 2019-09-23 | 2020-01-14 | 云南电网有限责任公司电力科学研究院 | Automatic power transmission line machine inspection image defect identification method based on yolo |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112508137B (en) * | 2021-02-09 | 2021-07-06 | 南方电网数字电网研究院有限公司 | Transformer abnormality detection method and device, computer equipment and storage medium |
| CN112508137A (en) * | 2021-02-09 | 2021-03-16 | 南方电网数字电网研究院有限公司 | Transformer abnormality detection method and device, computer equipment and storage medium |
| CN113077556A (en) * | 2021-03-29 | 2021-07-06 | 深圳大学 | Ticket checking system and method based on pedestrian re-identification |
| CN113610749B (en) * | 2021-04-21 | 2024-04-19 | 北京智慧空间科技有限责任公司 | Fan blade defect detection method based on neural network |
| CN113610749A (en) * | 2021-04-21 | 2021-11-05 | 北京智慧空间科技有限责任公司 | Fan blade defect detection method based on neural network |
| CN113313706A (en) * | 2021-06-28 | 2021-08-27 | 安徽南瑞继远电网技术有限公司 | Power equipment defect image detection method based on detection reference point offset analysis |
| CN113313706B (en) * | 2021-06-28 | 2022-04-15 | 安徽南瑞继远电网技术有限公司 | Power equipment defect image detection method based on detection reference point offset analysis |
| CN113379728A (en) * | 2021-07-02 | 2021-09-10 | 上海电气集团股份有限公司 | Method, system, equipment and readable storage medium for detecting defects on surface of rail |
| CN114202725A (en) * | 2021-12-07 | 2022-03-18 | 牧原食品股份有限公司 | Biogas slurry overflow monitoring method and system based on intelligent vision |
| CN114444550B (en) * | 2022-04-02 | 2022-07-29 | 中国人民解放军火箭军工程大学 | Structured data analysis method and device, electronic equipment and storage medium |
| CN114444550A (en) * | 2022-04-02 | 2022-05-06 | 中国人民解放军火箭军工程大学 | Structured data analysis method and device, electronic equipment and storage medium |
| CN115457297A (en) * | 2022-08-23 | 2022-12-09 | 中国航空油料集团有限公司 | Method and device for detecting oil leakage of aviation oil depot and aviation oil safety operation and maintenance system |
| CN115457297B (en) * | 2022-08-23 | 2023-09-26 | 中国航空油料集团有限公司 | Oil leakage detection method and device for aviation oil depot and aviation oil safety operation and maintenance system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112070135A (en) | Power equipment image detection method and device, power equipment and storage medium | |
| CN110059694B (en) | Intelligent identification method for character data in complex scene of power industry | |
| CN112580439B (en) | Large-format remote sensing image ship target detection method and system under small sample condition | |
| CN111563557B (en) | Method for detecting target in power cable tunnel | |
| CN112070134A (en) | Power equipment image classification method and device, power equipment and storage medium | |
| CN111444939A (en) | Small-scale equipment component detection method based on weak supervision cooperative learning in open scene of power field | |
| CN109558792B (en) | Method and system for detecting internet logo content based on samples and features | |
| CN112613569B (en) | Image recognition method, training method and device for image classification model | |
| CN113592822A (en) | Insulator defect positioning method for power inspection image | |
| CN111900694B (en) | Relay protection equipment information acquisition method and system based on automatic identification | |
| CN114511718B (en) | Intelligent management method and system for materials for building construction | |
| CN111815576B (en) | Method, device, equipment and storage medium for detecting corrosion condition of metal part | |
| CN116823793A (en) | Device defect detection method, device, electronic device and readable storage medium | |
| CN115131747A (en) | Knowledge distillation-based power transmission channel engineering vehicle target detection method and system | |
| CN114694130A (en) | Method and device for detecting telegraph poles and pole numbers along railway based on deep learning | |
| CN114972880A (en) | Label identification method and device, electronic equipment and storage medium | |
| CN111507398A (en) | Transformer substation metal instrument corrosion identification method based on target detection | |
| CN113408630A (en) | Transformer substation indicator lamp state identification method | |
| CN111062388B (en) | Advertisement character recognition method, system, medium and equipment based on deep learning | |
| CN112529836A (en) | High-voltage line defect detection method and device, storage medium and electronic equipment | |
| CN110765900A (en) | DSSD-based automatic illegal building detection method and system | |
| CN110163081A (en) | SSD-based real-time regional intrusion detection method, system and storage medium | |
| CN114882242A (en) | Violation image identification method and system based on computer vision | |
| CN113837178A (en) | Deep learning-based automatic positioning and unified segmentation method for meter of transformer substation | |
| CN112016515A (en) | File cabinet vacancy detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |