CN114219753A - Power equipment surface defect detection method based on deep learning and terminal - Google Patents

Power equipment surface defect detection method based on deep learning and terminal Download PDF

Info

Publication number
CN114219753A
CN114219753A CN202111256280.9A CN202111256280A CN114219753A CN 114219753 A CN114219753 A CN 114219753A CN 202111256280 A CN202111256280 A CN 202111256280A CN 114219753 A CN114219753 A CN 114219753A
Authority
CN
China
Prior art keywords
real
deep learning
template library
image
equipment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111256280.9A
Other languages
Chinese (zh)
Inventor
何志甘
范彦琨
张锦吉
付胜宪
陈德兴
李冠颖
林剑平
吕小伟
徐显烨
杨宏毅
李升晖
姚国华
林石
彭质斌
熊旭
张舒雅
黄东方
张颜真
许卉
严欣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Maintenance Branch of State Grid Fujian Electric Power Co Ltd
Original Assignee
Maintenance Branch of State Grid Fujian Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Maintenance Branch of State Grid Fujian Electric Power Co Ltd filed Critical Maintenance Branch of State Grid Fujian Electric Power Co Ltd
Priority to CN202111256280.9A priority Critical patent/CN114219753A/en
Publication of CN114219753A publication Critical patent/CN114219753A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Electricity, gas or water supply
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
    • G07C1/20—Checking timed patrols, e.g. of watchman

Abstract

The invention provides a method and a terminal for detecting surface defects of electric power equipment based on deep learning, wherein the method comprises the following steps: acquiring real-time pictures automatically and regularly acquired by the intelligent electric power inspection system; judging the real-time picture by adopting a deep learning template library, judging the real-time picture by adopting an image recognition mode if the judgment fails, and adopting manual marking processing if the judgment fails; judging the type, position and contour of the real-time picture automatic marking equipment which succeeds in distinguishing the deep learning template library or the image recognition mode; and importing the marked real-time picture into the deep learning template library for deep learning, and after updating the deep learning template library, continuing to identify the next real-time picture. The invention realizes the self-feedback image recognition processing method, fully utilizes a large number of pictures acquired by the intelligent inspection system, completes the automatic maintenance and automatic updating of the power equipment template library and realizes more accurate intelligent judgment of the working state of the power equipment.

Description

Power equipment surface defect detection method based on deep learning and terminal
Technical Field
The invention relates to the technical field of a power grid power transmission and transformation equipment state monitoring system, in particular to a power equipment surface defect detection method and a terminal based on deep learning.
Background
With the rapid development of automation technology, many links requiring manual operation in industrial production are gradually completed by machines, and industrial production automation also frees more and more workers from boring work, so that the workers can play greater value.
The detection of the surface defects of the power equipment is an important link in the safety production of a power grid, is a key step of automatic inspection of the intelligent inspection system, and can effectively improve the intelligent inspection quality and efficiency of the intelligent inspection system by means of a surface defect detection technology. The traditional surface defect detection algorithm structure obtains an image convenient to detect through image preprocessing, and then extracts image characteristics by means of a statistical machine learning method, so that the target of defect detection is realized. Wherein the image preprocessing step comprises: histogram equalization, filtering denoising, gray level binarization and secondary filtering to obtain simplified image information of foreground and background separation, and then completing defect marking and detection by utilizing algorithms such as mathematical morphology, Fourier transform, Gabor transform and the like and a machine learning model.
However, the traditional surface defect detection algorithm needs to manually design a complex algorithm flow, so that the surface defect detection of the electrical equipment based on the deep learning algorithm appears.
The deep learning algorithm adopts a Deep Convolutional Neural Network (DCNNs) combined with target detection algorithms such as SSD (Single Shot MultiBox Detector), Yolo (you Only Look one) and the like to construct a coarse-to-fine cascade detection network, comprises the steps of equipment positioning, defect detection and classification, and specifically comprises the following steps:
(1) and (3) equipment surface extraction: positioning a cantilever node in an image by virtue of an SSD frame which has good speed and precision, and positioning a sleeve based on a fast local framework of a Yolo frame;
(2) detecting and classifying the surface defects of the equipment: and judging the defects according to the detection of the equipment surface in the second stage, and classifying the defects through 4 convolution layers by means of DCNN.
The deep learning algorithm avoids the problem that a traditional algorithm needs a complex algorithm flow designed manually, has extremely high robustness and precision, but the essence of the deep learning is machine learning, and the most fundamental principle of the deep learning algorithm lies in statistics, namely, enough data are needed to be subjected to model definition, analysis data collection and injection training to improve the final output structure of the model, and the process is executed circularly to continuously improve the precision. In short, the deep learning algorithm needs to provide a sufficient number of a wide variety of device image templates for training and modeling the algorithm.
The equipment image template needs to be trained manually on equipment marks on the pictures, and the marks need to be operated manually, namely the equipment is marked on each picture, and the training of the deep learning template is completed through the marked pictures. In the process of adopting the deep learning template, the template picture → training → template → image recognition is a one-way process, in the recognition process, only the pre-trained template can be used, which easily causes inaccurate recognition, and for the picture which can not be recognized by the template, an artificial recognition mode is needed to be adopted, so that the recognition efficiency is reduced.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the method and the terminal for detecting the surface defects of the electric power equipment based on deep learning are provided, and more accurate intelligent judgment of the working state of the electric power equipment is realized through automatic maintenance and automatic updating of an electric power equipment template library.
In order to solve the technical problems, the invention adopts the technical scheme that: a method and a terminal for detecting surface defects of electric power equipment based on deep learning comprise the following steps:
s1, acquiring real-time pictures automatically and regularly acquired by the intelligent electric power inspection system;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
s3, distinguishing the real-time picture by adopting an image recognition mode, if the distinguishing fails, importing the real-time picture after being manually marked into the deep learning template library for deep learning, updating the deep learning template library, returning to S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
and S4, automatically marking the type, position and outline of equipment on the real-time picture, guiding the marked real-time picture into the deep learning template library for deep learning, and returning to S1 for identifying the next real-time picture after updating the deep learning template library.
In order to solve the technical problem, the invention adopts another technical scheme as follows: a deep learning based power equipment surface defect detection terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
s1, acquiring real-time pictures automatically and regularly acquired by the intelligent electric power inspection system;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
s3, distinguishing the real-time picture by adopting an image recognition mode, if the distinguishing fails, importing the real-time picture after being manually marked into the deep learning template library for deep learning, updating the deep learning template library, returning to S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
and S4, automatically marking the type, position and outline of equipment on the real-time picture, guiding the marked real-time picture into the deep learning template library for deep learning, and returning to S1 for identifying the next real-time picture after updating the deep learning template library.
The invention has the beneficial effects that: the invention provides a method and a terminal for detecting surface defects of electric power equipment based on deep learning, which are characterized in that firstly, real-time pictures automatically and regularly acquired by an intelligent electric power inspection system are identified based on a pre-obtained deep learning template library, the pictures which can not be identified are distinguished by adopting an image identification mode, both can realize automatic distinguishing of the type, the position and the outline of the marking equipment, and the pictures which can not be distinguished by the image recognition mode are manually marked, and the marked real-time pictures are imported into the deep learning template library for training after being marked each time, the method for recognizing and processing the self-feedback image by updating the deep learning template library fully utilizes a large number of pictures collected by the intelligent inspection system to complete automatic maintenance and automatic updating of the template library of the power equipment, and realizes more accurate intelligent judgment of the working state of the power equipment.
Drawings
Fig. 1 is a main flowchart of a method for detecting surface defects of an electrical device based on deep learning according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating a method for detecting surface defects of an electrical device based on deep learning according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of automatic marking on a real-time picture according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of marking a real-time picture with a manual mark according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a power equipment surface defect detection terminal based on deep learning according to an embodiment of the present invention.
Description of reference numerals:
1. a surface defect detection terminal of electrical equipment based on deep learning; 2. a memory; 3. a processor.
Detailed Description
In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.
Referring to fig. 1 to 4, a method for detecting surface defects of an electrical device based on deep learning includes the steps of:
s1, acquiring real-time pictures automatically and regularly acquired by the intelligent electric power inspection system;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
s3, distinguishing the real-time picture by adopting an image recognition mode, if the distinguishing fails, importing the real-time picture after being manually marked into the deep learning template library for deep learning, updating the deep learning template library, returning to S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
and S4, automatically marking the type, position and outline of equipment on the real-time picture, guiding the marked real-time picture into the deep learning template library for deep learning, and returning to S1 for identifying the next real-time picture after updating the deep learning template library.
As can be seen from the above description, the beneficial effects of the present invention are: firstly, real-time pictures automatically and regularly acquired by an intelligent electric power inspection system are identified based on a pre-obtained deep learning template library, pictures which cannot be identified are identified by adopting an image identification mode, the type, the position and the outline of marking equipment can be automatically identified by the two modes, the pictures which cannot be identified by the image identification mode are manually marked, the marked real-time pictures are imported into the deep learning template library for training after being marked each time, so that the deep learning template library is updated, a self-feedback image identification processing method is realized, a large number of pictures acquired by the intelligent inspection system are fully utilized, the automatic maintenance and the automatic updating of the electric power equipment template library are completed, and the intelligent identification of the working state of the electric power equipment is realized more accurately.
Further, in S3, an image recognition mode is used to determine the real-time picture, specifically:
and image recognition is carried out on the real-time pictures which cannot be recognized in the deep learning template library by adopting an image recognition template library so as to determine the type, the position and the outline of the equipment in the real-time pictures, the image recognition template library is formed by distinguishing all historical pictures which are automatically and regularly collected by an electric intelligent inspection system, and each historical picture comprises the type, the position and the outline of the corresponding equipment.
According to the description, the real-time pictures which cannot be identified by the deep learning template library are further identified by the image identification template library, so that the types, the positions and the contour marks of the equipment in the real-time pictures can be carried out, and accurate intelligent judgment of the working state of the power equipment is further realized.
Further, in S3, the step of importing the real-time picture after being manually marked into the deep learning template library for deep learning includes:
receiving the type and position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the real-time picture and the type, position and outline of the equipment corresponding to the real-time picture into the image recognition template library and the deep learning template library for deep learning.
According to the description, the real-time pictures which cannot be identified in the image identification template library are manually marked by the staff for the equipment types, positions and outlines in the real-time pictures, so that the intelligent identification of the working state of the power equipment is further realized, meanwhile, the marked real-time pictures are guided into the image identification template library to further enrich template materials in the image identification template library, namely, the image identification template library is continuously perfected, and the marking is not needed to be carried out through complex machine identification when similar real-time pictures are touched next time, but can be directly identified through the image identification template library, so that the intelligent identification efficiency is further improved.
Further, the step S4 of automatically marking the type, position and outline of the device on the real-time picture further includes:
and importing the real-time picture and the type and the outline of the corresponding equipment into the image recognition template library.
According to the description, the marked real-time pictures are imported into the image recognition template library after the equipment types, the positions and the outlines of the real-time pictures are marked every time, so that template materials in the image recognition template library can be further enriched, namely, the image recognition template library is continuously improved, and the intelligent distinguishing efficiency of the power equipment is improved.
Further, in S4, automatically marking the type, position, and contour of the device on the real-time picture, specifically including the following steps:
s41, obtaining the type of equipment in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library recognized in an image recognition mode, taking the historical picture as a reference image, and taking the real-time picture as an image to be processed;
s42, selecting four vertexes of the polygonal outline of the equipment on the reference image as feature points P1, P2, P3 and P4;
s43, taking a certain feature point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement, wherein the value of M is 34-40 pixels;
s44, calculating the data block with the highest similarity to the data block T by adopting a global search method in the image to be processed, wherein the calculation formula is as follows:
Figure BDA0003324259640000061
wherein, Pi,jRepresenting image brightness data with length and width of M pixel points by taking the position of coordinates (i, j) as the center in the image to be processed, wherein the value range of i is (M/2) to (W-M/2), the value range of j is (M/2) to (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed;
s45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed which is matched with the characteristic point in the reference image, repeating the steps S43 to S44 to respectively obtain target position points T1, T2, T3 and T4 which are matched with the characteristic points P1, P2, P3 and P4;
and S46, obtaining the position of the equipment in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtaining the polygonal contour of the equipment in the image to be processed by combining the polygonal contour of the equipment in the reference image.
According to the description, coordinate operation is carried out on the polygon of the outline of the equipment body in the real-time picture based on the four characteristic points, the minimum rectangle capable of containing all the vertexes of the polygon is calculated, and data used for marking the position of the equipment in the deep learning template can be obtained, so that the position and the outline of the equipment can be automatically and quickly marked, meanwhile, the position data and the outline data of the equipment can be applied to a deep learning template library for continuous training, the deep learning template library is further updated, and the accuracy of intelligent judgment of the power equipment is improved.
Referring to fig. 5, a deep learning-based power equipment surface defect detection terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the following steps when executing the computer program:
s1, acquiring real-time pictures automatically and regularly acquired by the intelligent electric power inspection system;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
s3, distinguishing the real-time picture by adopting an image recognition mode, if the distinguishing fails, importing the real-time picture after being manually marked into the deep learning template library for deep learning, updating the deep learning template library, returning to S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
and S4, automatically marking the type, position and outline of equipment on the real-time picture, guiding the marked real-time picture into the deep learning template library for deep learning, and returning to S1 for identifying the next real-time picture after updating the deep learning template library.
As can be seen from the above description, the beneficial effects of the present invention are: based on the same technical concept, the method for detecting the surface defect of the electric equipment based on the deep learning is matched with the method for detecting the surface defect of the electric equipment based on the deep learning, firstly, the real-time pictures automatically and regularly acquired by the intelligent electric inspection system are identified based on the deep learning template library obtained in advance, the pictures which cannot be identified are distinguished by adopting an image recognition mode, the types, the positions and the contours of marking equipment can be automatically distinguished, the pictures which cannot be distinguished by the image recognition mode are manually marked, the marked real-time pictures are guided into the deep learning template library for training after each marking, so that the deep learning template library is updated, the self-feedback image recognition processing method is realized, a large number of pictures acquired by the intelligent inspection system are fully utilized, and the automatic maintenance and the automatic updating of the electric equipment template library are completed, and more accurate intelligent judgment of the working state of the power equipment is realized.
Further, in S3, an image recognition mode is used to determine the real-time picture, specifically:
and image recognition is carried out on the real-time pictures which cannot be recognized in the deep learning template library by adopting an image recognition template library so as to determine the type, the position and the outline of the equipment in the real-time pictures, the image recognition template library is formed by distinguishing all historical pictures which are automatically and regularly collected by an electric intelligent inspection system, and each historical picture comprises the type, the position and the outline of the corresponding equipment.
According to the description, the real-time pictures which cannot be identified by the deep learning template library are further identified by the image identification template library, so that the types, the positions and the contour marks of the equipment in the real-time pictures can be carried out, and accurate intelligent judgment of the working state of the power equipment is further realized.
Further, in S3, the step of importing the real-time picture after being manually marked into the deep learning template library for deep learning includes:
receiving the type and position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the real-time picture and the type, position and outline of the equipment corresponding to the real-time picture into the image recognition template library and the deep learning template library for deep learning.
According to the description, the real-time pictures which cannot be identified in the image identification template library are manually marked by the staff for the equipment types, positions and outlines in the real-time pictures, so that the intelligent identification of the working state of the power equipment is further realized, meanwhile, the marked real-time pictures are guided into the image identification template library to further enrich template materials in the image identification template library, namely, the image identification template library is continuously perfected, and the marking is not needed to be carried out through complex machine identification when similar real-time pictures are touched next time, but can be directly identified through the image identification template library, so that the intelligent identification efficiency is further improved.
Further, the step S4 of automatically marking the type, position and outline of the device on the real-time picture further includes:
and importing the real-time picture and the type and the outline of the corresponding equipment into the image recognition template library.
According to the description, the marked real-time pictures are imported into the image recognition template library after the equipment types, the positions and the outlines of the real-time pictures are marked every time, so that template materials in the image recognition template library can be further enriched, namely, the image recognition template library is continuously improved, and the intelligent distinguishing efficiency of the power equipment is improved.
Further, in S4, automatically marking the type, position, and contour of the device on the real-time picture, specifically including the following steps:
s41, obtaining the type of equipment in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library recognized in an image recognition mode, taking the historical picture as a reference image, and taking the real-time picture as an image to be processed;
s42, selecting four vertexes of the polygonal outline of the equipment on the reference image as feature points P1, P2, P3 and P4;
s43, taking a certain feature point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement, wherein the value of M is 34-40 pixels;
s44, calculating the data block with the highest similarity to the data block T by adopting a global search method in the image to be processed, wherein the calculation formula is as follows:
Figure BDA0003324259640000091
wherein, Pi,jRepresenting image brightness data with length and width of M pixel points by taking the position of coordinates (i, j) as the center in the image to be processed, wherein the value range of i is (M/2) to (W-M/2), the value range of j is (M/2) to (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed;
s45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed which is matched with the characteristic point in the reference image, repeating the steps S43 to S44 to respectively obtain target position points T1, T2, T3 and T4 which are matched with the characteristic points P1, P2, P3 and P4;
and S46, obtaining the position of the equipment in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtaining the polygonal contour of the equipment in the image to be processed by combining the polygonal contour of the equipment in the reference image.
According to the description, coordinate operation is carried out on the polygon of the outline of the equipment body in the real-time picture based on the four characteristic points, the minimum rectangle capable of containing all the vertexes of the polygon is calculated, and data used for marking the position of the equipment in the deep learning template can be obtained, so that the position and the outline of the equipment can be automatically and quickly marked, meanwhile, the position data and the outline data of the equipment can be applied to a deep learning template library for continuous training, the deep learning template library is further updated, and the accuracy of intelligent judgment of the power equipment is improved.
Referring to fig. 1, a first embodiment of the present invention is:
a method for detecting surface defects of electric power equipment based on deep learning comprises the following steps:
and S1, acquiring real-time pictures automatically and regularly acquired by the intelligent electric power inspection system.
In this embodiment, a large number of intelligent patrol devices such as high-definition cameras, robots, or unmanned aerial vehicles are installed on the intelligent electric power patrol system, and these intelligent patrol devices regularly and fixedly acquire working images of various devices every day, where in this embodiment, each real-time picture may have a plurality of devices or only one device, but one real-time picture at least includes one device.
And S2, judging the real-time picture by adopting the deep learning template library, if the judgment fails, entering a step S3, and if the judgment fails, entering a step S4.
In the embodiment, the deep learning template library adopted firstly is a pre-trained template, and the real-time picture is analyzed and judged through the pre-trained deep learning template library firstly to determine the position and the outline of the equipment in the picture and the type of the equipment, so that the defect of the equipment is judged.
And S3, judging the real-time picture by adopting an image identification mode, if the judgment fails, importing the real-time picture after manual marking into a deep learning template library for deep learning, updating the deep learning template library, returning to the step S1 for identifying the next real-time picture, and otherwise, entering the step S4.
In this embodiment, the real-time pictures that cannot be distinguished by the deep learning template library obtained through pre-training are identified by using the image identification mode, and the pictures that cannot be distinguished by the image identification mode are manually marked, so that the intelligent judgment of the equipment is integrally realized.
And S4, automatically marking the type, position and outline of the equipment on the real-time picture, guiding the marked real-time picture into a deep learning template library for deep learning, and returning to S1 for identifying the next real-time picture after updating the deep learning template library.
In other words, in this embodiment, each time the real-time picture is distinguished, the device type, the position and the contour of the real-time picture are marked, and the deep learning template library is re-introduced for further deep learning training, so that the method replaces the existing method that the distinguished picture is only entered into the historical image library for historical data query without other use, and can continuously update and perfect the deep learning template library, thereby realizing the self-feedback image recognition processing method. A large amount of pictures collected by the intelligent inspection system are fully utilized, automatic maintenance and automatic updating of the power equipment template library are completed, and more accurate intelligent judgment of the working state of the power equipment is realized.
Referring to fig. 2 to 4, a second embodiment of the present invention is:
based on the first embodiment, in this embodiment, as shown in fig. 2, a specific flowchart of a method for detecting surface defects of an electrical device based on deep learning is shown.
In step S3, the image recognition mode is used to determine the real-time picture, specifically:
and performing image recognition on the real-time picture which cannot be recognized in the deep learning template library by adopting an image recognition template library so as to determine the type, the position and the outline of the equipment in the real-time picture. In this embodiment, the image recognition template library is formed by discriminating all history pictures acquired by the intelligent electric power inspection system automatically at regular time, and each history picture includes the type, position and contour of the corresponding device.
In other words, in this embodiment, the real-time pictures that cannot be recognized by the deep learning template library are further recognized by the image recognition template library, so that the types, positions and contours of the devices in the real-time pictures can be labeled, and accurate intelligent determination of the operating state of the power equipment is further realized.
In step S3, the real-time image after being manually marked is imported into a deep learning template library for deep learning, specifically:
receiving the type and position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the real-time picture and the type, position and outline of the equipment corresponding to the real-time picture into an image recognition template library and into a deep learning template library for deep learning.
In this embodiment, the worker manually marks the type, position, and contour of the device in the real-time picture, which cannot be recognized by the image recognition template library, as shown in fig. 5, that is, the worker copies the contour of the device in the real-time picture by using a polygon, so as to further realize intelligent recognition of the working state of the power device, and meanwhile, the marked real-time picture is guided into the image recognition template library to further enrich template materials in the image recognition template library, that is, the image recognition template library is continuously perfected, and when the next similar real-time picture is touched, the real-time picture is not marked by complex machine recognition, but can be directly recognized by the image recognition template library, so that the efficiency of intelligent recognition is further improved.
In step S4, the method includes automatically marking the type, position, and outline of the device on the real-time picture, and further includes:
and importing the real-time picture and the type and the outline of the corresponding equipment into an image identification template library.
That is, in this embodiment, like the marking process in step S3, the real-time pictures marked each time the device type, the location, and the contour are marked are imported into the image recognition template library, so as to further enrich the template materials in the image recognition template library, that is, to improve the image recognition template library, thereby improving the efficiency of intelligent identification of the power device.
In step S4, the method for automatically marking the type, position, and contour of the device on the real-time picture specifically includes the following steps:
and S41, obtaining the type of the equipment in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library recognized in the image recognition mode, and taking the historical picture as a reference image and the real-time picture as an image to be processed.
S42, four vertexes of the polygonal outline of the device on the reference image are selected as feature points P1, P2, P3 and P4.
S43, taking a certain feature point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement;
in this embodiment, the numerical value of M affects the calculation strength, and the larger the value of M is, the larger the consumed calculation resource is, the slower the calculation speed is, but the higher the recognition accuracy is; the smaller the value of M, the faster the operation speed, but the lower the recognition accuracy. Therefore, in practical use, the value of M can be adjusted as needed, and generally, the value of M is only required to be 34-40 pixels, and in the embodiment, the value of M is 37.
S44, calculating the data block with the highest similarity to the data block T by adopting a global search method in the image to be processed, wherein the calculation formula is as follows:
Figure BDA0003324259640000121
wherein, Pi,jImage brightness data representing M pixel points in length and width with coordinate (i, j) as center in the image to be processed,wherein the value ranges of i are (M/2) to (W-M/2), the value ranges of j are (M/2) to (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed.
S45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed which is matched with the characteristic point in the reference image, repeating the steps S43 to S44 to obtain target position points T1, T2, T3 and T4 which are matched with the characteristic points P1, P2, P3 and P4 respectively.
And S46, obtaining the position of the equipment in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtaining the polygonal contour of the equipment in the image to be processed by combining the polygonal contour of the equipment in the reference image.
In other words, in this embodiment, as shown in fig. 3, coordinate operation is performed on the polygon of the device body profile in the real-time picture based on four feature points, and a minimum rectangle that can accommodate vertices of all polygons is calculated, so that data for marking the device position in the deep learning template can be obtained, and thus, the device position and profile can be automatically and quickly marked, and meanwhile, the device position data and profile data can also be applied to a deep learning template library for continuous training, and the deep learning template library is further updated, so as to improve the accuracy of intelligent power device identification. In the embodiment, as shown in fig. 2, for the real-time image subjected to the manual labeling processing, the labeled contour, position, and the like are subjected to the coordinate operation based on the four feature points to convert the polygonal contour into the minimum rectangle so as to provide training data for the deep learning, and further update of the deep learning template library is realized.
In this embodiment, the image data is obtained automatically and regularly for training, so that the regular update and maintenance of the deep learning template library are realized, and the accuracy of intelligent judgment of the working state of the power equipment is further improved.
Referring to fig. 5, a third embodiment of the present invention is:
a power equipment surface defect detection terminal 1 based on deep learning comprises a memory 2, a processor 3 and a computer program which is stored on the memory 2 and can be executed on the processor 3, wherein in the embodiment, the processor 3 realizes the steps in the first embodiment or the second embodiment when executing the computer program.
In summary, according to the method and the terminal for detecting the surface defects of the electrical equipment based on the deep learning, provided by the invention, firstly, real-time pictures automatically collected by an intelligent electric power inspection system at regular time are identified based on a deep learning template library obtained in advance, and the image identification template library is used for further identifying pictures which cannot be identified, so that accurate intelligent judgment of the working state of the electrical equipment is further realized; and the pictures which can not be distinguished by the image recognition template library are manually marked so as to further realize the intelligent distinguishing of the working state of the power equipment. The method comprises the steps that for a real-time picture marked each time, an image recognition template library is imported to enrich template materials in the image recognition template library, so that the intelligent distinguishing efficiency of the power equipment is improved; and the real-time pictures which are judged each time can be reintroduced into the deep learning template base for further deep learning training, so that the method replaces the mode that the existing judged pictures only enter the historical image base for historical data query without other use, can continuously update and perfect the deep learning template base, and realizes the self-feedback image identification processing method. A large amount of pictures collected by the intelligent inspection system are fully utilized, automatic maintenance and automatic updating of the power equipment template library are completed, and more accurate intelligent judgment of the working state of the power equipment is realized.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to the related technical fields, are included in the scope of the present invention.

Claims (10)

1. A method for detecting surface defects of electric power equipment based on deep learning is characterized by comprising the following steps:
s1, acquiring real-time pictures automatically and regularly acquired by the intelligent electric power inspection system;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
s3, distinguishing the real-time picture by adopting an image recognition mode, if the distinguishing fails, importing the real-time picture after being manually marked into the deep learning template library for deep learning, updating the deep learning template library, returning to S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
and S4, automatically marking the type, position and outline of equipment on the real-time picture, guiding the marked real-time picture into the deep learning template library for deep learning, and returning to S1 for identifying the next real-time picture after updating the deep learning template library.
2. The method for detecting surface defects of electrical equipment based on deep learning of claim 1, wherein in S3, an image recognition mode is adopted to discriminate the real-time picture, specifically:
and image recognition is carried out on the real-time pictures which cannot be recognized in the deep learning template library by adopting an image recognition template library so as to determine the type, the position and the outline of the equipment in the real-time pictures, the image recognition template library is formed by distinguishing all historical pictures which are automatically and regularly collected by an electric intelligent inspection system, and each historical picture comprises the type, the position and the outline of the corresponding equipment.
3. The method for detecting surface defects of electrical equipment based on deep learning of claim 2, wherein in S3, the real-time pictures after being manually marked are imported into the deep learning template library for deep learning, specifically:
receiving the type and position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the real-time picture and the type, position and outline of the equipment corresponding to the real-time picture into the image recognition template library and the deep learning template library for deep learning.
4. The method as claimed in claim 2, wherein the step S4 of automatically marking the type, position and contour of the device on the real-time picture further comprises:
and importing the real-time picture and the type and the outline of the corresponding equipment into the image recognition template library.
5. The method as claimed in claim 2, wherein the step S4 of automatically marking the type, position and contour of the device on the real-time picture includes the following steps:
s41, obtaining the type of equipment in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library recognized in an image recognition mode, taking the historical picture as a reference image, and taking the real-time picture as an image to be processed;
s42, selecting four vertexes of the polygonal outline of the equipment on the reference image as feature points P1, P2, P3 and P4;
s43, taking a certain feature point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement, wherein the value of M is 34-40 pixels;
s44, calculating the data block with the highest similarity to the data block T by adopting a global search method in the image to be processed, wherein the calculation formula is as follows:
Figure FDA0003324259630000021
wherein, Pi,jThe length and width of the pixel points which are expressed in the image to be processed and take the coordinate (i, j) as the center are all MThe image brightness data of (a), wherein the value range of i is (M/2) to (W-M/2), the value range of j is (M/2) to (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed;
s45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed which is matched with the characteristic point in the reference image, repeating the steps S43 to S44 to respectively obtain target position points T1, T2, T3 and T4 which are matched with the characteristic points P1, P2, P3 and P4;
and S46, obtaining the position of the equipment in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtaining the polygonal contour of the equipment in the image to be processed by combining the polygonal contour of the equipment in the reference image.
6. A deep learning based power equipment surface defect detection terminal, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
s1, acquiring real-time pictures automatically and regularly acquired by the intelligent electric power inspection system;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
s3, distinguishing the real-time picture by adopting an image recognition mode, if the distinguishing fails, importing the real-time picture after being manually marked into the deep learning template library for deep learning, updating the deep learning template library, returning to S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
and S4, automatically marking the type, position and outline of equipment on the real-time picture, guiding the marked real-time picture into the deep learning template library for deep learning, and returning to S1 for identifying the next real-time picture after updating the deep learning template library.
7. The terminal of claim 6, wherein in S3, an image recognition mode is used to determine the real-time image, and specifically:
and image recognition is carried out on the real-time pictures which cannot be recognized in the deep learning template library by adopting an image recognition template library so as to determine the type, the position and the outline of the equipment in the real-time pictures, the image recognition template library is formed by distinguishing all historical pictures which are automatically and regularly collected by an electric intelligent inspection system, and each historical picture comprises the type, the position and the outline of the corresponding equipment.
8. The terminal of claim 7, wherein in S3, the real-time image after being manually marked is imported into the deep learning template library for deep learning, specifically:
receiving the type and position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the real-time picture and the type, position and outline of the equipment corresponding to the real-time picture into the image recognition template library and the deep learning template library for deep learning.
9. The deep learning based power equipment surface defect detection terminal of claim 7, wherein the step S4 of automatically marking the type, position and contour of the equipment on the real-time picture further comprises:
and importing the real-time picture and the type and the outline of the corresponding equipment into the image recognition template library.
10. The terminal of claim 7, wherein in the step S4, the type, position and contour of the device are automatically marked on the real-time picture, and the method specifically includes the following steps:
s41, obtaining the type of equipment in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library recognized in an image recognition mode, taking the historical picture as a reference image, and taking the real-time picture as an image to be processed;
s42, selecting four vertexes of the polygonal outline of the equipment on the reference image as feature points P1, P2, P3 and P4;
s43, taking a certain feature point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement, wherein the value of M is 34-40 pixels;
s44, calculating the data block with the highest similarity to the data block T by adopting a global search method in the image to be processed, wherein the calculation formula is as follows:
Figure FDA0003324259630000041
wherein, Pi,jRepresenting image brightness data with length and width of M pixel points by taking the position of coordinates (i, j) as the center in the image to be processed, wherein the value range of i is (M/2) to (W-M/2), the value range of j is (M/2) to (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed;
s45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed which is matched with the characteristic point in the reference image, repeating the steps S43 to S44 to respectively obtain target position points T1, T2, T3 and T4 which are matched with the characteristic points P1, P2, P3 and P4;
and S46, obtaining the position of the equipment in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtaining the polygonal contour of the equipment in the image to be processed by combining the polygonal contour of the equipment in the reference image.
CN202111256280.9A 2021-10-27 2021-10-27 Power equipment surface defect detection method based on deep learning and terminal Pending CN114219753A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111256280.9A CN114219753A (en) 2021-10-27 2021-10-27 Power equipment surface defect detection method based on deep learning and terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111256280.9A CN114219753A (en) 2021-10-27 2021-10-27 Power equipment surface defect detection method based on deep learning and terminal

Publications (1)

Publication Number Publication Date
CN114219753A true CN114219753A (en) 2022-03-22

Family

ID=80696182

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111256280.9A Pending CN114219753A (en) 2021-10-27 2021-10-27 Power equipment surface defect detection method based on deep learning and terminal

Country Status (1)

Country Link
CN (1) CN114219753A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116704267A (en) * 2023-08-01 2023-09-05 成都斐正能达科技有限责任公司 Deep learning 3D printing defect detection method based on improved YOLOX algorithm
CN117725942A (en) * 2024-02-06 2024-03-19 浙江码尚科技股份有限公司 Identification early warning method and system for label texture anti-counterfeiting

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116704267A (en) * 2023-08-01 2023-09-05 成都斐正能达科技有限责任公司 Deep learning 3D printing defect detection method based on improved YOLOX algorithm
CN116704267B (en) * 2023-08-01 2023-10-27 成都斐正能达科技有限责任公司 Deep learning 3D printing defect detection method based on improved YOLOX algorithm
CN117725942A (en) * 2024-02-06 2024-03-19 浙江码尚科技股份有限公司 Identification early warning method and system for label texture anti-counterfeiting

Similar Documents

Publication Publication Date Title
Li et al. Automatic pavement crack detection by multi-scale image fusion
Zheng et al. HLU 2-Net: a residual U-structure embedded U-Net with hybrid loss for tire defect inspection
Chang et al. Fabric defect detection based on pattern template correction
CN112270310A (en) Cross-camera pedestrian multi-target tracking method and device based on deep learning
CN114219753A (en) Power equipment surface defect detection method based on deep learning and terminal
Zhou et al. Exploring faster RCNN for fabric defect detection
CN115601355A (en) Method and device for detecting and classifying product surface defects and storage medium
CN109509222B (en) Method and device for detecting linear object
CN110909660A (en) Plastic bottle detection and positioning method based on target detection
CN111402224A (en) Target identification method for power equipment
CN114863464B (en) Second-order identification method for PID drawing picture information
CN115439458A (en) Industrial image defect target detection algorithm based on depth map attention
Zhang et al. Multi-scale defect detection of printed circuit board based on feature pyramid network
CN111461010B (en) Power equipment identification efficiency optimization method based on template tracking
CN110516527B (en) Visual SLAM loop detection improvement method based on instance segmentation
CN113706455B (en) Rapid detection method for damage of 330kV cable porcelain insulator sleeve
CN111563896A (en) Image processing method for catenary anomaly detection
CN111091534A (en) Target detection-based pcb defect detection and positioning method
CN117115147B (en) Textile detection method and system based on machine vision
CN110287970B (en) Weak supervision object positioning method based on CAM and covering
KR20210122429A (en) Method and System for Artificial Intelligence based Quality Inspection in Manufacturing Process using Machine Vision Deep Learning
CN115984759A (en) Substation switch state identification method and device, computer equipment and storage medium
Li et al. Online workpieces recognition for the robotic spray-painting production line with a low-cost RGB-D camera
US11978197B2 (en) Inspection method for inspecting an object and machine vision system
Song et al. Instrument recognition in transformer substation base on image recognition algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination