CN111008587A - Intelligent visual recognition system based on deep learning and applied to robot - Google Patents
Intelligent visual recognition system based on deep learning and applied to robot Download PDFInfo
- Publication number
- CN111008587A CN111008587A CN201911200746.6A CN201911200746A CN111008587A CN 111008587 A CN111008587 A CN 111008587A CN 201911200746 A CN201911200746 A CN 201911200746A CN 111008587 A CN111008587 A CN 111008587A
- Authority
- CN
- China
- Prior art keywords
- network
- primary equipment
- robot
- fine
- grained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses an intelligent visual identification system based on deep learning, which is applied to a robot and comprises a defect data set, wherein the defect data set comprises one characteristic of a transformer substation primary equipment object, one state caused by the attachment of the transformer substation primary equipment object and a large amount of rust of the transformer substation primary equipment; the target detection model framework CSG Net comprises a feature extraction part, an FPN network, an RPN network and a fine-grained network; Res-neXt is adopted in the characteristic extraction part; after an object is selected in a continuous target detection frame, the fine-grained network further extracts features of a defect part, and the fine-grained network is realized by adding an APN network; the APN network trains based on the extracted features to obtain attribute region information; taking out and amplifying the attribute region crop, and taking the amplified crop as the input of the next-level scale network; and finally obtaining a defect area. The method and the device can accurately identify the defects in the transformer substation.
Description
Technical Field
The invention relates to the technical field of recognition systems, in particular to an intelligent visual recognition system based on deep learning, which is applied to a robot.
Background
In China, with sharp decline of the age-suitable population and aging of the population structure of physical workers, industries such as intensive type and service industry compete for the gradually reduced young labor population, so that the use cost is increased integrally. The intensive manufacturing industry, simple and repeated work and severe and high-risk work need to maintain the current production and service scale, and must rapidly introduce automatic equipment to replace the deficiency of labor force, and the trend of changing robots is inevitably irreversible. In the aspect of an electric power system, robots such as an overhead line inspection robot, a cable track inspection robot and a transformer substation inspection robot are gradually used in recent years, and the effect is good.
At present, the research on domestic robots has a history of more than 20 years, and the robots are also vigorously developed in China. However, since the research on this aspect is started later in China, there are some differences in the level of robot technology, the degree of practicality, and the stability, compared with the countries such as the United states and Japan.
Aiming at a wheeled substation inspection robot, a national grid company also establishes an important laboratory of an electric power robot, the robot product is used for electric power production, the high and new technical level of an electric power system is reflected, the characteristics of the electric power industry are highlighted in the field of robot and mechanical and electrical integration, a large amount of manpower and material resources are input, and a series of researches are carried out on various special robots such as a substation inspection robot and a high-voltage live working robot. In 2001, the Shandong electric power research institute provided the idea of using mobile robot technology to inspect equipment in a transformer substation for the first time, and was officially listed in the key project construction of the national 863 plan in 2002. In 11 months in 2005, a prototype of the inspection robot of the Changqing substation is put into operation formally, an innovative technical detection means is provided for popularization and application of an unattended substation, and the reliable and stable operation level of a power grid is improved.
After years of exploration, domestic great progress is made in the field of polling and researching of wheeled substation robot equipment, and the practical level of polling image intelligent processing technology is improved to promote the deep application of artificial intelligence technology in substation equipment polling, and the combined automatic polling test point construction of high-definition video and intelligent polling robots of a company substation is promoted; therefore, research, development and deployment of the transformer substation equipment abnormity inspection system are used for releasing manpower, so that the artificial intelligence technology is applied to the ground and serves the power industry more and more important.
The power industry has numerous devices, various defects and foreign matters, and is complex and variable. The object type of the data set used by the universal target detection model is greatly different. Its data are taken from scene shooting, and the environment complexity is higher, and the discernment target is more many, not only discerns the object, still need some characteristics of discernment object, such as breakage, fuzzy etc.. If the difference between the data sets is not considered, the difference of the identification target is not considered, the defects and the characteristic types of the foreign matters are not distinguished, and the defects and the characteristic types of the foreign matters are directly applied, the pit often falls into a pit with model loss, no convergence and poor identification effect. Therefore, it is necessary to adopt model-specific enhancement and image-specific processing for the equipment and image recognition in the power industry.
Disclosure of Invention
The invention aims to solve the problems that: the intelligent visual recognition system based on deep learning is applied to the robot, and defects in the transformer substation can be accurately recognized.
The technical scheme provided by the invention for solving the problems is as follows: an intelligent visual recognition system based on deep learning for robot includes
A defect data set comprising a characteristic of a substation primary equipment object, a state caused by substation primary equipment object attachment, and a large amount of corrosion of substation primary equipment;
the method comprises the steps of obtaining a target detection model framework CSGNet, wherein the target detection model framework CSGNet comprises a feature extraction part, an FPN network, an RPN network and a fine-grained network; the feature extraction part adopts Res-neXt which is the extension of Res-Net; Res-neXt adopts VGG stacking idea and increment split-transform-merge idea at the same time, expands single module of resNet, uses a plurality of cardalities; the FPN network fuses the multi-resolution characteristics of the image, and the structure can be divided into three parts: a bottom-up convolutional neural network, a top-down process and side connections between features; after an object is selected in a continuous target detection frame, the fine-grained network further extracts features of a defect part, and the fine-grained network is realized by adding an APN network; the APN network trains based on the extracted features to obtain attribute region information; taking out and amplifying the attribute region crop, and taking the amplified crop as the input of the next-level scale network; and finally obtaining a defect area.
Preferably, one characteristic of the substation primary equipment object comprises meter breakage, dial plate blurring and silica gel discoloration.
Preferably, one condition caused by the attachment of the substation primary equipment object comprises surface oil contamination.
Preferably, the large amount of corrosion of transformer substation primary equipment includes the block terminal corrosion, the corrosion of table meter, steel frame construction corrosion, stand corrosion.
Compared with the prior art, the invention has the advantages that: the method can classify and identify the defects and the foreign matter types of the equipment in the power industry by establishing the defect data set, and can accurately identify the defects in the transformer substation by processing the data through the CSGNet (target detection model framework).
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a comparison of ResnexT and resNet according to the present invention;
FIG. 2 is a schematic diagram of the FPN network of the present invention;
FIG. 3 is a diagram of an RPN network of the present invention;
FIG. 4 is a schematic diagram of fine-grained network operation according to the present invention;
FIG. 5 is a schematic view of the oil stain identification on the surface of the equipment of the invention;
FIG. 6 is a schematic view of rust identification in accordance with the present invention;
FIG. 7 is a schematic view of insulator breakage recognition according to the present invention;
Detailed Description
The following detailed description of the embodiments of the present invention will be provided with reference to the accompanying drawings and examples, so that how to implement the embodiments of the present invention by using technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented.
An intelligent visual identification system based on deep learning applied to a robot comprises a defect data set, wherein the defect data set comprises one characteristic of a transformer substation primary equipment object, one state caused by the attachment of the transformer substation primary equipment object and a large amount of rust of the transformer substation primary equipment;
the method comprises the steps of obtaining a target detection model framework CSGNet, wherein the target detection model framework CSGNet comprises a feature extraction part, an FPN network, an RPN network and a fine-grained network; the feature extraction part adopts Res-neXt which is the extension of Res-Net; Res-neXt adopts VGG stacking idea and increment split-transform-merge idea at the same time, expands single module of resNet, uses a plurality of cardalities; the FPN network fuses the multi-resolution characteristics of the image, and the structure can be divided into three parts: a bottom-up convolutional neural network, a top-down process and side connections between features; in practice, we find that the resolution of the bottom-up feature layer is reduced layer by layer, and then the information is partially lost after multi-layer processing from the top to the bottom of the topmost layer. Experiments show that a fast channel (bypass) is added to the bottom information to the top information, so that the top information is enriched. Making the feature processing part more efficient.
The RPN network is mainly used for ROI (region of interest extraction), which is consistent with the fast-rcnn.
After an object is selected in a continuous target detection frame, the fine-grained network further extracts features of a defect part, and the fine-grained network is realized by adding an APN network; the APN network trains based on the extracted features to obtain attribute region information; taking out and amplifying the attribute region crop, and taking the amplified crop as the input of the next-level scale network; and finally obtaining a defect area.
One characteristic of the primary equipment object of the transformer substation comprises meter damage, fuzzy dial and silica gel discoloration. Meter breakage, meter blurring, silica gel discoloration and the like, which firstly detect corresponding equipment objects, such as meter breakage, the meter needs to be detected firstly, the respirator needs to be detected firstly when the silica gel is discolored, and the deep learning target detection algorithm is adopted in the part. A fine-grained object classification algorithm is adopted for the mark breakage, the mark blurring and the silica gel discoloration, and the fine-grained classification algorithm is different from a general target detection algorithm in that the fine-grained algorithm can further distinguish a target according to finer granularity (for example, the target detection can identify whether the target is a cat or a dog, and the fine-grained classification algorithm can identify whether the target is a Zhonghua countryside dog or a bitdog), so that the fine-grained classification can be carried out on the target, and the local features of the object are further extracted, such as ears and mouth of different dogs. In particular to defect identification, a meter is damaged, and certain parts of a meter, such as certain parts of glass, have cracks; the insulator is damaged, and the edge of the insulator is provided with a fracture opening; the meter is fuzzy, and the glass surface on the dial is difficult to observe due to rain and the like; the surface oil stain is characterized in that a certain part of an object is covered with the oil stain to cause surface pollution. These are also features to be extracted by fine-grained partitions. In the labeling, we need to label the rectangular frame where the object is located and the defect portion corresponding to the defect portion.
In the fine-grained classification, we select a deep learning model based on the visual attention mechanism. The visual attention mechanism is a signal processing mechanism specific to human vision. The method is characterized in that when a vision system looks at things, a target area needing attention is obtained by rapidly scanning a global image, and then other useless information is suppressed to obtain an interested target. In deep convolutional networks, attention models can also be used to find regions of interest or discriminative regions in the image, and the regions of interest that are of interest to the convolutional network are different for different tasks. In defect identification, in addition to detecting objects (such as a table, a respiratory gas and an insulator), the defects on the objects are more concerned, such as damage and the like, fuzziness, oil pollution and other fine-grained characteristics, and the normal sample and the defect sample are distinguished through the fine-grained characteristics.
One state caused by the attachment of the substation primary equipment objects includes surface oil contamination. The transformer substation primary equipment is heavily rusted without completely selecting background objects (if the background is a whole iron sheet), but the transformer substation primary equipment can be still split into several different types: the corrosion of the distribution box, the corrosion of the meter, the corrosion of a steel frame structure, the corrosion of the stand column and the like.
The foregoing is merely illustrative of the preferred embodiments of the present invention and is not to be construed as limiting the claims. The present invention is not limited to the above embodiments, and the specific structure thereof is allowed to vary. All changes which come within the scope of the invention as defined by the independent claims are intended to be embraced therein.
Claims (4)
1. The utility model provides an intelligent visual identification system based on deep learning for robot which characterized in that: comprises that
A defect data set comprising a characteristic of a substation primary equipment object, a state caused by substation primary equipment object attachment, and a large amount of corrosion of substation primary equipment;
the target detection model framework CSG Net comprises a feature extraction part, an FPN network, an RPN network and a fine-grained network; the feature extraction part adopts Res-neXt which is the extension of Res-Net; Res-neXt adopts VGG stacking idea and increment split-transform-merge idea at the same time, expands single module of resNet, uses a plurality of cardalities; the FPN network fuses the multi-resolution characteristics of the image, and the structure can be divided into three parts: a bottom-up convolutional neural network, a top-down process and side connections between features; after an object is selected in a continuous target detection frame, the fine-grained network further extracts features of a defect part, and the fine-grained network is realized by adding an APN network; the APN network trains based on the extracted features to obtain attribute region information; taking out and amplifying the attribute region crop, and taking the amplified crop as the input of the next-level scale network; and finally obtaining a defect area.
2. The intelligent visual recognition system based on deep learning applied to robot as claimed in claim 1, wherein: one characteristic of the primary equipment object of the transformer substation comprises meter damage, fuzzy dial and silica gel discoloration.
3. The intelligent visual recognition system based on deep learning applied to robot as claimed in claim 1, wherein: one state caused by the attachment of the substation primary equipment objects includes surface oil contamination.
4. The intelligent visual recognition system based on deep learning applied to robot as claimed in claim 1, wherein: a large amount of rust of transformer substation's primary equipment includes the block terminal corrosion, the corrosion of table meter, steel frame construction corrosion, stand corrosion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911200746.6A CN111008587A (en) | 2019-11-29 | 2019-11-29 | Intelligent visual recognition system based on deep learning and applied to robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911200746.6A CN111008587A (en) | 2019-11-29 | 2019-11-29 | Intelligent visual recognition system based on deep learning and applied to robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111008587A true CN111008587A (en) | 2020-04-14 |
Family
ID=70113404
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911200746.6A Withdrawn CN111008587A (en) | 2019-11-29 | 2019-11-29 | Intelligent visual recognition system based on deep learning and applied to robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111008587A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111738229A (en) * | 2020-08-05 | 2020-10-02 | 江西小马机器人有限公司 | Automatic reading method for scale of pointer dial |
| CN111797725A (en) * | 2020-06-18 | 2020-10-20 | 浙江大华技术股份有限公司 | Equipment state detection method, device and system |
| CN112464652A (en) * | 2020-11-24 | 2021-03-09 | 昆明理工大学 | Method for extracting illustration and title thereof from literature |
| CN112651954A (en) * | 2020-12-30 | 2021-04-13 | 广东电网有限责任公司电力科学研究院 | Method and device for detecting insulator string dropping area |
| CN113034502A (en) * | 2021-05-26 | 2021-06-25 | 深圳市勘察研究院有限公司 | Drainage pipeline defect redundancy removing method |
| CN115365196A (en) * | 2022-08-19 | 2022-11-22 | 国网福建省电力有限公司福州供电公司 | Dirty cleaning equipment of net distribution overhead line insulated terminal |
| CN115830518A (en) * | 2023-02-15 | 2023-03-21 | 南京瀚元科技有限公司 | Intelligent frame extraction method for power inspection video in infrared scene |
| CN116980571A (en) * | 2023-09-25 | 2023-10-31 | 苏州市瑞思特智能制造有限公司 | Anti-collision bucket-wheel stacker-reclaimer, anti-collision system and anti-collision method thereof |
-
2019
- 2019-11-29 CN CN201911200746.6A patent/CN111008587A/en not_active Withdrawn
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111797725A (en) * | 2020-06-18 | 2020-10-20 | 浙江大华技术股份有限公司 | Equipment state detection method, device and system |
| CN111738229A (en) * | 2020-08-05 | 2020-10-02 | 江西小马机器人有限公司 | Automatic reading method for scale of pointer dial |
| CN111738229B (en) * | 2020-08-05 | 2020-11-24 | 江西小马机器人有限公司 | Automatic reading method for scale of pointer dial |
| CN112464652A (en) * | 2020-11-24 | 2021-03-09 | 昆明理工大学 | Method for extracting illustration and title thereof from literature |
| CN112651954A (en) * | 2020-12-30 | 2021-04-13 | 广东电网有限责任公司电力科学研究院 | Method and device for detecting insulator string dropping area |
| CN113034502A (en) * | 2021-05-26 | 2021-06-25 | 深圳市勘察研究院有限公司 | Drainage pipeline defect redundancy removing method |
| CN115365196A (en) * | 2022-08-19 | 2022-11-22 | 国网福建省电力有限公司福州供电公司 | Dirty cleaning equipment of net distribution overhead line insulated terminal |
| CN115830518A (en) * | 2023-02-15 | 2023-03-21 | 南京瀚元科技有限公司 | Intelligent frame extraction method for power inspection video in infrared scene |
| CN115830518B (en) * | 2023-02-15 | 2023-05-09 | 南京瀚元科技有限公司 | Intelligent frame extraction method for power inspection video under infrared scene |
| CN116980571A (en) * | 2023-09-25 | 2023-10-31 | 苏州市瑞思特智能制造有限公司 | Anti-collision bucket-wheel stacker-reclaimer, anti-collision system and anti-collision method thereof |
| CN116980571B (en) * | 2023-09-25 | 2023-12-19 | 苏州市瑞思特智能制造有限公司 | Anti-collision bucket-wheel stacker-reclaimer, anti-collision system and anti-collision method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111008587A (en) | Intelligent visual recognition system based on deep learning and applied to robot | |
| CN114973002A (en) | Improved YOLOv 5-based ear detection method | |
| CN111723774A (en) | Target identification method for power transmission equipment based on unmanned aerial vehicle inspection | |
| CN111127448B (en) | Method for detecting air spring fault based on isolated forest | |
| CN106326808A (en) | Method for detecting bird nests in power transmission line poles based on unmanned plane images | |
| CN103903008A (en) | Power transmission line fog level recognition method and system based on images | |
| CN111695506A (en) | Wind-induced foreign matter short-circuit fault early warning method and system for power transmission line | |
| CN114241364A (en) | Method for quickly calibrating foreign object target of overhead transmission line | |
| CN112508911A (en) | Rail joint touch net suspension support component crack detection system based on inspection robot and detection method thereof | |
| CN113065484A (en) | Insulator contamination state assessment method based on ultraviolet spectrum | |
| CN108596196B (en) | Pollution state evaluation method based on insulator image feature dictionary | |
| CN115995056A (en) | Automatic bridge disease identification method based on deep learning | |
| CN111259770A (en) | Rapid cable force testing system and method based on unmanned aerial vehicle platform and deep learning under complex background | |
| CN116167999A (en) | Distribution line zero-value insulator infrared thermal image detection method based on image matching | |
| CN110717883A (en) | Cable part automatic identification method based on infrared image | |
| CN113657258A (en) | Contact network fault identification system and method based on image processing | |
| Awrangjeb et al. | Classifier-free detection of power line pylons from point cloud data | |
| CN105528790A (en) | Transmission line small part identification method | |
| CN116704387B (en) | Power line channel inspection system and method based on video structuring | |
| CN113298077A (en) | Transformer substation foreign matter identification and positioning method and device based on deep learning | |
| CN112683916A (en) | Method and device for identifying missing or mounting error of small hardware fittings of current collecting line tower | |
| CN110969081B (en) | Power transmission line external force damage detection method based on KL divergence of multi-module division | |
| CN108573233B (en) | Power grid ceramic insulator identification method based on image processing technology | |
| CN112329858A (en) | Image recognition method for breakage fault of anti-loosening iron wire of railway motor car | |
| CN115937555A (en) | Industrial defect detection algorithm based on standardized flow model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200414 |