CN114581388A - Contact net part defect detection method and device - Google Patents
Contact net part defect detection method and device Download PDFInfo
- Publication number
- CN114581388A CN114581388A CN202210175758.3A CN202210175758A CN114581388A CN 114581388 A CN114581388 A CN 114581388A CN 202210175758 A CN202210175758 A CN 202210175758A CN 114581388 A CN114581388 A CN 114581388A
- Authority
- CN
- China
- Prior art keywords
- image
- data
- model
- image block
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 85
- 230000007547 defect Effects 0.000 title claims abstract description 71
- 238000000605 extraction Methods 0.000 claims abstract description 73
- 238000000034 method Methods 0.000 claims abstract description 62
- 238000012545 processing Methods 0.000 claims abstract description 53
- 238000004590 computer program Methods 0.000 claims abstract description 24
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 24
- 238000010586 diagram Methods 0.000 claims description 35
- 238000012549 training Methods 0.000 claims description 26
- 230000004660 morphological change Effects 0.000 claims description 6
- 238000003709 image segmentation Methods 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 description 22
- 230000008569 process Effects 0.000 description 21
- 239000011159 matrix material Substances 0.000 description 10
- 238000013135 deep learning Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000002372 labelling Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 239000012212 insulator Substances 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000011176 pooling Methods 0.000 description 5
- 230000002829 reductive effect Effects 0.000 description 5
- 230000011218 segmentation Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000000513 principal component analysis Methods 0.000 description 3
- 239000000725 suspension Substances 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 229910052799 carbon Inorganic materials 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a method and a device for detecting defects of parts of a contact network, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring image data of the contact net parts; intercepting the image data to obtain a target template image; processing the image data based on a deep convolutional neural network model to obtain a feature extraction model; the deep convolutional neural network model comprises a convolutional self-coding model; and according to the feature extraction model, carrying out similarity matching on the target template image and the image of the part to be detected so as to output a detection result. By adopting the method, the accuracy of the defect detection result of the contact net part can be effectively improved.
Description
Technical Field
The application relates to the technical field of image detection processing, in particular to a method and a device for detecting defects of parts of a contact network.
Background
In a railway system, a detection monitoring device (also called as a 4C system) for a suspension state of a contact network detects related parts such as nuts, cotter pins and insulators, and automatic identification and analysis are required to be completed in a high-resolution image, so that a maintenance suggestion is formed and the maintenance of the contact network is guided. At present, defect identification almost entirely depends on manual labeling data sets, and then a target Region (ROI) is positioned and detected and defect identified by using a deep learning method Of a neural network.
However, the manual labeling work for large-scale images is time-consuming and labor-consuming, the effectiveness of the traditional method is highly dependent on the quality of data labeling, and the detection result is inaccurate.
Disclosure of Invention
In view of the above, it is necessary to provide a method, an apparatus, a computer device, a computer readable storage medium, and a computer program product for detecting defects of components of an overhead line system.
In a first aspect, the application provides a contact net part defect detection method. The method comprises the following steps:
acquiring image data of the contact net parts;
intercepting the image data to obtain a target template image;
processing the image data based on a deep convolutional neural network model to obtain a feature extraction model; the deep convolutional neural network model comprises a convolutional self-coding model;
and according to the feature extraction model, carrying out similarity matching on the target template image and the image of the part to be detected so as to output a detection result.
In one embodiment, the step of processing the image data based on the deep convolutional neural network model to obtain a feature extraction model includes:
carrying out sliding window dicing with a preset size on the image data to obtain image block data; the preset size comprises a large size;
pre-detecting the image block data to obtain target image block data;
performing block cutting on the target image block data to obtain sub-image block data;
and inputting the sub-image block data into the convolution self-coding model for training to obtain the feature extraction model.
In one embodiment, the step of performing pre-detection processing on the image block data to obtain target image block data includes:
carrying out graying processing on the image block data to obtain grayscale image block data;
based on an image segmentation model, carrying out binarization on the gray-scale image block data to obtain binary image block data;
performing morphological change processing on the binary image block data to obtain corresponding outline area data of each binary image block;
and determining the binary image blocks with the contour area data being larger than or equal to the contour area threshold as the target image block data.
In one embodiment, the convolutional self-coding model comprises a coding model and a decoding model; the step of inputting the sub-image block data into the convolutional self-coding model for training to obtain the feature extraction model comprises:
training the sub-image block data for preset times by sequentially adopting the coding model and the decoding model; and under the condition that the preset times of training is finished, removing the decoding model, and determining the reserved coding model as the feature extraction model.
In one embodiment, the step of performing similarity matching on the target template image and the part image to be detected according to the feature extraction model to output a detection result includes:
sequentially carrying out dicing, pre-detection and block cutting processing on the part image to be detected to obtain a sub-image block to be detected;
respectively inputting the target template image and the sub image block to be detected into the feature extraction model for forward propagation so as to extract a template intermediate layer feature map and a to-be-detected intermediate layer feature map;
and processing the template intermediate layer characteristic diagram and the intermediate layer characteristic diagram to be detected to obtain the detection result.
In one embodiment, the template intermediate layer feature map comprises a template content feature map and a template style feature map; the characteristic diagram of the intermediate layer to be detected comprises a characteristic diagram of content to be detected and a characteristic diagram of a style to be detected; the detection result comprises a defect classification result and a defect positioning result;
the step of processing the template intermediate layer characteristic diagram and the intermediate layer characteristic diagram to be detected to obtain the detection result comprises the following steps:
determining content distance data of each sub image block to be detected relative to each target template image based on the template content characteristic diagram and the content characteristic diagram to be detected;
determining pattern distance data of each sub image block to be detected relative to each target template image according to the template pattern feature map and the pattern feature map to be detected;
and sequentially obtaining the defect classification result and the defect positioning result based on the content distance data and the pattern distance data.
In a second aspect, the application further provides a device for detecting defects of parts of the overhead line system. The device comprises:
the data acquisition module is used for acquiring image data of the contact net parts;
the intercepting module is used for intercepting the image data to obtain a target template image;
the feature extraction module is used for processing the image data based on a deep convolution neural network model to obtain a feature extraction model;
and the detection processing module is used for matching the similarity of the target template image and the image of the part to be detected according to the feature extraction model so as to output a detection result.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the following steps when executing the computer program:
acquiring image data of the contact net parts;
intercepting the image data to obtain a target template image;
processing the image data based on a deep convolutional neural network model to obtain a feature extraction model; the deep convolutional neural network model comprises a convolutional self-coding model;
and according to the feature extraction model, carrying out similarity matching on the target template image and the image of the part to be detected so as to output a detection result.
In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring image data of the contact net parts;
intercepting the image data to obtain a target template image;
processing the image data based on a deep convolutional neural network model to obtain a feature extraction model; the deep convolutional neural network model comprises a convolutional self-coding model;
and according to the feature extraction model, carrying out similarity matching on the target template image and the image of the part to be detected so as to output a detection result.
In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which when executed by a processor performs the steps of:
acquiring image data of the contact net parts;
intercepting the image data to obtain a target template image;
processing the image data based on a deep convolutional neural network model to obtain a feature extraction model; the deep convolutional neural network model comprises a convolutional self-coding model;
and according to the feature extraction model, carrying out similarity matching on the target template image and the image of the part to be detected so as to output a detection result.
According to the method, the device, the computer equipment, the storage medium and the computer program product for detecting the defects of the parts of the contact network, the obtained image data of the parts of the contact network are intercepted to obtain the target template image, the deep convolution neural network model is adopted to process the image data to obtain the feature extraction model, and then the similarity matching is carried out on the target template image and the images of the parts to be detected according to the feature extraction model, so that the detection result is output; this application can obtain the extremely strong characteristic extraction network of generalization through unsupervised degree of depth learning model training, compares in the degree of depth learning model that needs artifical mark data set in order to be used for target detection task and picture classification task, and this application need not to rely on artifical mark data set, can greatly reduced model experience risk, and effectively improves the accuracy of contact net spare part defect testing result.
Drawings
Fig. 1 is a schematic flow chart of a contact net part defect detection method in one embodiment;
FIG. 2 is a schematic flowchart illustrating the steps of processing image data based on a deep convolutional neural network model to obtain a feature extraction model in one embodiment;
FIG. 3 is a schematic view of a sliding window cutout in one embodiment;
FIG. 4 is a flowchart illustrating steps of pre-detecting image block data to obtain target image block data according to an embodiment;
FIG. 5 is a diagram illustrating an exemplary structure of a convolutional self-coding model;
fig. 6 is a schematic structural diagram of a contact net part defect detection device in one embodiment.
Detailed Description
To facilitate an understanding of the present application, the present application will now be described more fully with reference to the accompanying drawings. Embodiments of the present application are set forth in the accompanying drawings. This application may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
Tian Wang and Yang Chen et al in the article "A fast and robust controlled functional network-based detection model in product quality control" propose that the upper left corner pixel point of the specified size is moved from left to right from the upper left corner of the target picture every time in a sliding window manner, so as to realize the slicing of the input image, and then the sliced image block is sent to a depth network for learning. During reasoning, position-by-position identification is carried out in a sliding window detection mode. The similar template matching algorithm is to search whether the template picture exists in the target picture according to the appointed template picture, and when one pixel point is reached, the pixel point is taken as the top left corner vertex to intercept an image with the same size as the template from the source image and carry out pixel two-dimensional related comparison operation with the template, so that the region with the highest similarity is found as a matching result.
Junwen Chen et al propose a three-stage Defect localization method in the "Automatic Defect Detection of Fasteners on the content Support Device Using Deep connected Neural Network". Firstly, a target detection technology is adopted, and an SSD (solid State disk) algorithm is adopted as a detection network to position a main structural part; then, using YOLO (You Only Look one, YOLO) to quickly detect the fastener identified in the first stage; and finally, cutting the detected image, sending the image to a classification network for classification, and analyzing whether a fastener is lacked.
Xian Tao et al, in the article "Automatic Metallic Surface Defect Detection and registration with volumetric Neural Networks", have designed a cascaded Automatic encoder (CASAE) structure for Defect segmentation and localization. The cascade network converts the input defect image into a pixel-level predictive mask based on semantic segmentation. The defective regions of the segmentation result are classified into specific classes using a compressed Convolutional Neural Network (CNN).
In the above documents, a deep learning-based image classification model is pre-trained, image classes are used as labels, image features are input into a feature extraction structure composed of a convolutional layer, a pooling layer and the like to obtain high-dimensional feature vectors, and the high-dimensional feature vectors are mapped into a class vector space in an output layer, so that the prediction probability of each class is obtained, and then a cross entropy loss function is optimized to train the classification model. However, the effectiveness of these methods is highly dependent on the quality and scale of the data annotation.
In the aspect of quickly detecting the geometric characteristics of an image based on picture pixel information, the catenary suspension defect identification designed by Wen et al in a high-speed rail catenary suspension fastener defect identification method based on a 2-order cascade lightweight convolutional neural network is composed of 2 parts, namely fastener detection and defect identification. The fastener detection portion includes three modules: the system comprises a lightweight feature extraction network, a global attention module, a classifier and a detector which are mutually enhanced. The classifier judges whether the image contains the fastener, detects the position and the category of the fastener, and realizes mutual enhancement of the image and the fastener through a multitask loss function. The positioning and sorting of the fasteners can be done by these three modules. And a fastener defect identifying section capable of identifying a fastener defect for a given fastener image. The whole process is to cut the input image into blocks and then input the blocks into a fastener detection network to realize the positioning of the fasteners. The output fastener image is then scaled to 72 x 72 pixels as input to the fastener defect identification network, completing the final defect identification.
In the article of 'steel rail fastener nut missing detection system based on computer vision', royal et al propose a fastener positioning algorithm based on pixel point scanning statistics in a specific region, extract nut image features by using Principal Component Analysis (PCA), and classify by using a minimum distance classifier. Wherein the location about the fastener is with 75 threshold value with the pixel point carry out binaryzation with the picture, then fix a position the pillow region according to the number of pixel value 1 in the region, and then fix a position to the position of fastener. After the fasteners are positioned, the PCA algorithm is used for reducing the dimensions of the fasteners, and then the pictures are classified according to the Euclidean distance between the features.
Dapeng et al put forward a fastener subgraph quick positioning algorithm based on a confidence map and a fastener defect image identification method based on semi-supervised deep learning in the 'ballastless track fastener defect image identification method based on semi-supervised deep learning'. The fastener subgraph quick positioning algorithm based on the confidence graph is that a certain fastener subgraph neighborhood is firstly formulated in an orbit image based on prior information to serve as a guide graph, and a texture graph of the region and a confidence graph of the fastener subgraph in the region are constructed. And then, extracting the neighborhood of the next fastener subgraph according to the approximate interval of adjacent fasteners in the track image space, constructing a texture map and a confidence map of the neighborhood, and positioning the fastener subgraph in the new region by calculating a maximum value point of a correlation function between the confidence map and the guide map. The identification of the defect images of the fasteners based on semi-supervised deep learning refers to that firstly, unsupervised learning is carried out on a large data set to obtain representation characteristics, and then supervised learning is carried out on a relatively small labeled data set to obtain classification capability.
A depth network model is proposed in patent No. 2020104102387 entitled "high-speed rail catenary fastener identification and location method based on attention mechanism", and includes two modules, namely an attention module and a region recommendation module. Firstly, data is input into an attention Network of ResNet (ResNet) 50, then the data is repeatedly input into an attention module, and finally a depth Network characteristic diagram with a contact Network fastener characteristic representation is obtained. And then inputting the deep network feature map into a region recommendation module, and realizing border regression according to a classification network of pixel points. What can be obtained as a result are the positions and the types of the different fasteners.
Another patent with the patent number of 2017108643563 and the patent name of "a method for identifying and detecting fasteners of cantilever connectors of a high-speed rail contact Network supporting device" proposes that firstly, the cantilever connectors are marked on an image, then, the positioning of the cantilever connectors is realized by using a fast-conditional Neural Network (R-CNN) Network model, then, the fasteners of the contact Network are labeled into 6 types according to the direction and the abnormity, and then, the identification and the positioning of the fasteners are realized by using the fast R-CNN Network model on the basis of the cantilever connectors.
Another patent with patent number 2020105497232 and patent name "an anomaly detection method for insulator fasteners of high-speed rail contact networks" proposes that parts are labeled first, and then fasteners are identified and positioned by using a Faster R-CNN network model. And marking the identified contact network components as 6 types according to the direction and whether the contact network components are abnormal or not, and then carrying out abnormality detection on the fastener images through the distance measurement depth network.
From the goal of 4C system fastener defect identification, the following disadvantages are prevalent in the prior art:
the method is characterized in that small-size sliding window blocks are directly adopted for an original image, the execution speed is low, a large amount of semantic-free background information in a 4C image is detected, but a template matching algorithm is directly adopted and is very sensitive to the feature distribution of a template image, which means that when the content of the template image actually exists in a target image and image deformation such as scaling of a high-aspect ratio occurs, the calculated optimal matching area cannot be accurately positioned, the accurate positioning of the target is greatly limited, and the matching positioning requirement of complex images in an actual scene is hardly met.
The image classification pre-training model based on deep learning is high in model complexity and large in parameter quantity, if fine adjustment is directly carried out at an output classification end, the fact that the identification and classification of defects are extremely high in accuracy rate is difficult to guarantee, and if all parameters are updated to obtain high accuracy, huge gradient calculation cost is consumed.
The image processing algorithm can extract a satisfactory ROI target from an image with prominent geometric features through a binary image transformation algorithm and a contour extraction algorithm, wherein the image binarization algorithm is sensitive to local gray information if threshold-based binarization is used, and in practical application, if the contrast between a foreground target and a background is not high, the image is difficult to be effectively segmented to obtain a binary image with clear features, which is not beneficial to subsequent feature contour delineation.
Besides the common defects of relatively depending on labeled data, relatively complex algorithm model, high cost and the like, the paper and the patent documents have limitations in other aspects, such as that the method provided by the paper 'steel rail fastener nut missing detection system based on computer vision' and the paper 'ballastless track fastener defect image identification method based on semi-supervised deep learning' is relatively special in condition, the positioning of the fastener cannot be applied to other conditions, the fastener of a high-speed rail catenary cannot be effectively positioned, and the classification effect of other data defects is poor. The deep network models proposed by the three patents are poor in data processing effect and efficiency for extremely high signal-to-noise ratio, wherein the classification and positioning of different fasteners are only realized in the patent of the high-speed rail contact network fastener identification and positioning method based on the attention mechanism and the patent of the high-speed rail contact network supporting device cantilever connecting piece fastener identification and detection method, and the abnormal detection of the fasteners is not realized.
The method for detecting the defects of the parts of the contact network does not need to rely on a manual labeling data set, can obtain the feature extraction network with extremely high generalization, greatly reduces labeling burden and reduces model experience risk compared with a deep learning model needing manual arrangement labeling for a target detection task and a picture classification task, and realizes effective improvement of the accuracy of the detection result of the defects of the parts of the contact network.
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
In an embodiment, as shown in fig. 1, a method for detecting defects of contact network components is provided, and this embodiment is illustrated by applying the method to a terminal, it is to be understood that the method may also be applied to a server, may also be applied to a system including the terminal and the server, and is implemented by interaction between the terminal and the server. In this embodiment, the method includes the steps of:
step S110, acquiring image data of parts of the overhead line system;
specifically, image data of the state of a suspended part (or a fastener) of a heavy haul railway contact network is acquired, in some examples, the image data can be obtained by shooting from a global or local angle through an imaging device on the roof of a patrol car, the imaging size can be 5120px × 5120px, wherein the patrol car starts from a specified line and shoots all high-definition images of the contact network along the line; further, the image data may include image data of a nut, a cotter pin, and an insulator; in some examples, the inspection line may be used as a root directory for storing, the second level directory is a quarter, the third level directory is a name of each part, and the acquired image data of each part is stored, so that the file structure stores the image data that we captured.
Step S120, intercepting the image data to obtain a target template image;
specifically, various parts with clear features are cut out from the acquired image data to serve as target template images, in some examples, whether the original image contains various types of parts or not can be observed, and the images of the parts containing detection targets are sorted out, further, the images of the parts needing to be detected are cut out (cut out), and the sizes of the cut-out images are unified to be 200px × 200px, for example, nut target template images related to normal nut and nut missing, loosening and wrong insertion direction, cotter target template images related to normal cotter and cotter missing, insufficient angle and wrong insertion direction, and insulator target template images related to normal insulators and insulator breakage can be cut out.
Step S130, processing the image data based on the deep convolutional neural network model to obtain a feature extraction model; the deep convolutional neural network model comprises a convolutional self-coding model;
specifically, a deep convolutional neural network model is trained by using image data, so that the features of the image in a target area can be extracted, and a feature extraction model is obtained; in some examples, a convolutional self-coding model may be chosen as the training model;
in one embodiment, as shown in fig. 2, the step S130 of processing the image data based on the deep convolutional neural network model to obtain the feature extraction model may include:
step S210, performing sliding window dicing of a preset size on the image data to obtain image block data; the preset size includes a large size;
specifically, all image data are subjected to sliding window dicing with a preset size, and further, a large-size sliding window can be adopted for dicing, so that image block data is obtained, wherein each image can be subjected to sliding window dicing to obtain a plurality of small image blocks; in some examples, since the sizes of the acquired image data are unified to 5120px × 5120px and the size of the target template image is unified to 200px × 200px, the preset size (block height and width) of the primary sliding window cut block may be set to 512px × 512px, and a 50% overlap is retained when cropping.
In one embodiment, a schematic diagram of a sliding window slice of image data is shown in fig. 3, where the size of the image data is 5120px × 5120px, the preset size of the initial sliding window slice is 512px × 512px, 50% overlap is retained when cropping, and the step size (stride) is 256 px.
The positioning target is obtained through the efficient sliding window cutting block, and self-adaptive picture cutting is achieved.
Step S220, pre-detecting the image block data to obtain target image block data;
specifically, in some examples, an image processing algorithm may be used to perform pre-detection processing on image block data, remove background image blocks, and retain image blocks in which detection targets may exist, so as to obtain target image block data.
In one embodiment, as shown in fig. 4, the step S220 of performing a pre-detection process on the image block data to obtain target image block data may include:
step S410, carrying out graying processing on the image block data to obtain grayscale image block data;
step S420, carrying out binarization on the gray image block data based on an image segmentation model to obtain binary image block data;
step S440, performing morphological change processing on the binary image block data to obtain corresponding outline area data of each binary image block;
step S440, determining the binary image block with the contour area data greater than or equal to the contour area threshold as the target image block data.
Specifically, averaging pixels of each channel of image block data of three color channels to obtain gray image block data of a single channel, and denoising the gray image block data through Gaussian smoothing of 3 × 3 kernel size, median smoothing of 5 × 5 kernel size and Sobel operator (Sobel operator) of 3 × 3 kernel size, so that the characteristic edge of a gray image is clearer; then, carrying out binarization on the gray-scale image block data by using an image segmentation model, and in some examples, obtaining binary image block data by using a local self-adaptive threshold segmentation algorithm;
further, the binary image block data is subjected to a morphological change process, and in some examples, a 9 × 1 rectangular template for the dilation operation and a 9 × 7 rectangular template for the erosion operation may be respectively constructed for the binary image block data; then, sequentially executing primary expansion, primary corrosion and tertiary expansion to fill local cavities and eliminate local isolated points in each binary image in the binary image block data, thereby forming a complete characteristic region for searching and extracting the outline;
further, contour extraction is carried out on the binary image blocks after morphological change processing, so that contour area data corresponding to each binary image block are obtained, and therefore the binary image blocks with the contour area data larger than or equal to a contour area threshold are determined as target image block data;
in some examples, since the size of the target template image is 200px × 200px, the contour area threshold may be set to 10000; and (3) performing a Suzuki algorithm (contour tracking algorithm) on the binary image block data subjected to the morphological change processing to find a maximum external contour and obtain a contour area, if the contour area is larger than a contour area threshold, retaining the corresponding binary image block, and determining the corresponding binary image block as a target image block for further dicing and positioning.
In the adopted positioning process of the detected target, the image block is obtained by adopting the large-size sliding window, and then the image block is subjected to pre-detection by using the image processing algorithm, so that background information without detection is omitted, the high time overhead of sequentially classifying generated small image blocks in the prior art is greatly reduced, meanwhile, the global search of the target detection model for the image is effectively avoided, the harsh requirements of the target processing algorithm for obtaining the target and the target characteristics of the original image are also avoided, for example, the original image is required to have good geometric characteristics or high signal-to-noise ratio and the like, and the image is easy to segment, and the application can judge whether the image block has the background information without detection in the image block only by simple ROI (region of interest) conversion, thereby effectively overcoming the defects that the image segmentation algorithm has high requirements for image quality in the binarization process and the target is difficult to search after segmentation, and the pre-detection method of the image processing algorithm can rapidly filter out a large amount of background information without detection requirements in the 4C system image And (4) information.
Step S230, performing block cutting on the target image block data to obtain sub-image block data;
specifically, each target image block in the target image block data is continuously cut in a blocking mode to obtain sub-image block data with smaller size; in some examples, since the current target image block has a height and width of 512px × 512px, setting this sub-block size to 256px × 256px preserves 50% overlap when clipping.
The image processing algorithm is adopted for pre-detection processing and then is gradually cut into blocks, so that the ROI can be more quickly and accurately positioned, and reliable data pre-processing is provided for training a convolution self-coding model.
Step S240, inputting the sub-image block data into the convolutional self-coding model for training, and obtaining a feature extraction model.
Specifically, the convolutional self-coding model with bidirectional conduction can process information of sub-image block data, and obtains a feature extraction model through unsupervised learning training.
The mode of training the convolution self-coding model through unmarked sub-image block data belongs to unsupervised learning, and theoretically infinite data can be trained, so that a feature extraction model with extremely strong generalization is obtained, the feature extraction model is not required to be used for classification, the marking overhead is reduced, compared with a deep learning model needing manual arrangement of marks for a target detection task and an image classification task, the marking burden is greatly reduced, and the model experience risk is reduced.
In one embodiment, the convolutional self-coding model comprises a coding model and a decoding model; the step S240 of inputting the sub-image block data into the convolutional self-coding model for training to obtain a feature extraction model includes:
training the sub-image block data for a preset number of times by sequentially adopting a coding model and a decoding model; and under the condition that the preset times of training is finished, removing the decoding model, and determining the reserved coding model as a feature extraction model.
Specifically, sub-image block data is input into an encoding model, the encoding model can compress according to the information of the sub-image block data to extract features, then feature map data extracted from the sub-image block data is input into a decoding model to be trained for a preset number of times, so that the output of the decoding model (decompression model) is approximate to the original sub-image block data, the decoding model is removed when the training is completed, and the remaining encoding model is the feature extraction model.
To better illustrate the present application, the following is described with reference to a specific example:
the coding model and the decoding model can be built by adopting a coding network and a decoding network respectively, assuming that an input sub image block is X, the coding network is represented by a function f, and the decoding network is represented by a function g, the output encoded of the coding network can be represented by the following formula (1), and the output decoded of the decoding network can be represented by the following formula (2):
encoded＝f(X) (1)
decoded＝g(encoded)＝g(f(X)) (2)
wherein the loss function is shown in the following formula (3):
loss(X，decoded)＝∑i,j(Xi,j-decodedi,j)2 (3)
wherein i, j represents a coordinate position; a smaller loss function means that the output of the decoding network is closer to the original image. And finally, removing a decoding network, namely a function g, and only keeping the coding network shown in the formula (1) as a feature extraction model, namely the feature extraction network.
In some examples, the coding network includes convolutional layers and maxporoling (max pooling) layers, where maxporoling is responsible for spatial downsampling, particularly taking the neural network model of VGG-19(Visual Geometry Group, VGG). The decoding network includes convolutional layer and upsampling, and specifically adopts a structure similar to that of VGG-19, except that in contrast to the coding network, in one example, a specific structural diagram of a convolutional self-coding model is shown in fig. 5, and in detail, the process of coding network training may include the following steps:
(1) assuming that the data dimension of the input sub image block is (batch size, 3, h, w), wherein the batch size represents the data size of one iteration, 3 represents the initial input channel number, h represents the height of the sub image block, and w represents the width of the sub image block; inputting sub-image block data into a convolutional layer with two convolution kernels of 3 × 3, and then passing through a once-through Maxpooling pooling layer, so that the height and width (hereinafter referred to as height and width) in the data dimension of the sub-image block are halved for the first time, the number of channels becomes 64, conv3-64 in fig. 5 represents the convolutional layer parameters of the process, wherein 3 represents the size of the convolution kernels, and 64 represents the number of channels;
(2) inputting the output in the step (1) into a convolution layer with two convolution kernels of 3 × 3, and then performing primary maxporoling pooling, wherein the height and the width are halved for the second time, the number of channels becomes 128, and conv3-128 in fig. 5 represents parameters of the convolution layer in the process;
(3) the output in the step (2) is further processed by a convolution layer and a first pooling layer of which the convolution kernels are 3 multiplied by 3, the height and the width are halved for the third time, the number of channels is 256, and conv3-256 in the process is represented by convolution layer parameters in the process in fig. 5;
(4) similarly, the output in step (3) passes through the convolutional layer with convolution kernel of 3 × 3 and the maxporoling layer once, then the height and width are halved for the fourth time, the number of channels becomes 512, and conv3-512 in fig. 5 represents the convolutional layer parameters of the process;
(5) finally, after the output in the step (4) passes through the convolution layer with convolution kernels of 3 × 3 and the maxpoloring layer once, the height and the width are halved for the fifth time, the number of channels remains unchanged and is still 512, and conv3-512 in fig. 5 represents the parameters of the convolution layer in the process; the output at this time is encoded.
Accordingly, the process of decoding the network training may include the steps of:
firstly, inputting the output of the step (5) in the coding network training, namely encoded into four convolution layers and one upsampling layer, wherein the height and the width are amplified twice for the first time, and the number of channels is still 512;
secondly, the output in the first step passes through a convolution layer four times and an upsampling layer one time, the height and the width are amplified twice for the second time, the number of channels is unchanged and is still 512;
thirdly, the output in the second step is similarly subjected to convolution layer and upsampling layer for four times, the height and the width are amplified twice for the third time, and the number of channels is changed into 256;
fourthly, inputting the output in the third step to two convolution layers and a primary upsampling layer, doubling the height and the width for the fourth time, and changing the number of channels into 128;
fifthly, the output in the step IV passes through two convolution layers, the number of channels is changed into 64, and the height and the width are amplified twice for the fifth time after passing through an upsampling layer for one time;
finally, the output in the fifth step passes through a convolution layer, the number of channels is changed back to 3, and the output at the moment is consistent with the data dimension of the initial sub-image block in terms of the number of channels and the height and width.
The preset number of training processes of the coding network and the decoding network can be represented by t, and specifically, the t parameter can be set according to the tolerable computation time, that is, assuming that the coding network and the decoding network compute 10s once, and the tolerable time is 600s, then t is 60.
And under the condition that training is completed for a preset number of times, removing the part of the neural network in the decoding network, only reserving the part of the coding network, wherein the reserved part is the convolution part of the VGG neural network, and the reserved part is used as a feature extraction network, and the encoded output of the feature extraction network is the feature map data extracted from the sub-image block data.
And step S140, performing similarity matching on the target template image and the image of the part to be detected according to the feature extraction model so as to output a detection result.
Specifically, the obtained feature extraction model is used for inputting the image of the part to be detected and the target template image into the feature extraction model respectively for similarity matching, so that a detection result is obtained.
The feature extraction model obtained by training the convolution self-coding model is used for feature similarity calculation, and the network calculation complexity is far lower than that of a target detection model in the prior art.
In one embodiment, the step of performing similarity matching on the target template image and the part image to be detected according to the feature extraction model to output a detection result comprises:
sequentially carrying out dicing, pre-detection and block cutting processing on the part image to be detected to obtain a sub-image block to be detected;
respectively inputting the target template image and the sub image block to be detected into a feature extraction model for forward propagation so as to extract a template intermediate layer feature map and a to-be-detected intermediate layer feature map;
and processing the template intermediate layer characteristic diagram and the intermediate layer characteristic diagram to be detected to obtain a detection result.
Specifically, firstly, a sliding window dicing operation with the same preset size as that in step S210 is performed on the part image to be detected, in some examples, the preset size of 512px × 512px may be selected for sliding window dicing, so as to obtain an image block to be detected, and a plurality of image blocks to be detected may be obtained from a single part image to be detected; further, performing the same pre-detection processing as that in step S220, specifically, performing the pre-detection processing on the image block to be detected by using an image processing algorithm; further, the image block to be detected after the pre-detection processing is subjected to the same block cutting processing as that in step S230, in some examples, the size of the block cutting is also set to 256px × 256px, so as to obtain a plurality of sub image blocks to be detected;
further, the obtained sub-image block to be detected and the target template image are respectively input into a feature extraction model for forward propagation, the target template image can extract a corresponding template intermediate layer feature map through the feature extraction model, the sub-image block to be detected can extract a corresponding intermediate layer feature map to be detected through the feature extraction model, and then the template intermediate layer feature map and the intermediate layer feature map to be detected are subjected to similarity matching, so that a detection result is obtained.
In one embodiment, the template middle layer feature map can comprise a template content feature map and a template style feature map; the characteristic diagram of the intermediate layer to be detected can comprise a content characteristic diagram to be detected and a style characteristic diagram to be detected; the detection result comprises a defect classification result and a defect positioning result;
processing the template intermediate layer characteristic diagram and the intermediate layer characteristic diagram to be detected to obtain a detection result, wherein the step of obtaining the detection result comprises the following steps:
determining content distance data of each sub image block to be detected relative to each target template image based on the template content characteristic diagram and the content characteristic diagram to be detected;
determining the pattern distance data of each sub image block to be detected relative to each target template image according to the template pattern feature map and the pattern feature map to be detected;
and obtaining a defect classification result and a defect positioning result in sequence based on the content distance data and the pattern distance data.
Specifically, in some examples, since the feature extraction network is a VGG-19 architecture, an intermediate layer at the top of the feature extraction network, i.e., [25] th layer, may be used as a content layer to calculate a large-scale feature similarity between the sub image block and the target template image, and determine content distance data of each sub image block to be detected with respect to each target template image according to the template content feature map extracted by the content layer and the content feature map to be detected;
in addition, the middle layer [0, 5, 10, 19, 28] at the bottom of the feature extraction network can be used as a pattern layer for calculating the local texture feature similarity between the sub image blocks and the target template image, and determining pattern distance data of each sub image block to be detected relative to each target template image according to the template pattern feature map extracted by the pattern layer and the pattern feature map to be detected;
further, according to the obtained content distance data and the pattern distance data, a defect classification result and a defect positioning result of the part to be detected are obtained.
In a specific example, the target template image may be pre-processed to be converted into a 4-dimensional small-batch tensor expressed by numerical values from 0 to 1, where each dimension is expressed as (number of templates, 3, height, and width), where 3 represents the number of initial input channels, and then the 4-dimensional small-batch tensor is input into the feature extraction network to be propagated forward, and the template content feature maps extracted by the content layer are stored, and each Gram matrix is calculated by each template style feature map extracted by the style layer, and the Gram matrices of each style layer are stored, and the specific calculation method is as follows:
because the feature map dimensions of the current pattern layer are (number of templates, c, h, w), wherein c represents the number of channels, h and w represent the height and width of a single feature map respectively, the last two dimensions in the dimensions are obtained by flattening transformation (number of templates, c, h, w), and then a Gram matrix of dot products between feature vectors represented under each channel in the last two dimensions is calculated, specifically, the Gram matrix can be calculated by the following formula (4):
Grami,j＝<ci，cj> (4)
wherein, ciDenotes the ith channel, cjRepresents the jth channel; correspondingly, obtaining a to-be-detected content characteristic diagram under the content layer of the sub-image block to be detected and each to-be-detected pattern characteristic diagram under each pattern layer according to the extraction method, and respectively calculating each Gram matrix according to each to-be-detected pattern characteristic diagram;
further, L2_ dist is calculated for the content feature sets of each target template image and each subimage block to be detectedcontentThe distance is calculated by the following formula (5):
and calculating L2_ dist according to the Gram matrix of the target template image and the sub image block to be detectedstyleThe distance is calculated by the following formula (6):
in the above formula (5) and the above formula (6), l represents the number of layers of the neural network model VGG-19, "-" represents subtraction by elements of the matrix, and square represents squaring by elements of the matrix, and C in the formula (5) represents a feature map matrix under each channel.
The obtained content distance dimension of each sub image block to be detected relative to each target template image is represented as (number of sub image blocks × number of templates, C, h, w), and the pattern distance dimension is represented as (number of sub image blocks × number of templates, Gram matrix); further, averaging the last three positions in the dimension of the content distance to obtain the content distance of each sub image block to be detected relative to each template; and taking an element mean value of a Gram matrix in the pattern distance dimension to obtain the pattern distance of each sub image block to be detected relative to each template, wherein the smaller the distance is, the greater the similarity is.
Further, selecting the class of a target template image with the maximum similarity to the sub-image block to be detected as a defect classification result of the currently input part image to be detected, specifically, selecting the target template image with the minimum pattern distance to the sub-image block to be detected as the defect class of the sub-image block to be detected in the maximum similarity judgment process, and if a plurality of target template images have the same pattern distance, selecting the target template image with the minimum content distance to the sub-image block to be detected from the plurality of target template images as the defect class of the sub-image block to be detected;
further, the defect classification results of the sub-image blocks to be detected can be obtained, in the sub-image blocks to be detected of which the defect types are judged, similar to the judgment process of the maximum similarity, the sub-image block with the maximum similarity to the target template image is selected under the defect types to serve as the defect positioning result of the currently input part image to be detected, and as the number of times of slicing and the number of effective blocks (including the image block of the detection target) are fewer, the part image to be detected is positioned more quickly.
According to the method for detecting the defects of the parts of the overhead line system, the non-labeling data set is used for training the convolution self-coding model, so that a feature extraction model for unsupervised extraction of features in the related images of the overhead line system is obtained, the similarity between the images is calculated through image pattern differences, the positions and defect types of the target parts of the input images are predicted according to the similarity, and the accuracy of the detection results of the defects of the parts of the overhead line system is greatly improved.
It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.
Based on the same inventive concept, the embodiment of the application also provides a defect detection device for the parts of the overhead line system, which is used for realizing the defect detection method for the parts of the overhead line system. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the method, so that specific limitations in an embodiment of the device for detecting defects of parts of the overhead line system provided below can be referred to the limitations on the method for detecting defects of parts of the overhead line system, and are not described herein again.
In one embodiment, as shown in fig. 6, there is provided a contact net part defect detecting apparatus, including: data acquisition module 610, intercept module 620, feature extraction module 630 and detection processing module 640, wherein:
the data acquisition module 610 is used for acquiring image data of parts of the overhead line system;
an intercepting module 620, configured to intercept the image data to obtain a target template image;
the feature extraction module 630 is configured to process the image data based on the deep convolutional neural network model to obtain a feature extraction model;
and the detection processing module 640 is used for performing similarity matching on the target template image and the part image to be detected according to the feature extraction model so as to output a detection result.
All modules in the defect detection device for the parts of the overhead line system can be completely or partially realized through software, hardware and a combination of the software and the hardware. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to realize the contact net part defect detection method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
In an embodiment, a computer program product is provided, comprising a computer program which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (10)
1. A contact net part defect detection method is characterized by comprising the following steps:
acquiring image data of the contact net parts;
intercepting the image data to obtain a target template image;
processing the image data based on a deep convolutional neural network model to obtain a feature extraction model; the deep convolutional neural network model comprises a convolutional self-coding model;
and according to the feature extraction model, carrying out similarity matching on the target template image and the image of the part to be detected so as to output a detection result.
2. The method of claim 1, wherein the step of processing the image data based on the deep convolutional neural network model to obtain a feature extraction model comprises:
carrying out sliding window dicing with a preset size on the image data to obtain image block data; the preset size comprises a large size;
pre-detecting the image block data to obtain target image block data;
performing block cutting on the target image block data to obtain sub-image block data;
and inputting the sub-image block data into the convolution self-coding model for training to obtain the feature extraction model.
3. The method according to claim 2, wherein the step of performing pre-detection processing on the image block data to obtain target image block data comprises:
carrying out graying processing on the image block data to obtain grayscale image block data;
based on an image segmentation model, carrying out binarization on the gray-scale image block data to obtain binary image block data;
performing morphological change processing on the binary image block data to obtain corresponding outline area data of each binary image block;
and determining the binary image blocks with the contour area data being larger than or equal to the contour area threshold as the target image block data.
4. The method of claim 2, wherein the convolutional self-coding model comprises a coding model and a decoding model; the step of inputting the sub-image block data into the convolutional self-coding model for training to obtain the feature extraction model comprises:
training the sub image block data for preset times by sequentially adopting the coding model and the decoding model; and under the condition that the preset times of training is finished, removing the decoding model, and determining the reserved coding model as the feature extraction model.
5. The method according to any one of claims 1 to 4, wherein the step of performing similarity matching on the target template image and the part image to be detected according to the feature extraction model to output a detection result comprises:
sequentially carrying out dicing, pre-detection and block cutting processing on the part image to be detected to obtain a sub-image block to be detected;
inputting the target template image and the sub image block to be detected into the feature extraction model respectively for forward propagation so as to extract a template intermediate layer feature map and a to-be-detected intermediate layer feature map;
and processing the template intermediate layer characteristic diagram and the intermediate layer characteristic diagram to be detected to obtain the detection result.
6. The method of claim 5, wherein the template intermediate layer feature map comprises a template content feature map and a template style feature map; the characteristic diagram of the middle layer to be detected comprises a content characteristic diagram to be detected and a style characteristic diagram to be detected; the detection result comprises a defect classification result and a defect positioning result;
the step of processing the template intermediate layer characteristic diagram and the intermediate layer characteristic diagram to be detected to obtain the detection result comprises the following steps:
determining content distance data of each sub image block to be detected relative to each target template image based on the template content characteristic diagram and the content characteristic diagram to be detected;
determining pattern distance data of each sub image block to be detected relative to each target template image according to the template pattern feature map and the pattern feature map to be detected;
and sequentially obtaining the defect classification result and the defect positioning result based on the content distance data and the pattern distance data.
7. The utility model provides a contact net spare part defect detecting device which characterized in that, the device includes:
the data acquisition module is used for acquiring image data of the contact net parts;
the intercepting module is used for intercepting the image data to obtain a target template image;
the feature extraction module is used for processing the image data based on a deep convolutional neural network model to obtain a feature extraction model;
and the detection processing module is used for matching the similarity of the target template image and the image of the part to be detected according to the feature extraction model so as to output a detection result.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210175758.3A CN114581388A (en) | 2022-02-24 | 2022-02-24 | Contact net part defect detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210175758.3A CN114581388A (en) | 2022-02-24 | 2022-02-24 | Contact net part defect detection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114581388A true CN114581388A (en) | 2022-06-03 |
Family
ID=81770940
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210175758.3A Pending CN114581388A (en) | 2022-02-24 | 2022-02-24 | Contact net part defect detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114581388A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114708267A (en) * | 2022-06-07 | 2022-07-05 | 浙江大学 | Image detection processing method for corrosion defect of tower stay wire on power transmission line |
| CN115272831A (en) * | 2022-09-27 | 2022-11-01 | 成都中轨轨道设备有限公司 | Transmission method and system for monitoring images of suspension state of contact network |
| CN116797602A (en) * | 2023-08-25 | 2023-09-22 | 东莞市爱培科技术有限公司 | Surface defect identification method and device for industrial product detection |
| CN116824271A (en) * | 2023-08-02 | 2023-09-29 | 上海互觉科技有限公司 | SMT chip defect detection system and method based on tri-modal vector space alignment |
| CN117036267A (en) * | 2023-08-01 | 2023-11-10 | 广州伊索自动化科技有限公司 | Curved surface printing detection method, system and storage medium |
| CN117115178A (en) * | 2023-08-23 | 2023-11-24 | 国网四川省电力公司电力科学研究院 | Semi-parameter sharing-based power infrared inspection image segmentation and detection method |
| WO2024000420A1 (en) * | 2022-06-30 | 2024-01-04 | 宁德时代新能源科技股份有限公司 | Defect detection method and device |
| CN117436012A (en) * | 2023-07-28 | 2024-01-23 | 中国建筑第二工程局有限公司 | Concrete square pile damage identification method based on self-adaptive multichannel convolutional neural network |
| CN117474916A (en) * | 2023-12-27 | 2024-01-30 | 苏州镁伽科技有限公司 | Image detection method, electronic equipment and storage medium |
| WO2024060917A1 (en) * | 2022-09-23 | 2024-03-28 | 中国电信股份有限公司 | Defect identification method, apparatus and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108982508A (en) * | 2018-05-23 | 2018-12-11 | 江苏农林职业技术学院 | A kind of plastic-sealed body IC chip defect inspection method based on feature templates matching and deep learning |
| CN109829903A (en) * | 2019-01-28 | 2019-05-31 | 合肥工业大学 | A kind of chip surface defect inspection method based on convolution denoising self-encoding encoder |
| CN111401384A (en) * | 2020-03-12 | 2020-07-10 | 安徽南瑞继远电网技术有限公司 | Transformer equipment defect image matching method |
| CN111784644A (en) * | 2020-06-11 | 2020-10-16 | 上海布眼人工智能科技有限公司 | Printing defect detection method and system based on deep learning |
| CN112991349A (en) * | 2019-12-17 | 2021-06-18 | 阿里巴巴集团控股有限公司 | Image processing method, device, equipment and storage medium |
| CN113034492A (en) * | 2021-04-19 | 2021-06-25 | 深圳市华汉伟业科技有限公司 | Printing quality defect detection method and storage medium |
-
2022
- 2022-02-24 CN CN202210175758.3A patent/CN114581388A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108982508A (en) * | 2018-05-23 | 2018-12-11 | 江苏农林职业技术学院 | A kind of plastic-sealed body IC chip defect inspection method based on feature templates matching and deep learning |
| CN109829903A (en) * | 2019-01-28 | 2019-05-31 | 合肥工业大学 | A kind of chip surface defect inspection method based on convolution denoising self-encoding encoder |
| CN112991349A (en) * | 2019-12-17 | 2021-06-18 | 阿里巴巴集团控股有限公司 | Image processing method, device, equipment and storage medium |
| CN111401384A (en) * | 2020-03-12 | 2020-07-10 | 安徽南瑞继远电网技术有限公司 | Transformer equipment defect image matching method |
| CN111784644A (en) * | 2020-06-11 | 2020-10-16 | 上海布眼人工智能科技有限公司 | Printing defect detection method and system based on deep learning |
| CN113034492A (en) * | 2021-04-19 | 2021-06-25 | 深圳市华汉伟业科技有限公司 | Printing quality defect detection method and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 李珍珍: "产品表面的无监督缺陷检测算法研究", 中国优秀硕士电子期刊网, no. 03, 15 March 2021 (2021-03-15), pages 138 - 537 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114708267A (en) * | 2022-06-07 | 2022-07-05 | 浙江大学 | Image detection processing method for corrosion defect of tower stay wire on power transmission line |
| WO2024000420A1 (en) * | 2022-06-30 | 2024-01-04 | 宁德时代新能源科技股份有限公司 | Defect detection method and device |
| WO2024060917A1 (en) * | 2022-09-23 | 2024-03-28 | 中国电信股份有限公司 | Defect identification method, apparatus and system |
| CN115272831A (en) * | 2022-09-27 | 2022-11-01 | 成都中轨轨道设备有限公司 | Transmission method and system for monitoring images of suspension state of contact network |
| CN115272831B (en) * | 2022-09-27 | 2022-12-09 | 成都中轨轨道设备有限公司 | Transmission method and system for monitoring images of suspension state of contact network |
| CN117436012A (en) * | 2023-07-28 | 2024-01-23 | 中国建筑第二工程局有限公司 | Concrete square pile damage identification method based on self-adaptive multichannel convolutional neural network |
| CN117036267A (en) * | 2023-08-01 | 2023-11-10 | 广州伊索自动化科技有限公司 | Curved surface printing detection method, system and storage medium |
| CN116824271A (en) * | 2023-08-02 | 2023-09-29 | 上海互觉科技有限公司 | SMT chip defect detection system and method based on tri-modal vector space alignment |
| CN116824271B (en) * | 2023-08-02 | 2024-02-09 | 上海互觉科技有限公司 | SMT chip defect detection system and method based on tri-modal vector space alignment |
| CN117115178A (en) * | 2023-08-23 | 2023-11-24 | 国网四川省电力公司电力科学研究院 | Semi-parameter sharing-based power infrared inspection image segmentation and detection method |
| CN117115178B (en) * | 2023-08-23 | 2024-05-14 | 国网四川省电力公司电力科学研究院 | Semi-parameter sharing-based power infrared inspection image segmentation and detection method |
| CN116797602A (en) * | 2023-08-25 | 2023-09-22 | 东莞市爱培科技术有限公司 | Surface defect identification method and device for industrial product detection |
| CN117474916B (en) * | 2023-12-27 | 2024-04-26 | 苏州镁伽科技有限公司 | Image detection method, electronic equipment and storage medium |
| CN117474916A (en) * | 2023-12-27 | 2024-01-30 | 苏州镁伽科技有限公司 | Image detection method, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114581388A (en) | Contact net part defect detection method and device | |
| CN108562589B (en) | Method for detecting surface defects of magnetic circuit material | |
| CN108460764B (en) | Ultrasonic image intelligent segmentation method based on automatic context and data enhancement | |
| Shi et al. | Automatic road crack detection using random structured forests | |
| CN111738342B (en) | Pantograph foreign matter detection method, storage medium and computer equipment | |
| CN113592845A (en) | Defect detection method and device for battery coating and storage medium | |
| CN116664559B (en) | Machine vision-based memory bank damage rapid detection method | |
| CN109840483B (en) | Landslide crack detection and identification method and device | |
| CN112883795B (en) | Rapid and automatic table extraction method based on deep neural network | |
| CN106815806B (en) | Single image SR reconstruction method based on compressed sensing and SVR | |
| CN110751619A (en) | Insulator defect detection method | |
| CN101976504A (en) | Multi-vehicle video tracking method based on color space information | |
| CN111915583A (en) | Vehicle and pedestrian detection method based on vehicle-mounted thermal infrared imager in complex scene | |
| CN115908363B (en) | Tumor cell statistics method, device, equipment and storage medium | |
| CN114155213A (en) | Chip defect detection method and device based on active learning | |
| Liang et al. | Automatic defect detection of texture surface with an efficient texture removal network | |
| CN110991374B (en) | Fingerprint singular point detection method based on RCNN | |
| CN113657225B (en) | Target detection method | |
| Zihao et al. | Printed Surface Defect Detection Model Based on Positive Samples. | |
| CN117746079A (en) | Clustering prediction method, system, storage medium and equipment for hyperspectral image | |
| CN115346206B (en) | License plate detection method based on improved super-resolution deep convolution feature recognition | |
| CN110322466B (en) | Supervised image segmentation method based on multi-layer region limitation | |
| Das et al. | Object Detection on Scene Images: A Novel Approach | |
| CN116228795A (en) | Ultrahigh resolution medical image segmentation method based on weak supervised learning | |
| CN113362347B (en) | Image defect region segmentation method and system based on super-pixel feature enhancement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |