CN116703811A - Defect identification and anomaly segmentation method for drainage pipeline defect - Google Patents
Defect identification and anomaly segmentation method for drainage pipeline defect Download PDFInfo
- Publication number
- CN116703811A CN116703811A CN202211275650.8A CN202211275650A CN116703811A CN 116703811 A CN116703811 A CN 116703811A CN 202211275650 A CN202211275650 A CN 202211275650A CN 116703811 A CN116703811 A CN 116703811A
- Authority
- CN
- China
- Prior art keywords
- network
- segmentation
- image
- defect
- original image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 62
- 230000011218 segmentation Effects 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title claims abstract description 30
- 230000002159 abnormal effect Effects 0.000 claims abstract description 11
- 238000000605 extraction Methods 0.000 claims abstract description 11
- 230000005856 abnormality Effects 0.000 claims abstract description 6
- 239000002131 composite material Substances 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 11
- 238000010801 machine learning Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000000877 morphologic effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000013441 quality evaluation Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000002950 deficient Effects 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 101150064138 MAP1 gene Proteins 0.000 description 1
- 241001647769 Mirza Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 101150077939 mapA gene Proteins 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a drainage pipeline defect identification and abnormality segmentation method, which comprises the following steps: s1, inputting an original image into a segmentation network to obtain a segmentation image; s2, inputting the segmentation map into a trained cGAN to obtain a synthetic map; s3, inputting the original image and the synthetic image into the segmentation network to obtain feature extraction values of the original image and the synthetic image; s4, inputting the feature extraction values of the original image and the synthetic image into a comparison module to realize defect identification and abnormal segmentation, solving the technical problem of low accuracy of defect identification and abnormal segmentation in the prior art, improving the reliability and safety of defect detection of the drainage pipeline, and having application value.
Description
Technical Field
The invention relates to the technical field of image segmentation, in particular to a defect identification and abnormality segmentation method for a drainage pipeline defect.
Background
Before the advent of machine learning methods, CCTV detection was the dominant practice in the field of pipeline defect identification. CCTV has appeared in the early 50 s of the 20 th century and has been basically mature in the 80 s, and the pipeline CCTV identification system needs to clean the pipeline before identification, and records and archives the position and orientation and defect types if abnormal points are found in the detection process. Because of the complicated and complicated structure of the pipeline in practice, the method has a plurality of inconveniences and is high in subjectivity. Beginning in the 1990 s, automatic defect recognition methods based on machine learning have emerged, which also underwent a development process from learning methods to non-learning methods. The traditional surface defect identification method based on machine vision often adopts a conventional image processing algorithm or a mode of manually designing a feature and a classifier. For example, sinha et al propose a linear feature extraction mechanism to accomplish the specific task of defect identification, and in subsequent studies they further propose an algorithm based on morphological operation sequences to segment the surface of pipe cracks, holes, joints and collapse. However, these morphological methods typically require a large number of processing steps and require complex design of feature extractors to address defects with obvious morphological features. Each feature extractor is directed to only one defect, which greatly limits the image processing efficiency. In recent years, machine learning has become a popular technique in the field of computer vision, and deep learning techniques such as convolutional neural networks have been widely used for crack image analysis. In the deep learning algorithm, the web learning automatically extracts features during the training process, and is more versatile than the conventional method because it omits the key and time-consuming feature engineering steps used in the conventional machine learning technique. These previous studies have focused mainly on defect classification and localization, and automatic detection of defect shapes and boundaries has been rarely studied. Wang et al for the first time proposed a semantic segmentation network named DilaSeg to segment pipeline defects. In the field of semantic segmentation by applying deep learning, a fully connected neural network is a breakthrough model, which can realize pixel-level conversion. Therefore, many image segmentation networks have been proposed in recent years, which are very similar to fully-connected neural networks, and PipeUNet (reference: pan G, zheng Y, guo S, et al Automatic sewer pipe defect semantic segmentation based on improved U-Net [ J ]. Automation in Construction,2020,2020 (119) ]) is one of them. The quality evaluation of the segmentation model refers to the evaluation of the overall quality of the segmentation model without using a label true value. When the model is at failure risk, the quality evaluation can give early warning in time. Uncertainty estimation or confidence estimation has been a popular topic in the field of machine learning for many years and can be directly applied to defect identification tasks. The quality evaluation of the existing segmentation model mainly comprises two modes: the first is to predict the segmentation quality of a medical image using a Bilinear Convolutional Neural Network (BCNN), and derive a regression result of the segmentation quality from depth features calculated from a pair of images and its segmentation map. The second employs an unsupervised learning method, using geometric features to evaluate segmentation quality. But this method is hardly applicable to natural images in view of the complexity and large shape variations of 2D scenes and objects.
Disclosure of Invention
The invention aims to provide a defect identification and abnormal segmentation method for a drainage pipeline defect, which aims to solve the technical problems of low defect identification and abnormal segmentation accuracy in the prior art.
The invention provides a defect identification and abnormality segmentation method for a drainage pipeline defect, which comprises the following steps: s1, inputting an original image into a segmentation network to obtain a segmentation image; s2, inputting the segmentation map into a trained cGAN (reference document: mirza M, osindero S.conditional generative adversarial nets [ J ]. ArXiv preprint arXiv:1411.1784,2014.), and obtaining a synthetic map; s3, inputting the original image and the synthetic image into the segmentation network to obtain feature extraction values of the original image and the synthetic image; s4, inputting the feature extraction values of the original image and the synthetic image into a comparison module to realize defect identification and abnormal segmentation.
Further, the cGAN in S2 includes a generating network and a discriminating network, and the specific training method of the cGAN includes: s21, generating a network and randomly initializing a discrimination network; s22, fixing parameters of the generating network, and updating parameters of the judging network to enable the grade given by the judging network to be higher to the original image and lower to the synthesized image generated by the generating network; s23, fixing parameters of the judging network, updating parameters of the generating network, and enabling the grading of the judging network to the synthetic graph generated by the generating network to achieve the maximum value.
Further, the comparison module in S4 includes a comparison function, and the difference feature of the original image and the composite image is defined by a cosine distance.
According to the defect identification and anomaly segmentation method for the drainage pipeline defects, the cGAN is trained in advance, the segmentation diagram of the segmentation network is used as input, the synthetic diagram is generated by using the generation network of the cGAN, and the synthetic diagram and the original diagram are input into the comparison function to obtain difference characteristics, so that defect identification and anomaly segmentation are realized, the technical problem of low defect identification and anomaly segmentation accuracy in the prior art is solved, the reliability and safety of drainage pipeline defect identification are improved, and the method has application value.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a defect identification and anomaly segmentation method for a drainage pipeline defect provided in the present embodiment;
FIG. 2 is a schematic diagram of the synthesis of a split map according to the present embodiment;
fig. 3 is a schematic diagram of a comparison module provided in the present embodiment;
fig. 4 is a schematic structural diagram of a bilinear convolutional neural network of the comparison module provided in this embodiment;
fig. 5 is a schematic structural diagram of a discrimination network of cGAN according to the present embodiment;
fig. 6 is a schematic structural diagram of a generating network according to the present embodiment;
FIG. 7 is an original image of a pipeline defect provided in the present embodiment, wherein the defect types of the pipeline defect are branch pipe hidden joints, offset, cracks and leakage in sequence from left to right;
FIG. 8 is a composite view of the corresponding composite module of FIG. 7;
fig. 9 is a diagram of the actual pipeline defect and the segmentation result provided in the present embodiment, and the original image, the segmentation label, and the segmentation result are sequentially shown from left to right;
fig. 10 shows the abnormality detection results of the segmentation map obtained by the different threshold values t provided in the present embodiment.
Detailed Description
The technical solutions of the present invention will be clearly and completely described in connection with the embodiments, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The defect identification and abnormal segmentation method for the drainage pipeline defect provided by the embodiment comprises the following steps:
s1, inputting original image xInto a semantic segmentation network M to obtain a segmentation graph
S2, dividing the graphInputting into trained cGAN to obtain synthetic map +.>The more excellent the semantic segmentation network M behaves, the more the segmentation map +.>Generated synthetic graph->The closer to the original x.
S3, combining the original image x and the synthetic imageInputting into semantic segmentation network M to obtain original image x and synthetic image->Is a feature extraction value of (1).
S4, combining the original image x and the synthetic imageThe feature extraction values of (2) are input into a comparison module F ()'s to obtain y and +.>The difference between the two was measured. By analysis of synthetic diagrams->And the difference characteristic between the original image x, locating the position of the defect, and realizing defect identification and abnormal segmentation.
For anomaly segmentation, a defective sample x will generate any labeled segmentation map in the segmentation network MDivide the labeled division map->Inputting into the generating network G in cGAN to obtain labeled synthetic diagram +.>Due to->Comprising defective objects with labels, synthetic map generated by cGAN +.>Will only be restored to the corresponding tagged defect object. Synthetic pattern->There will be a large difference between the original image x, which reflects the pixel-level semantic difference between the actual segmentation map and the theoretically derived segmentation map, which is the main basis for defect identification.
The comparison module comprises a comparison function F (), and the original image x and the composite image are defined through cosine distanceIs a characteristic of the difference of (2). Original x and synthetic->Inputting into the segmentation module M again, for original image x and synthetic image +.>Extracting features, and defining original image x and synthetic image +.>Differences in the last layer of features during downsampling. The comparison module mainly completes the two works, namely defect identification, defect position and defect type information, calculation of the intersection ratio of the original image and the synthetic image on each defect category, and comparison of the synthetic imageAnd the difference between the original image characteristic extraction results can locate the position of the defect, and the second is abnormal segmentation, namely judging whether the picture has the defect or not. Specifically, to a pipeline defect detection task, after feature extraction is performed on a synthetic image and an original image, differences between pixels of the synthetic image and pixels of the original image are measured by using a result obtained by cosine distance calculation, and different thresholds t are set. When the pixel difference is smaller than the threshold t, namely the cosine distance between the synthetic image and the original image is larger than the threshold 1-t, the pixel point is considered to belong to an abnormal object containing defects. The abnormality detection results of the divided map obtained by the different threshold values t in the present embodiment are shown in fig. 10.
cGAN in S2 includes generating network G and discriminating network D. Training cGAN using tagged images such that cGAN generates a composite mapAs close as possible to the original x. The specific training method of the sGAN comprises the following steps: s21, firstly, generating a network G and a discrimination network D for random initialization; s22, in each round of iteration, firstly, fixing parameters of a generated network G, and updating parameters of a discrimination network D so that the score D (x) given by the discrimination network to an original image x is higher, and the score D (G (z)) given by a synthetic image G (z) generated by the generated network G is lower; s23, fixing parameters of the discrimination network D, and updating parameters of the generation network G so that the score D (G (z)) of the discrimination network D on the synthetic graph G (z) generated by the generation network G is as large as possible. Through the above process, the generating network G and the discriminating network D are mutually opposed, and the final result is that the generating network G of cGAN can generate a picture as real as possible.
The original image of the pipeline defect image provided by the embodiment is shown in fig. 7, and fig. 8 is a corresponding composite image. In fig. 7 and 8, the defect types are branch pipe dark joints, offset, cracks, leaks in this order from left to right.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (3)
1. A defect identification and anomaly segmentation method for a drainage pipeline defect, characterized in that the defect identification and anomaly segmentation method comprises the following steps:
s1, inputting an original image into a segmentation network to obtain a segmentation image;
s2, inputting the segmentation map into a trained cGAN to obtain a synthetic map;
s3, inputting the original image and the synthetic image into the segmentation network to obtain feature extraction values of the original image and the synthetic image;
s4, inputting the feature extraction values of the original image and the synthetic image into a comparison module to realize defect identification and abnormal segmentation.
2. The model for identifying and abnormality of a drain pipe defect according to claim 1, wherein the cGAN in S2 includes a generating network and a discriminating network, and the specific training method of the cGAN includes:
s21, generating a network and randomly initializing a discrimination network;
s22, fixing parameters of the generating network, and updating parameters of the judging network to enable the grade given by the judging network to be higher to the original image and lower to the synthesized image generated by the generating network;
s23, fixing parameters of the judging network, updating parameters of the generating network, and enabling the grading of the judging network to the synthetic graph generated by the generating network to achieve the maximum value.
3. The model for identifying and dividing defects of drain pipeline defects according to claim 1, wherein the comparison module in S4 comprises a comparison function, and the difference characteristics of the original image and the composite image are defined by cosine distances.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211275650.8A CN116703811A (en) | 2022-10-18 | 2022-10-18 | Defect identification and anomaly segmentation method for drainage pipeline defect |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211275650.8A CN116703811A (en) | 2022-10-18 | 2022-10-18 | Defect identification and anomaly segmentation method for drainage pipeline defect |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116703811A true CN116703811A (en) | 2023-09-05 |
Family
ID=87842070
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211275650.8A Pending CN116703811A (en) | 2022-10-18 | 2022-10-18 | Defect identification and anomaly segmentation method for drainage pipeline defect |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116703811A (en) |
-
2022
- 2022-10-18 CN CN202211275650.8A patent/CN116703811A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Pan et al. | Automatic sewer pipe defect semantic segmentation based on improved U-Net | |
CN108460764B (en) | Ultrasonic image intelligent segmentation method based on automatic context and data enhancement | |
Xiao et al. | Surface defect detection using image pyramid | |
CN109214349B (en) | Object detection method based on semantic segmentation enhancement | |
CN111028217A (en) | Image crack segmentation method based on full convolution neural network | |
CN113436169B (en) | Industrial equipment surface crack detection method and system based on semi-supervised semantic segmentation | |
CN112395957B (en) | Online learning method for video target detection | |
Li et al. | A robust instance segmentation framework for underground sewer defect detection | |
CN110555831B (en) | Deep learning-based drainage pipeline defect segmentation method | |
CN110348342B (en) | Pipeline disease image segmentation method based on full convolution network | |
CN110009622B (en) | Display panel appearance defect detection network and defect detection method thereof | |
CN113962951B (en) | Training method and device for detecting segmentation model, and target detection method and device | |
Zhao et al. | Research on detection method for the leakage of underwater pipeline by YOLOv3 | |
Peng et al. | Non-uniform illumination image enhancement for surface damage detection of wind turbine blades | |
CN116205907A (en) | Decorative plate defect detection method based on machine vision | |
CN117252815A (en) | Industrial part defect detection method, system, equipment and storage medium based on 2D-3D multi-mode image | |
Oh et al. | Robust sewer defect detection with text analysis based on deep learning | |
Katsamenis et al. | A few-shot attention recurrent residual U-Net for crack segmentation | |
Dang et al. | Lightweight pixel-level semantic segmentation and analysis for sewer defects using deep learning | |
CN112669269A (en) | Pipeline defect classification and classification method and system based on image recognition | |
CN117274355A (en) | Drainage pipeline flow intelligent measurement method based on acceleration guidance area convolutional neural network and parallel multi-scale unified network | |
CN116596895A (en) | Substation equipment image defect identification method and system | |
CN116703811A (en) | Defect identification and anomaly segmentation method for drainage pipeline defect | |
Chang et al. | Deep Learning Approaches for Dynamic Object Understanding and Defect Detection | |
CN111882545B (en) | Fabric defect detection method based on bidirectional information transmission and feature fusion |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |