CN116703811A - Defect identification and anomaly segmentation method for drainage pipeline defect - Google Patents

Abstract

The invention provides a drainage pipeline defect identification and abnormality segmentation method, which comprises the following steps: s1, inputting an original image into a segmentation network to obtain a segmentation image; s2, inputting the segmentation map into a trained cGAN to obtain a synthetic map; s3, inputting the original image and the synthetic image into the segmentation network to obtain feature extraction values of the original image and the synthetic image; s4, inputting the feature extraction values of the original image and the synthetic image into a comparison module to realize defect identification and abnormal segmentation, solving the technical problem of low accuracy of defect identification and abnormal segmentation in the prior art, improving the reliability and safety of defect detection of the drainage pipeline, and having application value.

Description

Defect identification and anomaly segmentation method for drainage pipeline defect

Technical Field

The invention relates to the technical field of image segmentation, in particular to a defect identification and abnormality segmentation method for a drainage pipeline defect.

Background

Before the advent of machine learning methods, CCTV detection was the dominant practice in the field of pipeline defect identification. CCTV has appeared in the early 50 s of the 20 th century and has been basically mature in the 80 s, and the pipeline CCTV identification system needs to clean the pipeline before identification, and records and archives the position and orientation and defect types if abnormal points are found in the detection process. Because of the complicated and complicated structure of the pipeline in practice, the method has a plurality of inconveniences and is high in subjectivity. Beginning in the 1990 s, automatic defect recognition methods based on machine learning have emerged, which also underwent a development process from learning methods to non-learning methods. The traditional surface defect identification method based on machine vision often adopts a conventional image processing algorithm or a mode of manually designing a feature and a classifier. For example, sinha et al propose a linear feature extraction mechanism to accomplish the specific task of defect identification, and in subsequent studies they further propose an algorithm based on morphological operation sequences to segment the surface of pipe cracks, holes, joints and collapse. However, these morphological methods typically require a large number of processing steps and require complex design of feature extractors to address defects with obvious morphological features. Each feature extractor is directed to only one defect, which greatly limits the image processing efficiency. In recent years, machine learning has become a popular technique in the field of computer vision, and deep learning techniques such as convolutional neural networks have been widely used for crack image analysis. In the deep learning algorithm, the web learning automatically extracts features during the training process, and is more versatile than the conventional method because it omits the key and time-consuming feature engineering steps used in the conventional machine learning technique. These previous studies have focused mainly on defect classification and localization, and automatic detection of defect shapes and boundaries has been rarely studied. Wang et al for the first time proposed a semantic segmentation network named DilaSeg to segment pipeline defects. In the field of semantic segmentation by applying deep learning, a fully connected neural network is a breakthrough model, which can realize pixel-level conversion. Therefore, many image segmentation networks have been proposed in recent years, which are very similar to fully-connected neural networks, and PipeUNet (reference: pan G, zheng Y, guo S, et al Automatic sewer pipe defect semantic segmentation based on improved U-Net [ J ]. Automation in Construction,2020,2020 (119) ]) is one of them. The quality evaluation of the segmentation model refers to the evaluation of the overall quality of the segmentation model without using a label true value. When the model is at failure risk, the quality evaluation can give early warning in time. Uncertainty estimation or confidence estimation has been a popular topic in the field of machine learning for many years and can be directly applied to defect identification tasks. The quality evaluation of the existing segmentation model mainly comprises two modes: the first is to predict the segmentation quality of a medical image using a Bilinear Convolutional Neural Network (BCNN), and derive a regression result of the segmentation quality from depth features calculated from a pair of images and its segmentation map. The second employs an unsupervised learning method, using geometric features to evaluate segmentation quality. But this method is hardly applicable to natural images in view of the complexity and large shape variations of 2D scenes and objects.

Disclosure of Invention

The invention aims to provide a defect identification and abnormal segmentation method for a drainage pipeline defect, which aims to solve the technical problems of low defect identification and abnormal segmentation accuracy in the prior art.

The invention provides a defect identification and abnormality segmentation method for a drainage pipeline defect, which comprises the following steps: s1, inputting an original image into a segmentation network to obtain a segmentation image; s2, inputting the segmentation map into a trained cGAN (reference document: mirza M, osindero S.conditional generative adversarial nets [ J ]. ArXiv preprint arXiv:1411.1784,2014.), and obtaining a synthetic map; s3, inputting the original image and the synthetic image into the segmentation network to obtain feature extraction values of the original image and the synthetic image; s4, inputting the feature extraction values of the original image and the synthetic image into a comparison module to realize defect identification and abnormal segmentation.

Further, the cGAN in S2 includes a generating network and a discriminating network, and the specific training method of the cGAN includes: s21, generating a network and randomly initializing a discrimination network; s22, fixing parameters of the generating network, and updating parameters of the judging network to enable the grade given by the judging network to be higher to the original image and lower to the synthesized image generated by the generating network; s23, fixing parameters of the judging network, updating parameters of the generating network, and enabling the grading of the judging network to the synthetic graph generated by the generating network to achieve the maximum value.

Further, the comparison module in S4 includes a comparison function, and the difference feature of the original image and the composite image is defined by a cosine distance.

According to the defect identification and anomaly segmentation method for the drainage pipeline defects, the cGAN is trained in advance, the segmentation diagram of the segmentation network is used as input, the synthetic diagram is generated by using the generation network of the cGAN, and the synthetic diagram and the original diagram are input into the comparison function to obtain difference characteristics, so that defect identification and anomaly segmentation are realized, the technical problem of low defect identification and anomaly segmentation accuracy in the prior art is solved, the reliability and safety of drainage pipeline defect identification are improved, and the method has application value.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a defect identification and anomaly segmentation method for a drainage pipeline defect provided in the present embodiment;

FIG. 2 is a schematic diagram of the synthesis of a split map according to the present embodiment;

fig. 3 is a schematic diagram of a comparison module provided in the present embodiment;

fig. 4 is a schematic structural diagram of a bilinear convolutional neural network of the comparison module provided in this embodiment;

fig. 5 is a schematic structural diagram of a discrimination network of cGAN according to the present embodiment;

fig. 6 is a schematic structural diagram of a generating network according to the present embodiment;

FIG. 7 is an original image of a pipeline defect provided in the present embodiment, wherein the defect types of the pipeline defect are branch pipe hidden joints, offset, cracks and leakage in sequence from left to right;

FIG. 8 is a composite view of the corresponding composite module of FIG. 7;

fig. 9 is a diagram of the actual pipeline defect and the segmentation result provided in the present embodiment, and the original image, the segmentation label, and the segmentation result are sequentially shown from left to right;

fig. 10 shows the abnormality detection results of the segmentation map obtained by the different threshold values t provided in the present embodiment.

Detailed Description

The technical solutions of the present invention will be clearly and completely described in connection with the embodiments, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The defect identification and abnormal segmentation method for the drainage pipeline defect provided by the embodiment comprises the following steps:

s1, inputting original image xInto a semantic segmentation network M to obtain a segmentation graph

S2, dividing the graphInputting into trained cGAN to obtain synthetic map +.>The more excellent the semantic segmentation network M behaves, the more the segmentation map +.>Generated synthetic graph->The closer to the original x.

S3, combining the original image x and the synthetic imageInputting into semantic segmentation network M to obtain original image x and synthetic image->Is a feature extraction value of (1).

S4, combining the original image x and the synthetic imageThe feature extraction values of (2) are input into a comparison module F ()'s to obtain y and +.>The difference between the two was measured. By analysis of synthetic diagrams->And the difference characteristic between the original image x, locating the position of the defect, and realizing defect identification and abnormal segmentation.

For anomaly segmentation, a defective sample x will generate any labeled segmentation map in the segmentation network MDivide the labeled division map->Inputting into the generating network G in cGAN to obtain labeled synthetic diagram +.>Due to->Comprising defective objects with labels, synthetic map generated by cGAN +.>Will only be restored to the corresponding tagged defect object. Synthetic pattern->There will be a large difference between the original image x, which reflects the pixel-level semantic difference between the actual segmentation map and the theoretically derived segmentation map, which is the main basis for defect identification.

The comparison module comprises a comparison function F (), and the original image x and the composite image are defined through cosine distanceIs a characteristic of the difference of (2). Original x and synthetic->Inputting into the segmentation module M again, for original image x and synthetic image +.>Extracting features, and defining original image x and synthetic image +.>Differences in the last layer of features during downsampling. The comparison module mainly completes the two works, namely defect identification, defect position and defect type information, calculation of the intersection ratio of the original image and the synthetic image on each defect category, and comparison of the synthetic imageAnd the difference between the original image characteristic extraction results can locate the position of the defect, and the second is abnormal segmentation, namely judging whether the picture has the defect or not. Specifically, to a pipeline defect detection task, after feature extraction is performed on a synthetic image and an original image, differences between pixels of the synthetic image and pixels of the original image are measured by using a result obtained by cosine distance calculation, and different thresholds t are set. When the pixel difference is smaller than the threshold t, namely the cosine distance between the synthetic image and the original image is larger than the threshold 1-t, the pixel point is considered to belong to an abnormal object containing defects. The abnormality detection results of the divided map obtained by the different threshold values t in the present embodiment are shown in fig. 10.

cGAN in S2 includes generating network G and discriminating network D. Training cGAN using tagged images such that cGAN generates a composite mapAs close as possible to the original x. The specific training method of the sGAN comprises the following steps: s21, firstly, generating a network G and a discrimination network D for random initialization; s22, in each round of iteration, firstly, fixing parameters of a generated network G, and updating parameters of a discrimination network D so that the score D (x) given by the discrimination network to an original image x is higher, and the score D (G (z)) given by a synthetic image G (z) generated by the generated network G is lower; s23, fixing parameters of the discrimination network D, and updating parameters of the generation network G so that the score D (G (z)) of the discrimination network D on the synthetic graph G (z) generated by the generation network G is as large as possible. Through the above process, the generating network G and the discriminating network D are mutually opposed, and the final result is that the generating network G of cGAN can generate a picture as real as possible.

The original image of the pipeline defect image provided by the embodiment is shown in fig. 7, and fig. 8 is a corresponding composite image. In fig. 7 and 8, the defect types are branch pipe dark joints, offset, cracks, leaks in this order from left to right.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims (3)

1. A defect identification and anomaly segmentation method for a drainage pipeline defect, characterized in that the defect identification and anomaly segmentation method comprises the following steps:

s1, inputting an original image into a segmentation network to obtain a segmentation image;

s2, inputting the segmentation map into a trained cGAN to obtain a synthetic map;

s3, inputting the original image and the synthetic image into the segmentation network to obtain feature extraction values of the original image and the synthetic image;

s4, inputting the feature extraction values of the original image and the synthetic image into a comparison module to realize defect identification and abnormal segmentation.

2. The model for identifying and abnormality of a drain pipe defect according to claim 1, wherein the cGAN in S2 includes a generating network and a discriminating network, and the specific training method of the cGAN includes:

s21, generating a network and randomly initializing a discrimination network;

s22, fixing parameters of the generating network, and updating parameters of the judging network to enable the grade given by the judging network to be higher to the original image and lower to the synthesized image generated by the generating network;

s23, fixing parameters of the judging network, updating parameters of the generating network, and enabling the grading of the judging network to the synthetic graph generated by the generating network to achieve the maximum value.

3. The model for identifying and dividing defects of drain pipeline defects according to claim 1, wherein the comparison module in S4 comprises a comparison function, and the difference characteristics of the original image and the composite image are defined by cosine distances.