CN116703811A - A Defect Identification and Abnormal Segmentation Method for Drainage Pipeline Defects - Google Patents

Abstract

The present invention provides a drainage pipeline defect identification and abnormal segmentation method, the defect identification and abnormal segmentation method includes: S1. Input the original image into the segmentation network to obtain the segmented image; S2. Input the trained segmented image In the cGAN, a synthetic image is obtained; S3. inputting the original image and the synthetic image into the segmentation network to obtain the feature extraction values of the original image and the synthetic image; S4. inputting the feature extraction values of the original image and the synthetic image Defect identification and abnormal segmentation are implemented in the comparison module, which solves the technical problem of low accuracy of defect identification and abnormal segmentation in the prior art, improves the reliability and safety of drainage pipeline defect detection, and has application value.

Description

A Defect Identification and Abnormal Segmentation Method for Drainage Pipeline Defects

technical field

The invention relates to the technical field of image segmentation, in particular to a defect identification and abnormal segmentation method for drainage pipeline defects.

Background technique

Before the advent of machine learning methods, CCTV detection was the mainstream practice in the field of pipeline defect identification. As early as the 1950s, the pipeline closed-circuit television identification system CCTV had appeared, and it was basically mature in the 1980s. The pipeline closed-circuit television identification system needs to clean the pipeline before identification. If an abnormal point is found during the detection process, its position, orientation and defect type will be recorded and archive. Due to the intricate structure of the pipeline in practice, this method has many inconveniences and is highly subjective. Since the 1990s, automatic defect identification methods based on machine learning have emerged, and these methods have also undergone a development process from learning methods to non-learning methods. Traditional machine vision-based surface defect recognition methods often use conventional image processing algorithms or artificially designed features plus classifiers. For example, Sinha et al. proposed a linear feature extraction mechanism to complete the specific task of defect identification, and in a follow-up study, they further proposed an algorithm based on a sequence of morphological operations to segment pipeline cracks, holes, joints, and collapsed surface. However, these morphological methods usually require a large number of processing steps and require the design of sophisticated feature extractors to handle defects with distinct morphological features. Each feature extractor only targets one defect, which greatly limits the image processing efficiency. In recent years, machine learning has become a popular technology in the field of computer vision, and deep learning technologies such as convolutional neural networks have been widely used in crack image analysis. In deep learning algorithms, the network learns how to automatically extract features during training, and because deep learning omits the critical and time-consuming feature engineering step used in traditional machine learning techniques, it is more general than traditional methods. These previous studies mainly focus on defect classification and localization, and the automatic detection of defect shape and boundary is rarely studied. For the first time, Wang et al. proposed a semantic segmentation network named DilaSeg to segment pipeline defects. In the field of applying deep learning to semantic segmentation, the fully connected neural network is a breakthrough model, which can achieve pixel-level transformation. Therefore, many image segmentation networks have been proposed in recent years, which are very similar to fully connected neural networks, PipeUNet (references: Pan G, ZhengY, Guo S, et al. Automatic sewer pipe defect semantic segmentation based on improved U-Net[J ]. Automation in Construction, 2020, 2020 (119).) is one of them. Quality assessment of segmentation models refers to assessing the overall quality of segmentation models without using the ground-truth labels. When the model faces the risk of failure, the quality assessment can give early warning in time. Uncertainty estimation or confidence estimation has been a hot topic in the field of machine learning for many years and can be directly applied to defect identification tasks. There are two main ways to evaluate the quality of existing segmentation models: the first is to use bilinear convolutional neural network (BCNN) to predict the segmentation quality of medical images, the depth features calculated from a pair of images and their segmentation maps The regression results of the segmentation quality are obtained in . The second adopts an unsupervised learning method and utilizes geometric features to evaluate segmentation quality. But considering the complexity and large shape variation of 2D scenes and objects, this method is hardly suitable for natural images.

Contents of the invention

The purpose of the present invention is to provide a defect identification and abnormal segmentation method for drainage pipeline defects, so as to solve the technical problem of low defect identification and abnormal segmentation accuracy existing in the prior art.

A defect identification and abnormal segmentation method for drainage pipeline defects provided by the present invention includes: S1. inputting the original image into the segmentation network to obtain the segmented image; S2. inputting the segmented image into the trained cGAN (reference: Mirza M, Osindero S.Conditional generative adversarial nets[J].arXiv preprint arXiv:1411.1784,2014.) to obtain a synthetic image; S3. Input the original image and synthetic image into the segmentation network to obtain the original image and synthetic The feature extraction value of the image; S4. Input the feature extraction value of the original image and the synthetic image into the comparison module to realize defect identification and abnormal segmentation.

Further, the cGAN in the S2 includes a generation network and a discrimination network, and the specific training method of the cGAN includes: S21. The generation network and the discrimination network are randomly initialized; S22. The parameters of the generation network are fixed, and the parameters of the discrimination network are updated, so that The discriminant network gives a higher score to the original image, but gives a lower score to the synthetic image generated by the generating network; S23. Fix the parameters of the discriminant network, update the parameters of the generating network, so that the discriminant network can synthesize the generated network The score of the graph achieves the maximum value.

Further, the comparison module in S4 includes a comparison function, which defines the difference characteristics between the original image and the synthesized image through cosine distance.

A defect identification and abnormal segmentation method for drainage pipeline defects provided by the present invention, by pre-training cGAN, taking the segmentation map of the segmentation network as input, using the generation network of cGAN to generate a synthetic image, and inputting the synthetic image and the original image into the comparison function The difference features are obtained, defect identification and abnormal segmentation are realized, the technical problem of low accuracy of defect identification and abnormal segmentation in the prior art is solved, and the reliability and safety of drainage pipeline defect identification are improved, which has application value.

Description of drawings

In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the specific implementation or description of the prior art. Obviously, the accompanying drawings in the following description The drawings show some implementations of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

FIG. 1 is a flow chart of a defect identification and abnormal segmentation method for a drainage pipe defect provided in this embodiment;

FIG. 2 is a schematic diagram of synthesizing the segmented image provided by this embodiment into a composite image;

Fig. 3 is the schematic diagram of the comparison module provided by the present embodiment;

Fig. 4 is the schematic diagram of the bilinear convolutional neural network structure of the comparison module provided by the present embodiment;

FIG. 5 is a schematic structural diagram of the discriminant network of cGAN provided in this embodiment;

FIG. 6 is a schematic structural diagram of the generation network provided by this embodiment;

Figure 7 is the original picture of the pipeline defect image provided in this embodiment, and the defect types in the figure from left to right are branch pipe concealed connection, offset, crack, and leakage;

Fig. 8 is the synthesizing figure of corresponding synthesizing module in Fig. 7;

Figure 9 shows the real pipeline defects and segmentation results provided by this embodiment, from left to right are the original image, segmentation labels, and segmentation results;

Fig. 10 is the abnormality detection results of the segmentation map obtained by different thresholds t provided in this embodiment.

Detailed ways

The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

A defect identification and abnormal segmentation method for drainage pipeline defects provided in this embodiment includes:

S1. Input the original image x into the semantic segmentation network M to obtain the segmentation image

S2. Split the graph Input the trained cGAN to get the composite image/> The better the performance of the semantic segmentation network M, the better the segmentation map /> Generated composite map /> The closer it is to the original image x.

S3. The original image x and the composite image Enter the semantic segmentation network M to get the original image x and the composite image /> The feature extraction value of .

S4. The original image x and the composite image The feature extraction values of are input into the comparison module F(.) to obtain y and /> respectively Measure the difference between the two. Synthetic graph by analyzing /> The difference between the feature and the original image x, locate the location of the defect, and realize defect identification and abnormal segmentation.

For abnormal segmentation, a defect sample x will generate any labeled segmentation map in the segmentation network M The labeled split graph /> Input into the generation network G in cGAN to get a labeled composite map /> due to /> Composite map generated by cGAN containing labeled defective objects /> will only be reverted to the corresponding tagged defect object. Composite image/> There will be a large difference between x and the original image x, which reflects the pixel-level semantic difference between the actual segmentation image and the theoretical segmentation image, which is the main basis for defect identification.

The comparison module includes a comparison function F(.), which defines the original image x and the synthetic image by cosine distance difference characteristics. The original image x and composite image /> Input to the segmentation module M again, for the original image x and the synthetic image /> Perform feature extraction, and then use the cosine distance as an indicator to define the original image x and the composite image /> The difference on the features of the last layer during downsampling. The comparison module mainly completes two aspects of work, one is defect identification, including defect location and defect type information, by calculating the cross-merge ratio of the original image and the synthetic image on each defect category, and by comparing the feature extraction results of the synthetic image and the original image The difference between them can locate the location of the defect, and the second is abnormal segmentation, which is to judge whether there is a defect in the picture. Specific to the pipeline defect detection task, after feature extraction is performed on the synthetic image and the original image, the result obtained by cosine distance calculation is used to measure the pixel difference between the synthetic image and the original image, and a different threshold t is set. When the pixel difference is less than the threshold t, that is, the cosine distance between the composite image and the original image is greater than the threshold 1-t, the pixel is considered to be an abnormal object containing defects. The anomaly detection results of the segmentation maps obtained with different thresholds t in this embodiment are shown in FIG. 10 .

The cGAN in S2 includes a generative network G and a discriminative network D. Use labeled images to train cGAN, so that the synthetic image generated by cGAN As close as possible to the original image x. The specific training methods of sGAN include: S21. First, the generator network G and the discriminant network D are randomly initialized; S22. In each round of iteration, the parameters of the generator network G are fixed first, and the parameters of the discriminant network D are updated, so that the discriminant network is more accurate to the original The score D(x) given by graph x is higher, and the score D(G(z)) given to the synthetic graph G(z) generated by the generation network G is lower; S23. After fixing the parameters of the discriminant network D, Update the parameters of the generator network G so that the score D(G(z)) of the discriminant network D on the synthetic graph G(z) generated by the generator network G is as large as possible. After the above process, the generation network G and the discriminant network D compete with each other, and the final result is that the generation network G of cGAN can generate as real pictures as possible.

The original image of the pipeline defect image provided in this embodiment is shown in FIG. 7 , and FIG. 8 is the corresponding composite image. In Fig. 7 and Fig. 8, the defect types from left to right in the figure are blind connection, offset, crack and leakage of branch pipe.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (3)

1. A defect identification and abnormal segmentation method of drainage pipeline defects, characterized in that, the defect identification and abnormal segmentation method comprises: S1. Input the original image into the segmentation network to obtain the segmentation image; S2. Input the segmented image into the trained cGAN to obtain a composite image; S3. Inputting the original image and the composite image into the segmentation network to obtain feature extraction values of the original image and the composite image; S4. Input the feature extraction values of the original image and the synthesized image into the comparison module to realize defect identification and abnormal segmentation. 2. The defect recognition and abnormal segmentation model of a kind of drainage pipe defect according to claim 1, wherein the cGAN in the S2 includes a generating network and a discriminant network, and the specific training methods of the cGAN include: S21. Random initialization of the generation network and the discrimination network; S22. Fixing the parameters of the generating network and updating the parameters of the discriminant network, so that the discriminant network gives a higher score to the original image, but gives a lower score to the synthetic image generated by the generating network; S23. Fixing the parameters of the discriminant network, and updating the parameters of the generating network, so that the discriminant network can achieve a maximum score on the synthetic image generated by the generating network. 3. The defect identification and abnormal segmentation model of a drainage pipeline defect according to claim 1, wherein the comparison module in the S4 includes a comparison function, and the original image and the synthetic image are defined by a cosine distance difference characteristics.