CN112686217A - Mask R-CNN-based detection method for disease pixel level of underground drainage pipeline - Google Patents
Mask R-CNN-based detection method for disease pixel level of underground drainage pipeline Download PDFInfo
- Publication number
- CN112686217A CN112686217A CN202110152149.1A CN202110152149A CN112686217A CN 112686217 A CN112686217 A CN 112686217A CN 202110152149 A CN202110152149 A CN 202110152149A CN 112686217 A CN112686217 A CN 112686217A
- Authority
- CN
- China
- Prior art keywords
- network model
- disease
- drainage pipeline
- underground drainage
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 152
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 152
- 238000001514 detection method Methods 0.000 title claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 58
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 38
- 230000011218 segmentation Effects 0.000 claims abstract description 29
- 238000000034 method Methods 0.000 claims abstract description 23
- 238000012360 testing method Methods 0.000 claims abstract description 21
- 230000006870 function Effects 0.000 claims abstract description 15
- 238000011176 pooling Methods 0.000 claims abstract description 14
- 238000013135 deep learning Methods 0.000 claims abstract description 10
- 238000013526 transfer learning Methods 0.000 claims abstract description 8
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 238000005516 engineering process Methods 0.000 claims abstract description 6
- 238000004519 manufacturing process Methods 0.000 claims abstract description 6
- 230000000694 effects Effects 0.000 claims abstract description 5
- 238000012795 verification Methods 0.000 claims description 27
- 230000007797 corrosion Effects 0.000 claims description 9
- 238000005260 corrosion Methods 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 6
- 230000008021 deposition Effects 0.000 claims description 5
- 238000000151 deposition Methods 0.000 claims description 5
- 238000005336 cracking Methods 0.000 claims description 4
- 230000007613 environmental effect Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 230000002238 attenuated effect Effects 0.000 claims description 2
- 238000012937 correction Methods 0.000 claims description 2
- 238000007667 floating Methods 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 238000013139 quantization Methods 0.000 claims description 2
- 230000035945 sensitivity Effects 0.000 claims description 2
- 238000013528 artificial neural network Methods 0.000 abstract description 2
- 230000007547 defect Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 239000013049 sediment Substances 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 239000010865 sewage Substances 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 239000002341 toxic gas Substances 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention is suitable for the technical field of intelligent segmentation of pipeline diseases, and relates to a detection method for detecting the disease pixel level of an underground drainage pipeline based on Mask R-CNN, which comprises the following steps: manufacturing an underground drainage pipeline example segmentation data set by using the acquired drainage pipeline disease video; optimizing a loss function and an ROI pooling layer in a Mask R-CNN deep learning framework, and improving the detection precision of an example segmentation algorithm by using a ResNet101 network as a pipeline disease feature extraction network; initializing network model parameters by using a transfer learning technology, performing a hyper-parameter tuning test on the network model, and starting network model training; evaluating the performance of the training network model, and analyzing the intersection-parallel ratio of the predicted disease area and the real disease area of the network model; and judging whether the network model can achieve the pixel-level segmentation effect or not. The method is combined with a Mask R-CNN example segmentation frame, a ResNet101 residual error neural network and drainage pipeline disease big data, and quick, accurate and automatic identification and positioning of the underground drainage pipeline diseases are realized.
Description
Technical Field
The invention belongs to the technical field of intelligent segmentation of pipeline diseases, and particularly relates to a method for detecting the disease pixel level of an underground drainage pipeline based on Mask R-CNN.
Background
The underground drainage pipe network is an important infrastructure of modern cities, is a material foundation and an indispensable life line for the survival and development of the cities, and plays a role in discharging urban sewage and rainwater. Along with the development of economy in China, urban construction scale is larger and larger, laid underground pipelines are longer and longer, the remaining service life of the pipelines is shorter and shorter, potential safety hazards caused by ageing and overhauling of underground drainage pipelines are obvious, and insufficient drainage capacity of the drainage pipelines due to scaling, sediment siltation and other diseases is one of the main reasons for frequent urban waterlogging. Meanwhile, in recent years, road collapse accidents are endless, and the soil around the pipeline is hollowed due to structural diseases such as pipeline leakage and corrosion, the structural performance of the pipeline is reduced, and the like, which are one of the root causes of multiple urban road collapse events. Therefore, the method is necessary for detecting the diseases of the underground drainage pipeline and finding potential safety hazards in time and guiding the maintenance and reinforcement of the underground drainage pipeline.
The existing detection method for underground drainage pipeline diseases mainly comprises a manual observation method, a pipeline sonar detection method, a pipeline closed-circuit television detection method (CCTV) and the like. The manual observation method is that a professional detector enters a pipeline to detect diseases in the pipeline, but toxic gas in the pipeline easily causes casualties of the detector; the pipe sonar detection is based on ultrasonic waves to detect pipe diameters, sediments and the like of the cross section of a pipe, and the method cannot detect diseases such as corrosion and leakage of the pipe; CCTV detection is to enter a pipeline to shoot a video through a pipeline detection robot carrying a camera, and then analyze the video by a professional with rich experience to judge the disease condition in the pipeline. However, in the CCTV detection, the judgment efficiency of professionals on the pipeline diseases is low, the workload is large, and the analysis on the disease degree is easily influenced by personal experience. Therefore, how to rapidly, efficiently and accurately detect the diseases of the underground drainage pipelines becomes a key scientific and technological problem to be solved urgently in the field of underground pipeline maintenance in China.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a method for detecting the disease pixel level of the underground drainage pipeline based on Mask R-CNN, so as to solve the problems of low detection precision and low efficiency of the underground drainage pipeline disease in the prior art.
In order to solve the technical problems, the invention adopts the following technical scheme:
the invention provides a Mask R-CNN-based detection method for disease pixel levels of underground drainage pipelines, which comprises the following steps:
s10, acquiring a disease video of the underground drainage pipeline by using an underground drainage pipeline CCTV detection robot, extracting a disease image in the disease video of the drainage pipeline, preprocessing and manufacturing an underground drainage pipeline example segmentation data set;
s20, optimizing a loss function and an ROI pooling layer in a Mask R-CNN deep learning framework, and in a network trunk part, using a ResNet101 network as an underground drainage pipeline disease feature extraction network to improve the detection precision of an example segmentation algorithm;
s30, initializing network model parameters by using a transfer learning technology, performing a network model hyper-parameter tuning test on the training step length, the learning rate and the anchor size in the network model, and starting the network model training;
s40, in the network model training process, evaluating and training the network model performance by using a verification set, and analyzing the intersection ratio of the predicted disease area and the real disease area of the network model, namely IoU indexes;
and S50, after the training of the network model is finished, verifying the detection precision, universality and robustness of the trained network model by using a test set, and judging whether the network model can achieve the pixel-level segmentation effect.
Further, the specific steps in step S10 are:
s11, acquiring underground drainage pipeline disease video data by an underground drainage pipeline CCTV detection robot, and extracting underground drainage pipeline disease images from the acquired underground drainage pipeline video data every 30 frames based on a compiled matlab program;
s12, selecting the disease images of the underground drainage pipeline with 5 types of typical disease characteristics of stagger, deposition, corrosion, cracking and scaling as an underground drainage pipeline disease data set;
s13, accurately marking the disease outline of the underground drainage pipeline disease image by using an open-source Lableme data marking tool, and finally generating binary mask data of the underground drainage pipeline disease;
and S14, after the data marking is finished, randomly selecting 20% of the data as a deep learning verification set, using the rest 80% of the data as the network model training set, and finishing the manufacturing of the underground drainage pipeline disease data set.
Further, the specific steps in step S20 are:
s21, designing and constructing an improved Mask R-CNN instance segmentation algorithm, wherein the improved Mask R-CNN instance segmentation algorithm comprises a Convolutional Neural Network (CNN), a Region suggestion Network (RPN), a RoIAlign and a segmentation Network, and an activation function adopts a ReLU function;
s22, using a RoIAlign layer instead of ROI pooling, the RoIAlign avoiding any quantization of the ROI bounding box, allowing the existence of floating point numbers, and making the ROI exactly match the extracted disease features;
s23, replacing the ROI pooling by the RoIAlign layer in the S22, selecting 4 sampling points in each ROI area, performing area expansion by a bilinear interpolation algorithm, summarizing results by using average pooling, and completing accurate mapping of a target area of an image to be detected of the underground drainage pipeline and a characteristic map;
s24, a ResNet101 network based on a residual error learning unit is adopted, and a Feature Pyramid Network (FPN) is combined to serve as the underground drainage pipeline disease Feature extraction network, so that the problem that the traditional CNN network has gradient disappearance along with the deepening of the network depth is solved, and the disease Feature utilization rate is increased.
Further, the specific steps in step S30 are:
s31, initializing the network model based on a pre-trained weight network model and fine-tuning by using the transfer learning technology;
s32, performing network training and testing in a Linux environment, and performing network training and testing in a virtual environment built by TensorFlow 1.4 and Keras 2.1;
s33, performing an optimization test on the network model iteration step length and the initial learning rate by adopting a trial-and-error method, wherein the initial learning rate is attenuated by adopting an exponential attenuation mode, and the optimal parameter setting is selected from the initial learning rate;
s34, for the area suggestion network in S21, adopting anchors with different aspect ratios and different scales to slide on the extracted feature map to generate a plurality of candidate areas to realize sampling, and in order to adapt to disease areas with different shapes and different sizes, the invention adopts the anchor sizes of 48, 96, 192, 384 and 768 and the aspect ratios of 0.5, 1 and 2;
s35, when the network model is trained, extracting a fixed number of samples from a training set each time, inputting the samples into the network model, and starting the network model training by using a stochastic gradient descent and back propagation algorithm;
s36, finally, the network model achieves correction of the boundary frame of the candidate area through boundary frame regression, then achieves classification of disease categories through classification regression, finally outputs prediction masks of all diseases at the mask branch, and finally achieves segmentation of pixel levels of all drainage pipeline diseases.
Further, redundant regions are generated for the candidate regions extracted in S34, and a DIoU-NMS is used to replace a conventional non-maximum suppression algorithm, so as to remove the redundant candidate regions.
Further, the specific step of step S40 is:
s41, selecting the optimal network model parameters to train the network model, selecting a fixed amount of deep learning verification set data to input into the network model every 5000 training steps of the network model, and verifying the performance of the network model;
s42, randomly displaying the network model verification result in the network model training process, and evaluating the network model verification performance by using IoU indexes;
s43, calculating a loss function value of the actual output error of the network model and the target output error of the network model, if the loss meets a loss threshold preset by the network model, finishing training, and finally outputting an underground drainage pipeline disease detection network model; otherwise, the step S41 is returned to.
Further, the IoU evaluation index in the network model verification in the step S42 is an area ratio of an intersection area and a phase area of the predicted disease area and the real disease area.
Further, for the verification of the network model in the step S42, if the disease types of the predicted disease area and the real disease area are matched, and the IoU value of the predicted disease area and the real disease area is not less than 50%, the predicted area is regarded as a true positive area; otherwise, the prediction region is considered to be a false positive region.
Further, for the underground drainage pipeline disease detection network model in the step S43, the network model training loss curve, the verification loss curve, and the average precision value of the verification set are output to evaluate the performance of the network model.
Further, the specific step of step S50 is:
s51, after the training of the network model is completed, selecting a new image which does not participate in the training and verification of the network model to test the performance of the network model, and calculating the average accuracy and IoU value of the test;
s52, selecting underground drainage pipeline disease data under different environmental noises to test the robustness of the trained network model, analyzing the universality of the network model on underground drainage pipeline disease detection under different working conditions and the sensitivity to different environmental noises, and judging whether the trained network model can accurately segment the underground drainage pipeline disease at a pixel level.
Compared with the prior art, the detection method of the disease pixel level of the underground drainage pipeline based on Mask R-CNN provided by the invention at least has the following beneficial effects:
according to the method, by means of an improved Mask R-CNN instance segmentation algorithm, a network model training loss function is optimized, ROIAline is used for replacing ROI pooling, the detection precision of complex diseases of the underground drainage pipeline is improved, and the problem that the corrosion, scaling and other similar diseases are difficult to accurately identify by manual disease detection is solved; by using a ResNet101 network and a feature fusion algorithm based on a residual error structural unit, the utilization rate of the disease features of the low-layer pipeline is increased, and the disease detection precision is improved; by using classification regression and boundary frame regression, the accurate judgment of the drainage pipeline disease category is realized, and the disease position is accurately predicted; and the Mask prediction branch is used, so that the prediction binary Mask of the complex disease area of the underground drainage pipeline can be accurately output, and the pixel level segmentation of the disease area is realized.
Drawings
In order to illustrate the solution of the invention more clearly, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are some embodiments of the invention, and that other drawings may be derived from these drawings by a person skilled in the art without inventive effort.
FIG. 1 is a general flow diagram of a detection method for detecting underground drainage pipeline disease pixel level based on Mask R-CNN provided by the invention.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings. Preferred embodiments of the present invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
As shown in FIG. 1, the invention provides a detection method of underground drainage pipeline disease pixel level based on Mask R-CNN, which is characterized by comprising the following steps:
s10, acquiring a disease video of the underground drainage pipeline by using an underground drainage pipeline CCTV detection robot, extracting a disease image in the video, preprocessing and manufacturing an underground drainage pipeline example segmentation data set;
specifically, as a preferable scheme, in step S10 of this embodiment, the drainage pipe defect image includes five defects of stagger, deposition, corrosion, scaling and cracking. Firstly, acquiring a drainage pipeline disease video by using a CCTV underground drainage pipeline detection robot; secondly, extracting pipeline disease images from the collected video data every 30 frames based on the compiled matlab program; then, aiming at the obtained large number of drainage pipeline disease images, images with 5 types of typical disease characteristics of stagger, deposition, corrosion, cracking and scaling are manually selected to serve as an underground drainage pipeline disease data set.
Specifically, in step S10, the selected drain pipe disease data set is manually marked by using an open-source big data marking tool Labelme, and finally binary mask data of the drain pipe disease is generated, and the manufactured mask file marks a background pixel as 0, a staggered area pixel as 1, a deposition area pixel as 2, a corrosion area pixel as 3, a rupture area pixel as 4, and a fouling area pixel as 5. Each image only contains one background label, but there may be multiple disease damages, therefore, each disease mask must be separated from the background, and in the present invention, each disease has separate mask information, which also provides for further example segmentation task.
Specifically, in step S10, after the data marking is completed, 20% of the data are randomly selected as the deep learning verification set, and the remaining 80% are used as the network model training set, so that the data set is completed.
S20, optimizing a loss function and an ROI pooling layer in a Mask R-CNN deep learning framework, and in a network trunk part, using a ResNet101 network as an underground drainage pipeline disease feature extraction network to improve the detection precision of an example segmentation algorithm;
specifically, in step S20 of this embodiment, the modified Mask R-CNN network includes a convolutional neural network, a regional suggestion network, a roilign, and a segmentation network, and the activation function of the present invention adopts a ReLU function.
Preferably, the network model training loss comprises a classification loss LclsAnd a boundary frameLoss LboxAnd predicted mask loss Lmask。LclsAnd LboxCalculating according to the classification loss and the bounding box loss in fast R-CNN, respectively, LmaskThe definition is as follows: firstly, applying sigmoid function calculation to each pixel, and then taking the average cross entropy of all pixels after sigmoid on RoI as Lmask，LmaskThe calculation is as follows:
wherein x isiIs the predicted value of the ith pixel in positive RoI, biThe true value of the ith pixel in the positive RoI, and N is the number of pixel points in the positive RoI.
As a preferred scheme, a RoIAlign layer is used for replacing an ROI pooling layer, 4 sampling points in each RoI area are selected, a bilinear interpolation algorithm is adopted for area expansion, results are summarized by using average pooling, accurate mapping of a target area of an image to be detected of the drainage pipeline and a characteristic map is completed, and RoI is accurately matched with the extracted disease characteristics.
As a preferred scheme, the convolutional network adopts ResNet101 based on a residual error learning unit, and combines a characteristic Pyramid network (FPN) to perform a drainage pipeline disease Feature extraction network, so that high-resolution and low-semantic bottom features and low-resolution and high-semantic top features are fused, the utilization rate of disease features is increased, a network model can ensure detection efficiency and improve detection accuracy; ResNet101 includes 5 convolution modules.
S30, initializing network model parameters by using a transfer learning technology, performing an over-parameter tuning test on the training step length, the learning rate and the anchor size in the network model, and starting network model training;
specifically, in this embodiment, in order to solve the problem that supervised learning training requires a large number of labeled samples, and labeling samples consumes a large amount of manpower and time, a transfer learning technique is adopted, and a network model is initialized based on a pre-trained weight network model and fine-tuning. Compared with random parameter initialization, the transfer learning adopts a dark net53.conv.74 pre-training network model which is well represented in a past public data set to initialize a neural network, so that the training efficiency is accelerated, and the detection precision is improved.
Specifically, a Mask R-CNN deep learning framework is built on an Ubuntu operating system and is carried out in a virtual environment built by TensorFlow 1.4 and Keras 2.1, and the parameters of the used high-performance computer terminal equipment are as follows: intel E5-2630V4@2.20GHz, 10-core 20-thread processor, NVIDIA RTx2080Ti graphics processor.
Specifically, the training of the network model comprises forward propagation and backward propagation, batch data in a training set are input into the network model for training through the forward propagation, then the training loss of the network model is calculated, and then the weight parameters of the network model are updated in a backward propagation mode.
Specifically, in the network model training process, the superparameters have great influence on the accuracy, the convergence rate, the training time and the like of the training network model.
Specifically, as a preferred scheme, in this embodiment, the network model hyper-parameter setting is as follows: the iteration step size of each epoch is 1000, the verification step size is 50, the learning rate is 0.001, and the reduction is 10 times at 10000 steps. The momentum coefficient is 0.9 and the weight attenuation coefficient is 0.0001.
Specifically, the anchor with different aspect ratios and different scales is adopted in the area suggestion network to slide on the extracted feature map to generate a plurality of candidate areas to realize sampling, as a preferred scheme, the size of the anchor is 48, 96, 192, 384 and 768, the aspect ratio is 0.5, 1 and 2, and a DIoU-NMS is adopted to replace a traditional non-maximum suppression algorithm to remove redundant candidate areas.
S40, in the network model training process, evaluating the performance of the training network model by using a verification set, and analyzing the intersection and parallel ratio of the predicted disease area and the real disease area of the network model, namely IoU indexes;
specifically, in step S40 of this embodiment, an optimal network model parameter is selected for training, a fixed number of verification set data are selected for each 5000 training steps and input into the network model, the performance of the network model is verified, the verification result of the network model is randomly displayed, the verification result is analyzed by using IoU evaluation indexes, and the IoU evaluation index is an area ratio of an intersection area and a parallel area of a predicted disease area and a real disease area.
Specifically, when the network model is verified, if the disease types of the predicted disease area and the real disease area are matched, and the IoU value of the predicted disease area and the real disease area is not less than 50%, the predicted area is regarded as a true positive area, otherwise, the predicted area is regarded as a false positive area.
Specifically, error loss values of actual output and target output of the network model are closely observed in the training process, if the loss meets a loss threshold preset by the network model, the training is finished, and finally the underground drainage pipeline disease detection network model is output, otherwise, the step S30 is returned.
And S50, after the training of the network model is finished, verifying the detection precision, universality and robustness of the trained network model by using the test set, and judging whether the network model can achieve the pixel-level segmentation effect.
Specifically, as an optimal scheme, a new image in a test set which does not participate in network model training and verification is input into the network model for testing, the underground drainage pipeline diseases are detected, the detection result is compared with an artificial detection result, and whether the network model achieves an expected effect or not is judged.
Specifically, the network model evaluation index includes: the method comprises the steps of obtaining a plurality of evaluation indexes such as accuracy, detection efficiency, average precision value (AP) and mIoU, wherein the plurality of evaluation indexes can comprehensively evaluate the performance of a trained network model, the mIoU is an average cross-over ratio value, represents the coincidence degree of a prediction frame and an actual frame, and is an evaluation index of position detection accuracy, if the evaluation index tested by the network model meets the requirement, the network model has a good effect, and the structure and weight parameters of the network model at the moment are stored.
According to the detection method of the disease pixel level of the underground drainage pipeline based on the Mask R-CNN, disclosed by the embodiment, the network model training loss function is optimized through an improved Mask R-CNN example segmentation algorithm, ROIAline is used for replacing ROI pooling, the detection precision of complex diseases of the underground drainage pipeline is improved, and the problem that the corrosion, scaling and other similar diseases are difficult to accurately identify through manual disease detection is solved; by using a ResNet101 network and a feature fusion algorithm based on a residual error structural unit, the utilization rate of the disease features of the low-layer pipeline is increased, and the disease detection precision is improved; by using classification regression and boundary frame regression, the accurate judgment of the drainage pipeline disease category is realized, and the disease position is accurately predicted; the Mask is used for predicting the branch, the binary Mask for predicting the complex disease area of the underground drainage pipeline can be accurately output, the pixel level segmentation of the disease area is realized, the diseases of the underground drainage pipeline can be automatically identified and positioned quickly and accurately, and the intelligent development of the pipeline maintenance industry can be effectively promoted.
It is to be understood that the above-described embodiments are merely preferred embodiments of the present invention, and not all embodiments are shown in the drawings, which are set forth to limit the scope of the invention. This invention may be embodied in many different forms and, on the contrary, these embodiments are provided so that this disclosure will be thorough and complete. Although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that various changes in the embodiments and modifications can be made, and equivalents may be substituted for elements thereof. All equivalent structures made by using the contents of the specification and the attached drawings of the invention can be directly or indirectly applied to other related technical fields, and are also within the protection scope of the patent of the invention.
Claims (10)
1. A detection method for detecting underground drainage pipeline disease pixel level based on Mask R-CNN is characterized by comprising the following steps:
s10, acquiring a disease video of the underground drainage pipeline by using an underground drainage pipeline CCTV detection robot, extracting a disease image in the disease video of the drainage pipeline, preprocessing and manufacturing an underground drainage pipeline example segmentation data set;
s20, optimizing a loss function and an ROI pooling layer in a Mask R-CNN deep learning framework, and in a network trunk part, using a ResNet101 network as an underground drainage pipeline disease feature extraction network to improve the detection precision of an example segmentation algorithm;
s30, initializing network model parameters by using a transfer learning technology, performing a network model hyper-parameter tuning test on the training step length, the learning rate and the anchor size in the network model, and starting the network model training;
s40, in the network model training process, evaluating and training the network model performance by using a verification set, and analyzing the intersection ratio of the predicted disease area and the real disease area of the network model, namely IoU indexes;
and S50, after the training of the network model is finished, verifying the detection precision, universality and robustness of the trained network model by using a test set, and judging whether the network model can achieve the pixel-level segmentation effect.
2. The detection method for the pixel level of the diseases of the underground drainage pipeline based on the Mask R-CNN as claimed in claim 1, wherein the step S10 comprises the following steps:
s11, acquiring underground drainage pipeline disease video data by an underground drainage pipeline CCTV detection robot, and extracting underground drainage pipeline disease images from the acquired underground drainage pipeline video data every 30 frames based on a compiled matlab program;
s12, selecting the disease images of the underground drainage pipeline with 5 types of typical disease characteristics of stagger, deposition, corrosion, cracking and scaling as an underground drainage pipeline disease data set;
s13, accurately marking the disease outline of the underground drainage pipeline disease image by using an open-source Lableme data marking tool, and finally generating binary mask data of the underground drainage pipeline disease;
and S14, after the data marking is finished, randomly selecting 20% of the data as a deep learning verification set, using the rest 80% of the data as the network model training set, and finishing the manufacturing of the underground drainage pipeline disease data set.
3. The detection method for the pixel level of the diseases of the underground drainage pipeline based on the Mask R-CNN as claimed in claim 1, wherein the step S20 comprises the following steps:
s21, designing and constructing an improved Mask R-CNN instance segmentation algorithm, wherein the improved Mask R-CNN instance segmentation algorithm comprises a convolutional neural network, a region suggestion network, a RoIAlign and a segmentation network, and an activation function adopts a ReLU function;
s22, using a RoIAlign layer instead of ROI pooling, the RoIAlign avoiding any quantization of the ROI bounding box, allowing the existence of floating point numbers, and making the ROI exactly match the extracted disease features;
s23, replacing the ROI pooling by the RoIAlign layer in the S22, selecting 4 sampling points in each ROI area, performing area expansion by a bilinear interpolation algorithm, summarizing results by using average pooling, and completing accurate mapping of a target area of an image to be detected of the underground drainage pipeline and a characteristic map;
and S24, adopting a ResNet101 network based on a residual error learning unit, and combining a characteristic pyramid network as the disease characteristic extraction network of the underground drainage pipeline.
4. The detection method for the pixel level of the diseases of the underground drainage pipeline based on the Mask R-CNN as claimed in claim 1, wherein the step S30 comprises the following steps:
s31, initializing the network model based on a pre-trained weight network model and fine-tuning by using the transfer learning technology;
s32, performing network training and testing in a Linux environment, and performing network training and testing in a virtual environment built by TensorFlow 1.4 and Keras 2.1;
s33, performing an optimization test on the network model iteration step length and the initial learning rate by adopting a trial-and-error method, wherein the initial learning rate is attenuated by adopting an exponential attenuation mode, and the optimal parameter setting is selected from the initial learning rate;
s34, for the area suggestion network in S21, adopting anchors with different aspect ratios and different scales to slide on the extracted feature map to generate a plurality of candidate areas to realize sampling, and in order to adapt to disease areas with different shapes and different sizes, the invention adopts the anchor sizes of 48, 96, 192, 384 and 768 and the aspect ratios of 0.5, 1 and 2;
s35, when the network model is trained, extracting a fixed number of samples from a training set each time, inputting the samples into the network model, and starting the network model training by using a stochastic gradient descent and back propagation algorithm;
s36, finally, the network model achieves correction of the boundary frame of the candidate area through boundary frame regression, then achieves classification of disease categories through classification regression, finally outputs prediction masks of all diseases at the mask branch, and finally achieves segmentation of pixel levels of all drainage pipeline diseases.
5. The method for detecting the pixel level of the diseases in the underground drainage pipeline based on Mask R-CNN as claimed in claim 4, wherein redundant regions are generated in the candidate regions extracted in S34, and a DIoU-NMS is adopted to replace a traditional non-maximum suppression algorithm, so as to remove the redundant candidate regions.
6. The detection method for the pixel level of the diseases of the underground drainage pipeline based on Mask R-CNN as claimed in claim 1, wherein the concrete steps of step S40 are as follows:
s41, selecting the optimal network model parameters to train the network model, selecting a fixed amount of deep learning verification set data to input into the network model every 5000 training steps of the network model, and verifying the performance of the network model;
s42, randomly displaying the network model verification result in the network model training process, and evaluating the network model verification performance by using IoU indexes;
s43, calculating a loss function value of the actual output error of the network model and the target output error of the network model, if the loss meets a loss threshold preset by the network model, finishing training, and finally outputting an underground drainage pipeline disease detection network model; otherwise, the step S41 is returned to.
7. The method for detecting underground drainpipe disease pixel level based on Mask R-CNN as claimed in claim 6, wherein for IoU evaluation index in the network model verification in the step S42 is the area ratio of the intersection region and the phase-parallel region of the predicted disease region and the real disease region.
8. The method for detecting the pixel level of the diseases of the underground drainage pipeline based on Mask R-CNN as claimed in claim 6, wherein for the verification of the network model in the step S42, if the disease types of the predicted disease area and the real disease area are matched and the IoU value of the predicted disease area and the real disease area is not less than 50%, the predicted area is regarded as a true positive area; otherwise, the prediction region is considered to be a false positive region.
9. The method for detecting the pixel level of the diseases of the underground drainage pipeline based on the Mask R-CNN as claimed in claim 6, wherein for the network model for detecting the diseases of the underground drainage pipeline in the step S43, the average precision values of the training loss curve, the verification loss curve and the verification set of the network model are output to evaluate the performance of the network model.
10. The detection method for the pixel level of the diseases of the underground drainage pipeline based on Mask R-CNN as claimed in claim 1, wherein the concrete steps of step S50 are as follows:
s51, after the training of the network model is completed, selecting a new image which does not participate in the training and verification of the network model to test the performance of the network model, and calculating the average accuracy and IoU value of the test;
s52, selecting underground drainage pipeline disease data under different environmental noises to test the robustness of the trained network model, analyzing the universality of the network model on underground drainage pipeline disease detection under different working conditions and the sensitivity to different environmental noises, and judging whether the trained network model can accurately segment the underground drainage pipeline disease at a pixel level.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011203833 | 2020-11-02 | ||
| CN202011203833X | 2020-11-02 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112686217A true CN112686217A (en) | 2021-04-20 |
Family
ID=75459660
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110152149.1A Pending CN112686217A (en) | 2020-11-02 | 2021-02-04 | Mask R-CNN-based detection method for disease pixel level of underground drainage pipeline |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112686217A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096126A (en) * | 2021-06-03 | 2021-07-09 | 四川九通智路科技有限公司 | Road disease detection system and method based on image recognition deep learning |
| CN114366116A (en) * | 2022-01-28 | 2022-04-19 | 南方医科大学 | Parameter acquisition method based on Mask R-CNN network and electrocardiogram |
| CN114898204A (en) * | 2022-03-03 | 2022-08-12 | 中国铁路设计集团有限公司 | Rail transit peripheral hazard source detection method based on deep learning |
| CN115116026A (en) * | 2022-05-26 | 2022-09-27 | 江苏大学 | Automatic tracking method and system for logistics carrying robot |
| CN116205921A (en) * | 2023-05-05 | 2023-06-02 | 南京航空航天大学 | Pipeline disease identification method based on Yolov5 and Drools |
| CN114898203B (en) * | 2022-03-03 | 2023-10-17 | 中国铁路设计集团有限公司 | Rail transit peripheral dangerous source change detection method based on deep learning |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110942026A (en) * | 2019-11-26 | 2020-03-31 | 深圳大学 | Capsule robot drain pipe disease detection method and system based on deep learning |
| CN111861978A (en) * | 2020-05-29 | 2020-10-30 | 陕西师范大学 | Bridge crack example segmentation method based on Faster R-CNN |
-
2021
- 2021-02-04 CN CN202110152149.1A patent/CN112686217A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110942026A (en) * | 2019-11-26 | 2020-03-31 | 深圳大学 | Capsule robot drain pipe disease detection method and system based on deep learning |
| CN111861978A (en) * | 2020-05-29 | 2020-10-30 | 陕西师范大学 | Bridge crack example segmentation method based on Faster R-CNN |
Non-Patent Citations (2)
| Title |
|---|
| 张宇维: ""城市排水管内窥图像分类与病害智能检测研究"", 《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》 * |
| 梁博 等: ""基于卷积神经网络的多任务图像语义分割"", 《无线电工程》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096126A (en) * | 2021-06-03 | 2021-07-09 | 四川九通智路科技有限公司 | Road disease detection system and method based on image recognition deep learning |
| CN114366116A (en) * | 2022-01-28 | 2022-04-19 | 南方医科大学 | Parameter acquisition method based on Mask R-CNN network and electrocardiogram |
| CN114366116B (en) * | 2022-01-28 | 2023-08-25 | 南方医科大学 | Parameter acquisition method based on Mask R-CNN network and electrocardiogram |
| CN114898204A (en) * | 2022-03-03 | 2022-08-12 | 中国铁路设计集团有限公司 | Rail transit peripheral hazard source detection method based on deep learning |
| CN114898204B (en) * | 2022-03-03 | 2023-09-05 | 中国铁路设计集团有限公司 | Rail transit peripheral dangerous source detection method based on deep learning |
| CN114898203B (en) * | 2022-03-03 | 2023-10-17 | 中国铁路设计集团有限公司 | Rail transit peripheral dangerous source change detection method based on deep learning |
| CN115116026A (en) * | 2022-05-26 | 2022-09-27 | 江苏大学 | Automatic tracking method and system for logistics carrying robot |
| CN115116026B (en) * | 2022-05-26 | 2024-04-09 | 江苏大学 | Automatic tracking method and system for logistics transfer robot |
| CN116205921A (en) * | 2023-05-05 | 2023-06-02 | 南京航空航天大学 | Pipeline disease identification method based on Yolov5 and Drools |
| CN116205921B (en) * | 2023-05-05 | 2023-09-26 | 南京航空航天大学 | Pipeline disease identification method based on Yolov5 and Drools |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112686217A (en) | Mask R-CNN-based detection method for disease pixel level of underground drainage pipeline | |
| US20210319265A1 (en) | Method for segmentation of underground drainage pipeline defects based on full convolutional neural network | |
| KR102008973B1 (en) | Apparatus and Method for Detection defect of sewer pipe based on Deep Learning | |
| US10818003B2 (en) | High-precision intelligent detection method for bridge diseases based on spatial position | |
| CN107480611B (en) | Crack identification method based on deep learning convolutional neural network | |
| CN113469177B (en) | Deep learning-based drainage pipeline defect detection method and system | |
| Chen et al. | A self organizing map optimization based image recognition and processing model for bridge crack inspection | |
| US7215811B2 (en) | Method and apparatus for the automated detection and classification of defects in sewer pipes | |
| Koo et al. | Innovative method for assessment of underground sewer pipe condition | |
| JP7104799B2 (en) | Learning data collection device, learning data collection method, and program | |
| Wang et al. | Towards an automated condition assessment framework of underground sewer pipes based on closed-circuit television (CCTV) images | |
| CN110826588A (en) | Drainage pipeline defect detection method based on attention mechanism | |
| Zhou et al. | Automatic sewer defect detection and severity quantification based on pixel-level semantic segmentation | |
| Shahrokhinasab et al. | Performance of image-based crack detection systems in concrete structures | |
| Dang et al. | Deep learning-based masonry crack segmentation and real-life crack length measurement | |
| Dogan et al. | Detection of damages caused by earthquake and reinforcement corrosion in RC buildings with Deep Transfer Learning | |
| KR20230132686A (en) | A method for damage identification and volume quantification of concrete pipes based on PointNet++ neural network | |
| Bruno et al. | Decay detection in historic buildings through image-based deep learning | |
| O'Byrne et al. | Texture analysis based detection and classification of surface features on ageing infrastructure elements | |
| Khalifa et al. | A new image-based model for predicting cracks in sewer pipes | |
| Dirksen et al. | The role of uncertainty in urban drainage decisions: uncertainty in inspection data and their impact on rehabilitation decisions | |
| Qi et al. | A CNN-based method for concreate crack detection in underwater environments | |
| CN116051496A (en) | Real-time sewer defect detection method | |
| Wang et al. | Automatic Damage Segmentation Framework for Buried Sewer Pipes Based on Machine Vision: Case Study of Sewer Pipes in Zhengzhou, China | |
| CN113610957A (en) | BIM-based automatic management method for drainage pipeline three-dimensional defect information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210420 |