CN111784645B - Filling pipeline crack detection method - Google Patents
Filling pipeline crack detection method Download PDFInfo
- Publication number
- CN111784645B CN111784645B CN202010544835.9A CN202010544835A CN111784645B CN 111784645 B CN111784645 B CN 111784645B CN 202010544835 A CN202010544835 A CN 202010544835A CN 111784645 B CN111784645 B CN 111784645B
- Authority
- CN
- China
- Prior art keywords
- image
- pipeline
- training
- model
- crack
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 104
- 238000012549 training Methods 0.000 claims abstract description 121
- 238000000034 method Methods 0.000 claims abstract description 47
- 230000008569 process Effects 0.000 claims abstract description 16
- 238000012360 testing method Methods 0.000 claims description 66
- 230000008439 repair process Effects 0.000 claims description 29
- 230000009471 action Effects 0.000 claims description 16
- 230000000694 effects Effects 0.000 claims description 13
- 238000011282 treatment Methods 0.000 claims description 11
- 238000012544 monitoring process Methods 0.000 claims description 10
- 238000002372 labelling Methods 0.000 claims description 7
- 230000015654 memory Effects 0.000 claims description 7
- 239000003795 chemical substances by application Substances 0.000 claims description 4
- 239000011521 glass Substances 0.000 claims description 4
- 238000011276 addition treatment Methods 0.000 claims description 3
- 238000011084 recovery Methods 0.000 claims description 3
- 230000002787 reinforcement Effects 0.000 claims description 3
- 125000004122 cyclic group Chemical group 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 11
- 238000005065 mining Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 238000003860 storage Methods 0.000 description 7
- 230000007547 defect Effects 0.000 description 5
- 238000011176 pooling Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000011478 gradient descent method Methods 0.000 description 3
- 229910000831 Steel Inorganic materials 0.000 description 2
- 238000005299 abrasion Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 239000002184 metal Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 102100029469 WD repeat and HMG-box DNA-binding protein 1 Human genes 0.000 description 1
- 101710097421 WD repeat and HMG-box DNA-binding protein 1 Proteins 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000008485 antagonism Effects 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000009193 crawling Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005553 drilling Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000001125 extrusion Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000013101 initial test Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a crack detection method for a filling pipeline, and belongs to the field of mining industry. The method comprises the following steps: training an image denoising model by using the pipeline image with the crack and the pipeline image with the crack after the noise adding process; training an image raindrop removing model by using the pipeline image with the cracks and the pipeline image with the raindrops and the cracks at the same time; training a crack detection model by using a target detection training set, wherein each pipeline image in the target detection training set is marked with a crack position; and acquiring a pipeline image to be detected in real time, and sequentially denoising, raindrop removing processing and crack detecting the pipeline image to be detected by using the trained image denoising model, the image raindrop removing model and the crack detecting model. By adopting the method and the device, noise and raindrops in the image can be repaired, so that the pipeline crack detection precision and detection efficiency are improved.
Description
Technical Field
The invention relates to the field of mining industry, in particular to a crack detection method for a filling pipeline.
Background
The underground filling pipeline is a paste conveying pipeline communicated with the surface filling station and the underground stope. The precise management of the paste conveying state in the pipeline and the pipeline health condition is an important component part of the accurate filling operation. When the concentration of the paste is too high, the flow is too large or the non-full pipe flow exists, the impact, extrusion and friction between the paste and the pipeline can cause certain abrasion to the side wall of the pipeline, and serious industrial production accidents such as pipeline rupture and paste leakage can be finally caused. Therefore, the well-under pipeline health management work is an important ring for realizing the full-flow accurate operation of paste filling.
At present, pipeline detection systems at home and abroad mainly comprise an ultrasonic flaw detection system, a three-dimensional laser scanning imaging detection system, a video image detection system and the like. The ultrasonic flaw detection requires that the pipeline material has high sound transmission rate, is mainly used for steel pipes, steel corrugated pipes, plastic pipes and the like at present, and has certain limitation in detection. The three-dimensional laser scanning imaging detection system provides a brand new technical means, has the characteristics of non-contact, high precision, digitization and the like, but requires high-precision scanning point data, and then the result is processed by a computer to obtain the pipeline defect condition, so that the technical requirement is high and the economic cost is higher. The image detection system acquires a high-resolution video image through the digital camera and the illumination light source, data can be transmitted to the control console in a wireless or wired mode in real time, and staff can control the detection system to finish transmission, acquisition, storage and the like of the image data through the control console.
For pipeline image data acquired by using an image detection system, a plurality of image processing technologies are mainly adopted to detect faults at present, including an edge detection algorithm and an image processing technology based on mathematical morphological parameters, namely He Cunfu of Beijing university of industry et al [ He Cunfu, zhou Long, he Shouyin, et al, a CCD-based pipeline defect detection system of a mobile robot in a pipeline [ J ]. Mechanical and electronic, 2006 (10): 33-35 ] a pipeline crawling robot detection system with a CCD video image acquisition device is designed. The robot can run in a metal pipeline with a medium-small pipe diameter, the CCD video image acquisition system adopts a built-in light source with a simple structure, acquired data is transmitted to an upper computer in real time after being processed by PCI acquisition such as encoding and decoding, compression and the like, and the system can successfully acquire defect images in the pipe. Although the detection robot system can realize over-bending and adapting to variable pipe diameters, the pipeline defect image processing process is simple and the defect recognition effect is poor.
The underground pipeline is in a bad environment condition, the image sensor is not bright enough in view field, not uniform in brightness and various noises of components and parts of the circuit and reasons of mutual influence when shooting, various noises possibly exist in the image, and the signal interference exists when the filling pipeline is buried underground, the signal shielding effect of some metal pipelines seriously hinders data transmission, various environmental noises are added in the data transmission process, and the detection performance of the whole system is affected. Meanwhile, the monitoring camera is in underground for a long time, the underground environment is wet, and the monitoring camera is likely to have raindrops for a long time, so that the quality of a pipeline image shot by the monitoring camera can be seriously affected. The specificity of the downhole tubing environment makes detection of tubing cracks more difficult.
Disclosure of Invention
The embodiment of the invention provides a crack detection method for a filling pipeline, which can repair noise and raindrops in an image, so that the crack detection precision and detection efficiency of the pipeline are improved; the technical scheme is as follows:
in one aspect, there is provided a crack detection method of a filling pipe, the method being applied to an electronic device, the method comprising:
training an image denoising model by using the pipeline image with the crack and the pipeline image with the crack after the noise adding process;
training an image raindrop removing model by using the pipeline image with the cracks and the pipeline image with the raindrops and the cracks at the same time;
training a crack detection model by using a target detection training set, wherein each pipeline image in the target detection training set is marked with a crack position;
and acquiring a pipeline image to be detected in real time, and sequentially denoising, raindrop removing processing and crack detecting the pipeline image to be detected by using the trained image denoising model, the image raindrop removing model and the crack detecting model.
Further, before training the image denoising model by using the cracked pipeline image and the noise-added pipeline image, the method comprises:
collecting a pipeline image with cracks, and placing a glass block with raindrops in front of a camera for collecting the pipeline image with the raindrops and the cracks at the same time;
dividing the acquired pipeline image with the cracks into a training set train1 and a test set test1 according to a preset first proportion;
dividing the acquired pipeline image with raindrops and cracks into a training set train3 and a test set test3 according to a preset first proportion; wherein, train1 and train3 are used as a group of training data, which is denoted as train_train; test1 and test3 are taken as a group of test data and are marked as Rain_test;
respectively carrying out noise adding treatment on the training set train1 and the test set test1, and respectively marking the train1 and the test1 subjected to the noise adding treatment as train2 and test2; wherein, train1 and train2 are used as a group of training data, which is recorded as noise_train; test1 and test2 are taken as a group of test data and are marked as noise_test;
marking the training set train1 and the test set test1 after the labeling treatment as train4 and test4 respectively; wherein the labeling process refers to marking the crack position in each pipeline image and adding the crack name.
Further, training the image denoising model by using the cracked pipeline image and the noise-added pipeline image comprises the following steps:
establishing an image denoising model by adopting a method based on deep reinforcement learning;
and training the image denoising model by using training data noise_train so that the trained image denoising model can denoise and repair the pipeline image with Noise.
Further, after training the image denoising model using the cracked pipeline image and the noise-added cracked pipeline image, the method further comprises:
and testing the trained image denoising model by using the test data noise_test so as to detect the denoising restoration effect.
Further, training the image raindrop removal model using the cracked pipeline image and the pipeline image with raindrops and cracks at the same time comprises the following steps:
establishing an image raindrop removing model by adopting a method based on generating an countermeasure network;
and training the image raindrop removing model by using training data Rain_train so that the trained image raindrop removing model can carry out raindrop removing restoration on the pipeline image with raindrops.
Further, after training the image raindrop model using the cracked pipe image and the pipe image with both raindrops and cracks, the method further comprises:
and testing the trained image raindrop removing model by using test data Rain_test so as to detect the raindrop removing repair effect of the image raindrop removing model.
Further, training the crack detection model using the target detection training set includes:
establishing a crack detection model, wherein the crack detection model adopts a Faster RCNN network structure;
and training the crack detection model by taking the train4 as a target detection training set, so that the trained crack detection model can identify and position cracks in the pipeline image.
Further, after training the crack detection model with the target detection training set, the method further comprises:
and testing the trained crack detection model by using a test set test4.
Further, the acquiring the pipeline image to be detected in real time includes:
and arranging a monitoring camera on the filling pipeline, wherein the monitoring camera is used for acquiring the pipeline image to be detected in real time.
In one aspect, an electronic device is provided that includes a processor and a memory having at least one instruction stored therein that is loaded and executed by the processor to implement the above-described fill pipe crack detection method.
In one aspect, a computer readable storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement the above-described filling pipe crack detection method is provided.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
in the embodiment of the invention, training an image denoising model by using a pipeline image with cracks and the pipeline image with cracks after the noise adding treatment; training an image raindrop removing model by using the pipeline image with the cracks and the pipeline image with the raindrops and the cracks at the same time; training a crack detection model by using a target detection training set, wherein each pipeline image in the target detection training set is marked with a crack position; and acquiring a pipeline image to be detected in real time, and sequentially denoising, raindrop removing processing and crack detecting the pipeline image to be detected by using the trained image denoising model, the image raindrop removing model and the crack detecting model. Therefore, manual detection is not needed, labor can be saved, noise and raindrops in an image can be repaired firstly aiming at the severe environment where an underground pipeline is located after the pipeline image is acquired, the actual pipeline image quality is improved, and pipeline crack detection is carried out, so that the pipeline crack detection precision and detection efficiency can be further improved, the robustness of pipeline crack detection is enhanced, and the filling efficiency and filling quality are greatly guaranteed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a method for detecting cracks in a filling pipeline according to an embodiment of the present invention;
FIG. 2 is a diagram showing an example of repair effect when using different tool sequences according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a network structure of a denoising model according to an embodiment of the present invention
Fig. 4 is a schematic structural diagram of a generating network according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a discrimination network according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a network architecture of a Faster RCNN provided in an embodiment of the invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present invention provides a method for detecting a crack in a filling pipe, which may be implemented by an electronic device, where the electronic device may be a terminal or a server, and the method includes:
s101, training an image denoising model by using a pipeline image with cracks and the pipeline image with cracks after the noise addition treatment;
s102, training an image raindrop removing model by using a pipeline image with cracks and a pipeline image with raindrops and cracks at the same time;
s103, training a crack detection model by using a target detection training set, wherein each pipeline image in the target detection training set is marked with a crack position;
s104, acquiring the pipeline image to be detected in real time, and sequentially denoising, raindrop removal processing and crack detection on the pipeline image to be detected by using the trained image denoising model, the image raindrop removal model and the crack detection model.
According to the crack detection method for the filling pipeline, provided by the embodiment, an image denoising model is trained by utilizing the pipeline image with the crack and the pipeline image with the crack after the noise addition treatment; training an image raindrop removing model by using the pipeline image with the cracks and the pipeline image with the raindrops and the cracks at the same time; training a crack detection model by using a target detection training set, wherein each pipeline image in the target detection training set is marked with a crack position; and acquiring a pipeline image to be detected in real time, and sequentially denoising, raindrop removing processing and crack detecting the pipeline image to be detected by using the trained image denoising model, the image raindrop removing model and the crack detecting model. Therefore, manual detection is not needed, labor can be saved, noise and raindrops in an image can be repaired firstly aiming at the severe environment where an underground pipeline is located after the pipeline image is acquired, the actual pipeline image quality is improved, and pipeline crack detection is carried out, so that the pipeline crack detection precision and detection efficiency can be further improved, the robustness of pipeline crack detection is enhanced, and the filling efficiency and filling quality are greatly guaranteed.
In this embodiment, optionally, before training the image denoising model by using the pipeline image with a crack and the pipeline image with a crack after the noise adding process, a large number of pipeline images with a crack and pipeline images with raindrops and cracks are acquired in a real environment, and the acquired images are processed, so as to manufacture image data that can be used for training and testing an image restoration model (the image denoising model and the image raindrop removing model) and a crack detection model, which specifically includes the following steps:
a1, collecting a pipeline image with cracks, and placing a glass block with raindrops in front of a camera for collecting the pipeline image with the raindrops and the cracks at the same time;
in this embodiment, the method for acquiring the image of the pipe with the crack is as follows: in practice, a camera is used to record images of the pipe with cracks.
In this embodiment, the collection mode of the pipeline image with raindrops and cracks simultaneously is as follows: before shooting, a glass block with raindrops is placed in front of a camera so that the camera can collect images of pipe cracks with raindrops at the same time.
A2, dividing the acquired pipeline image with the cracks into a training set train1 and a test set test1 according to a preset first proportion;
in this embodiment, assuming that 2532 images of a pipeline with a crack are acquired in total, and then the 2532 images of the pipeline are divided into a training set and a test set according to a preset first ratio (for example, 7:3), 1773 images of the training set are denoted as train1, and 759 images of the test set are denoted as test1.
A3, dividing the acquired pipeline image with raindrops and cracks into a training set train3 and a test set test3 according to a preset first proportion; wherein, train1 and train3 are used as a group of training data, which is denoted as train_train; test1 and test3 are taken as a group of test data and are marked as Rain_test;
in this example, it is assumed that 2532 images of a pipe with raindrops and cracks are acquired simultaneously, and then the 2532 images of a pipe are also acquired according to 7: the 3 proportion is divided into a training set and a test set, 1773 training sets with raindrops and cracks are marked as train3, and the training sets train3 and train1 with raindrops and cracks are used as a group of training data and are marked as train_train. Similarly, 759 test sets with raindrops and cracks are denoted as test3, and test set test3 with raindrops and cracks and test set test1 are denoted as a set of test data as rain_test.
A4, respectively carrying out noise adding treatment on the training set train1 and the test set test1, and respectively marking the train1 and the test1 after the noise adding treatment as train2 and test2; wherein, train1 and train2 are used as a group of training data, which is recorded as noise_train; test1 and test2 are taken as a group of test data and are marked as noise_test;
in this embodiment, the training set train1 and the test set test1 obtained in the step A2 are subjected to noise adding processing, specifically: the gaussian Noise is added by setting the parameter "mode" in the existing function random_noise (img, mode= 'gaussian', mean=0, var=0.02) to "gaussian", the mean value is 0, the variance is 0.02, the training set after gaussian Noise is added is denoted as train2, the test set is denoted as test2, and train1 and train2 are taken as a set of data, denoted as noise_train, and test1 and test2 are taken as a set of data, denoted as noise_test.
A5, marking the training set train1 and the test set test1 after the labeling treatment as train4 and test4 respectively; wherein the labeling process refers to marking the crack position in each pipeline image and adding the crack name.
In this embodiment, a LabelImg tool may be used to make the training set train1 and the test set test1 obtained in the step A2 into a target detection data set (including a training set and a test set for target detection) through a labeling process, and specifically may include the following steps:
1) Downloading LabelImg, decompressing the downloaded LabelImg to obtain LabelImg-master files, and installing LabelImg software;
2) Checking positions to be marked in the pipeline images in the training set train1 and the test set test1, wherein only one type of the pipeline images is a crack, and the crack is named as a 'crack';
3) Opening a LabelImg tool, clicking an Open Dir (picture catalog), selecting an Image to be marked, then enclosing a crack by using a square frame, clicking a Change Save Dir (mark file catalog), namely setting a position for storing a mark file, clicking a create\nRectBox (creating mark), selecting a crack class name, finally clicking for storage, marking the Image, clicking Next Image, starting marking of the Next Image, finally, obtaining two folders after marking all the images, wherein one folder is original Image data, the other folder is label data corresponding to the Image data, namely an xml file, and the position and the class name of the crack are recorded. The labeled training set train1 is denoted as train4, while the labeled test set test1 is denoted as test4.
In this embodiment, the LabelImg tool is used to mark the location of the target object (i.e., crack) in the original image and a corresponding xml file is generated for each image to represent the location of the target box.
In this embodiment, optionally, the training the image denoising model using the cracked pipeline image and the noise-added cracked pipeline image includes:
b1, establishing an image denoising model by adopting a method based on deep reinforcement learning;
and B2, training the image denoising model by utilizing training data noise_train so that the trained image denoising model can denoise and repair the pipeline image with Noise.
In this embodiment, an image denoising model is built using a Toolbox (Toolbox) and a proxy network with a loop structure; the tool box comprises a set of tools which can be applied to damaged images, the tools are also called repair actions, each tool is formed by using a three-layer or eight-layer CNN network structure, namely a tool network, the tools are used for slightly damaging, the tools are used for severely damaging, the total number of the tools is 12, and each tool solves the problems of Gaussian blur, gaussian noise and JPEG distortion of a specific degree. The effect of the repair when using different tool sequences is shown in figure 2.
In fig. 2, dB is a unit measuring the image quality, i.e., peak signal to noise ratio, deblue represents a blurred image, denoise represents a noisy image, deJPEG represents a JPEG-compressed image, and the two images of the first line are images corrupted by two different distortion combinations. Using an appropriate tool chain, as shown in fig. 2 (c), (d), the image quality and peak signal to noise ratio (PSNR) values are sequentially improved. The tool order is then slightly rearranged as shown in fig. 2 (b), (e), or the recovery level of the tool is adjusted as shown in fig. 2 (a), (f). The results indicate that small changes in the tool chain can severely impact repair performance. In particular, the use of incorrect tools may result in unnatural outputs, such as oversharpening in fig. 2 (a) and blurring in fig. 2 (f). Even if the tool is properly selected, improper sequence can degrade performance as shown in fig. 2 (b), (e).
In this embodiment, the proxy network is used to select which of the 12 tools is used, which may be referred to as a repair action, and the proxy network finds the action with the largest action estimate vector, i.e., the best action, i.e., the best repair tool, and this tool is applied to image repair. An image repair is completed through a plurality of repair steps, each repair is performed by selecting a tool by the proxy network, and in each repair step, a agent (proxy network) is required to determine which tool in the Toolbox should be used, and the agent is composed of three parts, as shown in fig. 3: a feature extractor (Feature Extractor) comprising 4 convolution layers and1 full connection layer for converting input image into 32-dimensional feature vector; an One-hot Encoder (One-hot Encoder) whose input is the motion estimation vector of the previous step, and whose output converts it into a corresponding feature vector; a long-short-term memory network (Long short term memory, LSTM) which uses the outputs of the first two modules as inputs and outputs the estimated vector v of the current step t For selection of the restoration tool.
In this embodiment, in step B2, the batch size is set to be 64, the training frequency is 80, the learning rate is 0.0001, and the training data set noise_train is used to perform joint training on the constructed toolbox network and the proxy network, which specifically may include the following steps:
first, an original image with noise is input and is marked as I 1 At the time of first repair, characteristic information in the image is recorded as a state S 1 The initial test action is recorded asThe maximum action taken from the proxy network is noted as v 1 The action corresponding to the maximum action value is denoted as a 1 At repair Step T (Step T, i.e., at the time of the T-th repair), the proxy network is in a given input state S T The value of action a is evaluated, which may represent a bit:
v T ＝f ag (S T )
wherein f ag Representing a proxy network, vector v T A value representing the action S T And representing the characteristic information of the image subjected to the T-1 time repair, namely the characteristic information of the image to be subjected to the T time repair.
In the present embodiment, the action with the maximum value is selected as a T I.e. a T ＝argmax a v T A, wherein v T Value vector v representing action a T Once according to v T Maximum value obtaining action a in (2) T Then the corresponding tool, i.e. the repair action, is selected, which will be applied to the input image I T (referring to the image after the T-1 th repair) to obtain a new restored image: i T+1 ＝f r (I T ,a T ) Wherein f r Indicating a resume function. Using mean square error (Mean Square Error, MSE)As a loss function, where I gT Representing an image before repair, I T+1 An Image (Restored Image) after the T-th restoration is shown.
In this embodiment, adam (adaptive moment estimation ) may be used as an optimizer of the tool network in the training process, and a gradient descent method is used to adjust the weight, and parameters (including weight and offset) and network structure are adjusted in the training process until the training is completed, at which time the image denoising model is trained.
In this embodiment, optionally, after training the image denoising model using the cracked pipeline image and the noise-added pipeline image, the method further includes:
and testing the trained image denoising model by using the test data noise_test so as to detect the denoising restoration effect.
In this embodiment, the effect graph after denoising and repairing is compared with the corresponding image in the test set test1, and the effectiveness of the image denoising model is verified.
In this embodiment, optionally, training the image raindrop removal model using the pipeline image with the crack and the pipeline image with the raindrop and the crack simultaneously includes:
c1, establishing an image raindrop removing model by adopting a method based on an antagonism network generation;
and C2, training the image raindrop removing model by using training data rain_train so that the trained image raindrop removing model can carry out raindrop removing restoration on the pipeline image with raindrops.
In this embodiment, the method for creating the image raindrop removal model (C1) based on the method for creating the countermeasure network may specifically include the following steps:
first, a generation network is constructed. As shown in fig. 4, the generating network is constructed by using a context automatic encoder, and the purpose of the context automatic encoder is to generate a duct image without raindrops but with cracks, and the input of the context automatic encoder is the duct image with both raindrops and cracks. As shown in fig. 4, the context-aware encoder has 16 conv-relu blocks (conv-relu means convolution first and then activation using the relu function). The context automatic encoder generates an image without raindrops but with cracks, and the image and the real clean image (namely the real image without raindrops but with cracks) respectively pass through a VGG16 network structure for respectively extracting the respective characteristics, so as to calculate a loss function value Lg.
Then, a discrimination network is constructed, which verifies whether the image generated by the generation network appears to be authentic. As shown in fig. 5, the input image of the discrimination network is a pipeline image without raindrops but with cracks generated by the generation network, then 7 conv+relu layers are passed to obtain the characteristics of the image, and finally the image is sent to the full connection layer for discrimination.
Finally, a loss function is constructed.
In this embodiment, the loss function of the constructed discrimination network is L D Log (1-D (G (Z))) +ylogd (x)), where x represents a true pipe image without raindrops but with cracks, that is, true sample data, D (x) represents an output of the discrimination model, that is, a probability that input x is true data, Z represents a pipe image with raindrops and cracks at the same time, G (Z) represents an output of the generation network, output is a pipe image without raindrops but with cracks, log represents a base 10 logarithmic operation, D (G (Z)) represents a probability that the discrimination network D determines whether the pipe image generated by the generation network G is true, y is a coefficient parameter for adjusting the weight, and is a value greater than 0 and less than 1.
In this embodiment, the loss function of the generated network constructed is L g ＝(1-y)log(1-D(G(z)))。
In this embodiment, in step C2, the training frequency is set to 10000, in the training process, an Adam optimizer may be used as an optimizer for generating a network and for determining a network, and a gradient descent method is used to adjust the weight, and in the training process, parameters (including weight and offset) and network structure are adjusted until the determining network converges to 0.5. At the moment, the raindrop removing model of the image is trained well.
In this embodiment, optionally, after training the image raindrop model using the cracked pipeline image and the pipeline image with both raindrops and cracks, the method further includes:
and testing the trained image raindrop removing model by using test data Rain_test so as to detect the raindrop removing repair effect of the image raindrop removing model.
In this embodiment, the effect graph after the raindrop removal repair is compared with the corresponding image in the test set test1, and the effectiveness of the raindrop removal model of the image is verified.
In this embodiment, optionally, the training the crack detection model using the target detection training set includes:
d1, establishing a crack detection model, wherein the crack detection model adopts a Faster RCNN (Faster Region Convolutional Neural Networks, faster regional convolutional neural network) network structure;
and D2, training the crack detection model by taking the train4 as a target detection training set, so that the trained crack detection model can identify and position cracks in the pipeline image.
In this embodiment, as shown in fig. 6, the network structure of the fast RCNN mainly includes four parts:
(1) Feature extraction: comprising 13 conv layers: kernel_size=3, pad=1, stride=1; 13 relu layers, namely the activation function; 4 landing layers; wherein, the connection relation between each layer is: conv-reuu-pulling-conv-reuu-conv Relu-pulling-conv-flu-Relu-pulling-conv-relu-conv-relu-conv-relu-pulling-conv-relu-conv-relu-conv-relu.
(2) Regional generation network (Region Proposal Networks, RPN): after the feature map obtained through feature extraction enters an RPN network, the feature map is subjected to convolution once 3*3, then 1 full convolution kernel_size=1×1, pad=0, stride=1, then probability values of all classifications (only one classification of cracks in the embodiment) are calculated by using softmax, then a relatively accurate prediction frame is obtained through a suggestion (proposal) layer, and further the prediction frame is subjected to out-of-range rejection and overlapping frames are removed, so that a more accurate prediction frame is obtained.
(3) Pooling of region of interest (Region of interest Pooling, ROI Pooling): and inputting a prediction box obtained by the RPN network into the ROI Pooling layer for further classification and positioning.
(4) Regression and classification: after passing through the ROI Pooling layer, the feature map is fully connected through the fully connecting layer, and the category probability vector is output by calculating which category each prediction frame specifically belongs to through the fully connecting layer and the softmax; and simultaneously, the position offset of each prediction frame is obtained again by using the regression of the prediction frames, so as to obtain a more accurate target detection frame.
In this embodiment, the position loss of the prediction frame uses a smoothl 1 loss, and the classification loss uses a multi-classification cross entropy nn.
In this embodiment, in step D2, network parameters of the crack detection model are initialized: setting the learning rate of the network to lr=10- 3 The number of data of BATCH training is BATCH_SIZE=6, and the number of training rounds is 100 rounds; then, training the constructed crack detection model by utilizing a training set train4, using an Adam optimizer as an optimizer of a fast RCNN network in the training process, adopting a gradient descent method to adjust the weight, and adjusting parameters (comprising the weight and the offset) and a network structure in the training process until the training is finished.
In this embodiment, optionally, after training the crack detection model using the target detection training set, the method further includes:
and testing the trained crack detection model by using a test set test4, and detecting the detection effect of the crack detection model to verify the effectiveness of the crack detection model.
In the embodiment, when the image denoising model, the image raindrop removing model and the crack detection model are trained and then used in an actual industrial environment, a monitoring camera is arranged on a filling pipeline to acquire a pipeline image to be detected in real time, wherein the monitoring camera is mainly positioned at the bottom of a drilling hole, two ends of a pipe cable well, the pressure is high, the corner of the pipeline, the abrasion of the pipeline is high and the area where the pipeline type changes, and a monitoring object is the outer wall of the easily damaged side of the pipeline; and then, denoising and raindrop removing processing are sequentially carried out on the pipeline image to be detected by using the trained image denoising model, the image raindrop removing model and the crack detecting model, and cracks are detected.
Fig. 7 is a schematic structural diagram of an electronic device 600 according to an embodiment of the present invention, where the electronic device 600 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 601 and one or more memories 602, where at least one instruction is stored in the memories 602, and the at least one instruction is loaded and executed by the processors 601 to implement the filling pipe crack detection method described above.
In an exemplary embodiment, a computer readable storage medium, such as a memory comprising instructions executable by a processor in a terminal to perform the above-described filling pipe crack detection method is also provided. For example, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (5)
1. A method of crack detection in a filled pipe, comprising:
training an image denoising model by using the pipeline image with the crack and the pipeline image with the crack after the noise adding process;
training an image raindrop removing model by using the pipeline image with the cracks and the pipeline image with the raindrops and the cracks at the same time;
training a crack detection model by using a target detection training set, wherein each pipeline image in the target detection training set is marked with a crack position;
acquiring a pipeline image to be detected in real time, and sequentially denoising, raindrop removing treatment and crack detection the pipeline image to be detected by using a trained image denoising model, an image raindrop removing model and a crack detection model;
wherein, before training an image denoising model by using a cracked pipeline image and the cracked pipeline image after the noise addition treatment, the method comprises:
collecting a pipeline image with cracks, and placing a glass block with raindrops in front of a camera for collecting the pipeline image with the raindrops and the cracks at the same time;
dividing the acquired pipeline image with the cracks into a training set train1 and a test set test1 according to a preset first proportion;
dividing the acquired pipeline image with raindrops and cracks into a training set train3 and a test set test3 according to a preset first proportion; wherein, train1 and train3 are used as a group of training data, which is denoted as train_train; test1 and test3 are taken as a group of test data and are marked as Rain_test;
respectively carrying out noise adding treatment on the training set train1 and the test set test1, and respectively marking the train1 and the test1 subjected to the noise adding treatment as train2 and test2; wherein, train1 and train2 are used as a group of training data, which is recorded as noise_train; test1 and test2 are taken as a group of test data and are marked as noise_test;
marking the training set train1 and the test set test1 after the labeling treatment as train4 and test4 respectively; the labeling process refers to marking the crack position in each pipeline image and adding a crack name;
wherein training the image denoising model by using the cracked pipeline image and the pipeline image with the crack after the noise adding process comprises the following steps:
establishing an image denoising model by adopting a method based on deep reinforcement learning; the image denoising model is built by using a tool box and a proxy network with a cyclic structure, wherein the tool box has 12 tools in total, the tools are also called repair actions, each tool is formed by using a three-layer or eight-layer CNN network structure, which is called a tool network, each repair step needs a agent proxy network to judge which tool in a Toolbox should be used, and the agent consists of three parts:
the feature extractor comprises 4 convolution layers and 1 full connection layer and is used for converting an input image into a 32-dimensional feature vector;
an One-hot encoder whose input is the motion estimation vector of the previous step, and whose output converts it into a corresponding feature vector;
a long-short-term memory network, which uses the outputs of the feature extractor and the One-hot encoder as inputs, and finally outputs the estimated value vector v of the current step t The recovery tool is used for selecting a recovery tool;
training the image denoising model by utilizing training data noise_train so that the trained image denoising model can denoise and repair a pipeline image with Noise;
the training of the image raindrop removal model by using the pipeline image with the cracks and the pipeline image with the raindrops and the cracks simultaneously comprises the following steps:
establishing an image raindrop removing model by adopting a method based on generating an countermeasure network;
training the image raindrop removing model by utilizing training data rain_train so that the trained image raindrop removing model can carry out raindrop removing restoration on a pipeline image with raindrops;
wherein training the crack detection model using the target detection training set comprises:
establishing a crack detection model, wherein the crack detection model adopts a Faster RCNN network structure;
and training the crack detection model by taking the train4 as a target detection training set, so that the trained crack detection model can identify and position cracks in the pipeline image.
2. The method of claim 1, wherein after training an image denoising model using a cracked pipe image and the noisy pipe image, the method further comprises:
and testing the trained image denoising model by using the test data noise_test so as to detect the denoising restoration effect.
3. The method of claim 1, wherein after training the image raindrop removal model using the cracked pipe image and the pipe image with both raindrops and cracks, the method further comprises:
and testing the trained image raindrop removing model by using test data Rain_test so as to detect the raindrop removing repair effect of the image raindrop removing model.
4. The filled duct crack detection method of claim 1, wherein after training the crack detection model with the target detection training set, the method further comprises:
and testing the trained crack detection model by using a test set test4.
5. The method of claim 1, wherein the acquiring in real time the pipe image to be detected comprises:
and arranging a monitoring camera on the filling pipeline, wherein the monitoring camera is used for acquiring the pipeline image to be detected in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010544835.9A CN111784645B (en) | 2020-06-15 | 2020-06-15 | Filling pipeline crack detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010544835.9A CN111784645B (en) | 2020-06-15 | 2020-06-15 | Filling pipeline crack detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111784645A CN111784645A (en) | 2020-10-16 |
| CN111784645B true CN111784645B (en) | 2023-08-22 |
Family
ID=72756560
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010544835.9A Active CN111784645B (en) | 2020-06-15 | 2020-06-15 | Filling pipeline crack detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111784645B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2602495B (en) * | 2021-01-04 | 2023-01-25 | Darkvision Tech Inc | Machine Learning Model for Identifying Surfaces in a Tubular |
| CN112746847B (en) * | 2021-01-15 | 2023-11-24 | 中国恩菲工程技术有限公司 | Mining method for gently-inclined medium-thickness ore body |
| CN113283382B (en) * | 2021-06-15 | 2022-08-30 | 合肥工业大学 | Method and device for describing leakage scene of underground pipeline |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010028387A2 (en) * | 2008-09-08 | 2010-03-11 | Schlumberger Technology Corporation | System and method for detection of flexible pipe armor wire ruptures |
| CN103499023A (en) * | 2013-09-24 | 2014-01-08 | 常州大学 | Method and device for detecting and positioning gas pipeline leakage on line |
| CN107220950A (en) * | 2017-05-31 | 2017-09-29 | 常州工学院 | A kind of Underwater Target Detection image enchancing method of adaptive dark channel prior |
| CN108416755A (en) * | 2018-03-20 | 2018-08-17 | 南昌航空大学 | A kind of image de-noising method and system based on deep learning |
| CN109342424A (en) * | 2018-10-25 | 2019-02-15 | 南京水动力信息科技有限公司 | A kind of road drainage pipeline of view-based access control model detection makes an on-the-spot survey method |
| CN109559302A (en) * | 2018-11-23 | 2019-04-02 | 北京市新技术应用研究所 | Pipe video defect inspection method based on convolutional neural networks |
| CN110826588A (en) * | 2019-08-29 | 2020-02-21 | 天津大学 | Drainage pipeline defect detection method based on attention mechanism |
| CN110969611A (en) * | 2019-12-03 | 2020-04-07 | 广州特种承压设备检测研究院 | Pipeline weld defect detection method, device and system and storage medium |
| CN111008961A (en) * | 2019-11-25 | 2020-04-14 | 深圳供电局有限公司 | Transmission line equipment defect detection method and system, equipment and medium thereof |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201711412D0 (en) * | 2016-12-30 | 2017-08-30 | Maxu Tech Inc | Early entry |
| CA3076483A1 (en) * | 2017-11-16 | 2019-05-23 | MultiSensor Scientific, Inc. | Systems and methods for multispectral imaging and gas detection using a scanning illuminator and optical sensor |
-
2020
- 2020-06-15 CN CN202010544835.9A patent/CN111784645B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010028387A2 (en) * | 2008-09-08 | 2010-03-11 | Schlumberger Technology Corporation | System and method for detection of flexible pipe armor wire ruptures |
| CN103499023A (en) * | 2013-09-24 | 2014-01-08 | 常州大学 | Method and device for detecting and positioning gas pipeline leakage on line |
| CN107220950A (en) * | 2017-05-31 | 2017-09-29 | 常州工学院 | A kind of Underwater Target Detection image enchancing method of adaptive dark channel prior |
| CN108416755A (en) * | 2018-03-20 | 2018-08-17 | 南昌航空大学 | A kind of image de-noising method and system based on deep learning |
| CN109342424A (en) * | 2018-10-25 | 2019-02-15 | 南京水动力信息科技有限公司 | A kind of road drainage pipeline of view-based access control model detection makes an on-the-spot survey method |
| CN109559302A (en) * | 2018-11-23 | 2019-04-02 | 北京市新技术应用研究所 | Pipe video defect inspection method based on convolutional neural networks |
| CN110826588A (en) * | 2019-08-29 | 2020-02-21 | 天津大学 | Drainage pipeline defect detection method based on attention mechanism |
| CN111008961A (en) * | 2019-11-25 | 2020-04-14 | 深圳供电局有限公司 | Transmission line equipment defect detection method and system, equipment and medium thereof |
| CN110969611A (en) * | 2019-12-03 | 2020-04-07 | 广州特种承压设备检测研究院 | Pipeline weld defect detection method, device and system and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于强化学习的浓密机底流浓度在线控制算法;袁兆麟 等;自动化学报 网络首发;1558-1571 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111784645A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111784645B (en) | Filling pipeline crack detection method | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN110992349A (en) | Underground pipeline abnormity automatic positioning and identification method based on deep learning | |
| CN110136116B (en) | Injection pump defect detection method, device, equipment and storage medium | |
| CN109767426B (en) | Shield tunnel water leakage detection method based on image feature recognition | |
| CN111667470B (en) | Industrial pipeline flaw detection inner wall detection method based on digital image | |
| CN113516651A (en) | Welding joint defect detection method and device based on residual error network | |
| CN111489352A (en) | Tunnel gap detection and measurement method and device based on digital image processing | |
| CN113706472B (en) | Highway pavement disease detection method, device, equipment and storage medium | |
| CN111079822A (en) | Method for identifying dislocation fault image of middle rubber and upper and lower plates of axle box rubber pad | |
| CN112200790B (en) | Cloth defect detection method, device and medium | |
| CN118037726B (en) | Railway accessory defect detection method and system | |
| CN111539927A (en) | Detection process and algorithm of automobile plastic assembly fastening buckle lack-assembly detection device | |
| CN113012157A (en) | Visual detection method and system for equipment defects | |
| CN113971669A (en) | Three-dimensional detection system applied to pipeline damage identification | |
| CN113888477A (en) | Network model training method, metal surface defect detection method and electronic equipment | |
| CN115719326A (en) | PCB defect detection method and device | |
| CN113516652A (en) | Battery surface defect and adhesive detection method, device, medium and electronic equipment | |
| CN112215301A (en) | Image straight line detection method based on convolutional neural network | |
| CN107123105A (en) | Images match defect inspection method based on FAST algorithms | |
| CN113034432B (en) | Product defect detection method, system, device and storage medium | |
| US8306311B2 (en) | Method and system for automated ball-grid array void quantification | |
| CN115330743A (en) | Method for detecting defects based on double lights and corresponding system | |
| CN110634124A (en) | Method and equipment for area detection | |
| CN111047550A (en) | Product abnormity detection method, device and equipment based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240619 Address after: Room 101, 1st Floor, Block D, Building 15, Building 1-6, Courtyard 5, Jiangtai Road, Chaoyang District, Beijing, 100016 Patentee after: FENGNIAO INNOVATION (BEIJING) TECHNOLOGY Co.,Ltd. Country or region after: China Address before: 100083 No. 30, Haidian District, Beijing, Xueyuan Road Patentee before: University OF SCIENCE AND TECHNOLOGY BEIJING Country or region before: China |