CN109559302A - Pipe video defect inspection method based on convolutional neural networks - Google Patents
Pipe video defect inspection method based on convolutional neural networks Download PDFInfo
- Publication number
- CN109559302A CN109559302A CN201811403708.6A CN201811403708A CN109559302A CN 109559302 A CN109559302 A CN 109559302A CN 201811403708 A CN201811403708 A CN 201811403708A CN 109559302 A CN109559302 A CN 109559302A
- Authority
- CN
- China
- Prior art keywords
- cnn
- image
- video
- defect
- pipeline
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of pipe video defect inspection method based on convolutional neural networks, frame is taken out to video, the multiple CNN of training classify to every frame image, count the result that each CNN is returned, it determines the defect type of the frame, is input with pipeline closed-circuit television video, be successive image frame by video slicing, every frame image is sent into multiple trained CNN and carries out two classification, classification results are only included containing certain specified defect and zero defect.Invention significantly improves the accuracys rate of pipeline defect detection, a kind of feasible method is provided for video detection, the automatic detection efficiency of defect of pipeline not only can be improved, also it can reduce the labor intensity of staff, this method Detection accuracy is high and detection speed is fast, has very big application value in pipe video defects detection, and achieves more satisfied result, it can be used as the Technical Reference of pipeline defect detection worker, the needs of practical application can be met well.
Description
Technical field
The invention belongs to technical field of image processing, and in particular to a kind of pipe video defect based on convolutional neural networks
Detection method.
Background technique
Drainage pipeline is the important component of sewerage system, in use, pipeline caused by environmental factor
Functional defect and structural defect cause drainage pipeline cisco unity malfunction, and when heavy rain is attacked, rainwater cannot exclude in time,
The case where many big cities meet with " waterlogging is at sea ", makes troubles to urban construction and people's lives.In order to greatest extent
Ground plays the drainability of existing pipeline, is timely discovery drainage pipeline security risk to the periodic detection of existing drainage pipeline
Effective measures.
Currently, CCTV (Closed Circuit Television) detection technique is widely used in detection inside pipeline
In, pipe robot carrying camera, which is gone into the well, shoots video, and operator is quasi- without energy of going into the well by way of playing video
Really detect pipeline situation.But pipeline internal light is dim and situation is complicated, operator works long hours easy visual fatigue
Influence detection accuracy.Therefore, pipe video defects detection is a very challenging job.Traditional pipe detection
Method is based primarily upon manual features and extracts in conjunction with the engineering of neural network (ANN) and support vector machines (SVM) as classifier
Learning method.They are usually taken the method for artificial selected characteristic and feature vector feeding classifier are classified.However, this
The versatility of method is not strong, the generalization ability of image classification is not strong, and when image category is excessive, being difficult to find out one group can be described
The feature of all categories image, therefore it is not particularly suited for complex situations.
Summary of the invention
For above-mentioned problems of the prior art, it can avoid above-mentioned skill occur the purpose of the present invention is to provide one kind
The pipe video defect inspection method based on convolutional neural networks of art defect.
In order to achieve the above-mentioned object of the invention, technical solution provided by the invention is as follows:
A kind of pipe video defect inspection method based on convolutional neural networks takes out frame to video, and the multiple CNN of training are to every
Frame image is classified, and counts that each CNN is returned as a result, determining the defect type of the frame.
Further, it is input with pipeline closed-circuit television video, is successive image frame by video slicing, every frame image is sent
Enter and carry out two classification in multiple trained CNN, classification results are only included containing certain specified defect and zero defect.
Further, the CNN includes top half network structure and lower half portion network structure；Top half network knot
Structure is made of 4 layers of convolutional layer and 3 layers of pond layer, and lower half portion network structure is mentioned by the feature of a level 2 volume product structure composition
Take device；Merge two full articulamentums of top half network structure and lower half portion network structure, training softmax output category
Probability.
Further, the first convolutional layer of top half network structure uses 32 5 × 5 cores, and the second convolutional layer 2 uses 64
A 3 × 3 core, 21 × 1 convolution kernels, which are respectively adopted, in third convolutional layer reduces input dimension, is then respectively fed to two Volume Four products
Layer, last stack features figure are sent into full articulamentum；First convolutional layer of lower half portion network structure uses 32 5 × 5 cores, the
Two convolutional layers use 64 3 × 3 cores, and pond is only carried out after the first convolutional layer, is sent into characteristic pattern after the second convolutional layer
512 Hidden units connect entirely；Finally merge two 512 dimension Quan Lian of top half network structure and lower half portion network structure
Connect layer, training softmax output category probability.
Further, the activation primitive of the CNN is ELU, and the definition that ELU corrects linear unit is
α is an adjustable parameter, it controls when ELU negative loop is being saturated.
Further, 1 α.
Further, the pond of the CNN turns to random pool, distributes its selected probability according to the size of characteristic point,
Steps are as follows:
1) ask pond region statistics and
2) each pixel in pond region obtains the probability value of each pixel divided by the summation for counting and calculating
3) according to probability value stochastical sampling mj=ai, l~P (p1..., p | Rj|)；
RjFor the size of pond layer sliding window；K is the number of element in pond layer sliding window；aiFor pond window
Element value；L is in the window of pond according to piRandomly selected value.
Further, the sufficient data training CNN is inputted, enhances skill using data in the case where data volume is inadequate
Art carrys out dilated data set using one or more combinations by the geometric transformation of image.
Further, overturning, highlighted method EDS extended data set are taken, image is from left to right overturn first, so
Image is highlighted afterwards.
Further, training CNN includes that image is normalized, and takes and subtracts average pixel value and the mark divided by pixel
Image is normalized in the mode of quasi- difference, formula are as follows:
Wherein XnIt is normalization pixel, x is original pixels,The mean value and standard deviation of original image pixels are respectively referred to s.
Pipe video defect inspection method provided by the invention based on convolutional neural networks, significantly improves defect of pipeline
The accuracy rate of detection provides a kind of feasible method for video detection, and the automatic detection effect of defect of pipeline not only can be improved
Rate also can reduce the labor intensity of staff, and this method Detection accuracy is high and detection speed is fast, examines in pipe video defect
There is very big application value in survey, and achieve more satisfied as a result, can be used as the technology of pipeline defect detection worker
With reference to the needs of practical application can be met well.
Detailed description of the invention
Fig. 1 is the flow chart of pipe video defects detection；
Fig. 2 is improvement convolutional neural networks structure chart of the invention；
Fig. 3 (a) is ReLU curve graph；
Fig. 3 (b) is LReLU curve graph；
Fig. 3 (c) is ELU curve graph；
Fig. 4 is the line chart that 5- rolls over cross validation Average Accuracy.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawing and specific implementation
The present invention will be further described for example.It should be appreciated that described herein, specific examples are only used to explain the present invention, and does not have to
It is of the invention in limiting.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise
Under every other embodiment obtained, shall fall within the protection scope of the present invention.
A kind of pipe video defect inspection method based on convolutional neural networks, by taking out frame, training multiple two to video
Classification convolutional neural networks are detected and are classified to every frame image, count that each convolutional neural networks return as a result, determining
The defect type of the video frame finally counts the result of whole section of video frame.
Traditional convolutional neural networks (CNN) by input layer, convolutional layer (Convolutional layer), activation primitive,
Pond layer (Pooling Layer), full articulamentum (Fully-connected layer) composition.
Utilize multiple convolutional neural networks (convolutional neural network, CNN) model inspection drainpipe
Whether contain defective and analyzing defect type in road.Each a kind of specified defect of neural network learning, by image in training process
It is classified as including or being learnt not comprising specific type of defects.Testing process is input with pipeline closed-circuit television video, first
Video is split as successive image frame, and then every frame is sent into 6 trained convolutional neural networks and carries out two classification, classification
As a result the return value definitive result for finally combining each neural network containing certain specified defect and zero defect is only included.Fig. 1 gives
The process of pipe video defects detection is gone out.The structure of each CNN is identical in Fig. 1, and training label is different.Training data is divided into just
Sample (including the image of defect) and negative sample (image for not including the defect).
The cascade structure of multilayer is mainly used in CNN processing image process, such network design more considers image
Minutia, sampled images feature is refined by the convolution operation of multi-layer, to realize the overall performance to image.But
It is excessively to pay close attention to the minutia in local receptor field due to such network, be easy to be limited to image itself, when the mesh in image
When marking relatively small, relies solely on details and be unable to accurate description target.
On the contrary, better effect can be obtained by suitably increasing feature of overall importance；Only one channel traditional CNN and the first floor
Convolution kernel size is fixed, then single channel CNN there is due to convolution mask it is single, caused loss original image portion difference
The problem of scale feature.Therefore the present invention designs the network structure of two different levels and by the two neural network in parallel
Structure realizes drainage pipeline defects detection.The purpose done so not only can solve the insufficient problem of feature extraction can also whole handle
Hold image overall feature and minutia.
1 × 1 convolution is added among convolutional layer to reduce dimension in the present invention.1 × 1 convolution kernel can reduce the channel of input
Number reduces computational complexity, accelerates network query function speed.Using 1 × 1 convolution, linear group for realizing multiple feature map
It closes.1 × 1 convolutional layer is added in the present invention between convolutional layer 2,4, so that the feature map number of output drops to 16, can make next
The training parameter of convolutional layer drops to 18512 from 69760, and parameter amount is greatly decreased, and improves the calculating speed of network.
In CNN network structure shown in Fig. 2, top half network structure is by 4 layers of convolutional layer, 3 layers of pond layer composition.Wherein roll up
Lamination 1 uses 32 5 × 5 cores, and convolutional layer 2 uses 64 3 × 3 cores, and it is defeated that 21 × 1 convolution kernels reductions are respectively adopted in convolutional layer 3
Enter dimension, be then respectively fed to convolutional layer 4-1 and convolutional layer 4-2, last stack features figure is sent into full articulamentum.Lower half portion
Network structure is by the feature extractor of a level 2 volume product structure composition, and purpose is exactly to increase feature of overall importance, first layer convolution
Layer uses 32 5 × 5 cores, and the second layer uses 64 3 × 3 cores, pond is only carried out after first layer convolution, the purpose for the arrangement is that
Retain image global characteristics as far as possible.Characteristic pattern connects entirely with 512 Hidden units are sent into after second layer pond, finally melts
It closes upper layer and lower layer 512 and ties up full articulamentum, training softmax output category probability.English word in Fig. 2: filters was represented
Filter, Kernel represent core, and Conv represents convolutional layer, and Input represents input, and Full-connected and FC represent full connection
Layer, Output represent output, and Defect represents defect, and stochastic pooling represents random pool.
The feature that top half network structure convolution proposes is more abstract, the feature that lower half portion network structure convolution proposes
More global, the feature that lower half portion network structure convolution proposes can reflect the global feature in pipeline from the overall situation, retain
The global feature of pipe inside nozzle, deposition and barrier.
The present invention is by improving CNN structure, and the global characteristics learnt using shallow-layer network structure are to from further feature
It is supplemented, CNN is helped more fully to grasp defect characteristic, 1 × 1 convolutional layer is added at the same time, input feature vector is dropped
Network training parameter is greatly reduced in dimension, accelerates network query function speed, reduces the appearance of over-fitting.Improved CNN can effectively more
Traditional algorithm is mended in detection defect accuracy rate and detects the deficiency in speed.
In artificial neural network, commonly used in increasing non-linear factor, introducing is non-linear effectively to delay ReLU function
Gradient disappearance problem is solved, is defined such as formula (1)
In formula: x indicates the input of hidden layer, shown in ReLU curve such as Fig. 3 (a).ReLU is activated when x is greater than 0, gradient
It is 1, gradient saturation problem is not present.When input is negative, ReLU is not activated completely.Therefore, ReLU can
Keep gradient unattenuated in x > 0, to alleviate gradient disappearance problem.However as trained propulsion, part input can be fallen into
Hard saturation region causes respective weights that can not update, and influences the convergence of network.
For the hard saturation problem in x < 0, LReLU activation primitive can be used, is defined such as formula (2)
In formula: a is (1 ,+∞) interior preset parameter, shown in functional image such as Fig. 3 (b).
There is a smaller slope at LReLU function negative end, so that no longer there is 0 gradient in entire active region
The case where, alleviate the problem of hidden layer neuron is abandoned.A can be set as 5.5.
The definition such as formula (3) of ELU correction linear unit
In formula: α is an adjustable parameter, it controls when ELU negative loop is being saturated；α is arranged in the present embodiment
It is 1, shown in functional image such as Fig. 3 (c).
ELU can get negative value, this allow unit activating mean value can closer to 0, similar to the effect of regularization, still,
Only need lower computation complexity.LReLU also has negative value, but it does not guarantee in the state that input is negative to noise Shandong
Stick.ELU characteristic with soft saturation when input takes smaller value is reviewed, the robustness to noise is improved.Right linear part
ELU is enabled to alleviate gradient disappearance, and the soft saturation in left side can allow ELU more healthy and stronger to input variation or noise, therefore this
Preferred ELU is invented as activation primitive.
Common pond method has maximum pond, mean value pond.Maximum pond is to take it most by the feature in specified neighborhood
A little louder, the offset error that convolutional layer parameter error causes estimation mean value is reduced, texture information is more retained.Mean value pond is pair
Characteristic point is averaging in neighborhood.The error for reducing convolution layer parameter caused by Size of Neighborhood is limited, more retains the back of image
Scape information.Random poolization then falls between, and on the one hand maximizes the value that ensure that maximum value, on the one hand ensures all members
Element will not all be taken as maximum value, cause excessive distortion.In order to improve the generalization ability of sorter network, the present invention uses random pool
Method.
Random pool distributes its selected probability according to the size of characteristic point.Steps are as follows for calculating:
A) ask pond region statistics and
B) each pixel in pond region divided by pass through statistics that a) step is calculated and, obtain each pixel
Probability value
C) according to probability value stochastical sampling mj=ai, l~P (p1..., p | Rj|)
R in formulajFor the size of pond layer sliding window；K is the number of element in pond layer sliding window.aiFor pond window
The element value of mouth；L is in the window of pond according to piRandomly selected value.
Experimental data of the invention derives from New Technique Application Inst., Beijing City, by most common detection instrument pipe
Road closed-circuit television detection system CCTV is collected from a few big city subsoil drains in China.Picture format is jpg, resolution ratio
It is 700 × 572.Share 6 class defects include residual wall, deposition, tree root intrusion, foreign matter penetrate, barrier, branch pipe housed joint.Defect is fixed
Justice is referring to CJJ-181-2012- urban drainage pipe detection and assessment technology regulation.
In order to avoid neural network over-fitting, it usually needs input sufficient data training neural network.Data volume not
Enhance technology using data in the case where enough, by the geometric transformation of image, carrys out dilated data set using one or more combinations.
The present invention takes overturning, highlighted method EDS extended data set.Image is from left to right overturn first, then image is carried out
It is highlighted.It is expanded by data set, 22000 pictures participate in training, verifying and the test of model in total.This experiment is expanded based on data
It is carried out after filling.The following table 1 participates in training, the amount of images of verifying and test after showing amplification.
The training of table 1, the picture number of verifying and test
Convolutional neural networks are trained usually using image of the resolution ratio between 128 × 128 to 299 × 299 pixels.
All images of this experiment are unified for 256 × 256 pixels.Training neural network usually requires that image is normalized.Normalizing
The convergent speed of training, which not only can be improved, in change also can ensure that all images have approximate pixel value range, prevent reversed
Gradient is calculated during propagation fluctuation occurs.The present invention takes the mode for subtracting average pixel value and the standard deviation divided by pixel
Image is normalized, operation is as shown in Equation 4:
Wherein XnIt is normalization pixel, x is original pixels,The mean value and standard deviation of original image pixels are respectively referred to s.
Experimental situation: this experiment carries out on LINUX server, is accelerated using 1080 GPU of NVIDIA Titan.
Development language is Python, carries out building for CNN using TensorFlow packet.The Learning Step that batch gradient descent method is arranged is
0.001, setting epochs size is 500, Dropout=0.5, and batch processing size is 64, and data are sent into a manner of out-of-order
Neural metwork training is sent by batch.Repeatedly training and verifying iteration are executed to optimize hyper parameter, is finally selected on verifying collection
The highest model of classification accuracy.
The present invention carrys out the performance of assessment models using k- folding cross validation method.K- rolls over cross-validation method and data is divided into k
A different grouping, per next, individually grouping is for verifying model, other K-1 sample is for training.Cross validation repeats K
Secondary, each sample verifying is primary.The purpose done so can allow network insensitive to the division of data, be conducive to evaluation model
Generalization ability.K value of the present invention is 5.Training, verifying are divided into positive and negative.Correction data collection is by including specific type of defects
Image composition, and negative data collection is made of the image except specified defect type.For example, when classifying tree root intrusion defect, tree root
Invaded image is assigned front label, and penetrated containing residual wall, deposition, foreign matter, the image of barrier and branch pipe housed joint is assigned
For negative label.
The present invention makees model using accuracy rate (Accuracy), accurate rate (Precision) and recall rate (Recall)
Performance Evaluation.Accuracy rate is calculated using formula (5):
The positive class of TP expression determines the class that is positive in formula, and TN indicates that negative class determines the class that is negative, and FP indicates that negative class determines the class that is positive, FN
Indicate that positive class determines the class that is negative.
Accurate rate and recall rate are calculated using formula (6) (7):
TP indicates that the judgement of positive class is positive class, and the FP expression class judgement that is negative is positive class.Accurate rate indicates the sample that prediction is positive
In how many be real positive sample.
TP indicates that positive class determines the class that is positive, and FN indicates that positive class determines the class that is negative.Recall rate indicates how many positive example in sample
It is predicted correct.
Fig. 4 show 5- folding cross validation Average Accuracy, wherein the broken line of top, which represents, averagely trains accuracy rate, lower section
Broken line represent and averagely verify accuracy rate, it is average to verify it can be seen from the figure that accuracy rate is averagely trained to be no better than 100%
Accuracy rate is respectively 89.2%.The Average Accuracy of training and verifying difference about 10%, shows that there are over-fittings to show in network
As.Over-fitting Producing reason is since training set quantity is inadequate.It can further be verified by continuing amplification data collection.
The following table 2 is experimental result of the test set on the algorithm proposed.
Experimental result of 2 test set of table in 5- folding cross validation
It is obtained by the analysis of upper table 2, the Average Accuracy of 6 kinds of defects, average accuracy and average recall rate are respectively
87.3%, 85.6% and 89.7%.Wherein tree root invades the accuracy rate on test set, and accurate rate and recall rate are apparently higher than it
His several defects.Tree root intrusion have the reason of preferable classification results be tree root intrusion training sample is more and the shape of tree root
Shape, size, color and texture are more prominent compared with other several defects, and upper layer CNN can extract more minutias, simultaneously
Tree root distributed areas are more, are also beneficial to it for lower layer CNN and extract to the global feature of tree root.Barrier and deposition
Also relatively preferably, reason is that barrier is mostly stone or sundries to classification results, and profile and color are all obvious, and CNN can also
These features are arrived with good study.In contrast, the accuracy rate that branch pipe housed joint and foreign matter penetrate is relatively low, with other three kinds
The accuracy rate difference 8% or so of defect.Cause accuracy rate lower the reason is that since two kinds of defects have similitude.
Application activating function can introduce non-linear factor in neural network, and neural network is better solved
Challenge, different activation primitives have different shadows to the convergence rate in the nicety of grading and backpropagation of neural network
It rings.The present invention applies these three activation primitives of ReLU, LReLU and ELU respectively on improved CNN, counts improved CNN model
The number of iterations and the wrong relationship for dividing rate in training process.
Using the model of ELU activation primitive after iteration 150 times the mistake point rate of network just fall below 30% hereinafter, and
ReLU and LReLU needs 200 iteration to can be only achieved identical result.ELU reduces the gap of normal gradient and natural gradient
To accelerate e-learning.The experimental results showed that ELU is substantially better than other two kinds of activation primitives, network convergence is effectively shortened
Time reduces the mistake point rate of network, illustrates validity of the present invention to ELU activation primitive using strategy.
Table 3 lists test result of the method proposed by the present invention on experimental data set, while also listing traditional CNN
Pipeline defect detection method and other improvements CNN method and traditional shortcoming detection method based on HOG+LPB+HSV are in this reality
Test the testing result on data set.
3 contrast and experiment of table
More contrast convolutional neural networks, the time required for 5 road combined CNN is 5 times of single network, when leading to detection
Between it is too long, the detection time of single picture is up to 1.3s；Binary channels convolutional neural networks improve conventional one-channel neural network
Convolution mask is single, extracts the insufficient problem of depth characteristic, but its network is relatively deep, training parameter amount is big, detection time
It is also relatively long for 0.45s.Although conventional method when detecting between on it is most fast, feature considers insufficient, and accuracy rate is not high.
Pipe video data include a large amount of picture frame, have a very high requirement to detection speed and Detection accuracy, and this method is from whole
Image overall feature is held on body, and CNN is helped more fully to grasp defect of pipeline feature, while 1 × 1 be added in study
Convolutional layer reduces model parameter, accelerates calculating speed.
From table 3 it can be seen that accuracy of the mentioned algorithm on data set is 87.3%, accurate rate 85.6% and recall rate
89.7%.It is more excellent relative to the pipeline defect detection method performance based on CNN.With pair of a few class representativeness algorithms of the prior art
Than this method has been above mainstream detection method in detection speed and Detection accuracy.
In order to verify the adaptability of the method for the present invention in practical applications, the present invention carries out 10 sections of CCTV pipe videos
Simultaneously contrast and experiment is tested, as shown in table 4.
4 video data test result of table
As known from Table 4, the present invention uses the pipe video of 10 sections of a length of 30s of mean time as test data.Using per second 3
The pumping frame strategy of frame calculates the Detection accuracy of every section of video and records detection time.Pipe video is lacked using this method
Sunken verification and measurement ratio can achieve 90.2%, and the average detected time is 48s.The experimental results showed that traditional CNN compared with the prior art is calculated
Method and a variety of improvement CNN algorithms, this method have more advantage on the average classification time, are more suitable for the big spirogram in pipe video
As frame is detected.
Pipe video defect inspection method proposed by the present invention based on convolutional neural networks significantly improves defect of pipeline
The accuracy rate of detection provides a kind of feasible method for video detection, and the automatic detection effect of defect of pipeline not only can be improved
Rate also can reduce the labor intensity of staff.The experimental results showed that this method Detection accuracy is high and detection speed is fast,
There is very big application value in pipe video defects detection, and achieve examining as a result, can be used as defect of pipeline for more satisfaction
The Technical Reference of survey worker.
Embodiments of the present invention above described embodiment only expresses, the description thereof is more specific and detailed, but can not
Therefore limitations on the scope of the patent of the present invention are interpreted as.It should be pointed out that for those of ordinary skill in the art,
Without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to protection model of the invention
It encloses.Therefore, the scope of protection of the patent of the invention shall be subject to the appended claims.
Claims (10)
1. a kind of pipe video defect inspection method based on convolutional neural networks, which is characterized in that take out frame to video, training is more
A CNN classifies to every frame image, counts that each CNN is returned as a result, determining the defect type of the frame.
2. pipeline defect detection method according to claim 1, which is characterized in that with pipeline closed-circuit television video be it is defeated
Enter, be successive image frame by video slicing, every frame image is sent into multiple trained CNN and carries out two classification, classification results
It only includes containing certain specified defect and zero defect.
3. pipeline defect detection method according to claim 1, which is characterized in that the CNN includes top half network
Structure and lower half portion network structure；Top half network structure is made of 4 layers of convolutional layer and 3 layers of pond layer, lower half subnetting
Network structure is by the feature extractor of a level 2 volume product structure composition；Merge top half network structure and lower half portion network
Two full articulamentums of structure, training softmax output category probability.
4. pipeline defect detection method according to claim 1 to 3, which is characterized in that the first of top half network structure
Convolutional layer uses 32 5 × 5 cores, and the second convolutional layer 2 uses 64 3 × 3 cores, and 21 × 1 convolution are respectively adopted in third convolutional layer
Core reduces input dimension, is then respectively fed to two Volume Four laminations, last stack features figure is sent into full articulamentum；Lower half
First convolutional layer of subnetwork structure uses 32 5 × 5 cores, and the second convolutional layer uses 64 3 × 3 cores, only in the first convolutional layer
Pond is carried out afterwards, connects characteristic pattern 512 Hidden units of feeding entirely after the second convolutional layer；Finally merge upper half subnetting
The two 512 full articulamentums of dimension of network structure and lower half portion network structure, training softmax output category probability.
5. pipeline defect detection method according to claim 1, which is characterized in that the activation primitive of the CNN is ELU,
ELU correct linear unit definition be
α is an adjustable parameter, it controls when ELU negative loop is being saturated.
6. pipeline defect detection method described in -5 according to claim 1, which is characterized in that α 1.
7. pipeline defect detection method according to claim 1, which is characterized in that the pond of the CNN turns to random pool,
Its selected probability is distributed according to the size of characteristic point, steps are as follows:
1) ask pond region statistics and
2) each pixel in pond region obtains the probability value of each pixel divided by the summation for counting and calculating
3) according to probability value stochastical sampling mj=ai, l~P (p1..., p | Rj|)；
RjFor the size of pond layer sliding window；K is the number of element in pond layer sliding window；aiFor the element of pond window
Value；L is in the window of pond according to piRandomly selected value.
8. pipeline defect detection method according to claim 1, which is characterized in that input described in sufficient data training
CNN uses one or more groups by the geometric transformation of image using data enhancing technology in the case where data volume is inadequate
It closes and carrys out dilated data set.
9. pipeline defect detection method according to claim 8, which is characterized in that overturning, highlighted method is taken to expand
Data set first from left to right overturns image, is then highlighted to image.
10. pipeline defect detection method according to claim 1, which is characterized in that training CNN includes returning to image
One changes, and takes and subtracts average pixel value and image is normalized divided by the mode of the standard deviation of pixel, formula are as follows:
Wherein XnIt is normalization pixel, x is original pixels,The mean value and standard deviation of original image pixels are respectively referred to s.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811403708.6A CN109559302A (en) | 2018-11-23 | 2018-11-23 | Pipe video defect inspection method based on convolutional neural networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811403708.6A CN109559302A (en) | 2018-11-23 | 2018-11-23 | Pipe video defect inspection method based on convolutional neural networks |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109559302A true CN109559302A (en) | 2019-04-02 |
Family
ID=65866944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811403708.6A Pending CN109559302A (en) | 2018-11-23 | 2018-11-23 | Pipe video defect inspection method based on convolutional neural networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109559302A (en) |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110662A (en) * | 2019-05-07 | 2019-08-09 | 济南大学 | Driver eye movement behavioral value method, system, medium and equipment under Driving Scene |
| CN110443182A (en) * | 2019-07-30 | 2019-11-12 | 深圳市博铭维智能科技有限公司 | A kind of urban discharging pipeline video abnormality detection method based on more case-based learnings |
| CN110599460A (en) * | 2019-08-14 | 2019-12-20 | 深圳市勘察研究院有限公司 | Underground pipe network detection and evaluation cloud system based on hybrid convolutional neural network |
| CN110728654A (en) * | 2019-09-06 | 2020-01-24 | 台州学院 | Automatic pipeline detection and classification method based on deep residual error neural network |
| CN110838107A (en) * | 2019-10-31 | 2020-02-25 | 广东华中科技大学工业技术研究院 | Method and device for intelligently detecting defects of 3C transparent component by variable-angle optical video |
| CN110841142A (en) * | 2019-10-22 | 2020-02-28 | 江苏爱朋医疗科技股份有限公司 | Method and system for predicting blockage of infusion pipeline |
| CN110930377A (en) * | 2019-11-18 | 2020-03-27 | 福州大学 | Automatic detection method for drainage pipeline abnormal type based on multitask learning |
| CN111191593A (en) * | 2019-12-30 | 2020-05-22 | 成都云尚物联环境科技有限公司 | Image target detection method and device, storage medium and sewage pipeline detection device |
| CN111311572A (en) * | 2020-02-13 | 2020-06-19 | 桂林理工大学 | Pipeline detection method and device, storage medium and robot |
| CN111402203A (en) * | 2020-02-24 | 2020-07-10 | 杭州电子科技大学 | Fabric surface defect detection method based on convolutional neural network |
| CN111443095A (en) * | 2020-05-09 | 2020-07-24 | 苏州市平海排水服务有限公司 | Pipeline defect identification and judgment method |
| CN111507960A (en) * | 2020-04-16 | 2020-08-07 | 重庆三峡学院 | Camera module lens contamination detection method based on neural network visualization |
| CN111784645A (en) * | 2020-06-15 | 2020-10-16 | 北京科技大学 | Crack detection method for filling pipeline |
| CN111986188A (en) * | 2020-08-27 | 2020-11-24 | 深圳市智源空间创新科技有限公司 | Capsule robot drainage pipe network defect identification method based on Resnet and LSTM |
| CN112712497A (en) * | 2020-12-21 | 2021-04-27 | 南京惊蛰信息技术有限公司 | Stability detection method for cast iron pipeline joint based on local descriptor |
| CN113221710A (en) * | 2021-04-30 | 2021-08-06 | 深圳市水务工程检测有限公司 | Neural network-based drainage pipeline defect identification method, device, equipment and medium |
| CN113421252A (en) * | 2021-07-07 | 2021-09-21 | 南京思飞捷软件科技有限公司 | Actual detection method for vehicle body welding defects based on improved convolutional neural network |
| CN113469252A (en) * | 2021-07-02 | 2021-10-01 | 河海大学 | Extra-high voltage converter valve operation state evaluation method considering unbalanced samples |
| CN113592002A (en) * | 2021-08-04 | 2021-11-02 | 江苏网进科技股份有限公司 | Real-time garbage monitoring method and system |
| CN113610831A (en) * | 2021-08-19 | 2021-11-05 | 江西应用技术职业学院 | Wood defect detection method based on computer image technology and transfer learning |
| CN113761785A (en) * | 2021-06-11 | 2021-12-07 | 神华国能宁夏煤电有限公司 | Method and device for determining thinning value of boiler and storage medium |
| US20220167066A1 (en) * | 2020-11-26 | 2022-05-26 | Communication University of Zhejiang | Short video copyright storage method based on blockchain and expression identification |
| CN116596885A (en) * | 2023-05-19 | 2023-08-15 | 长江生态环保集团有限公司 | Real-time detection method and system for drainage pipe network state |
| CN116805416A (en) * | 2023-08-21 | 2023-09-26 | 中国电建集团华东勘测设计研究院有限公司 | Drainage pipeline defect identification model training method and drainage pipeline defect identification method |
| CN117576105A (en) * | 2024-01-17 | 2024-02-20 | 高科建材（咸阳）管道科技有限公司 | Pipeline production control method and system based on artificial intelligence |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104966104A (en) * | 2015-06-30 | 2015-10-07 | 孙建德 | Three-dimensional convolutional neural network based video classifying method |
| CN105938559A (en) * | 2015-03-04 | 2016-09-14 | 埃森哲环球服务有限公司 | Digital image processing using convolutional neural networks |
| CN107229942A (en) * | 2017-04-16 | 2017-10-03 | 北京工业大学 | A kind of convolutional neural networks rapid classification method based on multiple graders |
| CN107886133A (en) * | 2017-11-29 | 2018-04-06 | 南京市测绘勘察研究院股份有限公司 | A kind of underground piping defect inspection method based on deep learning |
| CN107991873A (en) * | 2017-12-11 | 2018-05-04 | 中国矿业大学 | A kind of metro stray current leakage class Forecasting Methodology based on convolutional neural networks and BP neural network |
| CN108038850A (en) * | 2017-12-08 | 2018-05-15 | 天津大学 | A kind of drainage pipeline Exception Type automatic testing method based on deep learning |
| CN108038466A (en) * | 2017-12-26 | 2018-05-15 | 河海大学 | Multichannel human eye closure recognition methods based on convolutional neural networks |
-
2018
- 2018-11-23 CN CN201811403708.6A patent/CN109559302A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105938559A (en) * | 2015-03-04 | 2016-09-14 | 埃森哲环球服务有限公司 | Digital image processing using convolutional neural networks |
| CN104966104A (en) * | 2015-06-30 | 2015-10-07 | 孙建德 | Three-dimensional convolutional neural network based video classifying method |
| CN107229942A (en) * | 2017-04-16 | 2017-10-03 | 北京工业大学 | A kind of convolutional neural networks rapid classification method based on multiple graders |
| CN107886133A (en) * | 2017-11-29 | 2018-04-06 | 南京市测绘勘察研究院股份有限公司 | A kind of underground piping defect inspection method based on deep learning |
| CN108038850A (en) * | 2017-12-08 | 2018-05-15 | 天津大学 | A kind of drainage pipeline Exception Type automatic testing method based on deep learning |
| CN107991873A (en) * | 2017-12-11 | 2018-05-04 | 中国矿业大学 | A kind of metro stray current leakage class Forecasting Methodology based on convolutional neural networks and BP neural network |
| CN108038466A (en) * | 2017-12-26 | 2018-05-15 | 河海大学 | Multichannel human eye closure recognition methods based on convolutional neural networks |
Non-Patent Citations (5)
| Title |
|---|
| JACK C.P.CHENG ET AL: "Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques", 《AUTOMATION IN CONSTRUCTION》 * |
| MINGZHU WANG ET AL: "Development and Improvement of Deep Learning Based Automated Defect Detection for Sewer Pipe Inspection Using Faster R-CNN", 《ADVANCED COMPUTING STRATEGIES FOR ENGINEERING》 * |
| SRINATH S. KUMAR ET AL: "Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks", 《AUTOMATION IN CONSTRUCTION》 * |
| 张永良 等: "基于改进卷积神经网络的周界入侵检测方法", 《计算机科学》 * |
| 楚敏南: "基于卷积神经网络的图像分类技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110662A (en) * | 2019-05-07 | 2019-08-09 | 济南大学 | Driver eye movement behavioral value method, system, medium and equipment under Driving Scene |
| CN110443182A (en) * | 2019-07-30 | 2019-11-12 | 深圳市博铭维智能科技有限公司 | A kind of urban discharging pipeline video abnormality detection method based on more case-based learnings |
| CN110443182B (en) * | 2019-07-30 | 2021-11-09 | 深圳市博铭维智能科技有限公司 | Urban drainage pipeline video anomaly detection method based on multi-instance learning |
| CN110599460A (en) * | 2019-08-14 | 2019-12-20 | 深圳市勘察研究院有限公司 | Underground pipe network detection and evaluation cloud system based on hybrid convolutional neural network |
| CN110728654A (en) * | 2019-09-06 | 2020-01-24 | 台州学院 | Automatic pipeline detection and classification method based on deep residual error neural network |
| CN110728654B (en) * | 2019-09-06 | 2023-01-10 | 台州学院 | Automatic pipeline detection and classification method based on deep residual error neural network |
| CN110841142B (en) * | 2019-10-22 | 2022-03-08 | 江苏爱朋医疗科技股份有限公司 | Method and system for predicting blockage of infusion pipeline |
| CN110841142A (en) * | 2019-10-22 | 2020-02-28 | 江苏爱朋医疗科技股份有限公司 | Method and system for predicting blockage of infusion pipeline |
| CN110838107A (en) * | 2019-10-31 | 2020-02-25 | 广东华中科技大学工业技术研究院 | Method and device for intelligently detecting defects of 3C transparent component by variable-angle optical video |
| CN110838107B (en) * | 2019-10-31 | 2023-02-17 | 广东华中科技大学工业技术研究院 | Method and device for intelligently detecting defects of 3C transparent component by variable-angle optical video |
| CN110930377B (en) * | 2019-11-18 | 2022-05-13 | 福州大学 | Drainage pipeline abnormal type automatic detection method based on multitask learning |
| CN110930377A (en) * | 2019-11-18 | 2020-03-27 | 福州大学 | Automatic detection method for drainage pipeline abnormal type based on multitask learning |
| CN111191593A (en) * | 2019-12-30 | 2020-05-22 | 成都云尚物联环境科技有限公司 | Image target detection method and device, storage medium and sewage pipeline detection device |
| CN111311572A (en) * | 2020-02-13 | 2020-06-19 | 桂林理工大学 | Pipeline detection method and device, storage medium and robot |
| CN111402203A (en) * | 2020-02-24 | 2020-07-10 | 杭州电子科技大学 | Fabric surface defect detection method based on convolutional neural network |
| CN111402203B (en) * | 2020-02-24 | 2024-03-01 | 杭州电子科技大学 | Fabric surface defect detection method based on convolutional neural network |
| CN111507960A (en) * | 2020-04-16 | 2020-08-07 | 重庆三峡学院 | Camera module lens contamination detection method based on neural network visualization |
| CN111443095A (en) * | 2020-05-09 | 2020-07-24 | 苏州市平海排水服务有限公司 | Pipeline defect identification and judgment method |
| CN111784645B (en) * | 2020-06-15 | 2023-08-22 | 北京科技大学 | Filling pipeline crack detection method |
| CN111784645A (en) * | 2020-06-15 | 2020-10-16 | 北京科技大学 | Crack detection method for filling pipeline |
| CN111986188A (en) * | 2020-08-27 | 2020-11-24 | 深圳市智源空间创新科技有限公司 | Capsule robot drainage pipe network defect identification method based on Resnet and LSTM |
| US20220167066A1 (en) * | 2020-11-26 | 2022-05-26 | Communication University of Zhejiang | Short video copyright storage method based on blockchain and expression identification |
| CN112712497A (en) * | 2020-12-21 | 2021-04-27 | 南京惊蛰信息技术有限公司 | Stability detection method for cast iron pipeline joint based on local descriptor |
| CN112712497B (en) * | 2020-12-21 | 2024-05-28 | 江门市江海区金垠塱钢管有限公司 | Cast iron pipeline joint stability detection method based on local descriptor |
| CN113221710A (en) * | 2021-04-30 | 2021-08-06 | 深圳市水务工程检测有限公司 | Neural network-based drainage pipeline defect identification method, device, equipment and medium |
| CN113761785A (en) * | 2021-06-11 | 2021-12-07 | 神华国能宁夏煤电有限公司 | Method and device for determining thinning value of boiler and storage medium |
| CN113469252A (en) * | 2021-07-02 | 2021-10-01 | 河海大学 | Extra-high voltage converter valve operation state evaluation method considering unbalanced samples |
| CN113421252A (en) * | 2021-07-07 | 2021-09-21 | 南京思飞捷软件科技有限公司 | Actual detection method for vehicle body welding defects based on improved convolutional neural network |
| CN113421252B (en) * | 2021-07-07 | 2024-04-19 | 南京思飞捷软件科技有限公司 | Improved convolutional neural network-based vehicle body welding defect detection method |
| CN113592002A (en) * | 2021-08-04 | 2021-11-02 | 江苏网进科技股份有限公司 | Real-time garbage monitoring method and system |
| CN113610831A (en) * | 2021-08-19 | 2021-11-05 | 江西应用技术职业学院 | Wood defect detection method based on computer image technology and transfer learning |
| CN116596885A (en) * | 2023-05-19 | 2023-08-15 | 长江生态环保集团有限公司 | Real-time detection method and system for drainage pipe network state |
| CN116805416A (en) * | 2023-08-21 | 2023-09-26 | 中国电建集团华东勘测设计研究院有限公司 | Drainage pipeline defect identification model training method and drainage pipeline defect identification method |
| CN117576105A (en) * | 2024-01-17 | 2024-02-20 | 高科建材（咸阳）管道科技有限公司 | Pipeline production control method and system based on artificial intelligence |
| CN117576105B (en) * | 2024-01-17 | 2024-03-29 | 高科建材（咸阳）管道科技有限公司 | Pipeline production control method and system based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109559302A (en) | Pipe video defect inspection method based on convolutional neural networks | |
| CN113065558B (en) | Lightweight small target detection method combined with attention mechanism | |
| Jia et al. | Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot | |
| CN109147254B (en) | Video field fire smoke real-time detection method based on convolutional neural network | |
| CN106650913B (en) | A kind of vehicle density method of estimation based on depth convolutional neural networks | |
| CN106778595B (en) | Method for detecting abnormal behaviors in crowd based on Gaussian mixture model | |
| CN111680706B (en) | Dual-channel output contour detection method based on coding and decoding structure | |
| CN110263705A (en) | Towards two phase of remote sensing technology field high-resolution remote sensing image change detecting method | |
| CN107229929A (en) | A kind of license plate locating method based on R CNN | |
| CN107316295A (en) | A kind of fabric defects detection method based on deep neural network | |
| CN110135296A (en) | Airfield runway FOD detection method based on convolutional neural networks | |
| CN109376747A (en) | A kind of video flame detecting method based on double-current convolutional neural networks | |
| CN106991666B (en) | A kind of disease geo-radar image recognition methods suitable for more size pictorial informations | |
| CN109558815A (en) | A kind of detection of real time multi-human face and tracking | |
| CN110378232B (en) | Improved test room examinee position rapid detection method of SSD dual-network | |
| CN112906485A (en) | Visual impairment person auxiliary obstacle perception method based on improved YOLO model | |
| CN113536972B (en) | Self-supervision cross-domain crowd counting method based on target domain pseudo label | |
| CN108564077A (en) | It is a kind of based on deep learning to detection and recognition methods digital in video or picture | |
| CN109635634A (en) | A kind of pedestrian based on stochastic linear interpolation identifies data enhancement methods again | |
| CN112287827A (en) | Complex environment pedestrian mask wearing detection method and system based on intelligent lamp pole | |
| CN112668375B (en) | Tourist distribution analysis system and method in scenic spot | |
| CN107038416A (en) | A kind of pedestrian detection method based on bianry image modified HOG features | |
| CN109241902A (en) | A kind of landslide detection method based on multi-scale feature fusion | |
| CN110390673A (en) | Cigarette automatic testing method based on deep learning under a kind of monitoring scene | |
| CN115131640A (en) | Target detection method and system utilizing illumination guide and attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190402 |