CN118072193A - Dam crack detection method based on unmanned aerial vehicle image and deep learning - Google Patents
Dam crack detection method based on unmanned aerial vehicle image and deep learning Download PDFInfo
- Publication number
- CN118072193A CN118072193A CN202311538487.4A CN202311538487A CN118072193A CN 118072193 A CN118072193 A CN 118072193A CN 202311538487 A CN202311538487 A CN 202311538487A CN 118072193 A CN118072193 A CN 118072193A
- Authority
- CN
- China
- Prior art keywords
- image
- crack
- dam
- unmanned aerial
- aerial vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 26
- 238000013135 deep learning Methods 0.000 title claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 44
- 230000011218 segmentation Effects 0.000 claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 23
- 238000004364 calculation method Methods 0.000 claims abstract description 16
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 238000011156 evaluation Methods 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 10
- 238000011176 pooling Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 5
- 238000005520 cutting process Methods 0.000 claims description 4
- 238000011282 treatment Methods 0.000 claims description 4
- 238000013434 data augmentation Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 201000010099 disease Diseases 0.000 abstract description 5
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract description 5
- 238000000605 extraction Methods 0.000 abstract description 3
- 238000013528 artificial neural network Methods 0.000 abstract description 2
- 238000012360 testing method Methods 0.000 description 7
- 238000010200 validation analysis Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 241000592183 Eidolon Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a dam crack detection method based on unmanned aerial vehicle image and deep learning, which comprises the following steps: acquiring a dam crack image by using an unmanned aerial vehicle; preprocessing a data set, and constructing and training a dam crack detection network model; evaluating generalization capability of the model; denoising the segmented image; the method and the device have the advantages that the unmanned aerial vehicle is utilized to collect the crack images of the dam surface, the crack images are segmented based on the neural network, and the geometric parameter calculation is carried out on the segmented cracks, so that the dam cracks are rapidly and automatically detected, the crack detection network model is used as a network encoder to improve the feature extraction capability, the network can fully obtain image information, the crack areas in the images are accurately segmented, the method and the device have the characteristics of high segmentation precision and accurate segmentation images, and the method and the device can be used for automatically segmenting the dam crack images and providing a judgment basis for subsequent dam crack disease evaluation.
Description
Technical Field
The invention belongs to the field of health monitoring and damage identification of dam structures, and particularly relates to a dam crack detection method based on unmanned aerial vehicle images and deep learning.
Background
The dam is an important water retaining building in water conservancy infrastructure, and has various functions such as flood control and shipping. During the service period of the dam, the dam bears various loads and sudden disasters and also bears corrosion and erosion from the environment, so that the local and overall safety performance of the dam body can be reduced with the passage of time, and further damage and other disease phenomena occur. The light disease reduces the service life of the dam, affects the operation efficiency, and causes disaster accidents such as dam break, dyke break and the like, thereby threatening the life and property safety of the masses. Therefore, the efficient, nondestructive and accurate detection of the dam diseases and the timely adoption of the repairing measures are urgent demands for guaranteeing the safety of the dam. Because the crack is a common risk source of the dam, it is important to detect the development degree of the crack in time. According to the form and geometric characteristics of the crack, the potential cause of the crack can be deduced, and reasonable guidance is provided for the diagnosis of the dam structure health
The traditional crack detection method is mainly manual detection. The manual detection is low in efficiency and high in safety risk, the monitoring range is often limited by environmental conditions, and comprehensive and rapid detection on the dykes and dams cannot be carried out. With the development of machine vision, the image detection and identification algorithm gradually becomes a preferred method for detecting cracks and automatically identifying the breakthrough direction of dam cracks due to the high-efficiency identification efficiency and the characteristics of nondestructive detection. Image acquisition is the basis of image recognition, and the flexibility of unmanned aerial vehicle operation and expansibility of carrying equipment provide a feasible means for acquiring photos of various parts of a dyke. By carrying various types of sensors, the unmanned aerial vehicle can acquire high-precision data of a dam part which is difficult to reach by a worker, and the data is returned to a ground workbench, so that crack identification is realized rapidly and accurately by combining an image detection and identification algorithm.
In the process of acquiring the crack image by using the unmanned aerial vehicle, the image is often subjected to the influence of objective factors such as shaking, light ray transformation, flying speed and the like, so that the problem of large noise is frequently caused, and meanwhile, the problem of unbalanced pixel point types of a target area and a background area exists in the crack image, so that the difficulty of identifying the crack is improved. Currently, students apply deep learning to crack segmentation, however, the existing crack detection based on deep learning has many objects such as pavement, bridge, building and the like, and has algorithm optimization problems in terms of segmentation precision aiming at the research of dam cracks, such as a dam crack segmentation method and device (CN 116402749A) based on a coding and decoding network, and comprises the steps of collecting target image data of a target dam; performing computer vision processing on the crack-free image data to generate a first dam crack segmentation data set; training a dam crack segmentation model through a coding and decoding network based on the first dam crack segmentation dataset and the crack-free image data; and performing dam crack segmentation on the dam to be identified, which is subjected to crack segmentation, through the trained dam crack segmentation model. The device realizes the timely, comprehensive and accurate automatic detection of the micro deformation and the crack of the dam, is beneficial to the later-stage timely maintenance and reduces the loss. But the crack segmentation accuracy is still not ideal. And the subsequent treatments such as extraction of the geometric information of the cracks are lacked, and a complete intelligent recognition method of the cracks is not formed. Therefore, research on intelligent detection methods of dam cracks based on unmanned aerial vehicles and deep learning is urgently needed.
Disclosure of Invention
The invention aims to provide a dam crack detection method based on unmanned aerial vehicle images and deep learning. The method can realize high-precision segmentation of the cracks, calculate geometric parameters of the cracks, and detect the cracks automatically, with high precision and high timeliness.
The invention is realized at least by one of the following technical schemes.
A dam crack detection method based on unmanned aerial vehicle image and deep learning comprises the following steps:
acquiring dam crack images by using an unmanned aerial vehicle, and establishing a data set;
preprocessing a data set;
constructing and training a dam crack detection network model for detecting dam cracks and evaluating the generalization capability of the model;
Denoising the segmented image based on connected domain analysis;
the geometric parameters of the cracks are calculated based on the denoising image and the image processing technology so as to detect dam cracks.
Further, acquiring a dam crack image with the unmanned aerial vehicle, the establishing a data set includes:
1) Designing an observation route, wherein the observation route comprises a flight range, a flight frame number and a sunny and backsunny area division of a dyke;
2) Setting flight parameters including shooting angle, heading overlapping degree and side overlapping degree and flight mode;
3) Image data is acquired.
Further, the preprocessing of the data set comprises unmanned aerial vehicle image preprocessing, cutting and screening, mask making and data augmentation.
Further, the unmanned aerial vehicle image preprocessing comprises image graying and image enhancement, wherein the image graying is realized by adopting a weighted average method, different weights are given to three channel components according to the characteristics of an image and the difference of the influence degree of the three channel components on the image, weighted average calculation is carried out according to corresponding weight values, and a calculation result is an image gray value, wherein the following formula is shown:
Gray(x,y)＝0.299R(x,y)+0.587G(x,y)+0.114B(x,y)
Wherein Gray (x, y) represents a Gray value of the image after graying, R (x, y) represents a luminance value of a pixel having coordinates (x, y) on a red channel, G (x, y) represents a luminance value of a pixel having coordinates (x, y) on a green channel, and B (x, y) represents a luminance value of a pixel having coordinates (x, y) on a blue channel;
The image enhancement adopts a linear gray level conversion method, the spatial relationship in the image is not changed, and gray level values of all pixels in the original image are obtained through the same conversion, so that the image is clear and the characteristics are obvious:
Wherein, the gray value after the image graying treatment is f (x, y), and the gray value range is [ b, c ]; the gray value after the image linear transformation is g (x, y), and the gray value range is converted into [ d, e ].
Further, the VGG19-Unet network comprises an encoder and a decoder; the VGG19-Unet network follows an encoding-decoding structure, the whole network is U-shaped, and an encoder compresses the image size and extracts target characteristics; the decoder restores the image size and restores the target information, and the encoder and the decoder are respectively distributed on the left side and the right side of the U-shape; in addition, four jump connections are arranged to fuse the shallow and deep features, so that the segmentation precision is improved;
The convolution layer in VGG19-Unet comprises a convolution calculation, a batch normalization and a nonlinear conversion; the nonlinear transformation uses a leak ReLU function.
Further, the encoder is a VGG19 network with a full connection layer removed, and specifically comprises 16 convolution layers and 4 largest pooling layers, wherein the convolution layers adopt 3×3 convolution kernels, the step length is 1, the filling is same, the largest pooling layers adopt 2×2 pooling kernels, and the step length is 2;
The decoder is a U-Net decoder and comprises 4 decoding convolution blocks and an output layer, wherein each decoding convolution block comprises a deconvolution layer with the size of 2 multiplied by 2, 2 convolution layers with the size of 3 multiplied by 3 and the step length of 1, the convolution kernel of the output layer is 1 multiplied by 1, the step length is 1, and the same is filled.
Further, in the dike crack detection network model of VGG19-Unet network, the specific process of constructing VGG19-Unet network jump connection is as follows: after the convolutional layer 1_2, the encoder profile is connected with the decoding profile of the upsampling module 4; after the convolutional layer 2_2, the encoder profile is connected with the decoding profile of the upsampling module 3; after the convolutional layer 3_4, the encoder profile is connected with the decoding profile of the upsampling module 2; after the convolutional layer 4_4, the encoder profile is connected with the decoding profile of the upsampling module 1. By jump connection, the crack texture features of the encoder and the crack semantic features of the decoder are fused, so that high-precision segmentation is realized.
Further, training the VGG19-Unet network includes the steps of:
1) Determining a loss function, determining a loss function L dice as follows:
Where p i represents a binary image of the prediction result, y i represents a binary image of the real label, |p i | represents the number of pixels in p i, |y i | represents the number of pixels in y i, and y i∩pi represents the number of pixels in the overlapping region of the two;
2) Determining VGG19-Unet network precision evaluation index
The precision P, the recall R, F score and IoU are selected as network precision evaluation indexes, and various indexes are determined according to the following formula:
wherein, the pixel with the label as the crack is positive example P, the pixel with the label as the background is negative example N, TP indicates that the label of the pixel is the crack, the prediction result is also the crack, TN indicates that the label of the pixel is the background, and the prediction result is also the background; FP indicates that the pixel label is background, but the predicted result is a crack; FN indicates that the pixel label is a crack, but the prediction is background;
3) Training VGG19-Unet networks
The training set is sent into a network for training, and is set to be 2 in batches; using AdamW optimizer, the weight decay was set to 0.005, the initial learning rate was set to 10 -5, and the performance of the model on the validation set was evaluated every time one epoch was trained. Iterating until the loss function converges and training loss and verification loss values are close;
5) Preserving an optimal model
In the process of training the VGG19-Unet network, continuously updating the weight by using a deep learning framework, and after the model converges, taking the model with the largest verification set IoU as the optimal model to be stored;
further, evaluating the generalization ability of the model includes the steps of: loading a weight file of the optimal model; inputting the test set image into an optimal model; and dividing the test set image, and outputting the division precision and the division image.
Further, denoising the segmented image based on connected domain analysis comprises the steps of:
2) Extracting the outline of the connected domain, and obtaining the characteristic parameters of each connected domain
And (3) performing eight-connected-domain labeling on the segmented image to extract the outline of the connected domain, and calculating the characteristic parameters of each connected domain according to the outline.
The selected characteristic parameters are an area A, a circularity E and a squareness J, and the characteristic parameters are determined according to the following formula:
A＝N×S
Wherein N represents the number of all pixel points in each connected domain; s represents the area of a single pixel; l represents the perimeter of the outline of the connected domain, namely the total number of pixel points of the outline; s 0 represents the area of the connected domain; s MER denotes the area of the smallest circumscribed rectangle of the connected domain.
2) Denoising according to threshold
Setting a threshold value for the characteristic parameter according to the characteristic parameter value of the connected domain so as to distinguish a crack region from a noise block; redrawing the contour remained after noise deletion onto a new image to obtain a denoising image:
Further, calculating the fracture geometry parameters based on the denoised image and image processing includes the steps of:
1) Calculation of crack Length
Extracting a crack skeleton in the denoising image by using a Zhang-Suen refinement algorithm; scanning the crack skeleton to obtain the number of pixels of the skeleton, wherein the crack length L c is determined by the following formula;
Lc＝Nl×r
wherein r represents the side length of the actual physical size of the pixels, and N l is the number of pixels of the skeleton;
2) Calculating crack width
And projecting the crack to an axis parallel to the crack, and taking the actual physical distance corresponding to the maximum projection value as the crack width. The pixel width W c is determined by the following formula;
Wc＝Nw×r
where r represents the side length of the actual physical size of the pixel and N w is the number of pixels of the maximum projection value.
Compared with the prior art, the technical scheme provided by the invention has the following technical effects:
According to the invention, the unmanned aerial vehicle is utilized to collect the crack image of the dam surface, the crack image is segmented based on the VGG19-Unet neural network, and geometric parameter calculation is carried out on the segmented cracks, so that the dam cracks are rapidly and automatically detected. The invention uses VGG19 network as network encoder to improve the feature extraction capability. Therefore, the network can fully acquire image information and accurately divide the crack area in the image. The method provided by the invention and the U-net are adopted to carry out a comparison experiment, and the comparison experiment shows that the method has the characteristics of high segmentation precision and accurate segmentation image, can be used for automatically segmenting dam crack images, and provides a judgment basis for subsequent dam crack disease evaluation.
Drawings
FIG. 1 is a flow chart of a dam crack detection method based on unmanned aerial vehicle images and deep learning according to an embodiment of the invention;
FIG. 2 is a diagram of a network architecture of VGG19-Unet, according to an embodiment of the present invention;
FIG. 3 is a graph showing two segmentation results of VGG19-Unet on a test set crack image according to an embodiment of the invention.
Detailed Description
The preferred embodiments of the present invention will be described with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
As shown in fig. 1, a dam crack detection method based on unmanned aerial vehicle image and deep learning of the present embodiment includes the following steps:
(1) Acquiring dam crack images by using unmanned aerial vehicle
As an example, this example uses a multi-spectral version of the grand eidolon 4, which is equipped with a 200-ten-thousand pixel visible camera, with a resolution of 1600 x 1300 pixels of RGB pictures taken. The camera focal length is 5.74mm.
1) And designing an observation route according to the flight environment, wherein the observation route comprises a flight range, a flight frame number and sunny area division of a dam.
2) And setting flight parameters including shooting angle, heading overlapping degree and side overlapping degree and flight mode. To avoid negative effects of image distortion on subsequent fracture geometry calculations, an orthographic image of the fracture should be acquired. So that the lens and the dam surface are kept parallel as much as possible during shooting; the overlapping degree is more than 60%; the flight mode is hover shooting.
3) Image data is acquired.
As an example, 130 total images were acquired, the image size being 1600×1300 pixels.
(2) Dataset preprocessing
1) Graying of images
As an example, the image is grayed using a weighted average method.
The method used for graying the image is a weighted average method. According to the characteristics of an image and the difference of the influence degree of three channel components on the image, the method endows the three channel components with different weights, then carries out weighted average calculation according to corresponding weight values, and the calculated result is an image gray value, wherein the calculated result is shown in the following formula:
Gray(x,y)＝0.299R(x,y)+0.587G(x,y)+0.114B(x,y)
Where Gray (x, y) represents the Gray value of the image after graying, R (x, y) represents the luminance value of the pixel with (x, y) on the red channel, G (x, y) represents the luminance value of the pixel with (x, y) on the green channel, and B (x, y) represents the luminance value of the pixel with (x, y) on the blue channel.
2) Image enhancement
The method used for image enhancement is linear gray scale transformation. The method does not change the spatial relationship in the image, and only enables the gray values of all pixels in the original image to obtain the gray values of corresponding output points through the same transformation, so that the image is clear and the characteristics are obvious. The method is shown as follows:
Wherein, the gray value after the image graying treatment is f (x, y), and the gray value range is [ b, c ]; the gray value after the image linear transformation is g (x, y), and the gray value range is converted into [ d, e ].
As an embodiment, the image enhancement is performed using a linear gray scale transformation.
3) Cutting and screening
As an embodiment, taking an image photographed by an unmanned aerial vehicle; from the upper left corner of the image, the 448 x 448 windows are sequentially slid in sequence from left to right and from top to bottom, the sliding step length is 400 pixels, and each sliding is cut once, so that a sub-image is obtained.
And screening out clear subgraphs. And cutting and screening to obtain 220 crack images.
4) Mask making
The crack region in the subgraph is marked using open source software labelme as a subgraph mask.
5) Data augmentation
And (3) turning, rotating, dithering and the like are performed on the subgraph and the corresponding mask to amplify the data, and the processed data are divided into a training set, a verification set and a test set according to the proportion of 8:1:1.
(2) Construction of VGG19-Unet network model
As shown in fig. 2, VGG19-Unet follows an encoding-decoding structure, and the network is U-shaped as a whole. The encoder compresses the image size and extracts the target feature; the decoder restores the image size and restores the target information. The encoder and decoder are distributed to the left and right of the U-shape, respectively. In addition, 4 jump connections are arranged to fuse the shallow and deep features, so that the segmentation precision is improved.
The network encoder is a VGG19 network with the full connection layer removed. This section consists of 16 convolutional layers and 4 max pooling layers. The convolution layer adopts a 3×3 convolution kernel, the step size is 1, and the same is filled. The maximum pooling layer adopts 2×2 pooling cores, and the step size is 2.
The network decoder is a U-Net decoder. This section consists of 4 decoded convolutional blocks and an output layer. Each decoded convolutional block comprises a deconvolution layer of size 2 x 2, 2 convolutional layers of size 3 x 3 step size 1. The convolution kernel size of the output layer is 1×1, the step size is 1, and the filling mode is same.
The specific process of constructing the VGG19-Unet network model hopping connection is as follows: after the convolutional layer 1_2, the encoder profile is connected with the decoding profile of the upsampling module 4; after the convolutional layer 2_2, the encoder profile is connected with the decoding profile of the upsampling module 3; after the convolutional layer 3_4, the encoder profile is connected with the decoding profile of the upsampling module 2; after the convolutional layer 4_4, the encoder profile is connected with the decoding profile of the upsampling module 1. By jump connection, the crack texture features of the encoder and the crack semantic features of the decoder are fused, so that high-precision segmentation is realized.
The convolution layer in VGG19-Unet comprises a convolution calculation, a batch normalization and a nonlinear conversion; the nonlinear transformation uses a leak ReLU function.
(3) Training VGG19-Unet networks
1) Determining a loss function
The loss function L dice is determined as follows:
Where p i represents a binary image of the prediction result, and y i represents a binary image of the real tag. P i represents the number of pixels in p i, y i represents the number of pixels in y i, and y i∩pi represents the number of pixels in the overlapping region of the two.
2) Determining model precision evaluation index
The precision P, the recall R, F score and IoU are selected as model precision evaluation indexes, and various indexes are determined according to the following formula:
Wherein, the pixel with the label as the crack is the positive example (P), and the pixel with the label as the background is the negative example (N). TP indicates that the label of the pixel is a crack and the prediction result is also a crack. TN indicates that the pixel label is background and the prediction result is background; FP indicates that the pixel label is background, but the predicted result is a crack; FN indicates that the pixel label is a crack, but the prediction is background.
3) Training VGG19-Unet networks
The training set is input into a network for training, wherein the tensor size of the input image is 448 x 1. Batch set to 2; using AdamW optimizer, the weight decay was set to 0.005, the initial learning rate was set to 10 -5, and the performance of the model on the validation set was evaluated every time one epoch was trained. Iteration is performed until the loss function converges and the training loss and validation loss values are close.
4) Preserving an optimal model
The weights are updated continuously with the deep learning framework during training of the VGG19-Unet network. After the model converges, the model with the largest validation set IoU is saved as the best model.
(4) Evaluating generalization ability of model
The test set is input into the best model for testing. And loading a weight file of the optimal model, dividing the test set image, and outputting the division precision and the division image, wherein the tensor size of the output image is 448 multiplied by 1.
And (3) carrying out a comparison experiment with U-net, wherein the segmentation precision of different algorithms is as follows:
as can be seen from the above table, the present example improves the segmentation accuracy by improving the encoder, which improves the ability of the model to extract features.
(5) Segmented image denoising based on connected domain analysis
The present embodiment uses a fixed focus lens for photographing, and the zoom ratio of the camera in image photographing is 1.47 mm/pixel.
1) Extracting the outline of the connected domain, and obtaining the characteristic parameters of each connected domain
And (3) performing eight-connected-domain labeling on the segmented image to extract the outline of the connected domain, and calculating the characteristic parameters of each connected domain according to the outline.
The selected characteristic parameters are an area A, a circularity E and a squareness J, and the characteristic parameters are determined according to the following formula:
A＝N×S
Wherein N represents the number of all pixel points in each connected domain; s represents the area of a single pixel; l represents the perimeter of the outline of the connected domain, namely the total number of pixel points of the outline; s 0 represents the area of the connected domain; s MER denotes the area of the smallest circumscribed rectangle of the connected domain.
2) Denoising according to threshold
Setting a threshold value for the characteristic parameter according to the characteristic parameter value of the connected domain so as to distinguish a crack region from a noise block; and redrawing the contour remained after the noise is deleted on a new image to obtain a denoising image. As one example, a connected domain having an area S <100, a circularity R >0.3, and a squareness P >0.4 is determined as noise.
(6) Calculating crack geometric parameters based on denoising image and image processing technology
1) Calculation of crack Length
Extracting a crack skeleton in the denoising image by using a Zhang-Suen refinement algorithm; the fracture skeleton is scanned to obtain the number of pixels of the skeleton. The fracture length L c is determined by the following formula;
Lc＝Nl×r
where r represents the side length of the actual physical size of the pixel, and N l is the number of pixels of the skeleton.
2) Calculating crack width
And projecting the crack to an axis parallel to the crack, and taking the actual physical distance corresponding to the maximum projection value as the crack width. The pixel width W c is determined by the following formula;
Wc＝Nw×r
where r represents the side length of the actual physical size of the pixel and N w is the number of pixels of the maximum projection value.
Two examples of cracks are given in fig. 3 (a), (b). Calculating the length and width of the crack in the divided image, comparing it with the corresponding length and width of the crack in the mask, and performing error analysis
As can be seen from the above table, the calculation error of the crack length is 10% or less, and the calculation error of the crack width is 30% or less. The effectiveness of the fracture geometry parameter calculation method and the accuracy of the segmentation model are proved.
Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The dam crack detection method based on unmanned aerial vehicle image and deep learning is characterized by comprising the following steps of:
acquiring dam crack images by using an unmanned aerial vehicle, and establishing a data set;
preprocessing a data set;
constructing and training a dam crack detection network model for detecting dam cracks and evaluating the generalization capability of the model;
Denoising the segmented image based on connected domain analysis;
the geometric parameters of the cracks are calculated based on the denoising image and the image processing technology so as to detect dam cracks.
2. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 1, wherein the method comprises the following steps: acquiring a dam crack image using an unmanned aerial vehicle, the establishing a data set comprising:
1) Designing an observation route, wherein the observation route comprises a flight range, a flight frame number and a sunny and backsunny area division of a dyke;
2) Setting flight parameters including shooting angle, heading overlapping degree and side overlapping degree and flight mode;
3) Image data is acquired.
3. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 1, wherein the method comprises the following steps: the preprocessing of the data set comprises unmanned aerial vehicle image preprocessing, cutting and screening, mask making and data augmentation.
4. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 3, wherein the method comprises the following steps: the unmanned aerial vehicle image preprocessing comprises image graying and image enhancement, wherein the image graying is realized by adopting a weighted average method, different weights are given to three channel components according to the characteristics of the image and the difference of the influence degree of the three channel components on the image, weighted average calculation is carried out according to corresponding weight values, and the calculated result is the image gray value, wherein the calculated result is shown in the following formula:
Gray(x,y)＝0.299R(x,y)+0.587G(x,y)+0.114B(x,y)
Wherein Gray (x, y) represents a Gray value of the image after graying, R (x, y) represents a luminance value of a pixel having coordinates (x, y) on a red channel, G (x, y) represents a luminance value of a pixel having coordinates (x, y) on a green channel, and B (x, y) represents a luminance value of a pixel having coordinates (x, y) on a blue channel;
The image enhancement adopts a linear gray level conversion method, the spatial relationship in the image is not changed, and gray level values of all pixels in the original image are obtained through the same conversion, so that the image is clear and the characteristics are obvious:
Wherein, the gray value after the image graying treatment is f (x, y), and the gray value range is [ b, c ]; the gray value after the image linear transformation is g (x, y), and the gray value range is converted into [ d, e ].
5. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 1, wherein the method comprises the following steps: the VGG19-Unet network comprises an encoder and a decoder; the VGG19-Unet network follows an encoding-decoding structure, the whole network is U-shaped, and an encoder compresses the image size and extracts target characteristics; the decoder restores the image size and restores the target information, and the encoder and the decoder are respectively distributed on the left side and the right side of the U-shape; in addition, four jump connections are arranged to fuse the shallow and deep features, so that the segmentation precision is improved;
The convolution layer in VGG19-Unet comprises a convolution calculation, a batch normalization and a nonlinear conversion; the nonlinear transformation uses a leak ReLU function.
6. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 4, wherein the method comprises the following steps: the encoder is a VGG19 network with a full connection layer removed, and specifically comprises 16 convolution layers and 4 largest pooling layers, wherein the convolution layers adopt 3×3 convolution kernels, the step length is 1, the filling is same, the largest pooling layers adopt 2×2 pooling kernels, and the step length is 2;
The decoder is a U-Net decoder and comprises 4 decoding convolution blocks and an output layer, wherein each decoding convolution block comprises a deconvolution layer with the size of 2 multiplied by 2, 2 convolution layers with the size of 3 multiplied by 3 and the step length of 1, the convolution kernel of the output layer is 1 multiplied by 1, the step length is 1, and the same is filled.
7. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 4, wherein the method comprises the following steps: in the dike crack detection network model of VGG19-Unet network, the specific process for constructing VGG19-Unet network jump connection is as follows: after the convolutional layer 1_2, the encoder profile is connected with the decoding profile of the upsampling module 4; after the convolutional layer 2_2, the encoder profile is connected with the decoding profile of the upsampling module 3; after the convolutional layer 3_4, the encoder profile is connected with the decoding profile of the upsampling module 2; after the convolution layer 4_4, connecting the encoder profile with the decoding profile of the upsampling module 1; by jump connection, the crack texture features of the encoder and the crack semantic features of the decoder are fused, so that high-precision segmentation is realized.
8. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 1, wherein the method comprises the following steps: training the VGG19-Unet network comprises the steps of:
1) Determining a loss function, determining a loss function L dice as follows:
Where p i represents a binary image of the prediction result, y i represents a binary image of the real label, |p i | represents the number of pixels in p i, |y i | represents the number of pixels in y i, and y i∩pi represents the number of pixels in the overlapping region of the two;
2) Determining VGG19-Unet network precision evaluation index
The precision P, the recall R, F score and IoU are selected as network precision evaluation indexes, and various indexes are determined according to the following formula:
wherein, the pixel with the label as the crack is positive example P, the pixel with the label as the background is negative example N, TP indicates that the label of the pixel is the crack, the prediction result is also the crack, TN indicates that the label of the pixel is the background, and the prediction result is also the background; FP indicates that the pixel label is background, but the predicted result is a crack; FN indicates that the pixel label is a crack, but the prediction is background;
3) Training VGG19-Unet networks
The training set is sent into a network for training, and is set to be 2 in batches; adopting AdamW optimizers, setting weight attenuation to be 0.005, setting initial learning rate to be 10 -5, and evaluating the performance of the model on a verification set every time one epoch is trained; iterating until the loss function converges and training loss and verification loss values are close;
4) Preserving an optimal model
In the process of training the VGG19-Unet network, the weight is continuously updated by using a deep learning framework, and after the model converges, the model with the largest verification set IoU is stored as the optimal model.
9. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 1, wherein the method comprises the following steps: the denoising of the segmented image based on connected domain analysis comprises the following steps:
1) Extracting the outline of the connected domain, and obtaining the characteristic parameters of each connected domain
Performing eight-connected-domain labeling on the segmented image to extract the connected-domain outline, and calculating characteristic parameters of each connected-domain according to the eight-connected-domain outline;
The selected characteristic parameters are an area A, a circularity E and a squareness J, and the characteristic parameters are determined according to the following formula:
A＝N×S
Wherein N represents the number of all pixel points in each connected domain; s represents the area of a single pixel; l represents the perimeter of the outline of the connected domain, namely the total number of pixel points of the outline; s 0 represents the area of the connected domain; s MER represents the area of the smallest circumscribed rectangle of the connected domain;
2) Denoising according to threshold
Setting a threshold value for the characteristic parameter according to the characteristic parameter value of the connected domain so as to distinguish a crack region from a noise block; and redrawing the contour remained after the noise is deleted on a new image to obtain a denoising image.
10. The method for detecting dam cracks based on unmanned aerial vehicle image and deep learning according to claim 1, wherein the method comprises the following steps: calculating the fracture geometry parameters based on the denoised image and image processing comprises the steps of:
1) Calculation of crack Length
Extracting a crack skeleton in the denoising image by using a Zhang-Suen refinement algorithm; scanning the crack skeleton to obtain the number of pixels of the skeleton, wherein the crack length L c is determined by the following formula;
Lc＝Nl×r
wherein r represents the side length of the actual physical size of the pixels, and N l is the number of pixels of the skeleton;
2) Calculating crack width
Projecting the crack to an axis parallel to the crack, taking the actual physical distance corresponding to the maximum projection value as the crack width, and determining the pixel width W c by the following formula;
Wc＝Nw×r
where r represents the side length of the actual physical size of the pixel and N w is the number of pixels of the maximum projection value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311538487.4A CN118072193A (en) | 2023-11-17 | 2023-11-17 | Dam crack detection method based on unmanned aerial vehicle image and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311538487.4A CN118072193A (en) | 2023-11-17 | 2023-11-17 | Dam crack detection method based on unmanned aerial vehicle image and deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118072193A true CN118072193A (en) | 2024-05-24 |
Family
ID=91110386
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311538487.4A Pending CN118072193A (en) | 2023-11-17 | 2023-11-17 | Dam crack detection method based on unmanned aerial vehicle image and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118072193A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118298338A (en) * | 2024-06-05 | 2024-07-05 | 安徽省交通规划设计研究总院股份有限公司 | Road crack rapid identification and calculation method based on unmanned aerial vehicle low-altitude photography |
| CN118298338B (en) * | 2024-06-05 | 2024-09-27 | 安徽省交通规划设计研究总院股份有限公司 | Road crack rapid identification and calculation method based on unmanned aerial vehicle low-altitude photography |
-
2023
- 2023-11-17 CN CN202311538487.4A patent/CN118072193A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118298338A (en) * | 2024-06-05 | 2024-07-05 | 安徽省交通规划设计研究总院股份有限公司 | Road crack rapid identification and calculation method based on unmanned aerial vehicle low-altitude photography |
| CN118298338B (en) * | 2024-06-05 | 2024-09-27 | 安徽省交通规划设计研究总院股份有限公司 | Road crack rapid identification and calculation method based on unmanned aerial vehicle low-altitude photography |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110059694B (en) | Intelligent identification method for character data in complex scene of power industry | |
| US20210319561A1 (en) | Image segmentation method and system for pavement disease based on deep learning | |
| Li et al. | Automatic bridge crack identification from concrete surface using ResNeXt with postprocessing | |
| CN110070008A (en) | Bridge disease identification method adopting unmanned aerial vehicle image | |
| CN113780296A (en) | Remote sensing image semantic segmentation method and system based on multi-scale information fusion | |
| CN112330593A (en) | Building surface crack detection method based on deep learning network | |
| CN116485709A (en) | Bridge concrete crack detection method based on YOLOv5 improved algorithm | |
| CN111797920B (en) | Remote sensing extraction method and system for depth network impervious surface with gate control feature fusion | |
| CN117291913B (en) | Apparent crack measuring method for hydraulic concrete structure | |
| CN111161224A (en) | Casting internal defect grading evaluation system and method based on deep learning | |
| CN114359542A (en) | Concrete structure crack identification method based on computer vision and deep learning | |
| CN114049356A (en) | Method, device and system for detecting structure apparent crack | |
| CN115797314B (en) | Method, system, equipment and storage medium for detecting surface defects of parts | |
| CN116524189A (en) | High-resolution remote sensing image semantic segmentation method based on coding and decoding indexing edge characterization | |
| CN111223087A (en) | Automatic bridge crack detection method based on generation countermeasure network | |
| CN116091426A (en) | Pavement crack detection method based on coder-decoder | |
| CN116597270A (en) | Road damage target detection method based on attention mechanism integrated learning network | |
| CN113516652A (en) | Battery surface defect and adhesive detection method, device, medium and electronic equipment | |
| CN113313678A (en) | Automatic sperm morphology analysis method based on multi-scale feature fusion | |
| CN117952898A (en) | Water delivery tunnel crack detection method based on UNet network | |
| CN112232102B (en) | Building target recognition method and system based on deep neural network and multi-task learning | |
| CN116863134A (en) | Method and system for detecting and dividing length and width of tunnel lining crack | |
| CN118072193A (en) | Dam crack detection method based on unmanned aerial vehicle image and deep learning | |
| CN115273009A (en) | Road crack detection method and system based on deep learning | |
| Jiang et al. | Construction Site Segmentation Using Drone-Based Ortho-Image and Convolutional Encoder-Decoder Network Model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |