CN110569730A - Road surface crack automatic identification method based on U-net neural network model - Google Patents
Road surface crack automatic identification method based on U-net neural network model Download PDFInfo
- Publication number
- CN110569730A CN110569730A CN201910723374.9A CN201910723374A CN110569730A CN 110569730 A CN110569730 A CN 110569730A CN 201910723374 A CN201910723374 A CN 201910723374A CN 110569730 A CN110569730 A CN 110569730A
- Authority
- CN
- China
- Prior art keywords
- model
- training
- image
- value
- crack
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a road surface crack automatic identification method based on a U-net neural network model, which is characterized in that a database is expanded by an elastic deformation technology on the basis of a road surface crack 2D laser image acquired by a vehicle-mounted laser road detection device; then adjusting the structure of the U-net model, and fine-tuning parameters to enable the model to realize accurate and automatic identification of the pavement crack; inputting the prepared data set into a network, and repeatedly training the capability of the model for automatically learning the crack pixel characteristics; and finally, training a relatively stable automatic identification model so as to improve the crack identification precision and speed. The method can realize rapid and efficient automatic identification of the pavement cracks, reduce the consumption of human resources in highway detection operation, avoid subjective errors of naked eye identification and improve the identification accuracy.
Description
Technical Field
The invention relates to the technical field of automatic road detection, in particular to a road surface crack automatic identification method based on a U-net neural network model.
Background
With the emergence of advanced road pavement disease detection equipment, a plurality of pavement crack automatic identification algorithms emerge. The early crack identification means is mainly observed by manual naked eyes, and the method has the advantages of high working strength, long time consumption, low efficiency and subjectivity; based on the algorithm of threshold segmentation, segmenting the image into a foreground and a background by determining an optimal threshold, thereby extracting crack information; the edge detection algorithm is also an algorithm which is widely researched in recent years, and the contour features of the target are extracted by calculating the image gradient, wherein the Prewitt algorithm and the Canny algorithm are typically researched. The traditional algorithm provides a thought for the research of the crack identification field, but is easily interfered by the road illumination and noise. With the wide application of neural networks in recent years, a plurality of road surface crack identification algorithms based on the neural networks are proposed.
The automatic pavement crack identification algorithm mainly comprises three algorithms: the first is an asphalt pavement recognition algorithm based on BP neural network, firstly, the asphalt pavement image is subjected to filtering enhancement and is divided into a plurality of 32-pixel multiplied by 32-pixel sub-regions, then image parameters in the region and neighborhood sub-block prediction results are extracted for neural network training, and finally, pixel lattices with crack information are extracted. The second is ADA3D algorithm, which extracts crack features by performing convolution operation on images and finally outputs crack information through a full-connection layer, but because the full-connection layer is used, model parameters are increased, so that the recognition speed is low, the efficiency is low, and the requirement on a computer is extremely high. The third method is a road surface crack automatic identification algorithm based on a full Convolutional neural network (FCN). Firstly, performing convolution and pooling on an image; then carrying out up-sampling; and finally, outputting a result through feature fusion, wherein the whole network does not use a full connection layer. The method has the defects of complicated characteristic fusion process, missing detection of cracks and more false detection quantity by only adopting a simple weighted summation mode. The methods generally have the defects of not strong generalization ability and not accurate crack target extraction under the complex pavement.
In summary, the existing automatic pavement crack identification method has certain limitations. The main problems are that the device is easily influenced by road illumination and noise, the processing speed is low, and the generalization capability is not strong enough.
disclosure of Invention
In view of this, the invention aims to provide a road surface crack automatic identification method based on a U-net neural network model, which effectively improves the accuracy and recall rate of road surface crack identification and reduces the number of crack omission detection and false detection.
The invention is realized by adopting the following scheme: a road surface crack automatic identification method based on a U-net neural network model comprises the following steps:
step S1: collecting two-dimensional laser image data of a road surface, namely a two-dimensional laser original image;
step S2: calibrating and enhancing the image with cracks after the pavement two-dimensional laser image is manually screened so as to construct a model training library;
Step S3: adjusting a Unet model and parameters and training the adjusted Unet model;
step S4: providing an image to be detected and identified, taking the weight of the Unet model trained in the step S3 as a feature extractor, and traversing each pixel on the image to be detected and identified by taking 1 as a step length; limiting the output confidence range of each pixel to be between 0 and 1 in combination with a Sigmoid activation function;
Step S5: performing confidence judgment on the output result, and making the value of the output result 1 and classifying the output result as crack information when the confidence is more than 0.5; if the value is less than 0.5, the value is 0 and classified as background or noise;
Step S6: filtering the noise in the step S5, after the step S5, the confidence of each point is not 1, that is, 0, and then multiplying the confidence by 255 as the pixel value of each point; wherein 0 represents background information, and 255 represents a crack region; and finally, outputting a crack identification result, namely a gray image, namely a black-and-white image, only retaining crack information and background information.
further, the step S2 specifically includes the following steps:
step S21: data calibration: data calibration is carried out in a point marking mode;
Step S22: making a label: after the data calibration is completed, preprocessing is carried out on the calibrated image, namely RGB (0,0,0) is used for representing the crack contour, and RGB (255 ) represents the background and is used as a label;
Step S23: data enhancement: and (3) enhancing the preprocessed label image and the corresponding two-dimensional laser original image by using an elastic deformation method, namely, sequentially rotating, mirroring and cutting to expand the database.
further, the step S3 specifically includes the following steps:
Step S31: selection of the number of network layers of the Unet model: training the Unet model, wherein when the number of network layers is 32, the Loss value (Loss) of the training is 0.04, and when the number of network layers is continuously increased, the Loss value is not changed any more, but the training time is longer, so that the number of network layers of the Unet model is determined to be 32;
Step S32: in the convolution operation process, an all-zero filling mode is adopted to avoid losing boundary information, and after four times of pooling layers with the step length of 2, the image size is reduced to 1/16 of the original size; after four upsampling operations with the step size of 2, the image is restored to the original size, so that the original size of the image is not changed.
step S33: avoiding overfitting of the model: a Dropout layer is added at the end of the fourth convolution layer and the end of the fifth convolution layer; at this time, the model framework of the Unet is determined as follows: twenty-three convolutional layers, four pooling layers, two Dropout layers, four upper sampling layers and four feature fusion operations;
Step S34: adjusting parameters of the Unet model: adjusting the learning rate and the training iteration round number parameters of the Unet model;
step S35: training the Unet model determined in step S33: the Unet model obtains a predicted value through forward propagation, and calculates the error between the actual value and the predicted value through a cross entropy loss function, namely a loss value calculation formula is shown as a formula (1);
wherein p represents the true distribution q represents the predicted value, and H (p, q) represents the loss value;
then the loss value is reduced by the back propagation operation and the adjustment of the learning rate and the training round number, and the adjusted learning rate is 10-6after 150 rounds of training, the loss value is 0.04, the number of training rounds is continuously increased or the learning rate is adjusted, and the loss value is not changed any more; at this time, the weight file generated by the Unet model is the optimal feature extractor obtained by training.
Further, the formula for controlling the output result range to [0,1] in step S4 is shown as (2):
wherein f (z) represents a sigmoid activation function and z represents an input.
Further, the specific content of step S5 is: after the trained Unet model traverses each pixel point on the image, the confidence degree judgment is carried out, and the calculation formula is shown as (3):
Score is the confidence; 1 represents fracture information; 0 represents noise information; f (z) is the Sigmoid activation function calculation value.
Further, the specific content of step S6 is: and when the confidence coefficient is 1, classifying the image as a crack target, multiplying 1 by 255 to obtain the pixel value of each point of the output gray level image, and finally outputting the image only retaining the crack target and the background information.
further, the specific content of step S34 is: training the Unet model, judging whether the setting of the learning rate is reasonable or not by observing the descending speed and the descending trend of the loss value, namely the convergence speed of the model at the initial stage of the model training, and if the convergence speed of the model is low, setting the learning rate to be too low; on the contrary, if the loss value is decreased quickly and the change fluctuation is large, the learning rate is too high; in the later stage of model training, whether the number of iteration rounds is reasonable or not is determined by observing the changes of the loss value and the accuracy, if the loss value and the accuracy both tend to be a stable value, the training of the model is terminated, and the weight of the model is automatically stored at the moment; on the contrary, if the model has already iterated and finished the number of rounds set but loss value and accuracy rate continue to present the trend of decline, increase training round number in order to continue training the model on the basis of the weight value that the last iteration finishes producing, until loss value and accuracy rate both tend to a stable value; continuously debugging, finally setting the learning rate of the model to be 10-6(ii) a And stopping training when the identification of the model reaches 90.83% after 150 training iteration rounds. Compared with the prior art, the invention has the following beneficial effects:
The invention can effectively improve the automatic identification accuracy and recall rate of the pavement diseases under the complex pavement background condition, thereby improving the working efficiency of the road maintenance department.
drawings
FIG. 1 is a flow chart of an embodiment of the present invention
fig. 2 is a diagram illustrating an example of a data tag diagram according to an embodiment of the present invention.
FIG. 3 is a diagram of an example of data enhancement according to an embodiment of the present invention.
Fig. 4 is a diagram of the structure of the Unet model according to the embodiment of the present invention.
FIG. 5 is a diagram of an example of the recognition results of U-net, Canny edge detection and Otsu threshold segmentation;
fig. 5(a) is an original image, fig. 5(b) is a Canny edge detection map, fig. 5(c) is an Otsu threshold segmentation map, fig. 5(d) is a Unet model map, and fig. 5(e) is a Unet model map according to the present embodiment.
Detailed Description
the invention is further explained below with reference to the drawings and the embodiments.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
it is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
As shown in fig. 1, the present embodiment provides a road surface crack automatic identification method based on a U-net neural network model, including the following steps:
step S1: collecting two-dimensional laser image data of a road surface, namely a two-dimensional laser original image;
Step S2: calibrating and enhancing the image with cracks after the pavement two-dimensional laser image is manually screened so as to construct a model training library;
step S3: adjusting a Unet model and parameters and training the adjusted U-net model;
step S4: providing an image to be detected and identified, taking the weight of the Unet model trained in the step S3 as a feature extractor, and traversing each pixel on the image to be detected and identified by taking 1 as a step length; limiting the output confidence range of each pixel to be between 0 and 1 in combination with a Sigmoid activation function;
Step S5: performing confidence judgment on the output result, setting the value of the output result to be 1 when the confidence coefficient is greater than 0.5, and determining the output result as crack information, and setting the output result to be 0 when the confidence coefficient is less than 0.5 and classifying the output result into background or noise;
Step S6: filtering the noise in the step S5, after the step S5, multiplying the confidence coefficient by 255 as the pixel value of each point, if the confidence coefficient of each point is not 1, that is, 0; wherein 0 represents background information, and 255 represents a crack region; the final output crack identification result is a gray image, namely a black-and-white image, which only retains crack information and background information.
in the embodiment, the road surface image data is collected by the vehicle-mounted laser road detection equipment
In this embodiment, the step S2 specifically includes the following steps:
Step S21: data calibration: data calibration is carried out in a point marking mode; the reason for selecting point marking is that the cracks are different from road surface diseases such as loose, pit and sink, the shapes of the cracks are fine and irregular, and the characteristics of the cracks cannot be expressed finely by adopting a Labelimage or Labelme polygonal marking tool;
Step S22: making a label: after the data calibration is completed, preprocessing is carried out on the calibrated image, namely RGB (0,0,0) is used for representing the crack contour, and RGB (255 ) represents the background and is used as a label;
Step S23: data enhancement: the quality and the quantity of training samples are crucial to the training effect of a neural network model, overfitting phenomena of the model can occur due to insufficient sample quantity, a crack data set is not disclosed in the network, and the label image and the two-dimensional laser original image after preprocessing are enhanced by utilizing an elastic deformation method, namely, rotation, mirror image and cutting are sequentially carried out so as to expand a database.
In this embodiment, the step S3 specifically includes the following steps:
Step S31: selection of the number of network layers of the Unet model: training the Unet model, wherein when the number of network layers is 32, the Loss value (Loss) of the training is 0.04, and when the number of network layers is continuously increased, the Loss value is not changed any more, but the training time is longer, so that the number of network layers of the Unet model is determined to be 32;
Step S32: size of output image: in the convolution operation process, an all-zero filling mode is adopted to avoid losing boundary information, after four times of pooling layers with the step length of 2, the image size is reduced to 1/16 of the original size, and then the image is restored to the original size through four times of up-sampling operation with the step length of 2;
step S33: avoiding overfitting of the model: the Unet model has a complex network and limited training samples, which may cause an overfitting phenomenon in the testing process. To avoid this phenomenon, study on adding a Dropout layer at the end of the fourth and fifth convolutional layers; at this time, the model framework of the Unet is determined as follows: twenty-three convolutional layers, four pooling layers, two Dropout layers, four upper sampling layers and four feature fusion operations;
step S34: adjusting parameters of the Unet model: : adjusting the learning rate and the training iteration round number parameters of the Unet model;
Step S35: training the Unet model determined in step S33: the Unet model obtains a predicted value through forward propagation, and calculates the error between the real value and the predicted value through a cross entropy Loss function, namely a Loss value (Loss) calculation formula is shown as a formula (1);
Wherein p represents the true distribution q represents the predicted value, and H (p, q) represents the loss value;
Then the loss value is reduced by the back propagation operation and the adjustment of the learning rate and the training round number, and the adjusted learning rate is 10-6After 150 rounds of training, the loss value is 0.04, the number of training rounds is continuously increased or the learning rate is adjusted, and the loss value is not changed any more; at this time, the weight file generated by the Unet model is the optimal feature extractor obtained by training.
in the present embodiment, the formula for controlling the output result range to [0,1] in step S4 is shown as (2):
wherein f (z) represents a sigmoid activation function and z represents an input.
in this embodiment, the specific content of step S5 is: after the trained Unet model traverses each pixel point on the image, the confidence degree judgment is carried out, and the calculation formula is shown as (3):
Score is the confidence; 1 represents fracture information; 0 represents noise information; f (z) is the Sigmoid activation function calculation value.
In this embodiment, the specific content of step S6 is: and when the confidence coefficient is 1, classifying the image as a crack target, multiplying 1 by 255 to obtain the pixel value of each point of the output gray level image, and finally outputting the image only retaining the crack target and the background information.
in this embodiment, the specific content of step S34 is: judging whether the learning rate is set reasonably by observing the descending speed and the descending trend (the convergence speed of the model) of the loss value at the initial stage of model training, and if the convergence speed of the model is low, setting the learning rate too low; conversely, if the loss value decreases very quickly but the variation is large, the learning rate is too high. In the later stage of model training, whether the number of iteration rounds is reasonable or not is determined by observing the changes of the loss value and the accuracy, if the loss value and the accuracy both tend to be a stable value, the training of the model can be terminated, and the weight of the model can be automatically stored at the moment; on the contrary, if the model has already iterated and finished the number of rounds set but the loss value and accuracy rate still continue to be the trend of decline, then can increase the number of rounds of training on the basis of the weight value that the last iteration finishes and produces in order to continue training the model, until loss value and accuracy rate both tend to a stable value. Continuously debugging, finally setting the learning rate of the model to be 10-6(ii) a And stopping training when the identification of the model reaches 90.83% after 150 training iteration rounds.
Preferably, the present embodiment further provides the following examples:
(1) Elastic deformation technology-based fracture database expansion
the research adopts two-dimensional line scanning laser imaging equipment to acquire data images. The data set is divided into two parts: training a sample database and verifying the database. The quality and the quantity of training samples are crucial to the training effect of the neural network model, and the overfitting phenomenon of the model can be caused due to the fact that the sample quantity is not enough. The embodiment expands the database by applying elastic deformation to the sample image subjected to crack labeling for image enhancement. The enhancement effect is shown in fig. 2.
(2) automatic pavement crack identification method based on U-net neural network model
the present example uses a model (as shown in fig. 5) composed of thirty-three layers of neural networks, including twenty-three convolutional layers, four pooling layers, two dropout layers, and four upsampling layers, for the characteristic of fine and irregular fracture contours, and expands the fracture database by using elastic deformation technique. Given the test data, the trained Unet model returns an output value (0 or 1). 0 represents background information and 1 represents crack information in the image. The automatic identification of the cracks can be realized through the classification model. Due to the diversity of the cracks in the training library, the model has higher recall rate, can better identify the fine and latticed cracks with unobvious depth change, and has stronger generalization capability.
specifically, the present embodiment provides a road surface crack automatic identification method based on a U-net neural network model, including the following steps:
step S1: collecting pavement image data by utilizing vehicle-mounted laser road detection equipment;
step S2: calibrating and enhancing the screened images with cracks to construct a model training library;
step S3: adjusting a Unet model and parameters and training the U-net model;
step S4: traversing each pixel on the image to be identified by taking 1 as a step length by taking the weight of the Unet model trained in the step S3 as a feature extractor; limiting the output confidence range of each pixel to be between 0 and 1 in combination with a Sigmoid activation function;
Step S5: performing confidence judgment on the output result, setting the value of the output result to be 1 when the confidence coefficient is greater than 0.5, and determining the output result as crack information, and setting the output result to be 0 when the confidence coefficient is less than 0.5 and classifying the output result into background or noise;
step S6: the noise in step S5 is filtered, after step S5, the confidence of each point is not 1, that is, 0, and then the confidence is multiplied by 255 to obtain the pixel value of each point (0 represents background information, and 255 represents a crack region), so that the finally output crack recognition result is a gray scale image (black and white image) that only retains crack information and background information.
In this example, the specific implementation is as follows:
(1) device parameters and working principle thereof
in the embodiment, the road detection equipment is adopted for data acquisition, the equipment consists of a laser sensor and an area-array camera, the pavement image information can be acquired at high precision, the coverable pavement range is 3.6m (width) multiplied by 2m (length), and the precision can reach 1 mm/pixel.
(2) automatic pavement crack identification method based on U-net neural network model
1) database construction
And (5) establishing a database. The method is characterized in that a pavement 2D laser image acquired by DHDV is taken as a basis, a database is expanded through an elastic deformation technology, a label is manufactured in a point marking mode, RGB (0,0,0) is used for representing a crack contour, RGB (255 ) is used for representing a background, and a label sample is shown in FIG. 3.
2) Unet model architecture
the key core of the model proposed in this example is the convolutional layer, pooling layer, upsampling layer and feature fusion process.
Step 1: add 1 convolutional layer after the upsampling operation is finished. The recognition using the original Unet model may have missed detections (as shown by the red oval in FIG. 5 (d)) and false detections (as shown by the orange rectangle in FIG. 5 (d)). The missed detection situation often occurs in a fine crack area, and the main reason for the missed detection is that fine crack information may be missed in the lower sampling process of the Unet. The main reason for false detection is that the model incorrectly identifies some targets with a high similarity to the crack as crack pixels. For example, the edge pixel of the patch (fig. 5(d) 3 rd sheet) is identified as a crack. For this situation, in this embodiment, after each upsampling is completed, a convolution operation is performed first, and then feature fusion is performed, so that the extracted features are more abstract, and the model learns to automatically remove noise information in the training process, and the modified model recognition effect is as shown in fig. 5 (e).
step 2: and (5) feature fusion. The feature fusion operation is a crucial step for obtaining an accurate recognition result, and in order to make extracted features more abstract, after the sampling operation on the model is finished, a convolution operation is performed first and then feature fusion is performed (the feature fusion operation in fig. 4).
And step 3: determining a Unet model network architecture. Since the crack profile is fine and irregular, it is not easy to perform accurate feature extraction on it. Through debugging, the model is optimal when the number of the neural network layers is 33, and when the network depth is continuously increased, the accuracy rate is basically stabilized at 90.83%, but larger computer memory is occupied. Therefore, the U-net model framework is finally determined as follows: twenty-three convolutional layers + four pooling layers + two Dropout layers + four upsampling layers + four feature fusion operations.
2) Model parameter adjustment and training
Step 1: parameter adjustment: the main parameters related to the Unet model are learning rate and training iteration number. Judging whether the learning rate is set reasonably by observing the descending speed and the descending trend (the convergence speed of the model) of the loss value at the initial stage of model training, and if the convergence speed of the model is low, setting the learning rate too low; conversely, if the loss value decreases very quickly but the variation is large, the learning rate is too high. In the later stage of model training, whether the number of iteration rounds is reasonable or not is determined by observing the changes of the loss value and the accuracy, if the loss value and the accuracy both tend to be a stable value, the training of the model can be terminated, and the weight of the model can be automatically stored at the moment; otherwise, if the model has been iterated to complete the set round number but the loss value and the accuracy rate continue to be in the descending trend, the training round number can be increased on the basis of the weight value generated at the end of the last iteration to continue training the model until the loss value and the accuracy rate are reachedthe rates all tend to a steady value. Continuously debugging, finally setting the learning rate of the model to be 10-6(ii) a Stopping training when the identification of the model reaches 90.83% after the number of training iteration rounds is 150;
step 2: determining the size of the output image: the size of the input image to be recognized is 512 × 512 in the embodiment, and the size of the input image after model recognition is 512 × 512, because an all-zero filling mode is adopted in the convolution operation process to avoid losing boundary information; and then, after four times of pooling layers with the step size of 2, the size of the test image is reduced to 1/16 of the original size, and then the test image is restored to the original size through four times of upsampling operation with the step size of 2, so that the original size of the test image is not changed finally after a series of calculation operations of the model.
And step 3: training the Unet model determined in step S33: the Unet model obtains a predicted value through forward propagation, and calculates the error between the real value and the predicted value through a cross entropy Loss function, namely a Loss value (Loss) calculation formula is shown as the following formula; then the loss value is reduced by the back propagation operation and the adjustment of the learning rate and the training round number, and the adjusted learning rate is 10-6After 150 rounds of training, the loss value is 0.04, the number of training rounds is continuously increased or the learning rate is adjusted, and the loss value is not changed any more; at this time, the weight file generated by the Unet model is the optimal feature extractor obtained by training.
Where p denotes the true distribution q denotes the predicted value and H (p, q) denotes the loss value.
And 4, step 4: determining the importance degree of output information, transmitting the output information of the last layer of convolution layer to a Sigmoid function, wherein the output value is between 0 and 1, classifying the pixel point into crack information when the confidence coefficient of the output of each pixel is greater than 0.5, classifying the pixel point into noise when the confidence coefficient is less than 0.5, and automatically filtering.
The trained model in the embodiment can automatically identify the crack information in the test sample, the accuracy rate P is 90.83%, the recall rate R is 89.97%, and the F value can reach 90.40%.
in this embodiment, a model architecture composed of 33 layers of neural networks is constructed by adjusting the Unet neural network, where the model architecture includes 23 convolutional layers, 4 pooling layers, 2 dropout layers, and 4 upsampling layers, so as to implement accurate automatic identification of a pavement crack. The model has the following advantages:
(1) The Unet is different from other segmentation models, and has no fully-connected layer and pre-trained classification models, so that the model is simplified, and the convergence rate is higher; the feature fusion operation is a very effective step for the accurate identification of fine-shaped cracks.
(2) by adjusting the model structure, the crack identification precision is higher and better than that of the traditional Otsu threshold segmentation algorithm and Canny edge detection on the whole, the F value is 90.97%, the batch input of images of any size is accepted under the permission of computer conditions, and the identification time of each image is only 127 ms. This can greatly improve the working efficiency of highway maintenance departments, especially highway detection operations.
(3) Compared with a plurality of mainstream crack identification algorithms, the model adopted by the embodiment has better performance for identifying the pavement crack under the complex background condition.
The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.
Claims (7)
1. A road surface crack automatic identification method based on a U-net neural network model is characterized by comprising the following steps: the method comprises the following steps:
Step S1: collecting two-dimensional laser image data of a road surface, namely a two-dimensional laser original image;
step S2: calibrating and enhancing the image with cracks after the pavement two-dimensional laser image is manually screened so as to construct a model training library;
Step S3: adjusting a Unet model and parameters and training the adjusted Unet model;
Step S4: providing an image to be detected and identified, taking the weight of the Unet model trained in the step S3 as a feature extractor, and traversing each pixel on the image to be detected and identified by taking 1 as a step length; limiting the output confidence range of each pixel to be between 0 and 1 in combination with a Sigmoid activation function;
Step S5: performing confidence judgment on the output result, and making the value of the output result 1 and classifying the output result as crack information when the confidence is more than 0.5; if the value is less than 0.5, the value is 0 and classified as background or noise;
Step S6: filtering the noise in the step S5, after the step S5, the confidence of each point is not 1, that is, 0, and then multiplying the confidence by 255 as the pixel value of each point; wherein 0 represents background information, and 255 represents a crack region; and finally, outputting a crack identification result, namely a gray image, namely a black-and-white image, only retaining crack information and background information.
2. the method for automatically identifying the pavement cracks based on the U-net neural network model according to claim 1, wherein the method comprises the following steps: the step S2 specifically includes the following steps:
step S21: data calibration: data calibration is carried out in a point marking mode;
step S22: making a label: after the data calibration is completed, preprocessing is carried out on the calibrated image, namely RGB (0,0,0) is used for representing the crack contour, and RGB (255 ) represents the background and is used as a label;
step S23: data enhancement: and (3) enhancing the preprocessed label image and the corresponding two-dimensional laser original image by using an elastic deformation method, namely, sequentially rotating, mirroring and cutting to expand the database.
3. The method for automatically identifying the pavement cracks based on the U-net neural network model according to claim 1, wherein the method comprises the following steps: the step S3 specifically includes the following steps:
step S31: selection of the number of network layers of the Unet model: training the Unet model, wherein when the number of network layers is 32, the loss value of the training is 0.04, and when the number of network layers is continuously increased, the loss value is not changed any more, but the training time is longer, so that the number of network layers of the Unet model is determined to be 32;
Step S32: in the convolution operation process, an all-zero filling mode is adopted to avoid losing boundary information, and after four times of pooling layers with the step length of 2, the image size is reduced to 1/16 of the original size; after four upsampling operations with the step size of 2, the image is restored to the original size, so that the original size of the image is not changed.
Step S33: avoiding overfitting of the model: a Dropout layer is added at the end of the fourth convolution layer and the end of the fifth convolution layer; at this time, the model framework of the Unet is determined as follows: twenty-three convolutional layers, four pooling layers, two Dropout layers, four upper sampling layers and four feature fusion operations;
step S34: adjusting parameters of the Unet model: adjusting the learning rate and the training iteration round number parameters of the Unet model;
step S35: training the Unet model determined in step S33: the Unet model obtains a predicted value through forward propagation, and calculates the error between the actual value and the predicted value through a cross entropy loss function, namely a loss value calculation formula is shown as a formula (1);
Wherein p represents the true distribution q represents the predicted value, and H (p, q) represents the loss value;
Then the loss value is reduced by the back propagation operation and the adjustment of the learning rate and the training round number, and the adjusted learning rate is 10-6After 150 rounds of training, the loss value is 0.04, the number of training rounds is continuously increased or the learning rate is adjusted, and the loss value is not changed any more; at this time, the weight file generated by the Unet model is the optimal feature extractor obtained by training.
4. the method for automatically identifying the pavement cracks based on the U-net neural network model according to claim 1, wherein the method comprises the following steps: the formula for controlling the output result range to [0,1] in step S4 is shown in (2):
wherein f (z) represents a sigmoid activation function and z represents an input.
5. The method for automatically identifying the pavement cracks based on the U-net neural network model according to claim 1, wherein the method comprises the following steps: the specific content of step S5 is: after the trained Unet model traverses each pixel point on the image, the confidence degree judgment is carried out, and the calculation formula is shown as (3):
Score is the confidence; 1 represents fracture information; 0 represents noise information; f (z) is the Sigmoid activation function calculation value.
6. The method for automatically identifying the pavement cracks based on the U-net neural network model according to claim 1, wherein the method comprises the following steps: the specific content of step S6 is: and when the confidence coefficient is 1, classifying the image as a crack target, multiplying 1 by 255 to obtain the pixel value of each point of the output gray level image, and finally outputting the image only retaining the crack target and the background information.
7. The method for automatically identifying the pavement cracks based on the U-net neural network model according to claim 3, wherein the method comprises the following steps: the specific content of step S34 is: training the Unet model, judging whether the setting of the learning rate is reasonable or not by observing the descending speed and the descending trend of the loss value, namely the convergence speed of the model at the initial stage of the model training, and if the convergence speed of the model is low, setting the learning rate to be too low; on the contrary, if the loss value is decreased quickly and the change fluctuation is large, the learning rate is too high; in the later stage of model training, whether the iteration turns are reasonable or not is determined by observing the changes of the loss value and the accuracy rate, and if the loss value and the accuracy rate are both reasonablewhen the model approaches a stable value, the training of the model is terminated, and the weight of the model is automatically saved; on the contrary, if the model has already iterated and finished the number of rounds set but loss value and accuracy rate continue to present the trend of decline, increase training round number in order to continue training the model on the basis of the weight value that the last iteration finishes producing, until loss value and accuracy rate both tend to a stable value; continuously debugging, finally setting the learning rate of the model to be 10-6(ii) a And stopping training when the identification of the model reaches 90.83% after 150 training iteration rounds.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910723374.9A CN110569730B (en) | 2019-08-06 | 2019-08-06 | Road surface crack automatic identification method based on U-net neural network model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910723374.9A CN110569730B (en) | 2019-08-06 | 2019-08-06 | Road surface crack automatic identification method based on U-net neural network model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110569730A true CN110569730A (en) | 2019-12-13 |
| CN110569730B CN110569730B (en) | 2022-11-15 |
Family
ID=68774739
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910723374.9A Active CN110569730B (en) | 2019-08-06 | 2019-08-06 | Road surface crack automatic identification method based on U-net neural network model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110569730B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111121639A (en) * | 2019-12-18 | 2020-05-08 | 同济大学 | Rigid-flexible integrated crack detection system for narrow building space |
| CN111126505A (en) * | 2019-12-28 | 2020-05-08 | 北京工业大学 | Pavement crack rapid identification method based on deep learning |
| CN111598845A (en) * | 2020-04-26 | 2020-08-28 | 北京科技大学 | Pavement crack detection and positioning method based on deep learning and NEO-6M positioning module |
| CN111612846A (en) * | 2020-04-29 | 2020-09-01 | 南京震坤物联网科技有限公司 | Concrete crack width measuring method based on U-net CNN image recognition and pixel calibration |
| CN111783686A (en) * | 2020-07-03 | 2020-10-16 | 中国交通通信信息中心 | Asphalt pavement health state monitoring system and method |
| CN111783974A (en) * | 2020-08-12 | 2020-10-16 | 成都佳华物链云科技有限公司 | Model construction and image processing method and device, hardware platform and storage medium |
| CN111861982A (en) * | 2020-06-07 | 2020-10-30 | 中国葛洲坝集团第一工程有限公司 | Visual image monitoring and identifying system |
| CN113095109A (en) * | 2019-12-23 | 2021-07-09 | 中移(成都)信息通信科技有限公司 | Crop leaf surface recognition model training method, recognition method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180225527A1 (en) * | 2015-08-03 | 2018-08-09 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, apparatus, storage medium and device for modeling lane line identification, and method, apparatus, storage medium and device for identifying lane line |
| CN109872318A (en) * | 2019-02-22 | 2019-06-11 | 中国石油大学(华东) | A kind of geology for deep learning is appeared crack data set production method |
| CN110060251A (en) * | 2019-04-26 | 2019-07-26 | 福州大学 | A kind of building surface crack detecting method based on U-Net |
-
2019
- 2019-08-06 CN CN201910723374.9A patent/CN110569730B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180225527A1 (en) * | 2015-08-03 | 2018-08-09 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, apparatus, storage medium and device for modeling lane line identification, and method, apparatus, storage medium and device for identifying lane line |
| CN109872318A (en) * | 2019-02-22 | 2019-06-11 | 中国石油大学(华东) | A kind of geology for deep learning is appeared crack data set production method |
| CN110060251A (en) * | 2019-04-26 | 2019-07-26 | 福州大学 | A kind of building surface crack detecting method based on U-Net |
Non-Patent Citations (1)
| Title |
|---|
| 陈泽斌: "基于改进U-net模型的路面裂缝智能识别", 《数据采集与处理》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111121639A (en) * | 2019-12-18 | 2020-05-08 | 同济大学 | Rigid-flexible integrated crack detection system for narrow building space |
| CN111121639B (en) * | 2019-12-18 | 2021-07-27 | 同济大学 | Rigid-flexible integrated crack detection system for narrow building space |
| CN113095109A (en) * | 2019-12-23 | 2021-07-09 | 中移(成都)信息通信科技有限公司 | Crop leaf surface recognition model training method, recognition method and device |
| CN111126505A (en) * | 2019-12-28 | 2020-05-08 | 北京工业大学 | Pavement crack rapid identification method based on deep learning |
| CN111598845A (en) * | 2020-04-26 | 2020-08-28 | 北京科技大学 | Pavement crack detection and positioning method based on deep learning and NEO-6M positioning module |
| CN111612846A (en) * | 2020-04-29 | 2020-09-01 | 南京震坤物联网科技有限公司 | Concrete crack width measuring method based on U-net CNN image recognition and pixel calibration |
| CN111861982A (en) * | 2020-06-07 | 2020-10-30 | 中国葛洲坝集团第一工程有限公司 | Visual image monitoring and identifying system |
| CN111861982B (en) * | 2020-06-07 | 2024-03-12 | 中国葛洲坝集团第一工程有限公司 | Visual image monitoring and identifying system |
| CN111783686A (en) * | 2020-07-03 | 2020-10-16 | 中国交通通信信息中心 | Asphalt pavement health state monitoring system and method |
| CN111783974A (en) * | 2020-08-12 | 2020-10-16 | 成都佳华物链云科技有限公司 | Model construction and image processing method and device, hardware platform and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110569730B (en) | 2022-11-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110569730B (en) | Road surface crack automatic identification method based on U-net neural network model | |
| CN106875373B (en) | Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm | |
| CN110263705A (en) | Towards two phase of remote sensing technology field high-resolution remote sensing image change detecting method | |
| CN112598672A (en) | Pavement disease image segmentation method and system based on deep learning | |
| CN114693615A (en) | Deep learning concrete bridge crack real-time detection method based on domain adaptation | |
| CN113469177A (en) | Drainage pipeline defect detection method and system based on deep learning | |
| CN110751644B (en) | Road surface crack detection method | |
| CN113065578A (en) | Image visual semantic segmentation method based on double-path region attention coding and decoding | |
| CN109034184A (en) | A kind of grading ring detection recognition method based on deep learning | |
| CN110276763A (en) | It is a kind of that drawing generating method is divided based on the retinal vessel of confidence level and deep learning | |
| CN112749654A (en) | Deep neural network model construction method, system and device for video fog monitoring | |
| Yusof et al. | Automated asphalt pavement crack detection and classification using deep convolution neural network | |
| CN112964712A (en) | Method for rapidly detecting state of asphalt pavement | |
| CN110458019B (en) | Water surface target detection method for eliminating reflection interference under scarce cognitive sample condition | |
| Zhao et al. | Image dehazing based on haze degree classification | |
| CN110033045A (en) | A kind of method and apparatus of trained identification image atomization | |
| CN115830302B (en) | Multi-scale feature extraction fusion power distribution network equipment positioning identification method | |
| CN117576564A (en) | Disease and pest identification early warning method and system for tea planting | |
| CN113850335B (en) | Data augmentation method for bathroom ceramic defect detection | |
| CN115439690A (en) | Flue-cured tobacco leaf image grading method combining CNN and Transformer | |
| CN114862883A (en) | Target edge extraction method, image segmentation method and system | |
| CN111507941B (en) | Composition characterization learning method for aesthetic quality evaluation | |
| CN114648659A (en) | Lightweight concrete bridge disease identification method based on transfer learning | |
| CN114821174A (en) | Power transmission line aerial image data cleaning method based on content perception | |
| CN114821098A (en) | High-speed pavement damage detection algorithm based on gray gradient fusion characteristics and CNN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |