CN114708190A - Road crack detection and evaluation algorithm based on deep learning - Google Patents

Abstract

The invention discloses a road crack detection and evaluation algorithm based on deep learning, which comprises the following steps: s1, training a neural network model FUnet; s2, road crack detection and evaluation are carried out by using the algorithm flow; compared with the prior art, the invention has the advantages that: the road crack is extracted and analyzed by utilizing the convolutional neural network model FUnet and a subsequent algorithm, a standard flow is provided, the problem of different standards caused by the difference of judgment between people in manual detection is avoided, the road crack is extracted and analyzed through an image, the equipment is simple to use, the cost is reduced, the algorithm flow can finish the extraction and analysis of the crack end to end, the classification task of a pixel level can be finished, meanwhile, the object level division and analysis can be carried out, the internal structure does not need to be manually concerned, and the requirement on operators is low.

Description

A road crack detection and evaluation algorithm based on deep learning

technical field

The invention relates to the technical field of crack identification, in particular to a road crack detection and evaluation algorithm based on deep learning.

Background technique

China has one of the largest road networks in the world. After the road is laid, it faces maintenance problems. If the road surface is damaged and not maintained in time, it will greatly reduce the service life of the road surface and cause safety hazards. However, highways have the characteristics of large number, long mileage, and wide distribution, so it is difficult to quickly and accurately evaluate the road damage in different sections. The traditional manual detection method is slow, the labor cost is high, and it also needs to rely on the relevant professional measurement equipment for more accurate detection and statistics. However, the professional road inspection vehicle has the problems of high cost, large volume, and the need for special personnel to operate because it is equipped with many professional measuring equipment.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a deep learning-based road crack detection statistical algorithm flow, which can extract cracks from road pavement pictures and obtain relevant information, so as to provide reference for road damage assessment.

In order to solve the above-mentioned technical problems, the technical solution provided by the present invention is: a road crack detection and evaluation algorithm based on deep learning, comprising the following steps:

S1. Train the neural network model FUnet;

S11. Obtain the public road crack dataset CrackForest dataset;

S12. Input a mini-batch data to the FUnet model each time, obtain the output of the FUnet model, and use the loss function composed of cross entropy and F1 score to calculate the loss value in combination with the annotation of the data set;

S13. Use the error back propagation algorithm to update the gradient of the FUnet model, use the gradient descent algorithm to perform gradient descent on the FUnet according to the calculated gradient, and update the parameters of the FUnet model;

S14. Repeat steps S12 and S13 until the final loss value does not decrease substantially;

S15. Save the parameters of the trained FUnet model for subsequent use;

S2. Use the algorithm flow to detect and evaluate road cracks;

S21. Build a neural network model FUnet, import the previously trained data, and set the model to evaluation mode, in which the values of parameters in the network will not be changed;

S22. Obtain a picture from a camera or a video, and preprocess the picture. In this algorithm, only normalize the picture;

S23. Send the preprocessed picture into the FUnet model, and obtain the output of the FUnet model, and the output Mask of the FUnet model is the classification of the pixel points;

S24. Input the output Mask into the dfs-based pavement crack target detection algorithm, and divide the cracks into instances;

S25. According to the crack instance data obtained by the dfs-based pavement crack target detection algorithm, calculate the overall and individual data of cracks in the subsequent algorithm, including data such as picture crack density and individual crack distribution density. Crack instances whose indicators do not meet the minimum requirements are filtered, which can effectively avoid noise interference on the picture, and only focus on cracks that exceed normal indicators.

Compared with the prior art, the present invention has the advantages that: using the convolutional neural network model FUnet and subsequent algorithms to extract and analyze road cracks, it has a standard process and avoids differences in standards caused by human-to-human judgment differences in manual detection. The problem is to extract and analyze road cracks through images. It is relatively simple to use equipment and reduce costs. The algorithm flow can complete the extraction and analysis of cracks end-to-end, which can not only complete pixel-level classification tasks, but also Level division and analysis, no need to manually pay attention to the internal structure, and low requirements for operators.

Preferably, the acquired data set in step S11 uses an image processing method to augment the data set. These operations are performed on the original image and annotations in the data set at the same time. The specific operations include: image flipping, cropping and splicing, local distortion, Add Gaussian noise, random bias to brightness.

Preferably, the calculation formula of step S12 is as follows:

Loss=CrossEntropyLoss+(1-F ₁ ) (1)

in

Preferably, in step S24, by adjusting the search range in the dfs, it is possible to control whether to merge the discontinuous fractures into one fracture instance, and also to adjust the merging interval, that is, within this interval range, two discontinuous fractures are grouped together. with the same crack instance. After being processed by the pavement crack target detection algorithm based on dfs, the crack instance can be obtained, including the distribution range of the crack instance in the figure [xmin, ymin, xmax, ymax], and the number of crack pixels of the crack instance.

Description of drawings

Fig. 1 is a training and detection flow chart of a deep learning-based road crack detection and evaluation algorithm;

Fig. 2 is the overall structure diagram of the neural network model FUnet in a road crack detection and evaluation algorithm based on deep learning;

Fig. 3 is a detailed structure diagram of the neural network model FUnet in a deep learning-based road crack detection and evaluation algorithm;

FIG. 4 is a sample image of a road crack detection and evaluation algorithm based on deep learning;

Figure 5 is the pixel-level classification picture obtained by the algorithm, the black is the background, and the white is the crack;

Figure 6 shows the crack instance obtained by the dfs pavement crack target detection algorithm when the interval distance is 0, and each frame corresponds to a crack instance;

Figure 7 shows the crack instances obtained by the dfs pavement crack target detection algorithm when the interval distance is 7, and each box corresponds to a crack instance.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings.

When the present invention is specifically implemented, a deep learning-based road crack detection and evaluation algorithm includes the following steps:

S1. Train the neural network model FUnet;

S11. Obtain the public road crack dataset CrackForest dataset;

S12. Input a mini-batch data to the FUnet model each time, obtain the output of the FUnet model, and use the loss function composed of cross entropy and F1 score to calculate the loss value in combination with the annotation of the data set;

S13. Use the error back propagation algorithm to update the gradient of the FUnet model, use the gradient descent algorithm to perform gradient descent on the FUnet according to the calculated gradient, and update the parameters of the FUnet model;

S14. Repeat steps S12 and S13 until the final loss value does not decrease substantially;

S15. Save the parameters of the trained FUnet model for subsequent use;

S2. Use the algorithm flow to detect and evaluate road cracks;

S21. Build a neural network model FUnet, import the previously trained data, and set the model to evaluation mode, in which the values of parameters in the network will not be changed;

S22. Obtain a picture from a camera or a video, and preprocess the picture. In this algorithm, only normalize the picture;

S23, send the preprocessed picture into the FUnet model, and obtain the output of the FUnet model, and the output Mask of the FUnet model is the classification of the pixel points;

S24. Input the output Mask into the dfs-based pavement crack target detection algorithm, and divide the cracks into instances;

S25. According to the crack instance data obtained by the dfs-based pavement crack target detection algorithm, calculate the overall and individual crack data in the subsequent algorithm, including the picture crack density, individual crack distribution density and other data. Crack instances whose indicators do not meet the minimum requirements are filtered, which can effectively avoid noise interference on the picture, and only focus on cracks that exceed normal indicators.

Preferably, the acquired data set in step S11 uses an image processing method to augment the data set. These operations are performed on the original image and annotations in the data set at the same time. The specific operations include: image flipping, cropping and splicing, local distortion, Add Gaussian noise, random bias to brightness.

Preferably, the calculation formula of step S12 is as follows:

Loss=CrossEntropyLoss+(1-F ₁ ) (1)

in

Preferably, in step S24, by adjusting the search range in the dfs, it is possible to control whether to merge the discontinuous fractures into one fracture instance, and also to adjust the merging interval, that is, within this interval range, two discontinuous fractures are grouped together. with the same crack instance. After being processed by the pavement crack target detection algorithm based on dfs, the crack instance can be obtained, including the distribution range of the crack instance in the figure [xmin, ymin, xmax, ymax], and the number of crack pixels of the crack instance.

The working principle of the present invention: the parameter file after the FUnet model training is completed is only 1.5MB, and the parameter quantity is small, and the training can be quickly completed in a short time with only a small amount of data. Only about 1GB of video memory is required to run a single test, and the test on the RTX1070 graphics card can reach about 50 frames per second. The time complexity of the subsequent dfs pavement crack target detection algorithm is O(mn), where m and n are the width and height of the image, respectively. From this, it can be seen that the algorithm has lower performance requirements for the on-board platform and faster speed. On the one hand, it reduces the cost of deploying the algorithm. On the other hand, due to its low platform requirements, it can be deployed on a variety of on-board platforms.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, the features defined with "first" and "second" may expressly or implicitly include one or more of the features, and in the description of the present invention, "multiple" means two or two above, unless otherwise expressly specifically defined.

In the present invention, unless otherwise expressly specified and limited, the terms "installed", "connected", "connected", "fixed" and other terms should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection Or integrally connected; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication between the two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

In the present invention, unless otherwise expressly specified and limited, a first feature "on" or "under" a second feature may include the first and second features in direct contact, or may include the first and second features Not directly but through additional features between them. Also, the first feature being "above", "over" and "above" the second feature includes the first feature being directly above and obliquely above the second feature, or simply means that the first feature is level higher than the second feature. The first feature is "below", "below" and "below" the second feature includes the first feature being directly below and diagonally below the second feature, or simply means that the first feature has a lower level than the second feature.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and those of ordinary skill in the art will not depart from the principles and spirit of the present invention Variations, modifications, substitutions, and alterations to the above-described embodiments are possible within the scope of the present invention without departing from the scope of the present invention.

Claims (4)

1. A road crack detection and evaluation algorithm based on deep learning is characterized by comprising the following steps:

s1, training a neural network model FUnet;

s11, acquiring a CrackForest data set of the public road crack data set;

S12, inputting mini-batch data into the FUnet model each time to obtain the output of the FUnet model, and calculating a loss value by using a loss function consisting of cross entropy and F1 fraction in combination with the labeling of a data set;

s13, updating the gradient of the FUnet model by using an error back propagation algorithm, performing gradient descent on the FUnet according to the calculated gradient by using a gradient descent algorithm, and updating the parameter of the FUnet model;

s14, repeatedly executing the step S12 and the step S13 until the final loss value is not reduced basically;

s15, storing the parameters of the trained FUnet model for subsequent use;

s2, road crack detection and evaluation are carried out by using the algorithm flow;

s21, constructing a neural network model FUnet, importing the trained data, and adjusting the model into an evaluation mode without changing the value of parameters in the network;

s22, acquiring pictures from the camera or the video, and preprocessing the pictures, wherein the algorithm only needs to perform standardized processing on the pictures;

s23, sending the preprocessed pictures into a FUnet model, and obtaining output of the FUnet model, wherein an output Mask of the FUnet model is the classification of pixel points;

s24, inputting the output Mask into a dfs-based pavement crack target detection algorithm, and dividing the crack into examples;

S25, calculating total and individual data of cracks in subsequent algorithms according to crack example data obtained by a dfs-based pavement crack target detection algorithm, wherein the total and individual data comprise data such as picture crack density, crack individual distribution density and the like, and filtering crack examples with certain indexes not meeting the minimum requirement on the basis, so that noise interference on pictures can be effectively avoided, and only cracks exceeding normal indexes are concerned.

2. The deep learning-based road crack detection and evaluation algorithm as claimed in claim 1, wherein: the acquiring data set of step S11 is augmented by using an image processing method, and these operations are performed on the original image and the label in the data set at the same time, and the specific operations include: turning over the image, cutting and splicing, locally distorting, adding Gaussian noise and randomly biasing the brightness.

3. The deep learning-based road crack detection and evaluation algorithm of claim 1, wherein: the calculation formula of step S12 is as follows:

Loss＝CrossEntropyLoss+(1-F₁) (1)

wherein

4. The deep learning-based road crack detection and evaluation algorithm of claim 1, wherein: step S24 may control whether to merge the discontinuous cracks into one crack instance by adjusting the search range in dfs, and may also adjust the merge distance, that is, two discontinuous cracks are drawn to the same crack instance in this distance range. And obtaining a crack example after being processed by a dfs-based pavement crack target detection algorithm, wherein the crack example comprises the distribution range [ xmin, ymin, xmax, ymax ] of the crack example in a graph and the number of crack pixel points of the crack example.

Images