CN109949290B

CN109949290B - Pavement crack detection method, device, equipment and storage medium

Info

Publication number: CN109949290B
Application number: CN201910202588.1A
Authority: CN
Inventors: 徐国胜; 徐国爱; 冯卉
Original assignee: Beijing University of Posts and Telecommunications
Current assignee: Zhonggong High Tech Bazhou Maintenance Technology Industry Co ltd
Priority date: 2019-03-18
Filing date: 2019-03-18
Publication date: 2021-02-26
Anticipated expiration: 2039-03-18
Also published as: CN109949290A

Abstract

The embodiment of the application provides a pavement crack detection method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a plurality of preset pavement sample images; preprocessing the multiple preset pavement sample images based on gamma correction; training the convolutional neural network model; predicting the trained convolutional neural network model; if the training is determined to stop according to the prediction result, obtaining a test convolutional neural network model; testing the test convolutional neural network model; if the test result is within a preset target result range, obtaining a target convolutional neural network model; and inputting a three-channel image corresponding to the road surface image to be detected into the target convolutional neural network model to obtain the road surface crack information in the road surface image to be detected. The method provided by the embodiment of the application can solve the problems that the recognition rate of the cracks cannot be guaranteed and time and resources are wasted in the prior art.

Description

Pavement crack detection method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of image recognition, in particular to a pavement crack detection method, a device, equipment and a storage medium.

Background

Cracks are important marks reflecting road damage conditions, and with the increasing development of road traffic systems in China, pavement crack detection is a very important task for daily overhaul and maintenance of roads.

The conventional image recognition means is used for recognizing the actual road surface collected image, so that the accuracy is low, and the performance requirements of actual engineering cannot be met. Therefore, the existing pavement crack detection method cannot ensure the recognition rate of cracks and wastes time and resources.

Disclosure of Invention

The embodiment of the application provides a pavement crack detection method, a pavement crack detection device, pavement crack detection equipment and a storage medium, and aims to solve the problems that the existing pavement crack detection method cannot guarantee the recognition rate of cracks and wastes time and resources.

In a first aspect, an embodiment of the present application provides a pavement crack detection method, including:

acquiring a plurality of preset pavement sample images;

preprocessing the multiple preset pavement sample images based on gamma correction to obtain three-channel images corresponding to each preset pavement sample image in the multiple preset pavement sample images, and dividing the three-channel images corresponding to all the preset pavement sample images into a training set, a verification set and a test set;

training a convolutional neural network model according to the three-channel image, the preset loss function and the preset optimization algorithm in the training set to obtain a trained convolutional neural network model, wherein the convolutional neural network is the convolutional neural network model optimized through a residual error network;

inputting the three-channel images in the verification set into the trained convolutional neural network model for prediction to obtain a prediction result;

if the training is determined to stop according to the prediction result, taking the trained convolutional neural network model as a test convolutional neural network model;

testing the test convolutional neural network model according to the test set to obtain a test result;

if the test result is within a preset evaluation range, taking the test convolutional neural network model as a target convolutional neural network model;

and inputting a three-channel image corresponding to the road surface image to be detected into the target convolutional neural network model to obtain the road surface crack information in the road surface image to be detected.

In one possible design, the preprocessing the multiple preset road surface sample images based on gamma correction to obtain a three-channel image corresponding to each preset road surface sample image in the multiple preset road surface sample images includes:

setting the gamma value to be smaller than a first preset value, and carrying out nonlinear transformation on the gray values corresponding to the low gray parts in the multiple preset pavement sample images to obtain a first single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images;

setting the gamma value to be larger than a first preset value, and carrying out nonlinear transformation on the gray value corresponding to the high gray part in the multiple preset pavement sample images to obtain a second single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images;

taking each preset pavement sample image in the multiple preset pavement sample images as a third single-channel image, and obtaining a three-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images according to a first single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images and a second single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images;

the gray values corresponding to the low gray portions in the preset road surface sample images are smaller than a second preset value, and the gray values corresponding to the high gray portions in the preset road surface sample images are larger than the second preset value.

In one possible design, the convolutional neural network model includes: a convolutional layer, a pooling layer, and a full-link layer;

the network structure in the convolutional neural network model comprises a plurality of combination layers, a convolutional layer and a full-connection layer which are connected in sequence, wherein each combination layer in the plurality of combination layers is formed by connecting two connected convolutional layers with one pooling layer;

the size of the convolution kernel of the convolution layer in the first combination layer in the network structure is equal to the size of the convolution kernel of the second convolution layer and is larger than the sizes of the convolution kernels of other convolution layers in the network structure, and the sizes of the convolution kernels of the convolution layers in the combination layers corresponding to the other convolution layers in the network structure are equal to each other and are larger than the sizes of the convolution kernels of the convolution layers connected with the full-connection layer;

the size of the kernel of the pooling layer in the network structure is smaller than the size of the convolution kernels of other convolution layers in the network structure;

the size of a full-connection layer in the network structure is determined according to the size of the preset pavement sample image and the size of a convolution kernel of a convolution layer connected with the full-connection layer;

initialized network parameters are set in the network structure, and the network parameters comprise weights and offsets.

In one possible design, the preset loss function is a cross entropy function, the preset optimization algorithm is an adaptive moment estimation Adam optimization algorithm, and three-channel images in the training set all contain crack identifications; the training of the convolutional neural network model according to the three-channel image, the preset loss function and the preset optimization algorithm in the training set comprises the following steps:

setting parameters of the convolutional neural network model, wherein the parameters of the convolutional neural network model comprise a learning rate and the number of training samples, and the training samples are three-channel images corresponding to each preset pavement sample image in the training set;

selecting training samples corresponding to the number of the training samples from the training set according to the number of the training samples, inputting the training samples corresponding to the number of the training samples into the convolutional neural network model in batches, and obtaining a prediction result of each training sample in the training set;

calculating an error between a prediction result of each training sample and a corresponding true value containing a crack identifier through the cross entropy function and the adaptive moment estimation optimization algorithm, wherein the prediction result of each training sample is a probability matrix containing the crack identifier in each training sample, and the true value is an initial probability matrix containing the crack identifier in each preset road sample image obtained when a plurality of preset road sample images are obtained;

according to the gradient of the network parameter of each network layer in the network structure, the learning rate and the error, the network parameter of the current network layer is adjusted through back propagation, and the adjusted network parameter of the current network layer is updated to the network parameter of the current network layer; and the convolution layer, the pooling layer and the full-connection layer are all the network layers.

In one possible design, the determining to stop training according to the prediction result includes:

inputting the three-channel images in the verification set into the trained convolutional neural network model to obtain a prediction result as a current prediction result;

judging whether the current prediction result is within a preset evaluation range or not according to the current prediction result, and if the current prediction result is within the preset evaluation range, comparing the current prediction result with a historical prediction result to obtain a comparison result;

and if the comparison result is within the preset target result range, determining to stop training.

In one possible design, the testing the test convolutional neural network model according to the test set to obtain a test result includes:

inputting the three-channel image in the test set into the test convolutional neural network model to obtain the prediction result corresponding to each preset pavement sample image in the test set;

and calculating the coincidence rate of the crack identifications corresponding to the test set according to the prediction result corresponding to each preset pavement sample image in the test set and the real value corresponding to each preset pavement sample image in the test set, and taking the coincidence rate of the crack identifications as the test result.

In a second aspect, an embodiment of the present application provides a road surface crack detection device, including:

the road surface sample image acquisition module is used for acquiring a plurality of preset road surface sample images;

the preprocessing module is used for preprocessing the preset road surface sample images based on gamma correction to obtain three-channel images corresponding to each preset road surface sample image in the preset road surface sample images, and dividing the three-channel images corresponding to all the preset road surface sample images into a training set, a verification set and a test set;

the training module is used for training a convolutional neural network model according to the three-channel image in the training set, a preset loss function and a preset optimization algorithm to obtain the trained convolutional neural network model, and the convolutional neural network is the convolutional neural network model optimized through a residual error network;

the verification module is used for inputting the three-channel images in the verification set into the trained convolutional neural network model for prediction to obtain a prediction result;

the test convolutional neural network model determining module is used for taking the trained convolutional neural network model as a test convolutional neural network model when the training is determined to be stopped according to the prediction result;

the test module is used for testing the test convolutional neural network model according to the test set to obtain a test result;

the target convolutional neural network model determining module is used for taking the test convolutional neural network model as a target convolutional neural network model when the test result is within a preset target result range;

and the crack detection module is used for inputting the three-channel image corresponding to the road surface image to be detected into the target convolutional neural network model to obtain the road surface crack information in the road surface image to be detected.

In one possible design, the preprocessing module is specifically configured to:

setting the gamma value to be larger than a first preset value, and carrying out nonlinear transformation on the gray value corresponding to the high gray part in the multiple preset road surface sample images to obtain a second single-channel image corresponding to each preset original image in the multiple preset original images;

taking each preset original image in the multiple preset original images as a third single-channel image, and obtaining a three-channel image corresponding to each preset road surface sample image in the multiple preset road surface sample images according to a first single-channel image corresponding to each preset original image in the multiple preset original images and a second single-channel image corresponding to each preset original image in the multiple preset original images;

In a third aspect, an embodiment of the present application provides a road surface crack detection device, including: at least one processor and memory;

the memory stores computer-executable instructions;

the at least one processor executes computer-executable instructions stored by the memory to cause the at least one processor to perform the method of pavement crack detection as set forth in the first aspect and various possible designs of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method for detecting a road surface crack according to the first aspect and various possible designs of the first aspect is implemented.

According to the pavement crack detection method, the pavement crack detection device, the pavement crack detection equipment and the storage medium, a plurality of preset pavement sample images are obtained firstly, then the preset pavement sample images are preprocessed according to gamma correction, so that the difference between cracks in the preset pavement sample images and a background can be highlighted, and a three-channel image corresponding to each preset pavement sample image in the preset pavement sample images is obtained; dividing three-channel images corresponding to all preset pavement sample images into a training set, a verification set and a test set, in the training process, firstly training a convolutional neural network model according to the three-channel images, a preset loss function and a preset optimization algorithm in the training set to obtain the trained convolutional neural network model, wherein the convolutional neural network is the convolutional neural network model optimized through a residual error network, secondly, inputting the three-channel images in the verification set into the trained convolutional neural network model for prediction to obtain a prediction result, so as to be convenient for determining whether the training of the convolutional neural network model is stopped, if the training is determined to be stopped according to the prediction result, taking the trained convolutional neural network model as a test convolutional neural network model, and then testing according to the test convolutional neural network model, obtaining a test result, and conveniently determining whether the test convolutional neural network model is a target convolutional neural network model, namely if the test result is within a preset evaluation range, taking the test convolutional neural network model as the target convolutional neural network model; and finally, inputting a three-channel image corresponding to the road surface image to be detected into the target convolutional neural network model by using the target convolutional neural network model to obtain the road surface crack information in the road surface image to be detected. According to the scheme, in the pavement crack detection method, the pavement image under any environment is collected to serve as the preset pavement sample image, the preset pavement sample image is preprocessed, the crack part in the preset pavement sample image can be strengthened, then the optimized convolutional neural network model is trained, verified and tested according to the preprocessed preset pavement sample image, the target convolutional neural network model is obtained, the convolutional neural network model is continuously adjusted in a micro mode in the training process, and the performance is relatively stable. The scheme can ensure the recognition rate of cracks and save time and resources.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1 is a schematic flow chart of a pavement crack detection method according to an embodiment of the present disclosure;

fig. 2 is a schematic flow chart of a pavement crack detection method according to another embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of a convolutional neural network in a pavement crack detection method according to yet another embodiment of the present application;

fig. 4 is a schematic flow chart of a pavement crack detection method according to yet another embodiment of the present disclosure;

fig. 5 is a schematic flow chart of a pavement crack detection method according to another embodiment of the present disclosure;

fig. 6 is a schematic flow chart of a pavement crack detection method according to another embodiment of the present application;

fig. 7 is a schematic structural diagram of a pavement crack detection device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a pavement crack detection device provided in the embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the above-described drawings (if any) are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Fig. 1 is a schematic flow chart of a pavement crack detection method according to an embodiment of the present application, where an execution main body of the embodiment may be a terminal or a server, and the execution main body is not limited herein.

Referring to fig. 1, the pavement crack detection method includes:

s101, acquiring a plurality of preset road surface sample images.

The multiple preset pavement sample images comprise preset pavement sample images containing crack identifications and preset pavement sample images not containing crack identifications, each preset pavement sample image in the multiple preset pavement sample images corresponds to a true value, and the true value is an initial probability matrix containing the crack identifications in each preset pavement sample image.

In practical application, the execution main body of this embodiment can be a road surface crack detection system for to the detection or the discernment of road surface crack in the work of actual highway maintenance, through the image acquisition device among the road surface crack detection system, gather many road surface sample images and mark the part that contains the crack in many road surface sample images, obtain many and predetermine road surface sample images, carry out local detection, the processing speed is fast. The server can also acquire a plurality of preset road surface sample images through the acquisition device to perform remote detection, and the processing precision is high.

S102, preprocessing the multiple preset road surface sample images based on gamma correction to obtain three-channel images corresponding to each preset road surface sample image in the multiple preset road surface sample images, and dividing the three-channel images corresponding to all the preset road surface sample images into a training set, a verification set and a test set.

In this embodiment, a preprocessing model is constructed based on gamma correction, and the plurality of preset road surface sample images are input into the preprocessing model, so that a three-channel image corresponding to each preset road surface sample image in the plurality of preset road surface sample images can be obtained. By setting the gamma value in the preprocessing model, the gray value of each preset pavement sample image in the multiple preset pavement sample images can be subjected to nonlinear change, and the difference between the crack and the background in the preset pavement sample images can be highlighted.

In the training process, the convolutional neural network model is trained through three channels corresponding to all preset pavement sample images in the training set, then the convolutional neural network model is verified through three channels corresponding to all preset pavement sample images in the verification set, multiple rounds of training and verification can be performed, namely, the training set is trained once and then verified or predicted once, if the prediction effect is not good, the training set is trained once continuously, the verification set is verified or predicted once again, and by analogy, the convolutional neural network is trained for multiple rounds, so that pavement crack information output by the convolutional neural network model is more accurate.

S103, training a convolutional neural network model according to the three-channel image in the training set, a preset loss function and a preset optimization algorithm to obtain the trained convolutional neural network model, wherein the convolutional neural network is the convolutional neural network model optimized through a residual error network.

In this embodiment, before the convolutional neural network model is trained, a basic convolutional neural network model is constructed, and then the basic convolutional neural network model is optimized through a residual error network, so as to obtain the convolutional neural network model. The original information of an image (a preset road sample image or a road image to be detected) can be better retained by using the residual error network, the problem of gradient disappearance can be avoided under the condition that the convolutional neural network model is deep in hierarchy, and deep model training is obtained.

Specifically, the training set may be formed by 3024 three-channel images corresponding to the preset road surface sample image. Inputting a three-channel image corresponding to any one selected preset road surface sample image into a convolutional neural network model, effectively calculating the error between a prediction result and an actual value (true value) through a preset loss function, and providing a basis for adjusting network parameters in back propagation. In the training process, the convolutional neural network model is optimized through a preset optimization algorithm, and the adjustment of network parameters to fine adjustment every time can be guaranteed, so that the performance of the convolutional neural network model is stable.

And S104, inputting the three-channel image in the verification set into the trained convolutional neural network model for prediction to obtain a prediction result.

In this embodiment, the verification set may be formed by 500 three-channel images corresponding to the preset road surface sample image. Inputting the three-channel images in the verification set into the trained convolutional neural network model for prediction, and performing performance evaluation on the current convolutional neural network model verified at this time according to the obtained prediction result.

And S105, if the training is determined to stop according to the prediction result, taking the trained convolutional neural network model as a test convolutional neural network model.

In this embodiment, if the performance evaluation of the current convolutional neural network model reaches the preset standard, it is determined to stop training, and then the current convolutional neural network model, that is, the trained convolutional neural network model, is used as the test convolutional neural network model. The preset standard may be a preset evaluation range, and the preset evaluation range may be [0.95, 1 ].

And S106, testing the test convolutional neural network model according to the test set to obtain a test result.

In this embodiment, the three-channel image in the test set is input into the test convolutional neural network model to obtain a prediction result corresponding to each preset pavement sample image in the test set, and the performance of the test convolutional neural network model is evaluated according to the prediction result corresponding to each preset pavement sample image to obtain a test result. The test set can be formed by 10000 three-channel images corresponding to the preset pavement sample images, and the preset pavement sample images corresponding to the three-channel images in the test set can comprise the preset pavement sample images containing crack identifications and the preset pavement sample images not containing crack identifications, so that the collected images of the actually detected pavement can be simulated really, the performance of the test convolutional neural network model can be tested effectively, and the prediction of the test convolutional neural network model is more accurate.

And S107, if the test result is within a preset evaluation range, taking the test convolutional neural network model as a target convolutional neural network model.

In this embodiment, the test result may be a coincidence rate of a prediction result of the test concentrated preset pavement sample image and an actual result (true value) containing the crack identifier in the test concentrated preset pavement sample image, where the prediction result is a probability matrix containing the crack identifier in each preset pavement image, and the true value is an initial probability matrix containing the crack identifier in each preset pavement image. And if the coincidence rate is within [0.98, 1], indicating that the performance of the test convolutional neural network model reaches the standard, and taking the test convolutional neural network model as a target convolutional neural network model.

And S108, inputting a three-channel image corresponding to the road surface image to be detected into the target convolutional neural network model to obtain road surface crack information in the road surface image to be detected.

In this embodiment, a three-channel image corresponding to a road surface image to be detected is input into the target convolutional neural network model, and a probability matrix containing a crack identifier in the road surface image to be detected is output, where the road surface crack information may be directly the output of the target convolutional neural network model, that is, the probability matrix containing the crack identifier in the road surface image to be detected, or may be the probability matrix containing the crack identifier in the road surface image to be detected, which is output by the target convolutional neural network model, so as to obtain the crack identifier marked in the road surface image to be detected.

According to the pavement crack detection method provided by the embodiment, the pavement image under any environment is collected to serve as the preset pavement sample image, the preset pavement sample image is preprocessed, the crack part in the preset pavement sample image can be strengthened, then the optimized convolutional neural network model is trained, verified and tested according to the preprocessed preset pavement sample image, the target convolutional neural network model is obtained, the convolutional neural network model is continuously adjusted in a micro mode in the training process, and the performance is relatively stable. The scheme can ensure the recognition rate of cracks and save time and resources.

Fig. 2 is a schematic flow chart of a pavement crack detection method according to another embodiment of the present application, and this embodiment describes step S102 in detail based on the embodiment described in fig. 1. As shown in fig. 2, the preprocessing the multiple preset road surface sample images based on gamma correction to obtain three channel images corresponding to each preset road surface sample image in the multiple preset road surface sample images includes:

s201, setting the gamma value to be smaller than a first preset value, and carrying out nonlinear transformation on the gray values corresponding to the low gray parts in the multiple preset pavement sample images to obtain a first single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images.

In this embodiment, the gamma correction may perform nonlinear transformation on the gray scale value, and perform image correction by stretching part of the gray scale and compressing the other part of the gray scale. The most obvious effect of gamma correction is to remove the influence of partial illumination, and highlight the part of the original image (i.e. the preset image sample image or the road surface image to be detected) with unobvious image information due to over-strong or over-weak illumination.

Specifically, the first preset value is 1, when the gamma value (gamma value) is set to be less than 1, the low-gray level section in each preset road surface sample image is stretched, and other parts are compressed, so that the texture details of the low-gray level part are highlighted. And when the gray value corresponding to the low-gray part is a gray value in an interval of [0, 100], namely the gamma value is set to be less than 1, carrying out nonlinear transformation on the gray value in the interval of [0, 100] in the plurality of preset pavement sample images to obtain a first single-channel image corresponding to each preset pavement sample image in the plurality of preset pavement sample images.

S202, setting the gamma value to be larger than a first preset value, and carrying out nonlinear transformation on the gray values corresponding to the high gray parts in the multiple preset road surface sample images to obtain a second single-channel image corresponding to each preset road surface sample image in the multiple preset road surface sample images.

In the embodiment, similarly, when the gamma value is set to be greater than 1, the high-gray level section in each preset road surface sample image is stretched, and other parts are compressed, so that the texture detail of the high-gray level part is highlighted. And when the gray value corresponding to the high-gray part is a gray value in an interval of [100, 255], namely the gamma value is set to be greater than 1, carrying out nonlinear transformation on the gray values in the interval of [100, 255] in the multiple preset pavement sample images to obtain a second single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images.

S203, taking each preset pavement sample image in the multiple preset pavement sample images as a third single-channel image, and obtaining a three-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images according to a first single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images and a second single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images; the gray values corresponding to the low gray portions in the preset road surface sample images are smaller than a second preset value, and the gray values corresponding to the high gray portions in the preset road surface sample images are larger than the second preset value.

In this embodiment, each preset pavement sample image in the preset pavement sample images is taken as a third single-channel image, and the three-channel image corresponding to each preset pavement sample image in the preset pavement sample images is a first single-channel image corresponding to each preset pavement sample image in the preset pavement sample images, a second single-channel image corresponding to each preset pavement sample image in the preset pavement sample images, and each preset pavement sample image in the preset pavement sample images is taken as a third single-channel image. By adding the information of the input image, namely adding the first single-channel image and the second single-channel image on the basis of the third single-channel image, the recognition effect is improved.

In practical application, a preprocessing model is constructed based on gamma correction, the multiple preset pavement sample images are input into the preprocessing model, and a three-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images can be obtained.

Specifically, a) by setting the gamma value in the preprocessing model to be greater than 1, stretching the part (namely the gray value in the interval of [100, 255 ]) with higher gray value of each preset pavement sample image in the preset pavement sample images, so that the contrast of the high gray part is enhanced, and the difference between the white crack part and the background information is highlighted.

b) And stretching the part (the gray value in the interval of [0, 100 ]) with lower gray value of each preset pavement sample image in the preset pavement sample images by setting the gamma value in the preprocessing model to be less than 1, so that the contrast of the low gray part is enhanced, and the difference between the crack parts such as transverse cracks, longitudinal cracks, reticular cracks and the like and background information is highlighted. Wherein the background information may include texture details of the removed crack portion and corresponding gray values.

c) Inputting the two-channel images obtained in the two steps a) and b) into a convolutional neural network for training. When gamma correction is used, the preset road surface sample image is respectively preprocessed by using the gamma value more than 1 and less than 1.

Fig. 3 is a schematic structural diagram of a convolutional neural network in a pavement crack detection method according to yet another embodiment of the present application, and on the basis of the above embodiment, for example, on the basis of the embodiment shown in fig. 1, the present embodiment describes in detail a convolutional neural network model in step S103. The convolutional neural network model comprises: a convolutional layer, a pooling layer, and a full-link layer; the network structure in the convolutional neural network model comprises a plurality of combination layers, a convolutional layer and a full-connection layer which are connected in sequence, wherein each combination layer in the plurality of combination layers is formed by connecting two connected convolutional layers with one pooling layer; the size of the convolution kernel of the convolution layer in the first combination layer in the network structure is equal to the size of the convolution kernel of the second convolution layer and is larger than the sizes of the convolution kernels of other convolution layers in the network structure, and the sizes of the convolution kernels of the convolution layers in the combination layers corresponding to the other convolution layers in the network structure are equal to each other and are larger than the sizes of the convolution kernels of the convolution layers connected with the full-connection layer; the size of the kernel of the pooling layer in the network structure is smaller than the size of the convolution kernels of other convolution layers in the network structure; the size of a full-connection layer in the network structure is determined according to the size of the preset pavement sample image and the size of a convolution kernel of a convolution layer connected with the full-connection layer; initialized network parameters are set in the network structure, and the network parameters comprise weights and offsets.

In this embodiment, the network layers of the convolutional neural network model are convolutional layers, pooling layers and full-link layers, the number of convolutional layers is a third preset value, the number of pooling layers is a fourth preset value, the number of full-link layers is a fifth preset value, the third preset value is greater than the fourth preset value, and the fourth preset value is greater than the fifth preset value. The third preset value is 11, the fourth preset value is 5, and the fifth preset value is 1, that is, the convolutional neural network model identifies the road surface image by using a convolutional neural network with 11 convolutional layers, 5 pooling layers and 1 full-connection layer, wherein the output of the full-connection layer is classified into two categories, that is, the probability of containing the crack identifier and the probability of not containing the crack identifier.

Specifically, referring to fig. 3, the network structure in the convolutional neural network model is set as a plurality of combination layers, a convolutional layer and a fully-connected layer which are connected in sequence, that is, the network layer in the network structure sequentially passes through two convolutional layers and a pooling layer, and after the pooling layer is set, a convolutional layer and a fully-connected layer are sequentially set, wherein each combination layer in the plurality of combination layers is formed by connecting two connected convolutional layers with one pooling layer, and the number of the combination layers can be 5. Setting the size of the convolution kernel of the first convolution layer and the size of the convolution kernel of the second convolution layer in the network structure to be larger than the sizes of the convolution kernels of other convolution layers in the network structure, namely setting the size of the convolution kernel of the first convolution layer in the network structure to be 5 x 5 and the number of the corresponding convolution kernels to be 32, and setting the size of the convolution kernel of the second convolution layer in the network structure to be 5 x 5 and the number of the corresponding convolution kernels to be 32; the sizes of convolution kernels of other convolution layers in the network structure are 3 x 3, wherein the number of corresponding convolution kernels in the second combination layer is 64, the number of corresponding convolution kernels in the third combination layer is 128, the number of corresponding convolution kernels in the fourth combination layer is 256, and the number of corresponding convolution kernels in the fifth combination layer is 256; the convolution kernels of the convolutional layers connected to the full-link layer (i.e., the last convolutional layer in fig. 3) have a size of 1 × 1 and the number of corresponding convolution kernels is 1.

And setting the size of the core of the pooling layer in the network structure to be smaller than the sizes of the convolution cores of other convolution layers in the network structure, namely setting the size of the cores of all pooling layers in the network structure to be 2 x 2.

The size of the full-connection layer in the network structure is determined according to the size of the preset road surface sample image and the size of a convolution kernel of the convolution layer connected with the full-connection layer, wherein the size of the input three-channel image is 1088 x 704, the size of the full-connection layer in the network structure is determined after the convolution layer and the pooling layer are carried out for multiple times, and the full-connection layer is arranged in the network structure to obtain the convolution neural network model, namely the size of the full-connection layer is 34 x 22. Initializing parameters of each network layer in the network structure, the parameters of each network layer including a weight and a bias.

Fig. 4 is a schematic flow chart of a pavement crack detection method according to yet another embodiment of the present application, and based on the above-mentioned embodiment, for example, based on the embodiment shown in fig. 3, the present embodiment describes in detail a specific implementation process of step S103. As shown in fig. 4, the training of the convolutional neural network model according to the three-channel image in the training set and the preset loss function and the preset optimization algorithm includes:

s401, setting parameters of the convolutional neural network model, wherein the parameters of the convolutional neural network model comprise a learning rate and the number of training samples, and the training samples are three-channel images corresponding to each preset pavement sample image in the training set;

s402, selecting training samples corresponding to the number of the training samples from the training set according to the number of the training samples, inputting the training samples corresponding to the number of the training samples into the convolutional neural network model in batches, and obtaining a prediction result of each training sample in the training set;

s403, calculating an error between a prediction result of each training sample and a corresponding true value containing a crack identifier through the cross entropy function and the adaptive moment estimation optimization algorithm, wherein the prediction result of each training sample is a probability matrix containing the crack identifier in each training sample, and the true value is an initial probability matrix containing the crack identifier in each preset road sample image obtained when a plurality of preset road sample images are obtained;

s404, adjusting the network parameters of the current network layer through back propagation according to the gradient of the network parameters of each network layer in the network structure, the learning rate and the error, and updating the adjusted network parameters of the current network layer into the network parameters of the current network layer; and the convolution layer, the pooling layer and the full-connection layer are all the network layers.

In this embodiment, the preset loss function is a cross entropy function, and the cross entropy function can effectively calculate a difference (error) between a current prediction value and an actual value (true value), and does not provide a basis for adjusting network parameters in back propagation. The preset optimization algorithm is an adaptive moment estimation optimization algorithm Adam (adaptive moment estimation), and the adaptive moment estimation optimization algorithm can ensure that the modification quantity of the network parameters is stable every time and automatically prevent the network parameters from changing too much, so that the learning rate does not need to be manually adjusted in the training process. In order to facilitate the training of the convolutional neural network model, the optimization effect is good, and three-channel images in the training set all contain crack identifications.

Specifically, at the time of training, the learning rate of the convolutional neural network model is set to 10^-4And the number of training samples is 8 to 10, i.e. 8 or 9 or 10. Randomly selecting training samples corresponding to the set number from the training set according to the number of the training samples, inputting the training samples corresponding to the number of the training samples into the convolutional neural network model in batches, namely inputting all the training samples into the convolutional neural network model according to the number of the training samples in batches and in batches to obtain a prediction result of each training sample in the training set, namely a probability matrix containing crack identifications, and calculating an error between the prediction result of each training sample and a corresponding true value containing the crack identifications by combining with an initial probability matrix corresponding to each training sample. And adjusting the network parameters of the current network layer through back propagation according to the gradient of the network parameters of each network layer in the network structure, the learning rate and the errors, and updating the adjusted network parameters of the current network layer into the network parameters of the current network layer to be used as the network parameters of the convolutional neural network model used in the next round of training.

Fig. 5 is a schematic flow chart of a pavement crack detection method according to another embodiment of the present application, and a specific implementation process of this embodiment is described in detail on the basis of the above embodiment, for example, on the basis of any one of the embodiments shown in fig. 1 to 4. As shown in fig. 5, the determining to stop training according to the prediction result includes:

s501, inputting the three-channel images in the verification set into the trained convolutional neural network model to obtain a prediction result as a current prediction result;

s502, judging whether the current prediction result is within a preset evaluation range according to the current prediction result, and if the current prediction result is within the preset evaluation range, comparing the current prediction result with a historical prediction result to obtain a comparison result;

and S503, if the comparison result is within a preset target result range, determining to stop training.

In this embodiment, the trained convolutional neural network model is verified by a verification set, and the recognition effect of the trained convolutional neural network model is evaluated. Specifically, the three-channel images in the verification set are input into the prediction result obtained from the trained convolutional neural network model, the prediction result is used as the current prediction result, whether the current prediction result is within a preset evaluation range is judged according to the current prediction result, that is, whether the output result corresponding to each preset road surface sample image in the verification set is within a range of [0.95, 1], if the output result corresponding to any preset road surface sample image in the verification set is within a range of [0.95, 1], the current prediction result is compared with the historical prediction result, that is, the probability matrix containing the crack identifier and the initial probability matrix corresponding to the verification set are used for calculating the crack identifier coincidence rate, and the crack identifier coincidence rate is used as the comparison result. If the coincidence rate of the crack identifications is within the preset target result range, the preset target result range is [0.98, 1], namely the coincidence rate of the crack identifications is not less than 0.98, the convolutional neural network model after training is good in stability, and the training can be stopped.

Referring to fig. 6, fig. 6 is a schematic flow chart of a pavement crack detection method according to another embodiment of the present application, and this embodiment describes a specific implementation process of step S106 in detail based on the embodiment shown in fig. 4. As shown in fig. 6, the testing the test convolutional neural network model according to the test set to obtain a test result includes:

s601, inputting the three-channel image in the test set into the test convolutional neural network model to obtain the prediction result corresponding to each preset pavement sample image in the test set;

s602, calculating the coincidence rate of the crack identifications corresponding to the test set according to the prediction result corresponding to each preset pavement sample image in the test set and the real value corresponding to each preset pavement sample image in the test set, and taking the coincidence rate of the crack identifications as the test result.

In this embodiment, the testing process and the verifying process are similar in implementation principle, and a prediction result is obtained first, after the prediction result is obtained, whether the prediction result is within the preset evaluation range is determined, and if the current prediction result is within the preset evaluation range, the prediction result is compared with a corresponding true value, and a crack identifier coincidence rate corresponding to the test set is calculated, where the crack identifier coincidence rate (the result of the comparison) is used as the test result. The difference is that the test set in the test process comprises a preset pavement sample image containing a crack mark and a preset pavement image not containing a crack mark, the test process is more likely to be applied to pavement crack detection, through the test process, whether the trained convolutional neural network model is a target convolutional neural network model or not can be effectively detected, the recognition rate of cracks can be ensured, in the using process of a subsequent target convolutional neural network model, pavement crack information of the pavement image to be detected can be output by directly inputting the pavement image to be detected, the test process is convenient and fast, and meanwhile, time and resources are saved.

According to the scheme, in the process of detecting the pavement cracks, the target convolutional neural network model is obtained through image preprocessing and training, verification and testing of the convolutional neural network model, the convolutional neural network model is continuously adjusted in a micro-adjusting mode in the training process, and the performance is relatively stable. The scheme can ensure the recognition rate of cracks and save time and resources.

Fig. 7 is a schematic structural diagram of a pavement crack detection device according to an embodiment of the present application. As shown in fig. 7, the road surface crack detection device 70 includes: the system comprises a road surface sample image acquisition module 701, a preprocessing module 702, a training module 703, a verification module 704, a test convolutional neural network model determination module 705, a test module 706, a target convolutional neural network model determination module 707 and a crack detection module 708. The road surface sample image acquiring module 701 is used for acquiring a plurality of preset road surface sample images; the preprocessing module 702 is configured to preprocess the multiple preset road surface sample images based on gamma correction, obtain a three-channel image corresponding to each preset road surface sample image in the multiple preset road surface sample images, and divide the three-channel images corresponding to all preset road surface sample images into a training set, a verification set, and a test set; a training module 703, configured to train the optimized convolutional neural network model according to the three-channel image in the training set, a preset loss function, and a preset optimization algorithm; the verification module 704 is used for inputting the three-channel images in the verification set into the trained convolutional neural network model for prediction to obtain a prediction result; a test convolutional neural network model determining module 705, configured to use the trained convolutional neural network model as a test convolutional neural network model when it is determined according to the prediction result that training is stopped; the test module 706 is configured to test the test convolutional neural network model according to the test set to obtain a test result; a target convolutional neural network model determining module 707, configured to use the test convolutional neural network model as a target convolutional neural network model when the test result is within a preset evaluation range; and the crack detection module 708 is configured to input a three-channel image corresponding to the road surface image to be detected into the target convolutional neural network model, so as to obtain road surface crack information in the road surface image to be detected.

The apparatus provided in this embodiment may be used to implement the technical solutions of the above method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.

In one possible design, the preprocessing module 702 is specifically configured to: setting the gamma value to be smaller than a first preset value, and carrying out nonlinear transformation on the gray values corresponding to the low gray parts in the multiple preset pavement sample images to obtain a first single-channel image corresponding to each preset pavement sample image in the multiple preset pavement sample images; setting the gamma value to be larger than a first preset value, and carrying out nonlinear transformation on the gray value corresponding to the high gray part in the multiple preset road surface sample images to obtain a second single-channel image corresponding to each preset original image in the multiple preset original images; taking each preset original image in the multiple preset original images as a third single-channel image, and obtaining a three-channel image corresponding to each preset road surface sample image in the multiple preset road surface sample images according to a first single-channel image corresponding to each preset original image in the multiple preset original images and a second single-channel image corresponding to each preset original image in the multiple preset original images; the gray values corresponding to the low gray portions in the preset road surface sample images are smaller than a second preset value, and the gray values corresponding to the high gray portions in the preset road surface sample images are larger than the second preset value.

In one possible design, the convolutional neural network model includes: a convolutional layer, a pooling layer, and a full-link layer; the network structure in the convolutional neural network model comprises a plurality of combination layers, a convolutional layer and a full-connection layer which are connected in sequence, wherein each combination layer in the plurality of combination layers is formed by connecting two connected convolutional layers with one pooling layer; the size of the convolution kernel of the convolution layer in the first combination layer in the network structure is equal to the size of the convolution kernel of the second convolution layer and is larger than the sizes of the convolution kernels of other convolution layers in the network structure, and the sizes of the convolution kernels of the convolution layers in the combination layers corresponding to the other convolution layers in the network structure are equal to each other and are larger than the sizes of the convolution kernels of the convolution layers connected with the full-connection layer; the size of the kernel of the pooling layer in the network structure is smaller than the size of the convolution kernels of other convolution layers in the network structure; the size of a full-connection layer in the network structure is determined according to the size of the preset pavement sample image and the size of a convolution kernel of a convolution layer connected with the full-connection layer; initialized network parameters are set in the network structure, and the network parameters comprise weights and offsets.

In one possible design, the training module 703 is specifically configured to: setting parameters of the convolutional neural network model, wherein the parameters of the convolutional neural network model comprise a learning rate and the number of training samples, and the training samples are three-channel images corresponding to each preset pavement sample image in the training set; selecting training samples corresponding to the number of the training samples from the training set according to the number of the training samples, inputting the training samples corresponding to the number of the training samples into the convolutional neural network model in batches, and obtaining a prediction result of each training sample in the training set; calculating an error between a prediction result of each training sample and a corresponding true value containing a crack identifier through the cross entropy function and the adaptive moment estimation optimization algorithm, wherein the prediction result of each training sample is a probability matrix containing the crack identifier in each training sample, and the true value is an initial probability matrix containing the crack identifier in each preset road sample image obtained when a plurality of preset road sample images are obtained; according to the gradient of the network parameter of each network layer in the network structure, the learning rate and the error, the network parameter of the current network layer is adjusted through back propagation, and the adjusted network parameter of the current network layer is updated to the network parameter of the current network layer; and the convolution layer, the pooling layer and the full-connection layer are all the network layers.

In one possible design, the test convolutional neural network model determining module 705 is specifically configured to: inputting the three-channel images in the verification set into the trained convolutional neural network model to obtain a prediction result as a current prediction result; judging whether the current prediction result is within a preset evaluation range or not according to the current prediction result, and comparing the current prediction result with a historical prediction result when the current prediction result is within the preset evaluation range to obtain a comparison result; and when the comparison result is within a preset target result range, determining to stop training.

In one possible design, the test module 706 is specifically configured to: inputting the three-channel image in the test set into the test convolutional neural network model to obtain the prediction result corresponding to each preset pavement sample image in the test set; and calculating the coincidence rate of the crack identifications corresponding to the test set according to the prediction result corresponding to each preset pavement sample image in the test set and the real value corresponding to each preset pavement sample image in the test set, and taking the coincidence rate of the crack identifications as the test result.

Fig. 8 is a schematic structural diagram of a pavement crack detection device provided in the embodiment of the present application. As shown in fig. 8, the road surface crack detecting apparatus 80 of the present embodiment includes: a processor 801 and a memory 802; a memory 802 for storing computer-executable instructions; the processor 801 is configured to execute the computer-executable instructions stored in the memory to implement the steps performed by the receiving device in the above embodiments. Reference may be made in particular to the description relating to the method embodiments described above.

The embodiment of the application also provides a computer-readable storage medium, wherein a computer executing instruction is stored in the computer-readable storage medium, and when a processor executes the computer executing instruction, the pavement crack detection method is realized.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form. In addition, functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware form, and can also be realized in a form of hardware and a software functional unit.

The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application. It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.

The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus. The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A pavement crack detection method is characterized by comprising the following steps:

acquiring a plurality of preset pavement sample images;

if the test result is within a preset target result range, taking the test convolutional neural network model as a target convolutional neural network model;

inputting a three-channel image corresponding to a pavement image to be detected into the target convolutional neural network model to obtain pavement crack information in the pavement image to be detected;

the preprocessing is carried out on the multiple preset pavement sample images based on gamma correction to obtain three-channel images corresponding to each preset pavement sample image in the multiple preset pavement sample images, and the method comprises the following steps:

2. The method of claim 1, wherein the convolutional neural network model comprises: a convolutional layer, a pooling layer, and a full-link layer;

3. The method according to claim 2, wherein the preset loss function is a cross entropy function, the preset optimization algorithm is an adaptive moment estimation Adam optimization algorithm, and the three-channel images in the training set all contain crack identifications; the training of the convolutional neural network model according to the three-channel image, the preset loss function and the preset optimization algorithm in the training set comprises the following steps:

4. The method according to any one of claims 1-3, wherein said determining to stop training based on said prediction comprises:

5. The method of claim 3, wherein said testing said test convolutional neural network model according to said test set to obtain a test result comprises:

inputting the three-channel image in the test set into the test convolutional neural network model to obtain a prediction result corresponding to each preset pavement sample image in the test set;

6. A pavement crack detection device, characterized by comprising:

the crack detection module is used for inputting a three-channel image corresponding to the pavement image to be detected into the target convolutional neural network model to obtain pavement crack information in the pavement image to be detected;

the preprocessing module is specifically configured to:

7. A pavement crack detection apparatus, comprising: at least one processor and memory;

the memory stores computer-executable instructions;

the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the pavement crack detection method of any of claims 1 to 5.

8. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, implement the road surface crack detection method according to any one of claims 1 to 5.