CN114882474A - Road disease detection method and system based on convolutional neural network - Google Patents

Abstract

The invention belongs to the technical field of road construction, and provides a road disease detection method and a system based on a convolutional neural network, wherein a shadow removal module based on a generated countermeasure network removes shadows of a road disease image to be detected; detecting and obtaining road disease types based on the image after shadow removal and the target detection model; the construction process of the target detection model comprises the following steps: adopting a Yolov5 target detection network fused with a convolution attention module to respectively execute an attention mechanism on the channel and space dimensions and extract characteristic graphs with different dimensions; based on the idea of bidirectional feature fusion, the feature graphs of different dimensions are subjected to weighted fusion by adopting a self-adaptive feature fusion method to obtain a fused feature graph. The method solves the defects of the traditional road disease detection scheme, and obviously improves the detection precision.

Description

Method and system for road disease detection based on convolutional neural network

technical field

The invention belongs to the technical field of road construction, and in particular relates to a road disease detection method and system based on a convolutional neural network.

Background technique

The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

With the rapid development of road construction, the importance of road disease detection has become increasingly prominent. Timely and accurate access to road disease information can save huge costs for road maintenance and reduce the possibility of road traffic accidents.

Traditional road disease detection mainly relies on inspectors to collect road disease data by means of parking inspections, photographing records, and manual measurement.

However, the existing problems are: on the one hand, high labor cost, low detection efficiency, and poor safety; on the other hand, the data is not objective and cannot effectively manage spatial information, which does not meet the requirements of modern road inspection management. In recent years, computer vision methods and deep learning algorithms have been increasingly applied to the field of road disease detection, which has significantly improved the industry level. However, the current algorithm still has shortcomings such as being greatly affected by noise and poor model stability, and the current algorithm needs to be optimized.

SUMMARY OF THE INVENTION

In order to solve at least one technical problem existing in the above background art, the present invention provides a road disease detection method and system based on a convolutional neural network, which solves the shortcomings of the traditional road disease detection scheme, and has high detection accuracy. obvious improvement.

In order to achieve the above object, the present invention adopts the following technical solutions:

A first aspect of the present invention provides a road disease detection method based on a convolutional neural network, comprising the following steps:

Obtain the image of the road disease to be detected;

The shadow removal module based on generative adversarial network removes the shadow of the road disease image to be detected;

The disease type of the road is detected based on the image after removing the shadow and the target detection model; wherein, the construction process of the target detection model is as follows: using the Yolov5 target detection network fused with the convolutional attention module, executes in the channel and space dimensions respectively. Attention mechanism to extract feature maps of different dimensions;

Based on the idea of two-way feature fusion, the adaptive feature fusion method is used to weight the feature maps of different dimensions to obtain the fused feature map, and the classification results of road diseases are obtained based on the feature recognition based on the fused feature map.

A second aspect of the present invention provides a road disease detection system based on a convolutional neural network, including:

A data acquisition module, used to acquire images of road diseases to be detected;

The shadow removal module is used to remove the shadow of the road disease image to be detected based on the shadow removal module of the generative adversarial network;

The road disease detection module is used to detect the type of road disease based on the image after removing the shadow and the target detection model; wherein, the construction process of the target detection model is: using the Yolov5 target detection network fused with the convolutional attention module, respectively in The attention mechanism is performed on the channel and spatial dimensions, and feature maps of different dimensions are extracted;

Based on the idea of two-way feature fusion, the adaptive feature fusion method is used to weight the feature maps of different dimensions to obtain the fused feature map; the feature recognition based on the fused feature map is used to obtain the classification result of road diseases.

A third aspect of the present invention provides a computer-readable storage medium.

A computer-readable storage medium on which a computer program is stored, when the program is executed by a processor, implements the steps in the above-mentioned method for detecting road diseases based on a convolutional neural network.

A fourth aspect of the present invention provides a computer apparatus.

A computer device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, when the processor executes the program, the above-mentioned road disease detection based on a convolutional neural network is realized steps in the method.

Compared with the prior art, the beneficial effects of the present invention are:

The present invention aims to solve the problem that the original Yolov5 model has poor robustness for road disease detection of different sizes during the target detection process of road diseases, especially the problem of over-focusing on small objects, such as road potholes with a diameter of less than 30 mm, the target does not belong to the road The scope of the disease, on the contrary, increases the workload. A method of fusing the attention module is proposed to perform the attention mechanism in the channel and spatial dimensions respectively, and the detection confidence of the correct category is generally higher than that of the original Yolov5 model.

The invention aims to solve the problem that the detection effect is easily disturbed by image shadows during the target detection process of road diseases, especially crack diseases are easily confused with tree shadows, so as to reduce the probability of missed detection and misjudgment. A single-image shadow removal method for generative adversarial networks based on channel attention is designed, and a shadow removal network is designed.

Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will become apparent from the description which follows, or may be learned by practice of the invention.

Description of drawings

The accompanying drawings forming a part of the present invention are used to provide further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation of the present invention.

1 is a schematic flowchart of a method for detecting road diseases based on a convolutional neural network according to an embodiment of the present invention;

Fig. 2(a)-Fig. 2(d) are a sample library of road disease information according to an embodiment of the present invention;

Fig. 3 (a)-Fig. 3 (d) is the example picture of shadow training data set of the embodiment of the present invention;

4(a)-FIG. 4(d) are shaded images to be tested according to an embodiment of the present invention;

Fig. 5(a)-Fig. 5(d) are the model test effects after the shadow is removed according to the embodiment of the present invention.

Fig. 6(a)-Fig. 6(j) are visualization results of the model training process after the improvement of the embodiment of the present invention.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and embodiments.

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

It should be noted that the terminology used herein is for the purpose of describing specific embodiments only, and is not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural as well, furthermore, it is to be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates that There are features, steps, operations, devices, components and/or combinations thereof.

The overall idea of the present invention is:

First, based on the two perspectives of vehicle-mounted camera and UAV aerial photography, the road disease image data set was collected and labeled, and an image sample library containing four types of road disease information was established; secondly, based on the Yolov5 target detection algorithm, the fusion-based generative confrontation network The shadow removal algorithm, attention mechanism and feature fusion module of the proposed method construct a better target detection model.

The invention solves the shortcomings of the traditional road disease detection scheme, and has a significant improvement in detection accuracy. The research results are mainly applied in the intelligent construction of road maintenance informatization, timely detection of road diseases, extension of roads and the use of related auxiliary facilities Longevity, and promote the innovation of the "Internet + road management" model.

Example 1

As shown in FIG. 1 , this embodiment provides a method for detecting road diseases based on a convolutional neural network, including the following steps:

Step 1: Obtain the road disease image to be detected;

Step 2: The shadow removal module based on the generative adversarial network removes the shadow of the road disease image to be detected;

Step 3: Detect the type of road disease based on the shadow-removed image and the target detection model;

Wherein, the construction process of the target detection model is as follows: adopt the Yolov5 target detection network fused with the convolutional attention module, perform the attention mechanism in the channel and space dimensions respectively, and extract feature maps of different dimensions;

Based on the idea of two-way feature fusion, the adaptive feature fusion method is used to weight the feature maps of different dimensions to obtain the fused feature map;

Based on the three feature maps of different resolutions obtained by fusion, the anchor boxes are used respectively, and the anchor boxes are regressed based on the confidence threshold and non-maximum suppression (NMS) methods, and finally generated with class probability, confidence score and The output vector of the target bounding box realizes the classification and localization of road disease information.

As one or more embodiments, in step 1, a comprehensive and standardized data set is the basis for deep learning model training. Aiming at the problem of road disease detection, this embodiment establishes a sample library of dual-view and four types of road disease information; for the problem of image shadow processing, a more targeted image shadow data set is organized.

As shown in Figure 2(a)-Figure 2(d), the detection targets are divided into four types of road diseases: longitudinal cracks, transverse cracks, moiré cracks and road potholes.

The dataset is divided into two perspectives, vehicle-mounted camera (HDV) and unmanned aerial vehicle (UAV), to provide greater flexibility for road maintenance work.

In addition, for the problem of shadow influence, this embodiment establishes an image shadow training data set including four parts: no shadow picture, shadow picture, shadow part mask, and shadow part edge, as shown in Figure 3(a)-Figure 3(d) ) shown.

As one or more embodiments, in step 2, the shadow removal module of the generative adversarial network includes a shadow detector, a shadow detection discriminator, a shadow remover, and a shadow removal discriminator;

The shadow detector and shadow remover adopt UNet++ network, which is composed of up-sampling, down-sampling and multiple nodes, each node is a convolution layer, batch normalization layer, Mish activation function and scSE module. residual block.

The shadow detector is provided with an additional structure after the network of UNet++, and the additional structure is composed of a 3×3 convolution layer, a batch normalization layer, an LReLU activation function and a Sigmoid activation function, which realizes the output shadow mask limit range, Limits the output to the range 0 to 1.

The shadow detection discriminator is stacked into four channels according to the marked shadow mask, the input image, and the shadow mask output by the shadow detector, and determines whether it is a shadow rather than a crack-like disease.

The shadow remover is attached with the structure ColorBlock after the network of UNet++. Since the shadow is strongly affected by the light wavelength, the light intensity that the camera can obtain varies according to the color of the target. Therefore, during the training process, attention should be paid to each color channel and its The relationship between. ColorBlock uses a fully connected layer to estimate the weight of each channel, which makes up for the defect that ordinary convolutional layers cannot effectively train the relationship between channels, so as to effectively train the physical properties of each wavelength and learn the relationship between the input image and the ground truth image. difference.

The removal process of the shadow remover includes:

According to the shadow model, the light intensity of any position and the light intensity of the shadow area are represented;

Obtain the difference between the input image and the ground truth image based on the light intensity at any location and the light intensity in the shaded area;

Removing shadows based on the difference between the input image and the ground truth image results in a shadow-removed image.

Wherein, the light intensity of any position and the light intensity of the shadow area represented by the shadow model are specifically:

Based on shadow model: I(x,λ)=L(x,λ)R(x,λ)

Among them, I is the light intensity, L is the illuminance, R is the reflectivity, and I, L, and R depend on the position x and wavelength λ of a certain point on the image.

From this we get:

The light intensity I ^lit at a point in the non-shaded area is expressed as:

I ^lit (x,λ)=L ^d (x,λ)R(x,λ)+L ^a (x,λ)R(x,λ)

The light intensity I ^shadow of a point in the shadow area is expressed as:

I ^shadow (x,λ)=L ^a (x,λ)R(x,λ)

In the formula, L ^d represents the illuminance of direct lighting, and ^La represents the illuminance of indirect lighting.

The difference between a point on the input image and the corresponding point on the ground truth image is expressed as:

Δ=I _gt -I _input

=P(I ^lit (x,λ)-I ^shadow (x,λ))

≈I ^lit (x,λ)-I ^shadow (x,λ)

=L ^d (x,λ)R(x,λ)

Among them, the function P represents the image processing of the camera image acquisition system, I _gt represents the ground truth image (ie, the image without shadow), and I _input represents the input image (ie, the image with shadow).

的估计。Since the image has only three color channels, R, G, and B, λ in the above formula can be approximately expressed as a function of R, G, and B. ColorBlock has the function of weighting each color channel with a specific value, which can be understood as the weighting of each color channel. The illuminance of the direct illumination of the three color channels of the image

's estimate.

Assuming that the direct light intensity from the light source is constant for the entire image, and L ^d is independent of the position x, the result Δ can be expressed as:

The shadow removal discriminator is composed of a convolution layer, a batch normalization layer and a Mish activation function layer, these layers are connected in sequence, the stride of the convolution layer is set, and the image after shadow removal is output.

The stride of the convolutional layer can be set according to actual requirements. In this embodiment, the stride of the convolutional layer is set to 2 between every two convolutional layers.

The loss function of the shadow removal discriminator can be expressed as:

I _input is the shadow image, M _gt is the shadow mask image, I _gt is the shadowless image, I _output is the unshaded image output by the shadow remover, V _real is a random value matrix with an average value of 0.5, and V _fake is a A matrix of random values with mean -0.5 that follows a Gaussian distribution.

The advantage of the above technology is that in the process of target detection of road diseases, in order to solve the problem that the detection effect is easily disturbed by image shadows, especially crack diseases are easily confused with the shadows of branches, in order to reduce missed detection, missed judgment and false detection. The probability of judgment is based on the channel attention-based generative adversarial network single image shadow removal method (CANet), and a shadow removal network is designed.

As one or more embodiments, in step 3, the Convolutional Block Attention Module (CBAM) includes two sub-modules, a channel attention (CAM) and a spatial attention (SAM), respectively, in the channel and The attention mechanism is executed in the spatial dimension.

In the process of target detection of road diseases, the original Yolov5 model has poor robustness for road disease detection of different sizes, especially the problem of over-focusing on small objects, such as road potholes with a diameter of less than 30 mm, which do not belong to the category of road diseases. , which in turn increases the workload.

The implementation of the attention mechanism in the channel and space dimensions respectively includes:

When compressing the dimension of the feature map F ₀ (H×W×C) extracted by the general convolutional layer, both average pooling and maximum pooling are used to obtain two one-dimensional feature maps F _1,2 (1×1 ×C), send F _{1 and 2} into a two-layer shared neural network (MLP) respectively, the features output by MLP are added based on element-wise, and finally through the sigmoid activation operation, the final channel attention feature is generated Mc;

Perform an element-wise multiplication operation on the channel attention feature Mc and the input feature map F ₀ to obtain F ₃ as the input feature of the SAM module. F ₃ performs global maximum pooling and global average pooling based on the channel, and obtains two one Dimensional feature map F _4,5 (H×W×1), the two feature maps are channel spliced, and a 7×7 convolution operation is used to achieve dimensionality reduction, and finally through the sigmoid activation operation, the spatial attention feature Ms is generated .

Finally, an element-wise multiplication operation is performed on the output Ms _of the spatial attention module and the input feature map F3 to obtain the final generated features.

Spatial attention aims to improve the feature expression of key regions. It essentially transforms the spatial information in the original image into another space and retains the key information through a spatial transformation module, generates a weight mask for each position, and Weight the output so that specific target regions of interest are enhanced while attenuating irrelevant background regions.

In this embodiment, in order not to change the network structure as much as possible, that is, in order to make more use of pre-trained parameters and let the model training process converge as soon as possible, the attention module is added outside the block.

In this embodiment, the C3 modules in the Yolov5 backbone network, which are mainly responsible for extracting residual features, are all replaced with "convolution fusion attention + C3" modules.

As one or more embodiments, in step 3, the weighted fusion of feature maps of different dimensions using an adaptive feature fusion method specifically includes:

Considering that only simple stacking and stacking operations cannot make full use of the feature maps of different stages, in order to better fuse and extract the features output by the backbone network and fully characterize the features of different sizes, this embodiment uses the original feature extraction of Yolov5. Modules were improved.

Based on the idea of two-way feature fusion, each layer of Adaptive Spatial Feature Fusion (ASFF) performs weighted fusion of the stages of the original feature structure. The fusion of different stage features uses an attention mechanism to control other stages. The contribution of the stage to the features of this stage.

Algorithm verification

This embodiment is implemented based on the Pytorch deep learning framework and uses the hardware platform GeForce RTX 2080Ti. The experiment uses the Yolov5s pre-training model. Since some pre-training parameters are not available for changing the network structure, it is necessary to train more rounds without overfitting.

The model training strategy is shown in Table 1:

Table 1 Model training strategy

Various evaluation criteria were used in the experiments:

Accuracy (Accuracy, Acc):

Precision (Precision, P):

Recall (Recall, R):

Among them, TP is the number of positive classes determined as positive classes, FP is the number of positive classes determined by negative classes, FN is the number of positive classes determined as negative classes, and TN is the number of negative classes determined by negative classes.

Average accuracy (Average Precision, AP):

AP is obtained on the basis of the Precision-Recall curve, where r1, r2...rn are the Recall values corresponding to the first interpolation of the Precision interpolation segment in ascending order.

Mean Average Precision (mAP) for all categories:

mAP_0.5:0.95 means to take the average value of IoU from 0.5 to 0.95, mAP_0.5 means to take the case of IoU of 0.5.

1. Image shadow processing effect

Using the SRD+ pre-training model, 1055 branch shadow images and 1398 ISTD open source images are used to form dataset 4 for training. Figure 4(a)-Figure 4(d) shows the shadowed image, and Figure 5(a)-Figure 5(d) shows the model test effect after removing the shadow.

It can be seen from the experiments that the shadow removal algorithm model based on the adversarial neural network can effectively remove the shadow of the image.

2. The effect of Yolov5 algorithm improvement

(1) In order to test the improvement effect of the model, 3000 vehicle-mounted perspective pictures were selected to form Dataset 1 for training. A total of 25 groups of experiments were designed for different improvement methods, and the training results are shown in Table 2:

Table 2 Dataset 1 training results

It can be seen from the experiments that after adding the CBAM attention mechanism and replacing the ASFF feature extraction module, Yolov5 has the best improvement effect, and the model training mAP increases by 3.5%.

(2) In order to improve the generalization performance of the model, a total of 14,055 images from multiple perspectives are selected to form dataset 2 for training. The training results are shown in Table 3, and the visualization of the improved model training process is shown in Figure 6(a)-Figure 6(j):

Table 3 Dataset 2 training results

Methods metrics/mAP_0.5 metrics/mAP_0.5:0.95 Initial Yolov5 0.973 0.748 Improved Yolov5 0.987 0.794

It can be seen from the experiments that the Yolov5 model using the improved method of ASFF+CBAM has a 4.6% increase in mAP compared to the original Yolov5 algorithm.

(3) In order to adapt to the actual road condition detection application, 100 multi-view road disease pictures were collected to form Data Set 3 for testing. The test results are shown in Table 4.

Table 4 Dataset 3 test results

Methods Accuracy Original Yolov5 0.86 Improved Yolov5 0.94

It can be seen from the experiment that the original Yolov5 model is prone to over-focusing on small objects during the detection process, such as road potholes with a diameter of less than 30 mm. The detection target does not constitute a road disease, on the contrary, it increases the workload. The improved Yolov5 model solves the problem very well. This problem; at the same time, the detection confidence of the improved Yolov5 model is generally higher than that of the original Yolov5 model in the detection confidence of the correct category.

Due to the overlap and complexity of the classification of road diseases, the model has a small number of misjudgments in the process of target detection.

3. Yolov5 detection model that introduces image shadow processing

In order to test the performance improvement effect of the target detection model after shadow processing, a dataset 5 of 100 images with shadows from different angles is made for testing. The test results are shown in Table 5.

Table 5 Dataset 3 test results

Methods Accuracy Yolov5 algorithm detects the image before shadow removal 0.83 Yolov5 algorithm detects the image after shadow removal 0.90 After improvement, Yolov5 detects the image before shadow removal 0.86 The improved Yolov5 detects the image after removing the shadow 0.94

It can be seen from the experiments that the images with the shadow interference removed significantly reduce the probability of misjudgment in road disease detection, and can find the location of road diseases more efficiently and accurately, which greatly improves the detection of four types of road diseases.

To sum up, compared with the traditional road disease detection algorithm, the new road disease detection algorithm proposed by the present invention has significantly improved detection accuracy. The detection results show that the solution of the present invention can accurately identify cracks with a width greater than 5 mm and potholes with a diameter greater than 50 mm, which meets the actual needs of the current road census.

Embodiment 2

This embodiment provides a road disease detection system based on a convolutional neural network, including:

A data acquisition module, used to acquire images of road diseases to be detected;

The shadow removal module is used to remove the shadow of the road disease image to be detected based on the shadow removal module of the generative adversarial network;

The road disease detection module is used to detect the type of road disease based on the image after removing the shadow and the target detection model; wherein, the construction process of the target detection model is: using the Yolov5 target detection network fused with the convolutional attention module, respectively in The attention mechanism is performed on the channel and spatial dimensions, and feature maps of different dimensions are extracted;

Based on the idea of two-way feature fusion, the adaptive feature fusion method is used to weight the feature maps of different dimensions to obtain the fused feature map; the feature recognition based on the fused feature map is used to obtain the classification result of road diseases.

Embodiment 3

This embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps in the above-mentioned method for detecting road diseases based on a convolutional neural network.

Embodiment 4

This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and running on the processor, when the processor executes the program, the above-mentioned convolutional neural-based computer program is implemented. Steps in a networked road disease detection method.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. During execution, the processes of the embodiments of the above-mentioned methods may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. The road disease detection method based on the convolutional neural network is characterized by comprising the following steps of:

acquiring a road disease image to be detected;

removing the shadow of the road disease image to be detected based on a shadow removal module for generating a countermeasure network;

detecting and obtaining road disease types based on the image after shadow removal and the target detection model; the construction process of the target detection model comprises the following steps: adopting a Yolov5 target detection network fused with a convolution attention module to respectively execute an attention mechanism on the channel and space dimensions and extract characteristic graphs with different dimensions;

based on the idea of feature bidirectional fusion, the feature maps with different dimensions are subjected to weighted fusion by adopting a self-adaptive feature fusion method to obtain a fused feature map, and feature identification is carried out based on the fused feature map to obtain a classification result of the road diseases.

2. The convolutional neural network-based road disease detection method of claim 1, wherein the performing an attention mechanism in channel and spatial dimensions, respectively, specifically comprises:

the method comprises the steps of simultaneously introducing average pooling and maximum pooling when dimension compression is carried out on an original feature map to obtain two one-dimensional feature maps, respectively sending the two one-dimensional feature maps into a two-layer shared neural network, and carrying out addition operation to generate channel attention features;

and multiplying the channel attention characteristic and the original characteristic graph to obtain a third characteristic graph, performing global maximum pooling and global average pooling on the basis of the channel to obtain two one-dimensional characteristic graphs, splicing the two one-dimensional characteristic graphs, and reducing the dimension by using convolution operation to generate the space attention characteristic.

3. The convolutional neural network-based road disease detection method as claimed in claim 1, wherein the shadow removal module based on the generation countermeasure network removes shadows of the road disease image to be detected,

the shadow removal module generating the countermeasure network includes a shadow eliminator, the shadow eliminator elimination process including:

representing the light intensity of any position and the light intensity of a shadow area according to a shadow model;

obtaining a difference between the input image and the ground truth image based on the light intensity of the arbitrary position and the light intensity of the shadow area;

the elimination of the shadow based on the difference between the input image and the ground truth image results in a shadow-removed image.

4. The convolutional neural network-based road disease detection method as claimed in claim 3, wherein the shadow canceller adopts a network structure of UNet + +, and is composed of upsampling, downsampling and a plurality of nodes, each node is a residual block composed of a convolutional layer, a batch normalization layer, a Mish activation function and a scSE module, an additional structure ColorBlock is arranged behind the network of UNet + +, and the weight of each color channel of the image is estimated by using a full connection layer.

5. The convolutional neural network-based road disease detection method of claim 3, wherein the difference between the input image and the ground truth image is represented as:

Δ＝I _gt -I _input

＝P(I ^lit (x,λ)-I ^shadow (x,λ))

≈I ^lit (x,λ)-I ^shadow (x,λ)

＝L ^d (x,λ)R(x,λ)

in the formula I _gt Representing an unshaded image, I _input Representing a shadowed image, a function P representing the image processing of the camera image acquisition system, I ^lit Is the light intensity at position x, I ^shadow Light intensity of the shaded area, L ^d The illuminance of direct illumination is shown, R is the reflectance, and λ is the wavelength.

6. The convolutional neural network-based road disease detection method of claim 1, wherein the types of road diseases include four types of road diseases, namely longitudinal cracks, transverse cracks, tortoiseshell cracks and road pits.

7. The convolutional neural network-based road disease detection method of claim 1, wherein the shadow removal module further comprises a shadow detector that sets an additional structure after the UNet + + network to limit the range of the output shadow mask to be in the range of 0 to 1.

8. Road disease detecting system based on convolutional neural network, its characterized in that includes:

the data acquisition module is used for acquiring a road disease image to be detected;

the shadow removing module is used for removing the shadow of the road disease image to be detected based on the shadow removing module for generating the confrontation network;

the road disease detection module is used for detecting and obtaining the type of the road disease based on the image after the shadow is removed and the target detection model; the construction process of the target detection model comprises the following steps: adopting a Yolov5 target detection network fused with a convolution attention module to respectively execute an attention mechanism on the channel and space dimensions and extract characteristic graphs with different dimensions;

based on the thought of bidirectional feature fusion, the feature maps with different dimensions are subjected to weighted fusion by adopting a self-adaptive feature fusion method to obtain a fused feature map, and feature recognition is carried out based on the fused feature map to obtain a classification result of the road diseases.

9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the convolutional neural network-based road disease detection method according to any one of claims 1 to 7.

10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps in the convolutional neural network-based road disease detection method of any one of claims 1-7 when executing the program.

Images