CN114882474A - Road disease detection method and system based on convolutional neural network - Google Patents
Road disease detection method and system based on convolutional neural network Download PDFInfo
- Publication number
- CN114882474A CN114882474A CN202210608746.5A CN202210608746A CN114882474A CN 114882474 A CN114882474 A CN 114882474A CN 202210608746 A CN202210608746 A CN 202210608746A CN 114882474 A CN114882474 A CN 114882474A
- Authority
- CN
- China
- Prior art keywords
- shadow
- image
- road
- road disease
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of road construction, and provides a road disease detection method and a system based on a convolutional neural network, wherein a shadow removal module based on a generated countermeasure network removes shadows of a road disease image to be detected; detecting and obtaining road disease types based on the image after shadow removal and the target detection model; the construction process of the target detection model comprises the following steps: adopting a Yolov5 target detection network fused with a convolution attention module to respectively execute an attention mechanism on the channel and space dimensions and extract characteristic graphs with different dimensions; based on the idea of bidirectional feature fusion, the feature graphs of different dimensions are subjected to weighted fusion by adopting a self-adaptive feature fusion method to obtain a fused feature graph. The method solves the defects of the traditional road disease detection scheme, and obviously improves the detection precision.
Description
Technical Field
The invention belongs to the technical field of road construction, and particularly relates to a road disease detection method and system based on a convolutional neural network.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Along with the rapid development of road construction, the importance of road disease detection work is increasingly prominent. The road disease information is timely and accurately acquired, so that great cost can be saved for road maintenance work, and the possibility of road traffic accidents is reduced.
Traditional road disease detection mainly relies on the inspection personnel, adopts the mode of stopping to patrol and examine, take a picture record, artifical measurationing to gather road disease data.
But it has problems that: on one hand, the labor cost is high, the detection efficiency is low, the safety is poor, on the other hand, the data is not objective, the spatial information cannot be effectively managed, and the requirements of modern road patrol management are not met. In recent years, computer vision methods and deep learning algorithms are increasingly applied to the field of road disease detection, and the industrial level is obviously improved. However, the current algorithm still has the defects of large noise influence, poor model stability and the like, and the current algorithm needs to be optimized.
Disclosure of Invention
In order to solve at least one technical problem in the background art, the invention provides a road disease detection method and system based on a convolutional neural network, which solve the disadvantages of the traditional road disease detection scheme and obviously improve the detection precision.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a road disease detection method based on a convolutional neural network, which comprises the following steps:
acquiring a road disease image to be detected;
removing the shadow of the road disease image to be detected based on a shadow removal module for generating a countermeasure network;
detecting and obtaining the disease type of the road based on the image after shadow removal and the target detection model; the construction process of the target detection model comprises the following steps: adopting a Yolov5 target detection network fused with a convolution attention module to respectively execute an attention mechanism on the channel and space dimensions and extract characteristic graphs with different dimensions;
based on the thought of bidirectional feature fusion, the feature maps with different dimensions are subjected to weighted fusion by adopting a self-adaptive feature fusion method to obtain a fused feature map, and feature recognition is carried out based on the fused feature map to obtain a classification result of road diseases.
A second aspect of the present invention provides a road disease detection system based on a convolutional neural network, comprising:
the data acquisition module is used for acquiring a road disease image to be detected;
the shadow removing module is used for removing the shadow of the road disease image to be detected based on the shadow removing module for generating the countermeasure network;
the road disease detection module is used for detecting and obtaining the type of the road disease based on the image after the shadow is removed and the target detection model; the construction process of the target detection model comprises the following steps: respectively executing an attention mechanism on a channel and a space dimension by adopting a Yolov5 target detection network fused with a convolution attention module, and extracting to obtain feature maps with different dimensions;
based on the thought of bidirectional feature fusion, a self-adaptive feature fusion method is adopted to perform weighted fusion on feature maps of different dimensions to obtain a fused feature map; and carrying out feature recognition based on the fused feature map to obtain a classification result of the road diseases.
A third aspect of the invention provides a computer-readable storage medium.
A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the steps of the convolutional neural network-based road disease detection method as described above.
A fourth aspect of the invention provides a computer apparatus.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the convolutional neural network-based road disease detection method as described above when executing the program.
Compared with the prior art, the invention has the beneficial effects that:
the method aims to solve the problems that in the process of detecting the target of the road disease, the original Yolov5 model has poor robustness for detecting the road disease with different sizes, and particularly focuses on small objects excessively, such as road pits with the diameter less than 30 mm, and the target does not belong to the category of the road disease, otherwise, the workload is increased. A method of fusing attention modules is proposed, which performs an attention mechanism in the channel and spatial dimensions, respectively, with detection confidence that is generally higher for the correct class than for the original Yolov5 model.
The invention aims to solve the problems that in the target detection process of road diseases, the detection effect is easily interfered by image shadows, especially crack diseases are easily confused with branch shadows, and the probability of missed detection and misdetection and misjudgment is reduced. A method for generating a single image shadow removal method of a confrontation network based on channel attention is designed, and a shadow removal network is designed.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a schematic flow chart of a road disease detection method based on a convolutional neural network according to an embodiment of the present invention;
2(a) -2 (d) are sample libraries of road disease information according to embodiments of the present invention;
3(a) -3 (d) are example pictures of a shadow training data set in accordance with an embodiment of the present invention;
4(a) -4 (d) are shadowy images of an embodiment of the invention to be tested;
fig. 5(a) -5 (d) show the test effect of the model after the shadow is removed according to the embodiment of the present invention.
6(a) -6 (j) are the visualization results of the model training process after the improvement of the embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The overall idea of the invention is as follows:
firstly, collecting and labeling road disease image data sets based on two visual angles of vehicle-mounted camera shooting and unmanned aerial vehicle aerial shooting, and establishing an image sample library containing four types of road disease information; secondly, based on a Yolov5 target detection algorithm, a shadow removal algorithm, an attention mechanism and a feature fusion module based on a generation countermeasure network are fused, and a target detection model with a better effect is constructed.
The invention solves the defects of the traditional road disease detection scheme, obviously improves the detection precision, is mainly applied to road maintenance informatization intelligent construction, discovers road diseases in time, prolongs the service life of roads and related auxiliary facilities, and promotes the innovation of an internet and road management mode.
Example one
As shown in fig. 1, the present embodiment provides a road disease detection method based on a convolutional neural network, including the following steps:
step 1: acquiring a road disease image to be detected;
step 2: removing the shadow of the road disease image to be detected based on a shadow removal module for generating a countermeasure network;
and step 3: detecting and obtaining road disease types based on the image after shadow removal and the target detection model;
the construction process of the target detection model comprises the following steps: respectively executing an attention mechanism on a channel and a space dimension by adopting a Yolov5 target detection network fused with a convolution attention module, and extracting to obtain feature maps with different dimensions;
based on the thought of bidirectional feature fusion, a self-adaptive feature fusion method is adopted to perform weighted fusion on feature maps of different dimensions to obtain a fused feature map;
and (3) performing regression calculation on the anchor frame by using the anchor frame based on the three feature maps with different resolutions obtained by fusion and based on a confidence threshold and a non-maximum suppression (NMS) method, and finally generating an output vector with a class probability, a confidence score and a target surrounding frame, thereby realizing classification and positioning of the road disease information.
As one or more embodiments, in step 1, a comprehensive, canonical data set is the basis for deep learning model training. Aiming at the problem of road disease detection, the embodiment establishes a sample library of double-view and four types of road disease information; aiming at the problem of image shadow processing, the image shadow data set with strong pertinence is sorted.
As shown in fig. 2(a) -2 (d), the detection targets are divided into four types of road diseases, namely longitudinal cracks, transverse cracks, tortoiseshell cracks and road pits.
The data set is divided into two visual angles of vehicle-mounted camera shooting (HDV) and unmanned aerial vehicle aerial photography (UAV), and greater flexibility is provided for road maintenance work.
In addition, for the shadow influence problem, the embodiment establishes an image shadow training data set including four parts, namely an unshaded picture, a shaded mask and a shaded edge, which are sequentially shown in fig. 3(a) -3 (d).
In one or more embodiments, in step 2, the shadow removal module generating the countermeasure network includes a shadow detector, a shadow detection discriminator, a shadow eliminator, and a shadow elimination discriminator;
the shadow detector and the shadow eliminator adopt a UNet + + network and are composed of an upsampling layer, a downsampling layer and a plurality of nodes, and each node is a residual block composed of a convolution layer, a batch normalization layer, a Mish activation function and a scSE module.
The shadow detector is provided with an additional structure behind a UNet + + network, wherein the additional structure consists of a 3 x 3 convolution layer, a batch normalization layer, an LReLU activation function and a Sigmoid activation function, so that the limitation range of the output shadow mask is realized, and the output is limited within the range of 0 to 1.
And the shadow detection discriminator stacks four channels according to the marked shadow mask, the input image and the shadow mask output by the shadow detector, and judges whether the shadow is a shadow instead of a crack disease.
The shadow eliminator adds a structure ColorBlock behind a network of UNet + +, and because the shadow is strongly influenced by the wavelength of light, the light intensity that can be obtained by a camera varies according to the color of a target, and therefore, attention should be paid to each color channel and the relationship between the color channels in the training process. ColorBlock estimates the weight of each channel by using a full-connection layer, and makes up for the defect that the common convolutional layer cannot effectively train the relationship between the channels, so that the physical characteristics of each wavelength can be effectively trained, and the difference between the input image and the ground truth image can be learned.
The elimination process of the shadow eliminator comprises:
representing the light intensity of any position and the light intensity of a shadow area according to a shadow model;
obtaining a difference between the input image and the ground truth image based on the light intensity of the arbitrary position and the light intensity of the shadow area;
the elimination of the shadow based on the difference between the input image and the ground truth image results in a shadow-removed image.
Wherein, the light intensity of any position and the light intensity of the shadow area expressed according to the shadow model are specifically as follows:
based on the shadow model: i (x, λ) ═ L (x, λ) R (x, λ)
Where I is the light intensity, L is the illumination intensity, R is the reflectance, I, L, R depends on the position x and wavelength λ of a point on the image.
This gives:
light intensity I of a point in the non-shadow region lit Expressed as:
I lit (x,λ)＝L d (x,λ)R(x,λ)+L a (x,λ)R(x,λ)
light intensity I of a point in the shadow region shadow Expressed as:
I shadow (x,λ)＝L a (x,λ)R(x,λ)
in the formula, L d Indicating the illuminance of the direct illumination, L a Representing the illuminance of the indirect illumination.
The difference between a point on the input image and a corresponding point on the ground truth image is represented as:
Δ＝I gt -I input
＝P(I lit (x,λ)-I shadow (x,λ))
≈I lit (x,λ)-I shadow (x,λ)
＝L d (x,λ)R(x,λ)
wherein the function P represents the image processing of the camera image acquisition system, I gt Representing ground truth images (i.e. unshaded images), I input Representing the input image (i.e., shaded image).
Since the image has only three color channels R, G, B, λ in the above equation can be approximately expressed as a function of R, G, B, ColorBlock has the effect of weighting each color channel by a specific value, which can be understood as the illuminance of direct illumination of the three color channels of each image
Is estimated.
L assuming that the direct light intensity from the light source is constant for the entire image d Regardless of the position x, the result Δ can be expressed as:
the shadow elimination discriminator consists of a convolution layer, a batch processing normalization layer and a Mish activation function layer, the layers are connected in sequence, the step length of the convolution layer is set, and an image with the shadow eliminated is output.
The stride of the convolutional layers can be set according to the actual requirement, and in this embodiment, the stride of the convolutional layers is set to be 2 between every two convolutional layers.
The penalty function of the shadow elimination arbiter can be expressed as:
I input for shadow images, M gt For shadow mask images, I gt For shadow-free images, I output De-shadow image, V, output for shadow eliminator real Is a random matrix of values with an average value of 0.5, V fake Is a random matrix of values with an average value of-0.5, which follows a gaussian distribution.
The technology has the advantages that in the process of detecting the target of the road disease, in order to solve the problem that the detection effect is easily interfered by the image shadow, particularly the crack disease is easily confused with the branch shadow, and in order to reduce the probability of missed detection and misdetection and misjudgment, a single image shadow removing method (CANet) of the generation countermeasure network based on channel attention is designed to form a shadow removing network.
As one or more embodiments, in step 3, the Convolutional fused Attention Module (CBAM) includes two sub-modules, a Channel Attention (CAM) and a Spatial Attention (SAM), which perform Attention mechanisms in channel and spatial dimensions, respectively.
In the process of detecting the target of the road disease, the original Yolov5 model has poor robustness for detecting the road diseases with different sizes, and particularly has the problem of over-focusing small objects, such as road pit slots with the diameter less than 30 mm, the target does not belong to the category of the road disease, otherwise, the workload is increased.
The performing an attention mechanism in the channel and spatial dimensions, respectively, specifically comprises:
feature map F extracted from general convolutional layer 0 When dimension compression is carried out on (H multiplied by W multiplied by C), average pooling and maximum pooling are simultaneously introduced to obtain two one-dimensional characteristic maps F 1,2 (1X 1 XC), mixing F 1,2 Respectively sending the data to a two-layer shared neural network (MLP), carrying out addition operation based on element-wise on features output by the MLP, and finally generating final channel attention features Mc through Sigmoid activation operation;
for channel attention feature Mc and input feature map F 0 Carrying out element-wise multiplication operation to obtain F 3 As input features of the SAM module, F 3 Performing global maximum pooling and global average pooling based on the channels to obtain two one-dimensional feature maps F 4,5 (H multiplied by W multiplied by 1), channel splicing is carried out on the 2 feature maps, dimension reduction is realized by using a 7 multiplied by 7 convolution operation, and finally the spatial attention feature Ms is generated through sigmoid activation operation.
Finally, output Ms and input profile F for spatial attention Module 3 And carrying out element-wise multiplication operation to obtain the finally generated characteristics.
Spatial attention is aimed at improving the feature expression of key regions, and essentially, spatial information in an original picture is transformed into another space through a spatial transformation module, the key information is retained, a weight mask (mask) is generated and weighted output is carried out for each position, and therefore a specific target region of interest is enhanced while irrelevant background regions are weakened.
In this embodiment, in order to keep the network structure as unchanged as possible, i.e., to use more pre-trained parameters and make the model training process converge as soon as possible, the attention module is added outside the block.
In the embodiment, all the C3 modules in the Yolov5 backbone network, which are mainly responsible for extracting residual features, are replaced by the "convolution fusion attention + C3" module.
As one or more embodiments, in step 3, the performing weighted fusion on feature maps of different dimensions by using an adaptive feature fusion method specifically includes:
in consideration of the fact that the feature maps at different stages cannot be fully utilized only by simple stacking and superposition operations, in order to better fuse and extract features output by the backbone network and fully represent features of different sizes, the embodiment improves the original feature extraction module of Yolov 5.
Based on the idea of Feature bidirectional Fusion, each layer of adaptive Feature Fusion (ASFF) performs weighted Fusion on the stages of the original Feature structure, wherein the Fusion of different stage features adopts an attention mechanism to control the contribution of other stages to the stage Feature.
Algorithm validation
The embodiment is realized based on a Pythrch deep learning framework, and a hardware platform GeForce RTX 2080Ti is used. Experiments using the Yolov5s pre-training model required more rounds of training without fitting, since changing the network structure part pre-training parameters was not available.
The model training strategy is shown in table 1:
TABLE 1 model training strategy
Various evaluation criteria were used in the experiments:
accuracy (Accuracy, Acc):
precision (Precision, P):
recall (Recall, R):
wherein, TP is positive class judgment positive class number, FP is negative class judgment positive class number, FN is positive class judgment negative class number, and TN is negative class judgment negative class number.
Average Accuracy (AP):
the AP is obtained on the basis of Precision-Recall curves, wherein r1 and r2 … rn are corresponding Recall values at the first interpolation of the Precison interpolation segments arranged in ascending order.
Mean Average accuracy of all classes (mAP):
mAP-0.5: 0.95 means that IoU is averaged from 0.5 to 0.95, and mAP-0.5 means that IoU is 0.5.
1. Shadow processing effect of image
Using the SRD + pre-training model, data set 4 was trained using 1055 branch shadow pictures and 1398 ISTD open source pictures. Fig. 4(a) to 4(d) show shaded images, and fig. 5(a) to 5(d) show the test effect of the model after shading is removed.
Experiments show that the shadow of the picture can be effectively removed based on the shadow removing algorithm model of the antagonistic neural network.
2. Effect of improvement of Yolov5 algorithm
(1) In order to test the improvement effect of the model, 3000 vehicle-mounted visual angle pictures are selected to form a data set 1 for training. A total of 25 experiments were designed for different modification methods, and the training results are shown in table 2:
table 2 data set 1 training results
Experiments show that the Yolov5 has the best promotion effect after a CBAM attention mechanism is added and an ASFF feature extraction module is replaced, and the model training mAP is increased by 3.5%.
(2) In order to improve the generalization performance of the model, 14055 pictures with multiple visual angles are selected to form a data set 2 for training. The training results are shown in table 3, and the visualization of the model training process after improvement is shown in fig. 6(a) -6 (j):
table 3 data set 2 training results
| Methods | metrics/mAP_0.5 | metrics/mAP_0.5:0.95 |
| Initial Yolov5 | 0.973 | 0.748 |
| Improved Yolov5 | 0.987 | 0.794 |
Experiments show that the Yolov5 model adopting the improved method of the ASFF + CBAM is improved by 4.6% compared with the original Yolov5 algorithm mAP.
(3) In order to adapt to the application of actual road condition detection, 100 multi-view road disease pictures are collected to form a data set 3 for testing. The test results are shown in table 4.
Table 4 data set 3 test results
| Methods | Accuracy |
| Original Yolov5 | 0.86 |
| Improved Yolov5 | 0.94 |
Experiments show that the original Yolov5 model is easy to excessively focus small objects, such as road pits with the diameter less than 30 mm, in the detection process, the detection target does not form a road disease, otherwise, the work load is increased, and the problem is well solved by the improved Yolov5 model; meanwhile, the improved Yolov5 model is generally higher in the detection confidence of the correct class than the original Yolov5 model.
Because the classification of the road diseases has certain overlapping property and complexity, the model has a small amount of class misjudgment in the target detection process.
3. Yolov5 detection model introducing image shading processing
In order to test the performance improvement effect of the target detection model after the shadow processing, 100 image data sets 5 with shadows at different angles are manufactured for testing. The test results are shown in table 5.
Table 5 data set 3 test results
| Methods | Accuracy |
| Yolov5 algorithm detects pre-shadow pictures | 0.83 |
| Yolov5 algorithm for detecting picture after shadow removal | 0.90 |
| Post-improvement Yolov5 detection of pre-shadow pictures | 0.86 |
| Improved Yolov5 detection of post-shadow pictures | 0.94 |
According to experiments, the image for removing shadow interference obviously reduces the misjudgment probability when detecting the road diseases, can more efficiently and accurately find the positions of the road diseases, and greatly improves the detection of the four road diseases.
In conclusion, compared with the traditional road disease detection algorithm, the novel road disease detection algorithm provided by the invention has the advantage that the detection precision is obviously improved. The detection result shows that the scheme of the invention can accurately identify the cracks with the width larger than 5 mm and the pits with the diameter larger than 50 mm, and meet the actual requirements of the current road general survey.
Example two
The embodiment provides a road disease detection system based on a convolutional neural network, which comprises:
the data acquisition module is used for acquiring a road disease image to be detected;
the shadow removing module is used for removing the shadow of the road disease image to be detected based on the shadow removing module for generating the countermeasure network;
the road disease detection module is used for detecting and obtaining the type of the road disease based on the image after the shadow is removed and the target detection model; the construction process of the target detection model comprises the following steps: adopting a Yolov5 target detection network fused with a convolution attention module to respectively execute an attention mechanism on the channel and space dimensions and extract characteristic graphs with different dimensions;
based on the thought of bidirectional feature fusion, a self-adaptive feature fusion method is adopted to perform weighted fusion on feature maps of different dimensions to obtain a fused feature map; and carrying out feature identification based on the fused feature map to obtain a classification result of the road diseases.
EXAMPLE III
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the convolutional neural network-based road disease detection method as described above.
Example four
The embodiment provides a computer device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the road disease detection method based on the convolutional neural network.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The road disease detection method based on the convolutional neural network is characterized by comprising the following steps of:
acquiring a road disease image to be detected;
removing the shadow of the road disease image to be detected based on a shadow removal module for generating a countermeasure network;
detecting and obtaining road disease types based on the image after shadow removal and the target detection model; the construction process of the target detection model comprises the following steps: adopting a Yolov5 target detection network fused with a convolution attention module to respectively execute an attention mechanism on the channel and space dimensions and extract characteristic graphs with different dimensions;
based on the idea of feature bidirectional fusion, the feature maps with different dimensions are subjected to weighted fusion by adopting a self-adaptive feature fusion method to obtain a fused feature map, and feature identification is carried out based on the fused feature map to obtain a classification result of the road diseases.
2. The convolutional neural network-based road disease detection method of claim 1, wherein the performing an attention mechanism in channel and spatial dimensions, respectively, specifically comprises:
the method comprises the steps of simultaneously introducing average pooling and maximum pooling when dimension compression is carried out on an original feature map to obtain two one-dimensional feature maps, respectively sending the two one-dimensional feature maps into a two-layer shared neural network, and carrying out addition operation to generate channel attention features;
and multiplying the channel attention characteristic and the original characteristic graph to obtain a third characteristic graph, performing global maximum pooling and global average pooling on the basis of the channel to obtain two one-dimensional characteristic graphs, splicing the two one-dimensional characteristic graphs, and reducing the dimension by using convolution operation to generate the space attention characteristic.
3. The convolutional neural network-based road disease detection method as claimed in claim 1, wherein the shadow removal module based on the generation countermeasure network removes shadows of the road disease image to be detected,
the shadow removal module generating the countermeasure network includes a shadow eliminator, the shadow eliminator elimination process including:
representing the light intensity of any position and the light intensity of a shadow area according to a shadow model;
obtaining a difference between the input image and the ground truth image based on the light intensity of the arbitrary position and the light intensity of the shadow area;
the elimination of the shadow based on the difference between the input image and the ground truth image results in a shadow-removed image.
4. The convolutional neural network-based road disease detection method as claimed in claim 3, wherein the shadow canceller adopts a network structure of UNet + +, and is composed of upsampling, downsampling and a plurality of nodes, each node is a residual block composed of a convolutional layer, a batch normalization layer, a Mish activation function and a scSE module, an additional structure ColorBlock is arranged behind the network of UNet + +, and the weight of each color channel of the image is estimated by using a full connection layer.
5. The convolutional neural network-based road disease detection method of claim 3, wherein the difference between the input image and the ground truth image is represented as:
Δ＝I gt -I input
＝P(I lit (x,λ)-I shadow (x,λ))
≈I lit (x,λ)-I shadow (x,λ)
＝L d (x,λ)R(x,λ)
in the formula I gt Representing an unshaded image, I input Representing a shadowed image, a function P representing the image processing of the camera image acquisition system, I lit Is the light intensity at position x, I shadow Light intensity of the shaded area, L d The illuminance of direct illumination is shown, R is the reflectance, and λ is the wavelength.
6. The convolutional neural network-based road disease detection method of claim 1, wherein the types of road diseases include four types of road diseases, namely longitudinal cracks, transverse cracks, tortoiseshell cracks and road pits.
7. The convolutional neural network-based road disease detection method of claim 1, wherein the shadow removal module further comprises a shadow detector that sets an additional structure after the UNet + + network to limit the range of the output shadow mask to be in the range of 0 to 1.
8. Road disease detecting system based on convolutional neural network, its characterized in that includes:
the data acquisition module is used for acquiring a road disease image to be detected;
the shadow removing module is used for removing the shadow of the road disease image to be detected based on the shadow removing module for generating the confrontation network;
the road disease detection module is used for detecting and obtaining the type of the road disease based on the image after the shadow is removed and the target detection model; the construction process of the target detection model comprises the following steps: adopting a Yolov5 target detection network fused with a convolution attention module to respectively execute an attention mechanism on the channel and space dimensions and extract characteristic graphs with different dimensions;
based on the thought of bidirectional feature fusion, the feature maps with different dimensions are subjected to weighted fusion by adopting a self-adaptive feature fusion method to obtain a fused feature map, and feature recognition is carried out based on the fused feature map to obtain a classification result of the road diseases.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the convolutional neural network-based road disease detection method according to any one of claims 1 to 7.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps in the convolutional neural network-based road disease detection method of any one of claims 1-7 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210608746.5A CN114882474A (en) | 2022-05-30 | 2022-05-30 | Road disease detection method and system based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210608746.5A CN114882474A (en) | 2022-05-30 | 2022-05-30 | Road disease detection method and system based on convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114882474A true CN114882474A (en) | 2022-08-09 |
Family
ID=82679718
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210608746.5A Pending CN114882474A (en) | 2022-05-30 | 2022-05-30 | Road disease detection method and system based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114882474A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116958825A (en) * | 2023-08-28 | 2023-10-27 | 中国公路工程咨询集团有限公司 | Mobile remote sensing image acquisition method and highway maintenance monitoring method |
-
2022
- 2022-05-30 CN CN202210608746.5A patent/CN114882474A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116958825A (en) * | 2023-08-28 | 2023-10-27 | 中国公路工程咨询集团有限公司 | Mobile remote sensing image acquisition method and highway maintenance monitoring method |
| CN116958825B (en) * | 2023-08-28 | 2024-03-22 | 中国公路工程咨询集团有限公司 | Mobile remote sensing image acquisition method and highway maintenance monitoring method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102373456B1 (en) | Learning method and learning device, and testing method and testing device for detecting parking spaces by using point regression results and relationship between points to thereby provide an auto-parking system | |
| US20210327042A1 (en) | Deep learning-based system and method for automatically determining degree of damage to each area of vehicle | |
| CN109816024B (en) | Real-time vehicle logo detection method based on multi-scale feature fusion and DCNN | |
| CN107563265B (en) | High beam detection method and device | |
| CN111046880A (en) | Infrared target image segmentation method and system, electronic device and storage medium | |
| CN104809443A (en) | Convolutional neural network-based license plate detection method and system | |
| CN112766195B (en) | Electrified railway bow net arcing visual detection method | |
| CN112651404A (en) | Green fruit efficient segmentation method and system based on anchor-frame-free detector | |
| CN113378763A (en) | SAR image-oriented target automatic detection method and system | |
| CN113221943B (en) | Diesel vehicle black smoke image identification method, system and storage medium | |
| CN111027347A (en) | Video identification method and device and computer equipment | |
| CN111079518A (en) | Fall-down abnormal behavior identification method based on scene of law enforcement and case handling area | |
| CN111274964B (en) | Detection method for analyzing water surface pollutants based on visual saliency of unmanned aerial vehicle | |
| CN113255580A (en) | Method and device for identifying sprinkled objects and vehicle sprinkling and leaking | |
| CN112784675B (en) | Target detection method and device, storage medium and terminal | |
| CN111062967A (en) | Electric power business hall passenger flow statistical method and system based on target dynamic tracking | |
| CN114882474A (en) | Road disease detection method and system based on convolutional neural network | |
| CN115984543A (en) | Target detection algorithm based on infrared and visible light images | |
| CN111915634A (en) | Target object edge detection method and system based on fusion strategy | |
| CN115272882A (en) | Discrete building detection method and system based on remote sensing image | |
| CN117671597B (en) | Method for constructing mouse detection model and mouse detection method and device | |
| CN117935200A (en) | Automatic driving road condition monitoring method based on improvement YOLOv8 | |
| CN116229379B (en) | Road attribute identification method and device, electronic equipment and storage medium | |
| CN112785610A (en) | Lane line semantic segmentation method fusing low-level features | |
| CN116797789A (en) | Scene semantic segmentation method based on attention architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |