CN114863122A - Intelligent high-precision pavement disease identification method based on artificial intelligence - Google Patents
Intelligent high-precision pavement disease identification method based on artificial intelligence Download PDFInfo
- Publication number
- CN114863122A CN114863122A CN202210481986.3A CN202210481986A CN114863122A CN 114863122 A CN114863122 A CN 114863122A CN 202210481986 A CN202210481986 A CN 202210481986A CN 114863122 A CN114863122 A CN 114863122A
- Authority
- CN
- China
- Prior art keywords
- pavement
- pavement disease
- disease
- features
- level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 154
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 154
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 8
- 238000001514 detection method Methods 0.000 claims abstract description 38
- 238000012549 training Methods 0.000 claims description 48
- 230000006870 function Effects 0.000 claims description 25
- 238000000605 extraction Methods 0.000 claims description 23
- 238000010606 normalization Methods 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 230000001172 regenerating effect Effects 0.000 claims description 6
- 238000013526 transfer learning Methods 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 5
- 238000012795 verification Methods 0.000 claims description 4
- 230000008929 regeneration Effects 0.000 claims description 3
- 238000011069 regeneration method Methods 0.000 claims description 3
- 230000003247 decreasing effect Effects 0.000 claims description 2
- 230000007613 environmental effect Effects 0.000 claims description 2
- 230000007547 defect Effects 0.000 description 5
- 239000000284 extract Substances 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the field of pavement disease detection, and particularly relates to an intelligent high-precision pavement disease identification method based on artificial intelligence, which comprises the following steps: providing a pavement disease identification model, wherein the pavement disease identification model corresponds an input pavement disease picture with an output pavement disease type; when the pavement disease features are generated, extracting the pavement disease features in multiple levels based on a pavement disease picture, extracting the pavement disease features based on an original pavement disease picture in a first level, and extracting the pavement disease features based on the previous level in each level; taking the pavement disease features extracted from the levels except the first level as low-level features; corresponding to the levels arranged in the front, taking the pavement damage features extracted from the levels arranged in the back as high-level features; and the pavement disease characteristics corresponding to the pavement disease types output by the pavement disease identification model are the characteristics generated after the low-layer characteristics and the high-layer characteristics are fused. The recognition model has high precision and efficiency.
Description
Technical Field
The invention belongs to the field of pavement disease detection, and particularly relates to an intelligent high-precision pavement disease identification method based on artificial intelligence.
Background
The highway is the life line of the country, and good highway environment is convenient for goods and personnel to flow, plays vital role to national economic construction. With the influence of vehicle-mounted and freeze-thaw cycle, the road surface diseases are gradually increased, if the road surface diseases are not repaired, the damage degree of the road surface is increased in the horizontal direction, and the vehicle speed and the driving comfort degree are seriously influenced; the damage degree of the road is deepened in the longitudinal direction and even extends to a roadbed, so that serious structural damage is caused to the road. Therefore, the pavement damage should be detected in time for judging the pavement damage degree and accurately maintaining the damage.
Traditional road disease detection methods based on computer vision can be mainly divided into two categories: the method is based on the traditional image processing-Canny edge detection method, and the method is based on the deep neural network pavement disease detection method. However, the conventional image processing method is sensitive to the environment, and the detection effect of the road surface diseases is very poor in severe environment. The current deep convolutional network detection algorithm extracts multi-level characteristic information of a target, can be suitable for detecting low-quality disease images in severe environments, but has obviously low positioning accuracy and can not accurately identify diseases.
Therefore, a method for detecting and identifying road surface diseases with high precision and high efficiency in severe environments such as "light pollution" and "fog days" is required.
Disclosure of Invention
The invention provides an intelligent high-precision pavement disease identification method for accurately identifying pavement diseases, which comprises the following steps:
providing a pavement disease identification model, wherein the pavement disease identification model corresponds an input pavement disease picture with an output pavement disease type;
the pavement disease identification model generates corresponding pavement disease characteristics based on pavement disease objects on the picture, and the pavement disease characteristics correspond to the output pavement disease types;
when the pavement disease features are generated, extracting the pavement disease features in multiple levels based on a pavement disease picture, extracting the pavement disease features based on an original pavement disease picture in a first level, and extracting the pavement disease features based on the previous level in each level;
taking the pavement disease features extracted from the levels except the first level as low-level features;
corresponding to the levels arranged in the front, taking the pavement damage features extracted from the levels arranged in the back as high-level features;
and the pavement disease characteristics corresponding to the pavement disease types output by the pavement disease identification model are the characteristics generated after the low-layer characteristics and the high-layer characteristics are fused.
Further, the features extracted from each level except the first level are all processed into a pavement damage feature map with a pavement damage object outline, the pavement damage feature map of the last level selects the specific position of the pavement damage object through a candidate window frame, and a candidate frame pavement damage feature map with the pavement damage object is generated; respectively fusing the pavement damage object main bodies of the candidate frame pavement damage characteristic maps with the pavement damage object outlines in the pavement damage characteristic maps processed by each level except the first level to generate corresponding fused characteristic maps; and combining the fused feature maps processed by each level except the first level into a corresponding group of feature map combined features, wherein each group of feature map combined features corresponds to the type of the pavement damage to be output by the pavement damage identification model, and the number of feature maps of each group of feature map combined features is equal to that of the levels except the first level.
Further, the model is constructed as follows:
taking a main feature extraction network of a pre-training network model in another source domain as a multi-level feature extraction network of a Faster RCNN network model of a pavement disease recognition target domain;
wherein, the pre-training network model is selected from one of convolution networks of VGG16, VGG19, VGG96, ResNet32, ResNet48, ResNet101 and ResNet 152;
the last layer of convolution layer of the trunk feature extraction network is connected with an RPN model and used for generating a candidate window by the RPN model so as to frame out the specific position of the road surface disease object main body;
each convolution layer of the trunk feature extraction network except the first layer is connected with an NAM module and used for regenerating channel weight and pixel value weight required by the contour of the pavement disease object;
and constructing a full connection layer, and after a group of feature map combined features are converted into feature vectors, enabling the feature vectors to correspond to the pavement disease types through the full connection layer.
Further, the training mode of the pavement disease recognition model is as follows:
constructing a data set comprising a pavement disease picture and a corresponding pavement disease label;
setting a main feature of a pre-training network model in a source domain to extract a hyper-parameter of a network;
constructing a multi-tasking loss function including class loss of full connectivity layer phase cl And loss of bezel position in RPN stage lc And target and background loss ob ；
The formula is shown as (1.5) and (1.6):
Loss all ＝loss cl +loss lc +loss ob (1.5)
wherein Loss all Is the deviation between the real data and the predicted data;
where k in equation (1.6) represents the magnitude of the predicted score, and T 0 Representing a confidence score, loss ob (k) Indicating whether the current is a detection target or not; if the result is that the function value is 1, the frame contains the target; when the function value is 0, the frame does not comprise the target;
and training the pavement damage recognition model based on a transfer learning algorithm according to the constructed data set, a pre-training network model and a multi-task loss function in a source domain, and reserving the specific pavement damage category determined in the lower full-connection layer stage and reserving the optimal frame region in the RPN stage in the training process.
Further, before extracting features on an original road surface disease picture, the first layer of convolution kernel in the trunk feature extraction network adopts an NMS algorithm to delete a candidate frame of a road surface disease object, and the specific steps are as follows:
sorting a plurality of candidate frames on the pavement disease object according to the respective confidence values; calculating the overlapping rate between the candidate frame with the highest confidence value and the rest candidate frames to be processed, and setting an overlapping rate threshold value T; re-generating a confidence value of the candidate frame to be processed by adopting a Gaussian weighting algorithm for the candidate frame to be processed with the overlapping rate not within the set threshold range; arranging the candidate frames according to the regenerated confidence values, and setting a regenerated confidence value threshold value to remove the candidate frames to be processed with the scores lower than the confidence value threshold value; the formula is as follows:
s i the confidence value of the candidate frame to be processed is obtained, M is the candidate frame with the highest current confidence value, and bi is the candidate frame to be processed; iou is the overlapping rate between the candidate frame with the highest confidence value and the candidate frame to be processed, and the larger iou of bi and M is, the lower confidence value representing the regeneration of bi is.
Further, the overlap rate threshold T is used as a parameter of the pavement damage recognition model, in multiple times of training, the deviation between the predicted value and the true value is calculated, a deviation allowable value is set, and if the deviation is larger than the deviation allowable value, the T is increased; if the deviation is less than the deviation tolerance, T is decreased and gradually adjusted to make the multitask loss function smaller.
Further, in the NAM module, channel batch normalization processing is performed on each channel output by convolution according to a formula (1.3), and a weight coefficient of each channel is generated by an activation function sigmoid; carrying out pixel value batch normalization processing on the feature map of each channel output by convolution according to a formula (1.4), regenerating the pixel weight of each pixel value on the feature map by using an activation function sigmoid, and processing the pavement disease feature map needing to be fused with the candidate frame pavement disease feature map according to the regenerated channel weight and pixel weight; formula (1.3) formula (1.4) is as follows:
μ i is the scaling factor of the current channel; phi is a j Is the scaling factor for each channel; w is a i Is the normalized weight of the current channel; lambda [ alpha ] i Is the scaling factor of the current pixel;
is the scaling factor for each pixel; beta is a i Is the normalized weight of the current pixel.
Further, each feature map in the group of feature map combination features is overlapped to form an overlapped feature map feature, as shown in formula (1.1), and is converted into a corresponding feature vector before being input into the full connection layer; formula (1.1) is as follows:
M＝M 2 ⊕......⊕M n (1.1)
wherein M is a superimposed feature map, M 2 Combining a first feature map of the features for a set of feature maps; m n Is the most importantThe latter feature map.
Further, in the constructed data set, the ratio of the training set to the test set is 9: 1, in the training set, 90% of data is used for training and 10% of data is used for verification; training is concentrated to include samples under the environments of light pollution and foggy days; the ratio of the "light contaminated" samples to the "foggy" environmental samples was 1: 1.
Further, the specific mode of the fusion is as follows: a series of prior frames with road surface disease objects are generated through the extracted features of each level except the first level, and then the prior frames are combined to generate a fusion module containing multi-level features.
The invention has the beneficial effects that:
1. in order to solve the problem that road surface disease detection is not accurate in severe environments in the prior art, the method and the system respectively collect pictures of different types of road diseases in normal environments and severe environments such as light pollution, foggy days and the like when a data set is constructed, train the improved network model, enable the network model to have higher accuracy rate on road surface disease image detection, and can resist interference of irrelevant features.
2. Based on the advantages of a transfer learning algorithm and combination of multi-model combination, a fast RCNN network model is improved, a network structure which integrates low-layer characteristic information and high-layer characteristic information is designed, the rich detail information of a low-layer network can be trained and learned fully, the semantic information of a high-layer network can be reserved, a local sensing area is enlarged, and meanwhile, the detail information is reserved, so that high-precision pavement disease detection and identification are achieved.
3. The constructed network model eliminates redundant frames through an NMS algorithm, keeps the best frame, finds the best object detection position, improves the positioning accuracy of the model, and conducts a large amount of training to improve the generalization capability of the model. The method can greatly improve the accuracy of pavement disease detection and identification, reduce the omission ratio, has good inhibition effect on the misjudgment ratio, and eliminates redundant detection times, thereby greatly reducing the detection cost.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a data set construction flowchart as described in example 1;
FIG. 3 is a schematic diagram of the NAM module described in example 1;
fig. 4 is a schematic diagram of a model network structure according to embodiment 1.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Examples
As shown in fig. 1 to 4, the present embodiment provides an artificial intelligence-based high-precision pavement damage identification method suitable for high-precision and high-efficiency detection in severe weather, which includes the following steps:
providing a pavement disease identification model, wherein the pavement disease identification model corresponds an input pavement disease picture with an output pavement disease type;
the pavement disease identification model generates corresponding pavement disease characteristics based on the pavement disease objects on the picture, and the pavement disease characteristics correspond to the output pavement disease types;
when the pavement disease features are generated, extracting the pavement disease features in multiple levels based on a pavement disease picture, extracting the pavement disease features based on an original pavement disease picture in a first level, and extracting the pavement disease features based on the previous level in each level;
taking the pavement disease features extracted from the levels except the first level as low-level features;
corresponding to the levels arranged in the front, taking the pavement damage features extracted from the levels arranged in the back as high-level features;
and the pavement disease characteristics corresponding to the pavement disease types output by the pavement disease identification model are the characteristics generated after the low-layer characteristics and the high-layer characteristics are fused.
The invention utilizes the advantages of a transfer learning algorithm and a multi-model combination to establish a detection model suitable for extracting pavement disease characteristics, can accurately extract the characteristic information of a pavement disease image and effectively identify the pavement disease type, constructs a multi-model combination network model by improving a pavement disease identification model, and fuses low-layer characteristics and high-layer characteristics, so that the detection model can be used for fully training and learning rich detailed information of a low-layer network and retaining semantic information of a high-layer network, and can be used for expanding a local sensing area and retaining detailed information, thereby identifying the pavement disease information more accurately and more effectively.
In this embodiment, the features extracted from each level except the first level are all processed as a pavement damage feature map having a pavement damage object outline, and the pavement damage feature map of the last level selects the specific position of the pavement damage object through the candidate window frame, and generates a candidate frame pavement damage feature map having the pavement damage object; respectively fusing the pavement damage object main bodies of the candidate frame pavement damage characteristic maps with the pavement damage object outlines in the pavement damage characteristic maps processed by each level except the first level to generate corresponding fused characteristic maps; and combining the fused feature maps processed by each level except the first level into a corresponding group of feature map combined features, wherein each group of feature map combined features corresponds to the type of the pavement damage to be output by the pavement damage identification model, and the number of feature maps of each group of feature map combined features is equal to that of the levels except the first level.
In this embodiment, the model is constructed as follows:
taking a main feature extraction network of a pre-training network model in another source domain as a multi-level feature extraction network of a Faster RCNN network model of a pavement disease recognition target domain;
the pre-training network model is selected as one of convolution networks of VGG16, VGG19, VGG96, ResNet32, ResNet48, ResNet101 and ResNet152, and the hyper-parameters are continuously adjusted, so that a candidate region with a target, namely an ROI, can be accurately screened out, and an optimal backbone feature extraction network, namely a backbone is determined;
the last layer of convolution layer of the trunk feature extraction network is connected with an RPN model and used for generating a candidate window by the RPN model so as to frame out the specific position of the road surface disease object main body;
each convolution layer of the trunk feature extraction network except the first layer is connected with an NAM module and used for regenerating channel weight and pixel value weight required by the contour of the pavement disease object;
and constructing a full connection layer, and after a group of feature map combined features are converted into feature vectors, enabling the feature vectors to correspond to the pavement disease types through the full connection layer.
In this embodiment, the road surface defect recognition model is trained as follows:
constructing a data set comprising a pavement disease picture and a corresponding pavement disease label;
the method comprises the steps of collecting pictures of different road diseases through a high-resolution camera, collecting 13342 data in total, further amplifying the data by utilizing rotation, brightness conversion and cutting modes, finishing information customization labeling of road disease frame positions and the like by using a LabelImg tool, and making into a VOC2007 data set, thereby realizing model supervision training and learning.
Setting a main feature of a pre-training network model in a source domain to extract a hyper-parameter of a network;
constructing a multi-tasking loss function including class loss of full connectivity layer phase cl And loss of bezel position in RPN stage lc And target and background loss ob ；
The formula is shown as (1.5) and (1.6):
Loss all ＝loss cl +loss lc +loss ob (1.5)
wherein Loss all Deviation between real data and predicted data;
where k in equation (1.6) represents the magnitude of the predicted score, and T 0 Representing a confidence score, loss ob (k) Indicating whether the current is a detection target or not; if the result is that the function value is 1, the frame contains the target; when the function value is 0, the frame does not comprise the target;
the loss function is used for calculating the deviation between real data and predicted data, the generalization capability of the model can be estimated according to the deviation, the loss function of the fast RCNN model is distributed in an RPN stage and a full connection layer stage and comprises category loss, frame position loss, target loss and background loss, loss functions corresponding to the category loss, frame position loss and target and background loss are respectively established, and network parameters are finely adjusted in the forward propagation process, so that the deviation of the loss function is as small as possible during training, the convergence of the model is realized, and the model is close to an expected threshold value.
And training the pavement damage recognition model based on a transfer learning algorithm according to the constructed data set, a pre-training network model and a multi-task loss function in a source domain, and reserving the specific pavement damage category determined in the lower full-connection layer stage and reserving the optimal frame region in the RPN stage in the training process.
In this embodiment, before extracting features on an original road surface defect picture, the candidate frame of a road surface defect object is deleted by using an NMS algorithm, which includes the following specific steps:
sorting a plurality of candidate frames on the pavement disease object according to the respective confidence values; calculating the overlapping rate between the candidate frame with the highest confidence value and the rest candidate frames to be processed, and setting an overlapping rate threshold value T; re-generating a confidence value of the candidate frame to be processed by adopting a Gaussian weighting algorithm for the candidate frame to be processed with the overlapping rate not within the set threshold range; arranging the candidate frames according to the regenerated confidence values, and setting a regenerated confidence value threshold value to remove the candidate frames to be processed with the scores lower than the confidence value threshold value; the formula is as follows:
s i the confidence value of the candidate frame to be processed is obtained, M is the candidate frame with the highest current confidence value, and bi is the candidate frame to be processed; iou is the overlapping rate between the candidate frame with the highest confidence value and the candidate frame to be processed, and the larger iou of bi and M is, the lower confidence value representing the regeneration of bi is.
And when the candidate frame is deleted, the confidence value of the candidate frame is inhibited by adopting a Gaussian weighting algorithm instead of simply and violently eliminating the candidate frame which is larger than the overlapping rate threshold value T, so that the detection and the identification are more accurate.
In the embodiment, an overlap rate threshold value T is used as a parameter of a pavement damage identification model, in multiple training, the deviation between a predicted value and a true value is calculated, a deviation allowable value is set, and if the deviation is larger than the deviation allowable value, the T is increased; if the deviation is smaller than the deviation tolerance value, the T is reduced, and the size of the T is gradually adjusted, so that the multitask loss function value is smaller, and the model has stronger generalization capability.
In this embodiment, in the NAM module, a channel batch normalization process is performed on each channel output by convolution according to equation (1.3), and a weight coefficient of each channel is generated by an activation function sigmoid; carrying out pixel value batch normalization processing on the feature map of each channel output by convolution according to a formula (1.4), regenerating the pixel weight of each pixel value on the feature map by using an activation function sigmoid, and processing the pavement disease feature map needing to be fused with the candidate frame pavement disease feature map according to the regenerated channel weight and pixel weight; formula (1.3) formula (1.4) is as follows:
u i is the scaling factor of the current channel; phi is a unit of j Is the scaling factor for each channel; w is a i Is the weight of the current channel normalization; lambda [ alpha ] i Is the scaling factor of the current pixel;
is the scaling factor for each pixel; beta is a i Is the weight of the current pixel normalization.
By introducing the NAM normalization attention module, the channel submodule is added on the channel weight, the weight coefficient of an unimportant channel is restrained, and the space submodule is added on the pixel weight, so that the weight coefficient of an important pixel is enhanced, the extraction characteristic of the model is more efficient, and the model has good light weight characteristic, and is convenient for further extracting characteristic information of different channels and spaces.
In this embodiment, each feature map in the set of feature map combination features is superimposed to form a superimposed feature map feature, as in equation (1.1), and is converted into a corresponding feature vector before being input into the full connection layer; formula (1.1) is as follows:
M＝M 2 ⊕......⊕M n (1.1)
wherein M is a superimposed feature map, M 2 Combining a first feature map of the features for a set of feature maps; m n Is the last feature map.
In this embodiment, in the constructed data set, the ratio of the training set to the test set is 9: 1, in the training set, 90% of the data is again used for training and 10% for validation.
The fast RCNN model has the ability of perceiving and judging the target after 100 times of iterative training, supervised learning is carried out by constructing a data set with a label, and the proportion of a training set to a testing set is set as 9: and 1, further scientifically optimizing model parameters, dividing data in a training set into training data and verification data, wherein the purpose of the verification set is to detect the performance of the model in the training process, and further adaptively adjusting network parameters to enable the network parameters to be in an optimal state.
The training set comprises samples under the environments of light pollution and foggy days; the proportion of the light pollution samples to the fog environment samples is 1:1, 3000 samples are respectively collected, the detection precision of the model in the light pollution and fog environment is enhanced, the accuracy of the model in the detection of the road surface disease images is high, interference of irrelevant features can be resisted, and accordingly high-efficiency and high-precision extraction of the road surface diseases is achieved.
In this embodiment, the specific way of fusion is: a series of prior frames with road surface defect objects are generated through the extracted features of each level except the first level, and then the prior frames are combined to generate a fusion module containing multi-level features.
The invention provides an intelligent high-precision pavement disease identification method based on artificial intelligence, and aims to solve the problem of low detection accuracy caused by severe environments such as light pollution, foggy days and the like or low-quality image acquisition. The total number of data acquisition is 13342, 3000 data are acquired in the environment of light pollution, 3000 data are acquired in the environment of foggy day, so that the higher detection precision of the model in the environments can be enhanced, the rest data are acquired in the normal environment, and the model is promoted to overcome the influences of light pollution, foggy day and the like in severe environments because the training sample data of the model comprise the data in the environment of light pollution and foggy day. The method improves and optimizes the fast RCNN model, so that the method has higher accuracy rate for detecting the road surface disease image, and can resist the interference of irrelevant characteristics, thereby realizing the high-efficiency and high-precision extraction of the road surface disease. A pavement disease detection algorithm based on a fast RCNN deep convolution model is designed, different backbones are set by using the advantages of a transfer learning algorithm and combination of multi-model combination, network parameters are adjusted, the classified samples are trained on the models, and a detection model suitable for pavement disease feature extraction is constructed. In addition, in order to fuse more semantic feature information, a multilayer network feature fusion module is added in the backbone structure, the low-level network feature extraction capability is improved, detail information is reserved, the high-level local sensing area is enlarged, and therefore high-precision pavement disease detection and identification are achieved. In order to further extract the feature information of different channels and spaces, an NAM (normalized attention) module is introduced behind each convolution layer except the first layer of the main feature extraction network, so that the extracted features of the model are more efficient, and the model has good light weight characteristics. The intelligent high-precision pavement disease detection and identification method can be applied to the pavement quality maintenance process, so that the high-efficiency control of the road engineering quality is realized.
Example 2
The invention provides an intelligent high-precision pavement damage identification method based on artificial intelligence, which can be applied to the technical field of intelligent traffic, so that the aim of road engineering quality control, such as road crack detection, road pit detection and the like, is fulfilled. The specific details are as follows: in the road quality detection, images of each road surface are collected through a vehicle-mounted camera, feature information of the images is extracted for many times through convolution and pooling based on a self-adaptive backbone network structure, then low-level information and high-level information are fused, semantic information of the images is kept as much as possible, and finally corresponding frame features are screened out through an NMS algorithm with an optimal threshold value, so that the type judgment and the positioning of the disease features are realized.
In the embodiment, the fusion scheme is that a series of prior frames which may have targets are respectively generated from feature maps obtained by extracting the second layer, the third layer and the fourth layer, and then the prior frames are combined, so that a module with multi-layer fusion features can be provided, and the probability of target loss is reduced.
Compared with the existing pavement disease detection and identification method, the method has the advantages that: and the detection and identification of the pavement defect image with higher precision can be realized. The method improves and optimizes the fast RCNN model, so that the method has higher accuracy rate for detecting the road surface disease image and can resist the interference of irrelevant characteristics. Because the training sample data contains data under the environments of light pollution and foggy days, the model is prompted to overcome the influences of light pollution, foggy days and the like in severe environments, and therefore the efficient and high-precision extraction and positioning of the pavement diseases are achieved. After a large amount of pavement disease data are trained, the improved Faster RCNN model has strong generalization performance, and if a new image is input, the area where the disease is located can be sensed efficiently, and the specific type can be judged. The original image passes through a backbone feature extraction network suitable for pavement disease detection, the influence of noise and other interference information is overcome, the features of all layers are fused, disease feature points are richer, and an optimal frame is reserved by using a self-adaptive NMS algorithm, so that the positioning accuracy of the model is improved.
In fig. 2, Voc2007 shows a data set format for training a model, indications show that marked xml format data is stored in this directory, ImageSets are used for storing path files required for training and testing, a sub-directory Main thereof stores txt files, and JPEGImages is jpg format image data for training.
The method can greatly improve the accuracy of pavement disease detection and identification, reduce the omission factor, well inhibit the misjudgment rate, eliminate redundant detection times and greatly reduce the detection cost. In addition, high-precision detection is an important branch of intelligent traffic technology development.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (10)
1. An intelligent high-precision pavement disease identification method based on artificial intelligence is characterized by comprising the following steps:
providing a pavement disease identification model, wherein the pavement disease identification model corresponds an input pavement disease picture with an output pavement disease type;
the pavement disease identification model generates corresponding pavement disease characteristics based on the pavement disease objects on the picture, and the pavement disease characteristics correspond to the output pavement disease types;
when the pavement disease features are generated, extracting the pavement disease features in multiple levels based on a pavement disease picture, extracting the pavement disease features based on an original pavement disease picture in a first level, and extracting the pavement disease features based on the previous level in each level;
taking the pavement disease features extracted from the levels except the first level as low-level features;
corresponding to the levels arranged in the front, taking the pavement damage features extracted from the levels arranged in the back as high-level features;
and the pavement disease characteristics corresponding to the pavement disease types output by the pavement disease identification model are the characteristics generated after the low-layer characteristics and the high-layer characteristics are fused.
2. The intelligent high-precision pavement damage recognition method according to claim 1, wherein the extracted features of each level except the first level are processed into a pavement damage feature map with a pavement damage object outline, and the pavement damage feature map of the last level selects the specific position of the pavement damage object through a candidate window frame and generates a candidate frame pavement damage feature map with the pavement damage object; respectively fusing the pavement damage object main bodies of the candidate frame pavement damage characteristic maps with the pavement damage object outlines in the pavement damage characteristic maps processed by each level except the first level to generate corresponding fused characteristic maps; and combining the fused feature maps processed by each level except the first level into a corresponding group of feature map combined features, wherein each group of feature map combined features corresponds to the type of the pavement damage to be output by the pavement damage identification model, and the number of feature maps of each group of feature map combined features is equal to that of the levels except the first level.
3. The intelligent high-precision pavement damage recognition method according to claim 2, wherein the model is constructed in the following manner:
taking a main feature extraction network of a pre-training network model in another source domain as a multi-level feature extraction network of a Faster RCNN network model of a pavement disease recognition target domain;
wherein, the pre-training network model is selected from one of convolution networks of VGG16, VGG19, VGG96, ResNet32, ResNet48, ResNet101 and ResNet 152;
the last layer of convolution layer of the trunk feature extraction network is connected with an RPN model and used for generating a candidate window by the RPN model so as to frame out the specific position of the road surface disease object main body;
each convolution layer of the trunk feature extraction network except the first layer is connected with an NAM module and used for regenerating channel weight and pixel value weight required by the contour of the pavement disease object;
and constructing a full connection layer, and after a group of feature map combined features are converted into feature vectors, enabling the feature vectors to correspond to the pavement disease types through the full connection layer.
4. The intelligent high-precision pavement damage recognition method according to claim 3, wherein the pavement damage recognition model is trained in the following way:
constructing a data set comprising a pavement disease picture and a corresponding pavement disease label;
setting a main feature of a pre-training network model in a source domain to extract a hyper-parameter of a network;
constructing a multi-tasking loss function including class loss of full connectivity layer phase cl And loss of bezel position in RPN stage lc And target and background loss ob ；
The formulas are shown as (1.5) and (1.6):
Loss all ＝loss cl +loss lc +loss ob (1.5)
wherein Loss all Deviation between real data and predicted data;
where k in equation (1.6) represents the magnitude of the prediction score, and T 0 Representing a confidence score, loss ob (k) Indicating whether the current is a detection target or not; if the result is that the function value is 1, the frame contains the target; when the function value is 0, the frame does not comprise the target;
and training the pavement damage recognition model based on a transfer learning algorithm according to the constructed data set, a pre-training network model and a multi-task loss function in a source domain, and reserving the specific pavement damage category determined in the lower full-connection layer stage and reserving the optimal frame region in the RPN stage in the training process.
5. The intelligent high-precision pavement disease identification method according to claim 4, wherein the first layer of convolution kernel in the trunk feature extraction network adopts an NMS algorithm to delete the candidate frames of the pavement disease objects before extracting the features on the original pavement disease picture, and the specific steps are as follows:
sorting a plurality of candidate frames on the pavement disease object according to the respective confidence values; calculating the overlapping rate between the candidate frame with the highest confidence value and the rest candidate frames to be processed, and setting an overlapping rate threshold value T; re-generating a confidence value of the candidate frame to be processed by adopting a Gaussian weighting algorithm for the candidate frame to be processed with the overlapping rate not within the set threshold range; arranging the candidate frames according to the regenerated confidence values, and setting a regenerated confidence value threshold value to remove the candidate frames to be processed with the scores lower than the confidence value threshold value; the formula is as follows:
s i the confidence value of the candidate box to be processed is obtained, M is the candidate box with the highest current confidence value, and bi is the candidate box to be processed; iou is the overlapping rate between the candidate frame with the highest confidence value and the candidate frame to be processed, and the larger iou of bi and M is, the lower confidence value representing the regeneration of bi is.
6. The intelligent high-precision pavement damage recognition method according to claim 5, wherein an overlap rate threshold T is used as a parameter of a pavement damage recognition model, in multiple training, the deviation between a predicted value and a true value is calculated, a deviation allowable value is set, and if the deviation is larger than the deviation allowable value, T is increased; if the deviation is smaller than the deviation tolerance, T is decreased and gradually adjusted in size to make the multitask loss function smaller.
7. The intelligent high-precision pavement damage identification method according to claim 3, wherein in the NAM module, batch normalization processing is performed on each channel output by convolution according to a formula (1.3), and a weight coefficient of each channel is generated by an activation function sigmoid; carrying out pixel value batch normalization processing on the feature map of each channel output by convolution according to a formula (1.4), regenerating the pixel weight of each pixel value on the feature map by using an activation function sigmoid, and processing the pavement disease feature map needing to be fused with the candidate frame pavement disease feature map according to the regenerated channel weight and pixel weight; formula (1.3) formula (1.4) is as follows:
μ i is the scaling factor of the current channel; phi is a j Is the scaling factor for each channel; w is a i Is the weight of the current channel normalization; lambda [ alpha ] i Is the scaling factor of the current pixel;
is the scaling factor for each pixel; beta is a i Is the weight of the current pixel normalization.
8. The intelligent high-precision pavement damage recognition method according to claim 4, wherein each feature map in the set of feature map combination features is superimposed to form a superimposed feature map feature, as shown in formula (1.1), and is converted into a corresponding feature vector before being input into the full-link layer; formula (1.1) is as follows:
M＝M 2 ⊕......⊕M n (1.1)
wherein M is a superimposed feature map, M 2 Combining a first feature map of the features for a set of feature maps; m n Is the last feature map.
9. The intelligent high-precision pavement damage recognition method according to claim 4, wherein in the constructed data set, the proportion of the training set to the test set is 9: 1, in a training set, 90% of data is used for training, 10% of data is used for verification, and the training set comprises samples in the environments of light pollution and foggy days; the ratio of "light contaminated" samples to "foggy" environmental samples was 1: 1.
10. The intelligent high-precision pavement disease identification method according to claim 1, wherein the fusion is specifically performed by: a series of prior frames with road surface disease objects are generated through the extracted features of each level except the first level, and then the prior frames are combined to generate a fusion module containing multi-level features.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210481986.3A CN114863122B (en) | 2022-05-05 | 2022-05-05 | Intelligent high-precision pavement disease identification method based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210481986.3A CN114863122B (en) | 2022-05-05 | 2022-05-05 | Intelligent high-precision pavement disease identification method based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114863122A true CN114863122A (en) | 2022-08-05 |
| CN114863122B CN114863122B (en) | 2024-06-28 |
Family
ID=82635544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210481986.3A Active CN114863122B (en) | 2022-05-05 | 2022-05-05 | Intelligent high-precision pavement disease identification method based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114863122B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118115859A (en) * | 2024-03-13 | 2024-05-31 | 甘肃恒石公路检测科技有限公司 | Method for online updating and editing pavement disease image automatic identification algorithm |
| WO2024139297A1 (en) * | 2022-12-30 | 2024-07-04 | 深圳云天励飞技术股份有限公司 | Road disease identification method and re-identification method, and related device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101818919B1 (en) * | 2017-05-11 | 2018-01-17 | 양경용 | Mending method for asphalt road using mending mat |
| CN110222701A (en) * | 2019-06-11 | 2019-09-10 | 北京新桥技术发展有限公司 | A kind of bridge defect automatic identifying method |
| CN113392849A (en) * | 2021-06-30 | 2021-09-14 | 哈尔滨理工大学 | R-CNN-based complex pavement crack identification method |
| CN113537037A (en) * | 2021-07-12 | 2021-10-22 | 北京洞微科技发展有限公司 | Pavement disease identification method, system, electronic device and storage medium |
-
2022
- 2022-05-05 CN CN202210481986.3A patent/CN114863122B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101818919B1 (en) * | 2017-05-11 | 2018-01-17 | 양경용 | Mending method for asphalt road using mending mat |
| CN110222701A (en) * | 2019-06-11 | 2019-09-10 | 北京新桥技术发展有限公司 | A kind of bridge defect automatic identifying method |
| CN113392849A (en) * | 2021-06-30 | 2021-09-14 | 哈尔滨理工大学 | R-CNN-based complex pavement crack identification method |
| CN113537037A (en) * | 2021-07-12 | 2021-10-22 | 北京洞微科技发展有限公司 | Pavement disease identification method, system, electronic device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 杨婧: "基于SSD的路面破损检测方法研究", 《中国优秀硕士学位论文全文数据库 工程科技II辑》, 15 August 2021 (2021-08-15), pages 034 - 62 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024139297A1 (en) * | 2022-12-30 | 2024-07-04 | 深圳云天励飞技术股份有限公司 | Road disease identification method and re-identification method, and related device |
| CN118115859A (en) * | 2024-03-13 | 2024-05-31 | 甘肃恒石公路检测科技有限公司 | Method for online updating and editing pavement disease image automatic identification algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114863122B (en) | 2024-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109977812B (en) | Vehicle-mounted video target detection method based on deep learning | |
| CN111368687B (en) | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation | |
| CN112200161B (en) | Face recognition detection method based on mixed attention mechanism | |
| CN109447018B (en) | Road environment visual perception method based on improved Faster R-CNN | |
| Wang et al. | Soft-weighted-average ensemble vehicle detection method based on single-stage and two-stage deep learning models | |
| CN111460919B (en) | Monocular vision road target detection and distance estimation method based on improved YOLOv3 | |
| EP3690741B1 (en) | Method for automatically evaluating labeling reliability of training images for use in deep learning network to analyze images, and reliability-evaluating device using the same | |
| CN111882620B (en) | Road drivable area segmentation method based on multi-scale information | |
| CN114863122A (en) | Intelligent high-precision pavement disease identification method based on artificial intelligence | |
| CN111274886B (en) | Deep learning-based pedestrian red light running illegal behavior analysis method and system | |
| CN113505670B (en) | Remote sensing image weak supervision building extraction method based on multi-scale CAM and super-pixels | |
| CN114627437B (en) | Traffic target identification method and system | |
| CN114549909A (en) | Pseudo label remote sensing image scene classification method based on self-adaptive threshold | |
| CN112434723A (en) | Day/night image classification and object detection method based on attention network | |
| CN112529065A (en) | Target detection method based on feature alignment and key point auxiliary excitation | |
| CN114913498A (en) | Parallel multi-scale feature aggregation lane line detection method based on key point estimation | |
| CN111832411A (en) | Track cataract obstacle detection method based on fusion of vision and laser radar | |
| CN117152513A (en) | Vehicle boundary positioning method for night scene | |
| CN114639067A (en) | Multi-scale full-scene monitoring target detection method based on attention mechanism | |
| CN112785610B (en) | Lane line semantic segmentation method integrating low-level features | |
| CN117935200A (en) | Automatic driving road condition monitoring method based on improvement YOLOv8 | |
| CN114998689B (en) | Track data set generation method, track identification method and system | |
| CN112085101A (en) | High-performance and high-reliability environment fusion sensing method and system | |
| CN116630683A (en) | Road damage detection method based on diffusion self-adaptive feature extraction | |
| CN116824537A (en) | Automatic driving multitasking visual perception method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |