CN115984174B - Pavement disease identification method based on rotary constant-change detector - Google Patents
Pavement disease identification method based on rotary constant-change detector Download PDFInfo
- Publication number
- CN115984174B CN115984174B CN202211533695.0A CN202211533695A CN115984174B CN 115984174 B CN115984174 B CN 115984174B CN 202211533695 A CN202211533695 A CN 202211533695A CN 115984174 B CN115984174 B CN 115984174B
- Authority
- CN
- China
- Prior art keywords
- model
- rotation
- disease
- pavement
- rotating frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 88
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 88
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012545 processing Methods 0.000 claims abstract description 36
- 238000012549 training Methods 0.000 claims abstract description 34
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 230000009466 transformation Effects 0.000 claims abstract description 11
- 239000013598 vector Substances 0.000 claims abstract description 6
- 230000008859 change Effects 0.000 claims description 20
- 238000001514 detection method Methods 0.000 claims description 17
- 238000002372 labelling Methods 0.000 claims description 13
- 238000012795 verification Methods 0.000 claims description 12
- 238000012360 testing method Methods 0.000 claims description 11
- 238000004458 analytical method Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 238000005457 optimization Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 7
- 238000012163 sequencing technique Methods 0.000 claims description 7
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 6
- 230000007547 defect Effects 0.000 claims description 6
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 4
- 238000009877 rendering Methods 0.000 claims description 4
- GPRLSGONYQIRFK-MNYXATJNSA-N triton Chemical compound [3H+] GPRLSGONYQIRFK-MNYXATJNSA-N 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 2
- 230000002708 enhancing effect Effects 0.000 claims description 2
- 238000003032 molecular docking Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 4
- 239000010426 asphalt Substances 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 4
- 230000008439 repair process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 210000001503 joint Anatomy 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a pavement disease recognition technology, in particular to a pavement disease recognition method based on a rotary constant-change detector, which comprises the steps of preprocessing acquired road image data; training the image sample pavement diseases by adopting a rotary constant-change detector, and identifying the pavement diseases: inputting a rotation isomorphism backbone network to the normalized road image data to obtain a rotation isomorphism characteristic map; entering an area recommendation network and an area transformation network to generate a rotation equal-change recommendation area; entering a rotation invariant alignment network, and extracting rotation invariant features; rotating the invariant features, entering a full connection layer, and respectively outputting class vectors and coordinates of the rotating frame; the service processing module calculates the disease influence area according to the category vector and the coordinates of each rotating frame; if the rotating frame type is linear diseases, calculating the diagonal length according to the coordinates for evaluating the disease influence area; if the rotating frame type is a planar disease, the influence area is directly calculated according to the coordinates.
Description
Technical Field
The invention relates to a pavement disease recognition technology, in particular to a pavement disease recognition method based on a rotary constant-change detector.
Background
Asphalt pavement is an important infrastructure for promoting economic development as an important component of highway engineering. With the increase of service life, asphalt pavement can generate diseases in different forms and different degrees. Previously, road surface diseases are detected mainly by judging whether asphalt road surfaces have diseases or not by a road detector through a manual walking way, and judging and recording the classification and severity level of the detected diseases. Along with the development of hardware such as computer technology and cameras, a large amount of high-quality data of the pavement can be quickly obtained in a short time by using the comprehensive detection vehicle, but the collected pavement image still depends on analysis and judgment of professionals to determine whether the asphalt pavement has diseases or not at present, and the problems of low working efficiency, high labor intensity, poor reproducibility, high labor cost, low accuracy and the like exist in manual analysis.
In recent years, a great deal of research is performed on identifying asphalt pavement diseases by using a digital image processing technology, and most of the methods adopt a target detection algorithm based on a horizontal frame. Although the method reduces the labor cost and improves the detection efficiency to a certain extent, the common Convolutional Neural Network (CNN) does not definitely model the directional change, and the shape and orientation of various diseases have great randomness, so that the accuracy of the method for identifying the diseases is still not high, and the influence area of the diseases is difficult to evaluate.
Disclosure of Invention
The invention provides a pavement disease identification method based on a rotary constant-change detector, aiming at the defects of the prior art. For pavement diseases such as cracks, repairs and the like which are in an elongated strip shape and are oriented randomly, the method can reduce the difference caused by orientation in the same kind and increase the difference caused by shape among different kinds, and improves the recognition precision.
The invention adopts the technical scheme that:
the invention relates to a pavement disease identification method based on a rotary constant-change detector, which mainly comprises three stages:
1. preprocessing road image data;
the road image data preprocessing stage mainly comprises road image sample collection, sample labeling, sample training/verification/test set division and data enhancement expansion adopted by training.
Road/road surface image sample collection uses linear array or area array camera to install in the detection vehicle rear, and can adopt LED lamp etc. to carry out supplementary light filling. And meanwhile, the trigger signal of the mileage sensor controls the camera to shoot at intervals to acquire road surface images of corresponding mileage.
Sample marking, namely, marking a rotating frame OBB according to the following rule by adopting labelme marking software: marking by using a minimum circumscribed rectangular frame, finding corner points of short sides transiting to long sides according to the clockwise direction, defining the corner point at the left as a point 1 (namely coordinates (x 1, y 1)), and sequencing the rest 3 points according to the clockwise direction. The labeling of the rotating border OBB as shown in fig. 2-1 is somewhat more complex than the horizontal border HBB, but the expression for disease location and area of influence is more accurate.
Sample training/verification/test set division, the samples are randomly ordered, and the samples are divided into training, verification and test samples according to the proportion of 8:1:1.
Data enhancement is mainly divided into a single sample data enhancement mode and a sample data enhancement mode. Single sample data enhancement as shown in fig. 3 is relatively simple, including HSV gamut enhancement, random affine transformation, random clipping, random rotation, random scale transformation, random flipping; multisample data enhancement mainly consists of mosaics, mixup, which is used in combination because mosaics enhancement may create pixel holes when generating enhanced samples, adding Mixup helps to fill this defect.
2. Training a pavement disease rotation constant-change detector;
the training phase of the pavement disease rotation constant-variation detector mainly comprises a model structure optimization part, a model training part and a model output part.
Model structure optimization, taking a ReDet model as a prototype (shown in fig. 4). Firstly, the model adds a rotary constant network RPN-RT in a Backbone (Backbone) to generate rotary constant characteristics, so that the direction of a measured object can be accurately predicted, and the complexity of direction change modeling is reduced. In order to extract rotation invariant features from rotation invariant features, the model proposes a new rotation invariant RoI alignment method (rimoi alignment) which can warp regional features according to the bounding box of rotation RoI in the spatial dimension, and extract features aligned to the dimension by repeatedly switching the orientation channels and interpolating features. Finally, the rotated alike backbond and the rimoi alignment are combined to extract the complete rotation invariant feature for accurate rotation bezel object detection.
The model training and optimizing device adopts Adam, and has the advantages of high convergence rate and strong adaptability to sparse gradients. The main parameters are learning rate of the model, loss balance parameters, data enhancement corresponding parameters and the like. In particular, since the multi-sample data enhanced mosaics change the input distribution of the original data, the multi-sample data enhancement continuous training model is turned off at the last 3 epochs of training to achieve higher accuracy.
And outputting a model, wherein a mAP@0.5 index is adopted for model verification, wherein the index is an AP value which is correctly identified by the model when the intersection ratio threshold of a rotating frame is set to 0.5, and then the average value of the APs of all diseases is calculated. Where AP refers to the area enclosed by the Precision-Recall curve generated by testing under different confidence thresholds. During model verification, multiple candidate models trained under multiple parameters are evaluated according to the index, and the optimal model is selected as a final version.
3. The pavement disease rotation alike detector application is deployed.
The application deployment stage of the pavement disease rotation constant-change detector mainly comprises two parts of algorithm model deployment and background business processing. The model deployment module mainly configures operation resources such as a GPU, builds an algorithm model interface and deploys algorithm services. And the background business processing module is used for receiving external requests, analyzing and preprocessing images, interfacing algorithm model interfaces and calculating, finely processing model analysis results, calculating disease parameters, displaying effect processing, outputting results and the like.
The algorithm model deployment is mainly realized based on a torch and triton framework. The image rotation frame target detection model mainly has the following functions: and inputting the normalized image for forward reasoning, extracting rotation and other variable characteristics by the backbone network, and obtaining the category information and the coordinate information of each rotation by regression of the branch network. The algorithm model deployment flow is as follows: performing model format conversion through a torchscript toolkit; configuring the input and output sizes and formats of the models, the number of the deployed models and the occupation of computing resources; building a model interface according to the input and output format of the model; and deploying the model instance by using the model deployment image file and starting the service.
Background business processing is mainly realized by a flask, gunicorn framework, and functions of request input processing, a butt joint algorithm model interface, image base64 coding analysis, image size conversion, image normalization processing, road surface disease parameter calculation, result rendering, output and the like are realized by combining an opencv and other method libraries.
The invention has the beneficial effects that:
1. the pavement disease identification method based on the rotary constant-change detector effectively improves the disease identification precision. The method is mainly based on a ReDet model, and rotation isomorphism and rotation invariance are explicitly encoded by adopting a rotation isomorphism detector, so that rotation isomorphism characteristics can be extracted. For pavement diseases which are in long and thin strips and are random in orientation such as cracks and repair, the method can reduce the difference caused by orientation in the same kind and increase the difference caused by shape among different kinds, and effectively improves the pavement disease identification precision.
2. The pavement disease identification method based on the rotary constant-change detector can describe the influence area of the disease more accurately. Compared with the traditional method, the rotating frame generated based on the ReDet model can describe the influence area of diseases more accurately.
3. The pavement disease identification method based on the rotary constant-change detector has smaller model volume. The ReDet backbone network adopted by the method is ReResNet, and compared with the traditional ResNet, the ReResNet greatly reduces the weight parameter quantity of the rotating backbone network due to the characteristic of rotating weight sharing under the condition that the output channel number is the same. Typically, the ReResNet model volume is only 1/8 of that of the conventional model with similar recognition accuracy.
Drawings
FIG. 1 is a diagram showing a system architecture of a pavement damage identification method according to the present invention;
FIGS. 2-1, 2-2 and 2-3 illustrate the labeling of the rotating frame OBB by using labelme sample labeling software;
FIG. 3 is a schematic diagram showing data enhancement using a combination of single-sample and multiple-sample data enhancement modes;
FIG. 4 is a schematic diagram of a pavement slab rotation constant change detector application deployment;
FIG. 5 shows the flow of the pavement image processing and disease recognition method of the present invention;
fig. 6 shows a conventional detector in comparison with a rotary alike detector.
Detailed Description
In order to make the technical conception and advantages of the invention to achieve the objects of the invention more apparent, the technical scheme of the invention is further described in detail below with reference to the accompanying drawings. It is to be understood that the following examples are intended to illustrate and describe preferred embodiments of the invention and should not be construed as limiting the scope of the invention as claimed.
Example 1
Referring to fig. 5, the pavement disease recognition method based on the rotary constant change detector of the invention comprises the following implementation steps:
1) Preprocessing the acquired road image data, wherein the process comprises the following steps:
(1) Equidistant frame cutting is carried out on the video according to the mileage coding information on all the collected image samples, and a road image data set for training, verification and test is obtained;
in practical application, all image samples are collected by a road detection vehicle, and are shot by adopting a DS-2CD3T87WDV 3-L6.0 mm camera;
(2) Performing rotary frame OBB labeling on the image sample;
(3) Randomly sequencing samples, and dividing the samples into training, verifying and testing samples according to the proportion of 8:1:1;
(4) Enhancing data;
2) The method is characterized in that a rotary constant-change detector is adopted for training the sample pavement diseases, and the process comprises the following steps:
(1) Taking a ReDet model as a prototype, and performing model structure optimization;
(2) Model training is carried out by adopting an Adam optimizer;
(3) Model output:
firstly, calculating an AP value of each disease category correctly identified by a model when the intersection ratio threshold of the rotating frame is set to 0.5, and then, averaging the APs of all the disease categories; during model verification, evaluating a plurality of alternative models trained under various parameters by using the index, and selecting an optimal model as a final version;
3) Pavement defect identification (pavement image processing and defect identification process are shown in fig. 5):
(1) The road image data is normalized (normalized);
(2) The road image data after normalization processing is input into a Rotation constant backbone network (backbone) to obtain a Rotation constant characteristic map (Rotation-equivariance features);
(3) Rotating the isomorphous feature map, entering a regional recommendation network (RPN) and a regional transformation network (RT), and generating a rotating isomorphous recommendation region (RRoIs);
(4) Rotating the equal-change recommended area, entering a rotation-invariant alignment network (RiRoIALign), and extracting rotation-invariant features;
(5) And rotating the invariant features, entering a full connection layer (FC), and respectively outputting the category vectors and the coordinates of the rotating frame.
(6) And the service processing module calculates the disease influence area according to the category vector and the coordinates of each rotating frame. If the rotating frame type is linear diseases, calculating the diagonal length according to the coordinates for evaluating the disease influence area; if the rotating frame type is a planar disease, the influence area is directly calculated according to the coordinates.
In the step 1), labelme sample labeling software is adopted to label the rotating frame OBB. Sample labeling is carried out by labelme labeling software, and rotating frame OBB labeling is carried out according to the following rules: marking by using a minimum circumscribed rectangular frame, finding corner points of short sides transiting to long sides according to the clockwise direction, defining the corner point at the left as a point 1, and sequencing the rest 3 points according to the clockwise direction. As shown in fig. 2-1.
As shown in fig. 2-2, the labeling of the rotating frame OBB is slightly more complex than that of the horizontal frame HBB, and compared with the traditional target detection horizontal frame, the rotating frame is more accurate in disease positioning and influence area evaluation. The conventional horizontal frame B inevitably has more redundant areas than the rotating frame a, and the redundant areas do not actually detect diseases, but the total evaluation is finally included as the influence area of the diseases due to the limitation of the frame characteristics. The accumulation of redundant areas of a full road segment may then result in a road segment that is rated much lower than the actual value. The use of a rotating bezel can therefore be made closer to the actual value.
When multiple diseases are rechecked and evaluated, there is often an overlap of two or more disease selection boxes. As shown in fig. 2-3, in the evaluation of the impact area of the screen crack (frame C) in the figure, the impact area already occupied by the strip repair (frame AorB) needs to be removed. If the traditional horizontal frame B is used, the redundant area is too much, so that the residual area after the frame B covers the frame C is far smaller than the actual value; when the rotating frame A is used, redundant areas are reduced as far as possible, so that the evaluation of the influence area of the frame C network cracks is closer to an actual value.
Example 2
In the pavement disease recognition method based on the rotation constant change detector of the embodiment, unlike in the embodiment 1), in the step 1), data enhancement is performed by adopting two modes of single sample data enhancement and sample data enhancement, wherein the single sample data enhancement comprises HSV color gamut enhancement, random affine transformation, random clipping, random rotation, random scale transformation and random overturn; multisample data enhancement mainly consists of mosaics, mixup.
The process of data enhancement using a combination of single sample and multiple sample data enhancement modes is shown in fig. 3. Such a combination is used because the Mosaic enhancement may create pixel voids when the enhancement samples are generated, and adding mix up helps to fill this deficiency.
Example 3
In the pavement damage recognition method based on the rotation constant detector of the present embodiment, unlike in embodiment 1 or embodiment 2, in step 2), the model structure optimization process is as follows:
firstly, taking a ReDet model as a prototype, and adding a rotation constant change network RPN-RT into an input rotation constant change backbone network to generate rotation constant change characteristics;
secondly, extracting rotation invariant features from rotation invariant features by adopting a rotation invariant RoI alignment method RiRoI alignment;
finally, the rotation-alike backbond and the rimoi alignment are combined to extract complete rotation-invariant features for accurate rotation bezel object detection.
Model structure optimization, taking a ReDet model as a prototype. Firstly, the model adds a rotary constant network RPN-RT in a Backbone (Backbone) to generate rotary constant characteristics, so that the direction of a measured object can be accurately predicted, and the complexity of direction change modeling is reduced. In order to extract rotation invariant features from rotation invariant features, the model proposes a new rotation invariant RoI alignment method (rimoi alignment) which can warp regional features according to the bounding box of rotation RoI in the spatial dimension, and extract features aligned to the dimension by repeatedly switching the orientation channels and interpolating features. Finally, the rotated alike backbond and the rimoi alignment are combined to extract the complete rotation invariant feature for accurate rotation bezel object detection. As shown in fig. 4.
Model training, wherein the optimizer adopts Adam, and mainly adjusts parameters such as learning rate, loss balance parameters and data enhancement corresponding parameters of the model, and the model is continuously trained by closing multi-sample data enhancement at the end of training so as to achieve higher precision.
The road image data is input into a ReDet model, rotation invariant features are generated, the types and coordinates of the rotation frames are obtained based on the features, and finally the influence area of the diseases is obtained through conversion.
Example 4
As shown in fig. 1, the technical scheme of the present invention mainly includes three stages: 1. preprocessing road image data; 2. training a pavement disease rotation constant-change detector; 3. the pavement disease rotation alike detector application is deployed.
1. Road image data preprocessing
The road image data preprocessing stage mainly comprises road image sample collection, sample labeling, sample training/verification/test set division and data enhancement expansion adopted by training.
Road surface image sample collection uses linear array or area array camera to install in the detection vehicle rear, and can adopt LED lamp etc. to carry out supplementary light. And meanwhile, the trigger signal of the mileage sensor controls the camera to shoot at intervals to acquire road surface images of corresponding mileage.
Sample marking, namely, marking a rotating frame OBB according to the following rule by adopting labelme marking software: marking by using a minimum circumscribed rectangular frame, finding corner points of short sides transiting to long sides according to the clockwise direction, defining the corner point at the left as a point 1 (namely coordinates (x 1, y 1)), and sequencing the rest 3 points according to the clockwise direction. The labeling of the rotating frame OBB is slightly more complex than that of the horizontal frame HBB, but the disease positioning and the representation of the affected area are more accurate.
The method comprises the steps of dividing a sample set, randomly sequencing samples, and dividing the samples into training, verifying and testing samples according to the proportion of 8:1:1.
Data enhancement is mainly divided into a single sample data enhancement mode and a sample data enhancement mode. The single sample data enhancement is relatively simple, and comprises HSV color gamut enhancement, random affine transformation, random clipping, random rotation, random scale transformation and random overturning; multisample data enhancement mainly consists of mosaics, mixup, which is used in combination because mosaics enhancement may create pixel holes when generating enhanced samples, adding Mixup helps to fill this defect.
2. Pavement disease rotation constant change detector training
The training phase of the pavement disease rotation constant-variation detector mainly comprises a model structure optimization part, a model training part and a model output part.
Model structure optimization takes a ReDet model as a prototype (shown in the following figure). Firstly, the model adds a rotary constant network RPN-RT in a Backbone (Backbone) to generate rotary constant characteristics, so that the direction of a measured object can be accurately predicted, and the complexity of direction change modeling is reduced. In order to extract rotation invariant features from rotation invariant features, the model proposes a new rotation invariant RoI alignment method (rimoi alignment) which can warp regional features according to the bounding box of rotation RoI in the spatial dimension, and extract features aligned to the dimension by repeatedly switching the orientation channels and interpolating features. Finally, the rotated alike backbond and the rimoi alignment are combined to extract the complete rotation invariant feature for accurate rotation bezel object detection.
The model training and optimizing device adopts Adam, and has the advantages of high convergence rate and strong adaptability to sparse gradients. The main parameters are learning rate of the model, loss balance parameters, data enhancement corresponding parameters and the like. In particular, since the multi-sample data enhanced mosaics change the input distribution of the original data, the multi-sample data enhancement continuous training model is turned off at the last 3 epochs of training to achieve higher accuracy.
And outputting a model, wherein a mAP@0.5 index is adopted for model verification, wherein the index is an AP value which is correctly identified by the model when the intersection ratio threshold of a rotating frame is set to 0.5, and then the average value of the APs of all diseases is calculated. Where AP refers to the area enclosed by the Precision-Recall curve generated by testing under different confidence thresholds. During model verification, multiple candidate models trained under multiple parameters are evaluated according to the index, and the optimal model is selected as a final version.
The application deployment stage of the pavement disease rotation constant-change detector mainly comprises two parts of algorithm model deployment and background business processing. The model deployment module mainly configures operation resources such as a GPU, builds an algorithm model interface and deploys algorithm services. And the background business processing module is used for receiving external requests, analyzing and preprocessing images, interfacing algorithm model interfaces and calculating, finely processing model analysis results, calculating disease parameters, displaying effect processing, outputting results and the like.
The algorithm model deployment is mainly realized based on a torch and triton framework. The image rotation frame target detection model mainly has the following functions: and inputting the normalized image for forward reasoning, extracting rotation and other variable characteristics by the backbone network, and obtaining the category information and the coordinate information of each rotation by regression of the branch network. The algorithm model deployment flow is as follows: performing model format conversion through a torchscript toolkit; configuring the input and output sizes and formats of the models, the number of the deployed models and the occupation of computing resources; building a model interface according to the input and output format of the model; and deploying the model instance by using the model deployment image file and starting the service.
Background business processing is mainly realized by a flask, gunicorn framework, and functions of request input processing, a butt joint algorithm model interface, image base64 coding analysis, image size conversion, image normalization processing, road surface disease parameter calculation, result rendering, output and the like are realized by combining an opencv and other method libraries.
The pavement disease recognition method based on the rotation alike detector is mainly based on a ReDet model, and the rotation alike detector is adopted to explicitly code rotation alike changes and rotation invariance, namely rotation alike change characteristics can be extracted. For pavement diseases which are in an elongated strip shape and are random in orientation such as cracks and repair, the difference caused by orientation in the same kind of pavement diseases can be reduced, the difference caused by shape among different kinds of pavement diseases can be increased, the recognition precision is improved, and the generated rotating frame can describe the influence area of the diseases more accurately.
3. Application deployment of pavement disease rotation constant-change detector
The application deployment stage of the pavement disease rotation constant-change detector mainly comprises two parts of algorithm model deployment and background business processing. The model deployment module mainly configures operation resources such as a GPU, builds an algorithm model interface and deploys algorithm services. And the background business processing module is used for receiving external requests, analyzing and preprocessing images, interfacing algorithm model interfaces and calculating, finely processing model analysis results, calculating disease parameters, displaying effect processing, outputting results and the like.
The algorithm model deployment is mainly realized based on a torch and triton framework. The image rotation frame target detection model mainly has the following functions: and inputting the normalized image for forward reasoning, extracting rotation and other variable characteristics by the backbone network, and obtaining the category information and the coordinate information of each rotation by regression of the branch network. The algorithm model deployment flow is as follows: performing model format conversion through a torchscript toolkit; configuring the input and output sizes and formats of the models, the number of the deployed models and the occupation of computing resources; building a model interface according to the input and output format of the model; and deploying the model instance by using the model deployment image file and starting the service.
Background business processing is mainly realized by a flask, gunicorn framework, and functions of request input processing, a butt joint algorithm model interface, image base64 coding analysis, image size conversion, image normalization processing, road surface disease parameter calculation, result rendering, output and the like are realized by combining an opencv and other method libraries.
As shown in FIG. 6, the present invention employs a rotating alike detector that requires fewer sample labels during the training process to achieve the same generalization capability than conventional detectors. As shown in the left side of fig. 6, the conventional detector needs to learn crack labeling samples in all directions to adapt to the complex direction and angle change of cracks in actual detection; however, as shown in the right side of fig. 6, the rotation alike detector can learn and generalize from a single-direction sample to obtain detection capabilities of various angles due to the characteristic of rotation alike.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention. Other modifications of the practice of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention without the need for inventive faculty, and any modification or substitution of equivalents which fall within the spirit and principles of the invention, or which are obvious to those skilled in the art, are intended to be encompassed within the scope of the invention.
Claims (4)
1. A pavement disease recognition method based on a rotary constant-change detector is characterized by comprising the following steps of: the method comprises the following steps:
1) Preprocessing the acquired road image data, wherein the process comprises the following steps:
(1) Equidistant frame cutting is carried out on the video according to the mileage coding information on all the collected image samples, and a road image data set for training, verification and test is obtained;
(2) Carrying out rotating frame OBB labeling on the image sample:
labeling the image sample by labelme labeling software, and labeling the rotating frame OBB according to the following rules:
marking by using a minimum circumscribed rectangular frame, finding corner points of short sides transiting to long sides according to the clockwise direction, defining the corner point at the left as a point 1, and sequencing the rest 3 points according to the clockwise direction;
(3) Randomly sequencing samples, and dividing the samples into training, verifying and testing samples according to the proportion of 8:1:1;
(4) Enhancing data;
2) The method comprises the steps of training the pavement diseases of the image sample by adopting a rotary constant-change detector, inputting road image data into a ReDet model, generating a rotary constant characteristic, obtaining the category and the coordinate of a rotary frame based on the characteristic, and finally obtaining the influence area of the diseases by conversion, wherein the process comprises the following steps:
(1) Taking a ReDet model as a prototype, and performing model structure optimization;
(2) Model training is carried out by adopting an Adam optimizer;
(3) Model output:
firstly, calculating an AP value of each disease category correctly identified by a model when the intersection ratio threshold of the rotating frame is set to 0.5, and then, averaging the APs of all the disease categories;
during model verification, evaluating a plurality of alternative models trained under various parameters by using the AP value index, and selecting an optimal model as a final version;
3) The application and deployment stage of the pavement disease rotation constant-change detector;
the application deployment stage of the pavement disease rotation constant-change detector mainly comprises two parts of algorithm model deployment and background business processing;
the algorithm model deployment is realized based on a torch and triton frame, GPU operation resources are configured, an algorithm model interface is built, and algorithm service is deployed; the algorithm model deployment flow is as follows:
performing model format conversion through a torchscript toolkit; configuring the input and output sizes and formats of the models, the number of the deployed models and the occupation of computing resources; building a model interface according to the input and output format of the model; deploying an image file deployment model instance by using the model deployment and starting a service;
the algorithm model mainly comprises the following functions: inputting normalized images for forward reasoning, extracting rotation and other variable characteristics by a backbone network, and obtaining category information and coordinate information of each rotation by regression of a branch network;
the background business processing is mainly realized by a flask, gunicorn framework, and is used for receiving external requests, analyzing and preprocessing images, interfacing algorithm model interfaces and calculating, finely processing model analysis results, calculating disease parameters, displaying effect processing and outputting results; combining an opencv method library to realize request input processing, a docking algorithm model interface, image base64 coding analysis, image size conversion, image normalization processing, road surface disease parameter calculation, result rendering and output;
the pavement disease identification process comprises the following steps:
(1) Carrying out normalization processing on the road image data;
(2) The road image data after normalization processing is input into a rotary isovariational backbone network backup to obtain a rotary isovariational characteristic map;
(3) Rotating the isomorphous feature map, entering an area recommendation network RPN and an area transformation network RT, and generating a rotating isomorphous recommendation area RRoIs;
(4) Rotating the equal-change recommended area, entering a rotation-invariant alignment network RiRoIALign, and extracting rotation-invariant features;
(5) Rotating the invariant feature, entering a full connection layer FC, and respectively outputting a class vector and a coordinate of the rotating frame;
(6) The service processing module calculates the disease influence area according to the category vector and the coordinates of each rotating frame; if the rotating frame type is linear diseases, calculating the diagonal length according to the coordinates for evaluating the disease influence area; if the rotating frame type is a planar disease, the influence area is directly calculated according to the coordinates.
2. The method for identifying road surface damage based on a rotary constant change detector according to claim 1, wherein: in the step 1), data enhancement is carried out by adopting two modes of single sample data enhancement and multiple sample data enhancement, wherein the single sample data enhancement comprises HSV color gamut enhancement, random affine transformation, random clipping, random rotation, random scale transformation and random overturning; multisample data enhancement mainly consists of mosaics, mixup.
3. The pavement defect identification method based on the rotary constant change detector according to claim 1 or 2, characterized in that: in the step 2), the model structure optimization process is as follows:
firstly, taking a ReDet model as a prototype, and adding a rotation constant change network RPN-RT into an input rotation constant change backbone network to generate rotation constant change characteristics;
secondly, extracting rotation invariant features from rotation invariant features by adopting a rotation invariant RoI alignment method RiRoI alignment;
finally, the rotation-alike backbond and the rimoi alignment are combined to extract complete rotation-invariant features for accurate rotation bezel object detection.
4. The method for identifying road surface damage based on rotary alike detector according to claim 3, wherein: model training, wherein the optimizer adopts Adam, and mainly adjusts parameters such as learning rate, loss balance parameters and data enhancement corresponding parameters of the model, and the model is continuously trained by closing multi-sample data enhancement at the end of training so as to achieve higher precision.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211533695.0A CN115984174B (en) | 2022-12-02 | 2022-12-02 | Pavement disease identification method based on rotary constant-change detector |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211533695.0A CN115984174B (en) | 2022-12-02 | 2022-12-02 | Pavement disease identification method based on rotary constant-change detector |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115984174A CN115984174A (en) | 2023-04-18 |
| CN115984174B true CN115984174B (en) | 2024-03-22 |
Family
ID=85972897
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211533695.0A Active CN115984174B (en) | 2022-12-02 | 2022-12-02 | Pavement disease identification method based on rotary constant-change detector |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115984174B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113723208A (en) * | 2021-08-05 | 2021-11-30 | 北京大学 | Three-dimensional object shape classification method based on normative equal transformation conversion sub-neural network |
| CN114170487A (en) * | 2021-12-08 | 2022-03-11 | 北京计算机技术及应用研究所 | Vision-based water surface oil stain detection method |
| CN115393642A (en) * | 2022-08-22 | 2022-11-25 | 东南大学 | Highway asphalt pavement disease detection method based on target detection |
-
2022
- 2022-12-02 CN CN202211533695.0A patent/CN115984174B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113723208A (en) * | 2021-08-05 | 2021-11-30 | 北京大学 | Three-dimensional object shape classification method based on normative equal transformation conversion sub-neural network |
| CN114170487A (en) * | 2021-12-08 | 2022-03-11 | 北京计算机技术及应用研究所 | Vision-based water surface oil stain detection method |
| CN115393642A (en) * | 2022-08-22 | 2022-11-25 | 东南大学 | Highway asphalt pavement disease detection method based on target detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115984174A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | CCTSDB 2021: a more comprehensive traffic sign detection benchmark | |
| Lei et al. | Automated pavement distress detection and deterioration analysis using street view map | |
| CN109977782B (en) | Cross-store operation behavior detection method based on target position information reasoning | |
| CN110619279B (en) | Road traffic sign instance segmentation method based on tracking | |
| CN108921120B (en) | Cigarette identification method suitable for wide retail scene | |
| CN113160192A (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| CN114998852A (en) | Intelligent detection method for road pavement diseases based on deep learning | |
| CN110956207B (en) | Method for detecting full-element change of optical remote sensing image | |
| CN111476084A (en) | Deep learning-based parking lot dynamic parking space condition identification method | |
| CN113780200A (en) | Computer vision-based pavement multi-disease area detection and positioning method | |
| CN114677501A (en) | License plate detection method based on two-dimensional Gaussian bounding box overlapping degree measurement | |
| Bu et al. | A UAV photography–based detection method for defective road marking | |
| CN117422699A (en) | Highway detection method, highway detection device, computer equipment and storage medium | |
| CN115035251A (en) | Bridge deck vehicle real-time tracking method based on domain-enhanced synthetic data set | |
| CN117789148A (en) | Lane line detection method, system, equipment and storage medium | |
| CN117437615A (en) | Foggy day traffic sign detection method and device, storage medium and electronic equipment | |
| CN115346206B (en) | License plate detection method based on improved super-resolution deep convolution feature recognition | |
| Chang et al. | Semi-supervised learning for YOLOv4 object detection in license plate recognition system | |
| CN115984174B (en) | Pavement disease identification method based on rotary constant-change detector | |
| CN112053407A (en) | Automatic lane line detection method based on AI technology in traffic law enforcement image | |
| CN114882490B (en) | Unlimited scene license plate detection and classification method based on point-guided positioning | |
| CN115223114A (en) | End-to-end vehicle attitude estimation method based on bidirectional fusion feature pyramid | |
| CN111583341B (en) | Cloud deck camera shift detection method | |
| CN114332006A (en) | Automatic quantitative assessment method for urban battlement loss | |
| CN115063684A (en) | Agricultural machinery track identification method based on remote sensing image scene division and application method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |