CN117422699A - Highway detection method, highway detection device, computer equipment and storage medium - Google Patents
Highway detection method, highway detection device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN117422699A CN117422699A CN202311499044.9A CN202311499044A CN117422699A CN 117422699 A CN117422699 A CN 117422699A CN 202311499044 A CN202311499044 A CN 202311499044A CN 117422699 A CN117422699 A CN 117422699A
- Authority
- CN
- China
- Prior art keywords
- road
- road surface
- pavement
- image
- disease
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 304
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 201000010099 disease Diseases 0.000 claims abstract description 159
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 159
- 230000001133 acceleration Effects 0.000 claims abstract description 55
- 238000004364 calculation method Methods 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims description 104
- 238000000034 method Methods 0.000 claims description 69
- 238000012423 maintenance Methods 0.000 claims description 36
- 238000009826 distribution Methods 0.000 claims description 32
- 238000010586 diagram Methods 0.000 claims description 27
- 230000015654 memory Effects 0.000 claims description 27
- 238000007781 pre-processing Methods 0.000 claims description 19
- 230000007547 defect Effects 0.000 claims description 12
- 238000012937 correction Methods 0.000 claims description 10
- 238000013441 quality evaluation Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 4
- 238000007689 inspection Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 18
- 230000006870 function Effects 0.000 description 13
- 239000010426 asphalt Substances 0.000 description 11
- 239000004568 cement Substances 0.000 description 11
- 239000013598 vector Substances 0.000 description 10
- 238000013461 design Methods 0.000 description 9
- 239000000463 material Substances 0.000 description 9
- 230000011218 segmentation Effects 0.000 description 9
- 230000005484 gravity Effects 0.000 description 8
- 238000012360 testing method Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 6
- 230000001629 suppression Effects 0.000 description 6
- 230000009466 transformation Effects 0.000 description 6
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 230000008447 perception Effects 0.000 description 5
- 238000013145 classification model Methods 0.000 description 4
- 238000005286 illumination Methods 0.000 description 4
- 230000001965 increasing effect Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000008439 repair process Effects 0.000 description 3
- 239000011800 void material Substances 0.000 description 3
- 238000013016 damping Methods 0.000 description 2
- 239000000428 dust Substances 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 101100518501 Mus musculus Spp1 gene Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000036039 immunity Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003121 nonmonotonic effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Abstract
The invention relates to the technical field of deep learning, and discloses a highway detection method, a highway detection device, a highway detection computer device and a highway storage medium, wherein the highway detection method comprises the following steps: acquiring a pavement image to be measured; constructing a pavement disease detection model; inputting the pavement image to be detected into a pavement disease detection model, and detecting to obtain the pavement type and the disease area of the pavement to be detected; collecting acceleration data and sensor attitude data of a vehicle passing through a road surface to be measured; the road technical condition detection result of the road to be detected is obtained by calculation based on the acceleration data, the sensor posture data, the road type and the disease area of the road to be detected, and the road technical condition detection method can automatically identify and obtain the disease of the road to be detected, so that the interference of human factors is reduced, and the detection accuracy is improved. The road detection method is used for comprehensively covering various complex scenes of road detection, and the detection result is more accurate and comprehensive. Comprehensive road technical condition assessment is performed comprehensively, comprehensive road condition information is timely provided, the road is convenient to maintain timely, and unnecessary property loss is avoided.
Description
Technical Field
The invention relates to the technical field of deep learning, in particular to a highway detection method, a highway detection device, a highway detection computer device and a highway storage medium.
Background
The rural highway is used as an important traffic infrastructure for connecting cities and rural areas, and plays a key role in rural economic development, people's life, material transportation and the like. However, rural highways generally face the problems of limited maintenance resources, various types, high detection difficulty and the like, and in order to ensure the safety and smoothness of the rural highways, it is particularly important to discover and repair diseases in time.
In the related art, because the rural highway detection scene has complexity, the highway detection usually relies on manual detection and evaluation, the existing highway detection method is insufficient to cover the complex scene of the rural highway detection, the manual detection has the problem of low efficiency, and the problem of inaccurate detection results caused by human factors, so that the disease condition and the highway road condition of the rural highway cannot be comprehensively and accurately mastered in time, and the rural highway cannot be timely maintained, thereby causing unnecessary property loss.
Disclosure of Invention
In view of the above, the present invention provides a road detection method, apparatus, computer device and storage medium, so as to solve the problem that the disease condition and road condition of rural roads cannot be comprehensively and accurately mastered in time.
In a first aspect, the present invention provides a road detection method, the method comprising:
acquiring a pavement image to be measured;
constructing a pavement disease detection model;
inputting the pavement image to be detected into a pavement disease detection model, and detecting to obtain the pavement type and the disease area of the pavement to be detected;
collecting acceleration data and sensor attitude data of a vehicle passing through a road surface to be measured;
and calculating to obtain a road technical condition detection result of the road surface to be detected based on the acceleration data, the sensor posture data, the road surface type and the disease area of the road surface to be detected.
According to the invention, the road surface disease detection model is constructed, so that the road surface disease to be detected can be automatically identified, the interference of human factors is reduced, and the detection accuracy is improved. The road surface disease detection model obtained based on a large number of road surface disease image training can cover various complex scenes of road detection more comprehensively, and the detection result is more accurate and comprehensive. By comprehensively evaluating the technical condition of the road, comprehensive road condition information is provided more timely, more targeted support is provided for future maintenance schemes, the road is convenient to maintain in time, and unnecessary property loss is avoided.
In an alternative embodiment, constructing a pavement damage detection model includes:
acquiring a road surface image dataset comprising: the method comprises the steps of providing a pavement image with defects, a real pavement type corresponding to the pavement image and real frame coordinates of the defect position on the pavement image;
preprocessing and data enhancement are carried out on the pavement image to obtain a disease training image set;
inputting the disease training image set into an initial pavement disease detection model to obtain a predicted pavement type and predicted frame coordinates of the disease training image set;
calculating to obtain the total training loss based on the predicted road surface type and predicted frame coordinates, the real road surface type and real frame coordinates of the disease training image set;
and training the initial pavement disease detection model based on the training total loss to obtain a target pavement disease detection model.
In the mode, the pavement disease detection model is trained by utilizing a large number of disease images, the disease result obtained by the detection of the pavement disease detection model is more accurate, and the generalization and the robustness of the pavement disease detection model are further improved by preprocessing the pavement images and combining the structural design of the pavement disease detection model.
In an alternative embodiment, inputting the image of the road surface to be detected into the road surface disease detection model, detecting the road surface type and the disease area of the road surface to be detected, including:
inputting the road surface image to be detected into a target road surface disease detection model to obtain the road surface type and disease coordinates of the road surface to be detected;
acquiring a calibration plate data set, and calculating to obtain an area thermal distribution map of the real world based on the calibration plate data set, wherein the calibration plate data set comprises: calibration plate images containing calibration plates are collected under different angles and positions;
and calculating the disease area of the pavement to be measured based on the area thermal distribution diagram and by combining the disease coordinates.
In the mode, the area distance measurement system is established by setting the calibration image, so that the area and the length of the pavement damage can be detected more accurately, and the accuracy of pavement damage detection is improved.
In an alternative embodiment, a calibration plate dataset is obtained, and a real world area thermal profile is calculated based on the calibration plate dataset, comprising:
marking the position of the calibration plate in the calibration plate image to obtain a boundary frame corresponding to the calibration plate, wherein the boundary frame comprises the position and the size of the calibration plate in the calibration plate image;
Carrying out edge correction on the boundary frame, determining the inclination angles of the edges of the calibration plate, and calculating to obtain the relation between the unit pixel area in the calibration image and the coordinates in the real world;
and calculating area values of all pixels in the calibration plate image based on the relation between the unit pixel area in the calibration image and the coordinates in the real world, and obtaining the real-world area thermal distribution map.
In the method, a boundary frame is obtained by marking the calibration image, operations such as segmentation and edge correction are performed on the boundary frame, the area of each pixel point in the image corresponding to the real world is determined, and then the area distribution thermodynamic diagram of the real world is determined, so that the disease area of the road surface to be measured in the real world can be conveniently obtained through subsequent calculation.
In an alternative embodiment, the calculating to obtain the road technical condition detection result of the road to be detected based on the acceleration data, the sensor posture data, the road type and the disease area of the road to be detected includes:
calculating to obtain a road surface flatness index of the road surface to be measured based on the acceleration data and the sensor attitude information;
calculating to obtain the road surface running quality index of the road surface to be measured based on the road surface flatness index;
Calculating to obtain a road surface damage condition index of the road surface to be detected based on the road surface type and the disease area of the road surface to be detected;
and calculating to obtain a road technical condition detection result of the road surface to be detected based on the road surface running quality index and the road surface damage condition index.
In the mode, the road surface running quality detection is carried out by collecting information through the sensor, the road technical condition detection result of the road surface to be detected is obtained through calculation, the light-weight and automatic road flatness detection is realized, the carrying and the deployment are convenient, the manpower and the material resources are greatly saved, and the working efficiency is improved.
In an alternative embodiment, the method further comprises:
constructing a line facility detection model;
inputting the road surface image into a line facility detection model, and detecting to obtain line facilities in the road surface image;
and determining the position information of the pavement defect based on the line facilities in the pavement image.
In the mode, the line facility detection model is constructed, so that the line facility of the road surface to be detected can be automatically identified, interference of human factors is reduced, and the accuracy of the line facility detection is improved. And by determining the position information of the pavement diseases, the follow-up accurate maintenance of the diseases of the pavement to be tested is facilitated.
In an alternative embodiment, constructing a line facility detection model includes:
acquiring a line facility image dataset comprising: the method comprises the steps of providing a line facility image, a real category corresponding to the line facility image and real frame coordinates;
preprocessing and data enhancement are carried out on the along-line facility images to obtain a along-line facility training image set;
inputting the line facility training image set into an initial line facility detection model to obtain a prediction frame type and a prediction frame coordinate of the line facility training image set;
calculating to obtain total loss of the facility training along the line based on the predicted frame type and predicted frame coordinates, the real type and real frame coordinates of the facility training image set along the line;
training the initial along-line facility detection model based on the total loss of the along-line facility training to obtain a target along-line facility detection model.
In the mode, the line facility detection model is trained by utilizing a large number of line facility images, the line facility result obtained by detecting the line facility detection model is more accurate, and the generalization and the robustness of the line facility detection model are further improved by preprocessing the line facility images and combining the structural design of the line facility detection model.
In an alternative embodiment, the method further comprises:
and carrying out quality evaluation on the road to be tested based on the road technical condition detection result of the road to be tested, and determining a maintenance scheme of the road to be tested.
In the method, through overall technical condition assessment according to the detection result, more comprehensive road condition information can be provided for users, more targeted suggestions are given to future maintenance methods and maintenance plans, and accurate pavement maintenance is facilitated in time.
In a second aspect, the present invention provides a road detection apparatus, the apparatus comprising:
the image acquisition module is used for acquiring a pavement image to be detected;
the model construction module is used for constructing a pavement disease detection model;
the pavement damage detection module is used for inputting the pavement image to be detected into the pavement damage detection model, and detecting the pavement type and the damage area of the pavement to be detected;
the sensor data acquisition module is used for acquiring acceleration data and sensor attitude data of the vehicle passing through the road surface to be detected;
the technical condition calculation module is used for calculating and obtaining a road technical condition detection result of the road to be detected based on the acceleration data, the sensor attitude data, the road type and the disease area of the road to be detected.
In a third aspect, the present invention provides a computer device comprising: the road detection system comprises a memory and a processor, wherein the memory and the processor are in communication connection, the memory stores computer instructions, and the processor executes the computer instructions, so that the road detection method of the first aspect or any corresponding implementation mode is executed.
In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the road detection method of the first aspect or any of its corresponding embodiments.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a road detection method according to an embodiment of the invention.
Fig. 2 is a flow chart of a lightweight road detection method according to an embodiment of the invention.
Fig. 3 is a flowchart of another road detection method according to an embodiment of the invention.
Fig. 4 is a construction diagram of a road surface damage detection model of a lightweight road detection method according to an embodiment of the present invention.
Fig. 5 is a block diagram of a backbone network in a road surface damage detection model of a lightweight road detection method according to an embodiment of the present invention.
Fig. 6 is a structural view of a neck network in a road surface damage detection model of a lightweight road detection method according to an embodiment of the present invention.
Fig. 7 is a block diagram of a head network in a road surface damage detection model of a lightweight road detection method according to an embodiment of the present invention.
FIG. 8 is a calibration plate style diagram according to an embodiment of the present invention.
Fig. 9 is a flowchart of still another road detection method according to an embodiment of the invention.
Fig. 10 is a flowchart of still another road detection method according to an embodiment of the invention.
Fig. 11 is a schematic diagram of a line facility deduction standard of a lightweight road detection method according to an embodiment of the present invention.
Fig. 12 is a block diagram of a road detecting apparatus according to an embodiment of the present invention.
Fig. 13 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the related art, because the rural highway detection scene has complexity, the highway detection usually relies on manual detection and evaluation, the existing highway detection method is insufficient to cover the complex scene of the rural highway detection, the manual detection has the problem of low efficiency, and the problem of inaccurate detection results caused by human factors, so that the disease condition and the highway road condition of the rural highway cannot be comprehensively and accurately mastered in time, and the rural highway cannot be timely maintained, thereby causing unnecessary property loss.
In order to solve the above-mentioned problems, in the embodiments of the present invention, a highway detection method is provided for a computer device, and it should be noted that an execution body of the highway detection method may be a highway detection device, and the device may be implemented as part or all of the computer device in a manner of software, hardware or a combination of software and hardware, where the computer device may be a terminal, a client, or a server, and the server may be a server, or may be a server cluster formed by multiple servers. In the following method embodiments, the execution subject is a computer device.
The computer equipment in the embodiment is suitable for use scenes for detecting possible diseases on the pavement of rural highways. By the road detection method, the road disease detection model is constructed, so that the road disease to be detected can be automatically identified, interference of human factors is reduced, and detection accuracy is improved. The road surface disease detection model obtained based on a large number of road surface disease image training can cover various complex scenes of road detection more comprehensively, and the detection result is more accurate and comprehensive. By comprehensively evaluating the technical condition of the road, comprehensive road condition information is provided more timely, more targeted support is provided for future maintenance schemes, the road is convenient to maintain in time, and unnecessary property loss is avoided.
In accordance with an embodiment of the present invention, there is provided a highway inspection method embodiment, it being noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system, such as a set of computer executable instructions, and, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order other than that illustrated herein.
In this embodiment, a highway detection method is provided, which may be used in the above terminal, such as a mobile phone, a tablet pc, etc., fig. 1 is a flowchart of a highway detection method according to an embodiment of the present invention, and as shown in fig. 1, the flowchart includes the following steps:
step S101, obtaining a pavement image to be detected.
In one example, road images required for training are collected, a machine vision algorithm assists a inspector in screening high-quality images, and then two data sets are made, wherein the two data sets respectively comprise a line facility color image collected by a front camera and a road surface disease gray level image collected by a rear infrared camera, and the line facility color image and the road surface disease gray level image are named as a line facility data set and a road surface disease data set.
Preferably, the method also comprises the step of preprocessing the facility images and the road surface disease images along the line, and mainly adopts data enhancement methods such as white balance, image noise reduction, USM sharpening, affine transformation, random clipping, super resolution, image mixing, histogram equalization, mosaic and the like to improve the image quality and detection accuracy.
In the disease detection process, different illumination conditions can influence the overall exposure degree and contrast of an image, and uneven illumination can generate shadows and high-light areas, so that disease features are difficult to distinguish; meanwhile, the collected images are always provided with noise mainly from dust attached to a lens, raised dust on the road surface and strong light reflected by the road surface, and the noise can reduce the quality of the images and influence the visibility of details and the identification accuracy of features, so that the noise can be reduced to effectively improve the image quality, and the image analysis is facilitated; in addition, the camera or the shooting object moves fast in the exposure process, and image distortion possibly caused can generate dynamic blurring to influence the detection result. Therefore, the problems of excessive exposure and insufficient exposure of the image are solved by adopting a color constancy algorithm, the image contrast is improved by adopting a self-adaptive histogram equalization method, image denoising and image super-resolution are performed by adopting a Swin transform, the diversity of detection targets and the sensitivity to targets with different scales are improved by adopting an image mixing and Mosaic algorithm, the blurring is detected by adopting Fourier transform to analyze the frequency domain characteristics of the image, the blurring degree of the image is evaluated by calculating the frequency spectrum energy distribution after Fourier transform, the preprocessed road surface image to be detected is obtained, and the preprocessed road surface image to be detected is utilized for subsequent road surface disease detection.
And S102, constructing a pavement disease detection model.
In an example, a pavement damage detection model for detecting pavement damage is constructed, the pavement damage detection model is trained by using a damage data set, super parameters of the pavement damage model are adjusted to perform multiple training, the accuracy, recall and average precision values are compared, and an optimal model result is stored as a target pavement damage detection model. The pavement disease detection model structure is based on a YoloV8 model, and an improved pavement disease detection method and an optimized design of a training process are added on the basis of the YoloV8 model.
Step S103, inputting the pavement image to be detected into a pavement damage detection model, and detecting to obtain the pavement type and the damage area of the pavement to be detected.
In an example, the road surface image to be measured is input into a trained road surface damage detection model, and the road surface type and the damage area of the road surface to be measured are obtained through output.
Step S104, collecting acceleration data and sensor attitude data of the vehicle passing through the road surface to be detected.
In one example, road flatness and road running quality index of the road surface to be measured are measured by nine-axis sensors. The nine-axis sensor consists of a three-axis acceleration sensor, a three-axis gyroscope and a three-axis geomagnetic sensor. The collecting acceleration data and sensor attitude data of the vehicle passing through the road surface to be measured comprises the following steps: and acquiring triaxial acceleration data and sensor attitude information of the vehicle passing through the road surface to be measured. Specifically, fixing a nine-axis sensor in a detection vehicle for initial state calibration, measuring an initial value of gravity on the sensor, and recording; measuring roads with not less than three sections and not less than thirty meters in each section by a three-meter ruler method, and calculating the flatness index of each road; then, the detecting vehicle passes through the road section at a constant speed, and nine-axis numerical values of the sensor are recorded; finally, calculating the vertical acceleration, and establishing a relation between the vertical acceleration data and the flatness index, wherein the formula is as follows:
a＝0+a x i+a y j+a z k
A＝u x i+u y j+u z k
Wherein a is x ，a y ，a z The three-axis acceleration of the nine-axis sensor detected by the sensor is respectively measured in the transverse, longitudinal and vertical directions, a is the quaternion representation of the three-axis acceleration, g is the quaternion representation of the gravity,representing the conjugate matrix of q, u x ，u y ，u z The unit vectors are in the transverse direction, the longitudinal direction and the vertical direction respectively, A is the unit vector representation of the rotation axis, theta is the rotation angle of the sensor relative to the initial horizontal state, and i, j and k are imaginary units. a 'and g' are acceleration quaternions after rotation and gravity quaternions after rotation. Wherein, the following relations are satisfied among i, j and k:
i 2 ＝j 2 ＝k 2 ＝-1
ij＝-ji＝k
jk＝-kj＝i
ik＝-ki＝j
finally obtaining a 'and g' which are acceleration quaternion after rotation and gravity after rotationThe element number, for which the imaginary part coefficient a about k is taken respectively k And g k Subtracting a from the two h ＝a k -g k Obtaining the final vertical acceleration a h . Filtering all vertical accelerations by using a trapezoidal window function and combining a high-pass filter, eliminating low-frequency white noise and noise (such as vehicle body vibration after a vehicle is started, errors of a sensor, environmental influence and the like), reserving acceleration information generated by uneven pavement, and calculating to obtain a pavement evenness index IRI and a pavement running quality index RQI, wherein the following formula is shown:
Wherein f is the sampling frequency of the sensor, defaults to 100Hz, a m The M-th acceleration data of the sensor are all acceleration data collected by the sensor, S is the driving distance, and the unit is km; establishing a calibration flatness alpha IRI And manually detecting flatnessThe relationship between ω and β is a set of approximate solutions to the relationship between ω and β, thus removing the impact of the vehicle damping performance on the flatness measurement.
Step S105, calculating to obtain a road technical condition detection result of the road surface to be detected based on the acceleration data, the sensor attitude data, the road surface type and the disease area of the road surface to be detected.
In one example, the road technical condition index PQI is calculated by calculating the line facility condition index TCI, the road running quality index RQI calculated by using the nine-axis sensor, the road damage condition index PCI, and the road running quality index RQI according to the line facility detection result, and the low-level rural road technical condition index MQI is comprehensively obtained by combining the manually assessed roadbed technical condition index SCI and the bridge and tunnel structure technical condition index BCI.
In an implementation scenario, fig. 2 is a flowchart of a lightweight road detection method according to an embodiment of the invention. As shown in fig. 2, the lightweight road detection method includes: collecting and marking road surface pictures, and then preprocessing; inputting the pictures into a target detection model for training; adjusting the super parameters of the target detection model and repeating training to obtain a final model; and carrying out road technology assessment according to the result, generating a detection report, and giving out road maintenance suggestions. Wherein the road technology assessment comprises: establishing a real world coordinate system according to the calibration image; calibrating and measuring road flatness; the design and training is for target detection along line facility detection.
Specifically, the lightweight road detection method includes: (1) The method comprises the steps of collecting road images required by training, assisting a detector to screen high-quality images by a machine vision algorithm, then manufacturing two data sets, respectively comprising a line facility color image collected by a front camera and a road surface disease gray level image collected by a rear infrared camera, naming the line facility color image and the road surface disease gray level image as a line facility data set and a road surface disease data set, and then dividing the two data sets into a training data set and a test data set.
(2) The method is characterized by mainly adopting data enhancement methods such as white balance, image noise reduction, USM sharpening, affine transformation, random clipping, super-resolution, image mixing, histogram equalization, mosaics and the like to preprocess the line facility images and the road surface disease images.
(3) And manufacturing a calibration plate, collecting an image of the calibration plate by using a rear infrared camera, marking the calibration plate in the image by means of marking software, manufacturing a calibration plate data set, and dividing the calibration plate data set into a training data set and a test data set.
(4) And designing an example segmentation model, transmitting calibration plate training data into the model for training, then performing result evaluation by using the calibration plate test data, repeatedly adjusting model super-parameters, and optimizing output results.
(5) And determining the position information, the corner coordinates and the edge contour of the calibration plate in the calibration picture through a corner detection algorithm, a camera distortion correction algorithm, an edge detection algorithm and an example segmentation model, and then calculating the real world area thermodynamic diagram according to the information.
(6) And training a road surface classification model by using the road surface data set, classifying the cement road surface, the asphalt road surface and the gravel road surface, and then repeatedly adjusting super parameters to obtain an optimal model.
(7) The method comprises the steps of designing a pavement damage detection model for detecting pavement damage, optimizing the model aiming at pavement damage features based on a YOLO structure, adjusting the structure, adding switchable cavity convolution and deformable convolution, adding softening non-maximum suppression and bounding box regression loss based on a dynamic focusing mechanism, and optimizing a training flow of the model.
(8) And training a pavement disease detection model by using the disease data set, adjusting model superparameters to perform multiple training, comparing the accuracy, recall rate and average accuracy values, and storing an optimal model result.
(9) By analyzing the known road surface condition information, an expert system customized based on a natural language large model is used for giving road surface quality evaluation and maintenance advice and predicting future maintenance trend.
(10) And designing a target detection model for detecting facilities along rural highways and performing structural optimization aiming at target characteristics.
(11) Training a line facility detection model by using a line facility set, adjusting model superparameters to perform multiple training, comparing accuracy, recall and average accuracy values, and storing an optimal model result.
(12) And obtaining information such as the type, the area, the severity degree, the pile number, the area and the like of the diseases according to the disease detection result and the three-dimensional world reconstruction result, and in addition, calculating the road surface damage condition index PCI and the road surface damage condition rating.
(13) And collecting and detecting acceleration change information in the running process of the vehicle by using a nine-axis sensor, and calculating a road surface flatness index IRI and a road surface running quality index RQI according to the acceleration.
(14) The road technical condition index PQI is calculated by calculating a road facility condition index TCI along the line according to the detection result of the facility along the line, a road running quality index RQI calculated by using a nine-axis sensor, a road damage condition index PCI and a road running quality index RQI, and the low-grade rural road technical condition index MQI is comprehensively obtained by combining the manually assessed roadbed technical condition index SCI and the bridge tunnel structure technical condition index BCI.
According to the road detection method, the road disease detection model is built, so that the road disease to be detected can be automatically identified, interference of human factors is reduced, and detection accuracy is improved. The road surface disease detection model obtained based on a large number of road surface disease image training can cover various complex scenes of road detection more comprehensively, and the detection result is more accurate and comprehensive. By comprehensively evaluating the technical condition of the road, comprehensive road condition information is provided more timely, more targeted support is provided for future maintenance schemes, the road is convenient to maintain in time, and unnecessary property loss is avoided. In this embodiment, a road detection method is provided, which may be used in the above mobile terminal, such as a mobile phone, a tablet pc, etc., and fig. 3 is a flowchart of another road detection method according to an embodiment of the present invention, as shown in fig. 3, where the flowchart includes the following steps:
step S301, obtaining a pavement image to be measured. Please refer to step S101 in the embodiment shown in fig. 1 in detail, which is not described herein.
And step S302, constructing a pavement disease detection model.
Specifically, the step S302 includes:
In step S3021, a road surface image dataset is acquired.
In an embodiment of the present invention, a pavement image dataset includes: the road surface image with the defects, the real road surface type corresponding to the road surface image and the real frame coordinates of the defect positions on the road surface image.
In one example, acquiring the pavement image dataset may include: road surface images are acquired by front and rear cameras mounted on the inspection vehicle. The front camera can acquire color images of facilities along the road, the resolution is 2560 multiplied by 1440, the acquisition frame rate is 25 frames per second, the rear infrared camera can acquire infrared images of road surface diseases, the resolution is 1920 multiplied by 1080, the acquisition frame rate is 60 frames per second, and the infrared images can effectively cope with the influence of insufficient light on the identification effect. In the detection process, the vehicle shoots a line facility picture and a pavement damage picture every 5 meters according to the pile number, and stores the pictures in a line facility data set and a pavement damage data set.
After the data set is collected, the blurred, light-too-dark (such as culvert and evening images shot in the non-starting infrared mode) and overexposed images are removed by utilizing a machine vision algorithm, and then the images which do not comprise obvious diseases in the pavement disease data set and the images which do not comprise the line facilities in the line facility data set are deleted by a detector. Finally, the two data sets are divided into a training set and a test set according to the ratio of 8:2, namely 80% of images are used for training, 20% of images are used for testing, and no overlapping exists between the training set and the test set.
And step S3022, preprocessing and data enhancement are carried out on the road surface image, and a disease training image set is obtained.
In an example, the image preprocessing of the dataset mainly adopts data enhancement methods such as white balance, affine transformation, random clipping, super resolution, image mixing, histogram equalization, fourier transformation, and Mosaic, and the specific operation steps are as follows:
first, the line facility dataset and the pavement damage dataset were preprocessed using White Patch White balance algorithm, un harp Mask (USM) sharpening enhancement algorithm, gaussian filter algorithm, and histogram equalization algorithm. Setting the maximum value in an RGB channel as the illumination color in the image in the White Patch algorithm, so as to change the overall color balance of the image and enable the image to keep consistent colors under different illumination conditions; the USM sharpening enhancement algorithm subtracts a blurred version from the original image and then adds the result back to the original image to highlight the edges and details of the image; the histogram equalization algorithm is a method for improving the contrast of an image by stretching the range of pixel intensity distribution to increase the global contrast of the image; the gaussian filter is a linear filter that performs weighted averaging of pixel values over a small region of the image, the weights being determined by a gaussian function, thereby effectively removing gaussian noise from the image. After the pretreatment, the image quality is obviously improved, the interference noise is removed, the detection target characteristics are more obvious, the detection performance of the model is improved, and the detection precision is improved by 4.71%.
Secondly, affine transformation, random clipping and super-resolution algorithm are used for images in the road surface disease data set, so that sample diversity is increased, model performance and robustness are improved, and detection accuracy is increased by 1.16%. Wherein affine transformation changes the position, size and orientation of the image, but does not change the parallelism of the image, mainly including rotation, scaling, translation and warping; random cropping is a common data enhancement method that helps a model learn different parts of an image by randomly cropping a fixed-size sub-image from the image and improves the generalization ability of the model. Super-resolution is a technology for improving image resolution, and SWIN convertors are used to reconstruct high-resolution images from low-resolution images, so that the definition of the images is improved, and disease features are easier to distinguish.
And finally, image mixing, mosaics and other data enhancement algorithms are used for images in the line facility data set and the pavement disease data set, so that the sensitivity of the model to different scale features and the anti-interference performance to complex scenes are enhanced, the model performance is improved, and the detection precision is improved by 0.97%. The image mixing algorithm (MixUp) carries out linear interpolation on the two images and the labels of the two images to generate a new training sample, so that the model can better process the inherent change and noise of the data in the learning process; the mosaics data enhancement strategy generates a new training sample by splicing four (or nine) pictures together, so that the diversity of scenes can be increased, and the model is helped to learn the features with different scales better.
Step S3023, inputting the disease training image set into the initial road surface disease detection model to obtain the predicted road surface type and the predicted frame coordinates of the disease training image set.
Step S3024, calculating the total training loss based on the predicted road surface type and the predicted frame coordinates, the real road surface type and the real frame coordinates of the disease training image set.
Step S3025, training the initial road surface damage detection model based on the training total loss to obtain a target road surface damage detection model.
In one example, the training process for the road surface disease detection model further comprises: the pavement quality classification training is carried out on the pavement disease detection model, and the specific steps are as follows:
and training a pavement material classification model to distinguish a cement pavement, an asphalt pavement and a gravel pavement.
The main aim of the classification model is to predict the material type of the pavement according to the input pavement image, and the method of combining a wide residual error network with an adaptive sharpness perception minimization algorithm (ASAM) is used, so that the accuracy and the robustness of the classification model are ensured, and the specific operation steps are as follows:
(1) The pavement disease gray level images acquired by the rear infrared cameras are manually screened and divided into three categories, namely cement pavement images, asphalt pavement images and sand pavement images.
(2) Preprocessing the classified images, including operations such as cutting, scaling, rotating and white balancing of the images, and converting original image data into a format suitable for model training;
(3) A wide residual error network is established as a basic model, and a self-adaptive sharpness perception minimization algorithm is added to improve the model training process. The self-adaptive sharpness perception minimization algorithm is a learning algorithm of a deep neural network, is an effective generalized gap measure based on loss of surface sharpness, and can perform non-scaling learning on different scales so as to better adapt to data of different scales. And then, training the image input model, and repeatedly debugging the super parameters to obtain an optimal result.
In an example, the pavement damage detection model structure is based on a YoloV8 model, and an improved method for pavement damage detection and an optimized design for a training process are added on the basis. Fig. 4 is a structural diagram of a pavement damage detection model of a lightweight road detection method according to an embodiment of the present invention, where the model structure is shown in fig. 4, and in order to improve the bounding box fitting capability of the model and improve the training efficiency of the model, a bounding box loss based on a dynamic non-monotonic focusing mechanism is added, and the model loss is evaluated by using outliers instead of intersection ratios, and is expressed as:
Wherein L is bbox Representing bounding box loss, L IoU Representing the cross-ratio loss, A and B representing the candidate frame and the real frame, x and y representing the center coordinates of the candidate frame, x gt And y gt Representing the center coordinates of the real frame, W and H are the smallest rectangles that can frame a and B, α and σ are two preset hyper-parameters, default α=1.6, σ=4.The exponential sliding average, representing the amount of entrainment, is a dynamic value that enables a smoother, faster optimization process by combining historical gradient information with the current gradient. />Represents a monotonic focusing coefficient for measuring focusing effect.
In order to improve detection performance and reduce the omission ratio and false detection rate, a soft non-maximum suppression algorithm is used to replace traditional non-maximum suppression, soft non-maximum suppression adopts a soft suppression strategy, reduces confidence scores of other boundary frames with higher overlapping degree with a currently selected boundary frame according to the calculated intersection ratio, generally reduces the scores by adopting a Gaussian function or a linear function, and the confidence score updating process is expressed by a formula:
wherein the left side s of the formula i Representing the confidence score of the i candidate frame after updating, the right side s of the formula i The confidence score of the ith candidate frame before updating is represented, epsilon represents the standard deviation of a Gaussian function and is used for controlling the shape of the function, and the bigger the epsilon value is, the slower the speed of lowering the confidence score is, the smaller the epsilon value is, and the faster the speed of lowering the confidence score is; b i Representing the ith candidate box, M is the candidate box ordered from high to low by confidence score.
Because the pavement diseases have various shapes and different damage degrees, the characteristics of unobvious characteristics, difficult resolution, large shape difference and the like are common, and the method improves the detection accuracy rate, introduces deformable convolution to replace partial convolution layers in a backbone network in order to more effectively utilize object characteristics, weaken the influence of background characteristics on results. The deformable convolution can adapt the shape and position of the convolution kernel by training to accommodate irregular object features in the input image, thereby having higher performance in processing objects with complex deformations, expressed as:
wherein f (p i ) To output the value of the i position in the feature map, x (p i +p m +Δp m ) Representing an input feature map p i +p m +Δp m Value of position, w pi Representing p in the convolution kernel i R represents the receptive field corresponding to this convolution kernel, Δp m Is the offset of the convolution valued position, delta is the modulation factor, and is used for controlling the influence weight of the position offset on the output.
In order to better detect a large range of diseases and enhance the noise immunity of the model, a multi-head attention mechanism is introduced to improve the detection precision and reduce the interference of other sundries on the road surface on the model. The multi-head attention mechanism can better process global information of the image, effectively improve detection accuracy of targets with various scales and shapes, and help to eliminate influence of complex environments on detection results, and is expressed as follows:
h＝A(Q·W Q ,K·W K ,V·W V )
M(Q,K,V)＝Concat(h i ,...,h m )·W
Wherein A represents a calculation formula of attention, Q is a query vector and represents a region to be focused in an image; k is a key vector representing each region in the image; v is a value vector representing characteristic information of each region in the graph; Q.K represents the similarity between Q and K, i.e., the attention score. W (W) Q ，W K ，W V W represents a learnable weight matrix, d k Representing the dimensions, k, of the key vector T Representing the transposed vector of K, M representing the calculation formula of multi-head attention, concat representing vector dimension stitching operation, h i Representing the output result of the ith attention head, m represents the number of attention heads.
In order to improve the model efficiency and the detection performance thereof in a complex environment, switchable cavity convolution is added. The switchable cavity convolution can dynamically adjust the receptive field size of the convolution kernel according to semantic information and spatial information of input features so as to adapt to the context information of the picture, thereby improving the expression capability and the perception capability of the model for multi-scale object detection, and effectively reducing the calculated amount and the memory consumption, and the method is expressed as follows:
C(x,w,p)＝S(x)·C(x,w,p)+(1-S(x))·C(x,w+Δw,r)
wherein, C represents a convolution network, x represents an input feature map, S is a switchable convolution, and consists of an average pooling layer of 5x5 and a convolution layer of 1x1, which are related to input and position, deltaw is a trainable weight, default 0,w is the weight of the convolution layer, and p represents the void fraction of a common convolution layer, default 1; and r is the void ratio of the void convolution.
Specifically, the pavement disease detection model flow is as follows:
(1) Reading m images X= { X in pavement disease data set 1 ，x 2 ，……，x m -class label and real frame coordinate label y= { Y corresponding thereto 1 ，y 2 ，……，y m M represents the batch size, defaulting to 8. Preprocessing the read image.
(2) Image data X is transmitted to a Backbone network Backbone, and fig. 5 is a block diagram of a Backbone network in a road surface disease detection model of a lightweight road detection method according to an embodiment of the present invention, and the Backbone network structure is shown in fig. 5. Extracting features f of different scales from an image 1 ，f 2 ，f 3 。
(3) Transmitting the characteristics into a Neck network Neck, wherein FIG. 6 is a structure diagram of the Neck network in a road surface disease detection model of the lightweight road detection method according to an embodiment of the invention, the structure of the Neck network is shown in FIG. 6, and a multi-scale fusion characteristic eta is output 1 ，η 2 ，η 3 . The feature pyramid structure in the Neck integrates the high-resolution features of the shallow network and the features of the high-semantic information of the deep network, and the pixel aggregation network enables the low-level feature map to receive information from all the high-level feature maps, so that the enhancement of the semantic information of the low level is facilitated, and the detection precision of small objects is improved.
(4) Features are transmitted to a Head network Head, fig. 7 is a structural diagram of the Head network in a road surface disease detection model of the lightweight road detection method according to an embodiment of the present invention, and the structure of the Head network is shown in fig. 7. And decoupling the features, respectively transmitting the features into a classification branch and a regression branch, and outputting coordinates and categories of the prediction frame.
(5) According to the coordinates and the category of the prediction frame and the real frame, calculating the classification loss L cls Distributed focal loss L dfl And dynamic cross-ratio loss L based on dynamic focusing mechanism iou . Wherein, the classification lossThe method is used for measuring the gap between the predicted category and the real category, and the distributed focus loss and the dynamic cross ratio loss are used for calculating the regression loss of the boundary box.
(6) Multiplying the distributed focus loss, the classification loss and the dynamic cross-ratio loss by the dynamic weight w df l、w cls 、w iou Obtaining a training total loss L, and carrying out back propagation according to the L to calculate a gradient of a model parameter, wherein the gradient represents the change direction and the change speed of a loss function under the current parameter value; and then updating parameters of the model by using a random gradient descent optimization function. Dynamic weights are used to balance the effect of individual losses on model back propagation process, initial w iou Is 7.5, w cls Is 0.7, w dfl 1.5, according to the training progress, adjusting the loss weight, and finally w iou Is 6.85, w cls Is 1.0, w dfl 1.25, expressed as:
L＝w cls L cls +w dfl L dfl +w iou L iou
L cls ＝-w n [l n logσ(s n )+(1-l n )log(1-σ(s n ))]
at L iou Wherein L is IoU Representing the cross-ratio loss, A and B representing the candidate frame and the real frame, x and y representing the center coordinates of the candidate frame, x gt And y gt Representing the center coordinates of the real frame, W and H are the smallest rectangles that can frame a and B, α and σ are two preset hyper-parameters, default α=1.6, σ=4. The exponential sliding average, representing the amount of entrainment, is a dynamic value that enables a smoother, faster optimization process by combining historical gradient information with the current gradient. />Represents a monotonic focusing coefficient for measuring focusing effect.
At L dfl In p t And p t+1 Is the probability of predicting the two locations in the distribution closest to the tag y, β is an adjustment factor, typically 2, and this loss function can cause the network to focus more quickly on values near the target, increasing their probability, thereby improving the quality and accuracy of the bounding box.
At L cls Where σ is the sigmoid activation function, w is the weight of the nth sample, l is the true label, and s is the output of the model.
(7) Repeating the steps (1) to (6) until the preset iteration times are reached or the stop training condition of early stopping is met.
(8) And screening the detection frames according to the confidence, deleting the detection frames smaller than the confidence threshold, and deleting redundant overlapped detection frames by using a soft non-maximum suppression method.
(9) Based on the detection frame and the real labels in the data set, a series of accuracy indexes including accuracy, recall and average value precision are calculated, model performance is evaluated, and results and model hyper-parameters are recorded.
(10) And (3) adjusting the super parameters, repeating the steps (1) to (9), and comparing the results for a plurality of times until an optimal result is obtained.
In the mode, the pavement disease detection model is trained by utilizing a large number of disease images, the disease result obtained by the detection of the pavement disease detection model is more accurate, and the generalization and the robustness of the pavement disease detection model are further improved by preprocessing the pavement images and combining the structural design of the pavement disease detection model.
Step S303, inputting the pavement image to be detected into a pavement damage detection model, and detecting to obtain the pavement type and the damage area of the pavement to be detected.
Specifically, the step S303 includes:
step S3031, a calibration plate data set is acquired, and an area thermal distribution map of the real world is calculated based on the calibration plate data set.
In some optional embodiments, step S3031 includes:
and a1, marking the position of the calibration plate in the calibration plate image to obtain a boundary frame corresponding to the calibration plate, wherein the boundary frame comprises the position and the size of the calibration plate in the calibration plate image.
And a2, carrying out edge correction on the boundary frame, determining the inclination angles of the edges of the calibration plate, and calculating to obtain the relation between the unit pixel area in the calibration image and the coordinates in the real world.
And a3, calculating area values of all pixels in the calibration plate image based on the relation between the unit pixel area in the calibration image and the coordinates in the real world, and obtaining an area thermal distribution map of the real world.
In one example, training the example segmentation model to obtain the real world area thermal profile by creating a calibration plate dataset includes: manufacturing a calibration plate and collecting images: the calibration plate has obvious patterns which are easy to identify in the image, such as black and white checkerboard patterns and staggered oblique stripe patterns, and the total of four patterns is available. FIG. 8 is a diagram of a calibration plate pattern according to an embodiment of the present invention, as shown in FIG. 8, the calibration image captured by the camera is an image containing calibration plates, each of which has a size of 80cm by 80cm, and the images captured by the rear infrared cameras from a plurality of different angles and positions are collected to make a calibration plate dataset. The position of the calibration plate in the image is manually marked by using marking software LabelMe, and the marking result is a group of bounding boxes containing the position and size information of the calibration plate in the image. Then, mask RCNN is selected as a basic instance segmentation model, and fine tuning is performed according to the task. And finally, inputting the calibration plate image as training data to a model for training, evaluating by using test data, and optimizing an output result by repeatedly adjusting the super parameters of the model. The real world area thermodynamic diagram is calculated by combining a corner detection algorithm, a camera distortion correction algorithm, an edge detection algorithm and an example segmentation model, and the specific implementation steps are as follows:
(1) Firstly, an example segmentation algorithm is used for finding the position of a calibration plate, the corner points and the contours found by combining Harris corner point detection and Canny contour detection algorithm are used for carrying out edge correction, then the inclination angles of the edges of the calibration plate are calculated, so that the transverse and longitudinal distances corresponding to each pixel on the edges of the calibration plate are obtained, and the transverse and longitudinal distances are expressed as follows by a formula:
w×(x 2 -x 1 )＝l×cosθ
q×(y 2 -y 1 )＝l×sinθ
where θ represents the inclination of the edge of the calibration plate, l is the length of the edge of the calibration plate, here 80cm, (x) 1 ,y 1 ) And (x) 2 ,y 2 ) Representing the abscissa of two vertices of either edge of the calibration plate, w and q represent the coordinate distribution coefficients of the current edge. Assuming that the pixel area variation is uniformly nonlinear in the image, the coefficient of the unit pixel area variation with the coordinates can be found, expressed as:
/>
wherein alpha and beta are area distribution coefficients of two optional edges, w, of the current calibration plate 1 ，w 2 ，q 1 ，q 2 The coefficients of the coordinate distribution, cx, representing the two edges respectively 1 ，cy 1 ，cx 2 ，cy 2 Representing two edgesAnd calculating area distribution coefficients for the other two edges after the midpoint coordinates are obtained, and taking the average value as the area distribution coefficient of the current calibration plate.
Calculating the area distribution coefficients of the four calibration plates, and then calculating the area distribution thermodynamic diagram of the calibration picture, wherein the calculation method is expressed as follows by a formula:
M ij ＝max(d ij1 ,d ij2 ,d ij3 ,d ij4 ),i∈W,j∈H
Wherein W and H represent the width and height of the picture, i and j represent the coordinates of the pixel on the image, e represents the natural index, k represents the numbers of the four calibration plates, and p x And p y Representing the center coordinates (x, y) of the calibration plate; epsilon is a preset constant greater than 1 and represents a distance penalty coefficient, meaning that the farther the distance between two pixel points is, the smaller the influence on each other is; k represents the number of calibration plates, w k And q k Representing the coordinate distribution coefficient of the calibration plate with the number of k, alpha and beta represent the area distribution coefficient of the current calibration plate, and x ij And y ij Representing the abscissa of the ith column and jth row of pixels on the image; s is S ij The estimated area of the current pixel in the three-dimensional world. By this method, the area distribution thermodynamic diagram is obtained by finding the area values of all pixels on the image.
In the method, a boundary frame is obtained by marking the calibration image, operations such as segmentation and edge correction are performed on the boundary frame, the area of each pixel point in the image corresponding to the real world is determined, and then the area distribution thermodynamic diagram of the real world is determined, so that the disease area of the road surface to be measured in the real world can be conveniently obtained through subsequent calculation.
Step S3022, calculating the disease area of the road surface to be measured based on the area thermal distribution diagram and the disease coordinates.
In an example, the damaged area of the road surface to be detected is calculated by calculating the integral of the damaged area on the area distribution thermodynamic diagram of any damaged area detected on the road surface image to be detected.
In the mode, the area distance measurement system is established by setting the calibration image, so that the area and the length of the pavement damage can be detected more accurately, and the accuracy of pavement damage detection is improved.
Step S304, collecting acceleration data and sensor attitude data of the vehicle passing through the road surface to be detected. Please refer to step S104 in the embodiment shown in fig. 1 in detail, which is not described herein.
Step S305, calculating to obtain a road technical condition detection result of the road surface to be detected based on the acceleration data, the sensor attitude data, the road surface type and the disease area of the road surface to be detected. Please refer to step S105 in the embodiment shown in fig. 1 in detail, which is not described herein.
According to the road detection method, the road disease detection model is trained by utilizing a large number of disease images, the disease result obtained by detecting the road disease detection model is more accurate, and the generalization and the robustness of the road disease detection model are further improved by preprocessing the road image and combining with the structural design of the road disease detection model. By setting the calibration image and establishing an area distance measurement system, the area and the length of the pavement damage can be detected more accurately, and the accuracy of pavement damage detection is improved. The boundary frame is obtained by marking the calibration image, operations such as segmentation and edge correction are carried out on the boundary frame, the area of each pixel point in the image corresponding to the real world is determined, and then the area distribution thermodynamic diagram of the real world is determined, so that the disease area of the road surface to be measured in the real world can be conveniently obtained through subsequent calculation.
In this embodiment, a highway detection method is provided, which may be used in the above terminal, such as a mobile phone, a tablet pc, etc., and fig. 9 is a flowchart of another highway detection method according to an embodiment of the present invention, and as shown in fig. 9, the flowchart includes the following steps:
step S901, obtaining a road surface image to be measured. Please refer to step S301 in the embodiment shown in fig. 3 in detail, which is not described herein.
And step S902, constructing a pavement disease detection model. Please refer to step S302 in the embodiment shown in fig. 3 in detail, which is not described herein.
Step S903, inputting the road surface image to be detected into a road surface disease detection model, and detecting to obtain the road surface type and the disease area of the road surface to be detected. Please refer to step S303 in the embodiment shown in fig. 3 in detail, which is not described herein.
Step S904, collecting acceleration data and sensor attitude data of the vehicle passing through the road surface to be measured. Please refer to step S304 in the embodiment shown in fig. 3 in detail, which is not described herein.
Step S905, calculating to obtain a road technical condition detection result of the road surface to be detected based on the acceleration data, the sensor posture data, the road surface type and the disease area of the road surface to be detected.
Specifically, after the above step S905, the road detection method further includes:
Step S906, constructing a line facility detection model.
Specifically, the step S906 includes:
step S9061, acquiring a line facility image dataset.
And step S9062, preprocessing and data enhancement are carried out on the along-line facility images to obtain a along-line facility training image set.
Step S9063, inputting the line facility training image set into an initial line facility detection model to obtain the predicted frame type and the predicted frame coordinates of the line facility training image set.
Step S9064, calculating to obtain the total loss of the line facility training based on the predicted frame type and predicted frame coordinates and the real type and real frame coordinates of the line facility training image set.
Step S9065, training the initial along-line facility detection model based on the total loss of the along-line facility training, to obtain a target along-line facility detection model.
In one example, a line facility detection model for detecting facilities along a road is constructed, a line facility data set training model is used, super parameters of the line facility detection model are adjusted to conduct multiple training, accuracy, recall and average precision values are compared, and an optimal line facility detection model is stored. Because the road has the characteristics of more interference objects, complex and various environments, large target scale difference and various characteristics along the road, the structure and the super parameters of the detection model of the facilities along the road need to be adjusted, and the adjustment method comprises the following steps: and carrying out random scaling on the input image, improving the target confidence threshold value and adding dynamic region perception convolution. The dynamic region sensing network can dynamically adjust the model of the region characteristics, is suitable for processing complex and changeable spatial information distribution, has good effect on scenes with a large number of interfering objects, and comprises two modules: a learnable boot mask module and a filter generation module. The pilot mask module generates a multi-channel feature map through a convolution layer, each channel corresponds to a region, and then obtains which region each position belongs to through solving a maximum value index. The filter generation module generates a multi-channel feature map through two convolution layers, each channel corresponds to one filter, and then different filters are distributed to different areas according to the guide mask to carry out convolution operation.
Specifically, the operation flow of the line facility detection model for detecting the line facilities of the road is as follows:
(1) And reading m images and corresponding labels in the line facility data set.
(2) And transmitting the image data into a Backbone network Backbone, and extracting features f1, f2 and f3 with different scales from the image.
(3) And (3) transmitting the features with different scales into a Neck network Neck, and outputting multi-scale fusion features eta 1, eta 2 and eta 3.
(4) And transmitting the fusion characteristics into a Head network Head, decoupling the characteristics, and outputting coordinates and categories of the prediction frame.
(5) And calculating classification loss, distributed focus loss and complete cross ratio loss according to the coordinates and the category of the prediction frame and the real frame, wherein the distributed focus loss and the complete cross ratio loss are used for calculating the boundary frame regression loss.
(6) Multiplying the distributed focus loss, the classification loss and the dynamic cross ratio loss by dynamic weights to obtain the total training loss, counter-propagating to calculate the gradient of model parameters, and updating the parameters of the model by using a random gradient descent optimization function.
(7) Repeating the steps (1) to (6) until the preset iteration times are reached or the stop training condition of early stopping is met.
(8) And evaluating the quality of the model, including accuracy, recall rate and average mean value precision, evaluating the performance of the model, and recording the result and the model hyper-parameters.
(9) And (3) adjusting the super parameters, repeating the steps (1) to (8), and comparing the results for a plurality of times until the optimal line facility model is obtained.
In the mode, the line facility detection model is trained by utilizing a large number of line facility images, the line facility result obtained by detecting the line facility detection model is more accurate, and the generalization and the robustness of the line facility detection model are further improved by preprocessing the line facility images and combining the structural design of the line facility detection model.
Step S907, inputting the road surface image into a line facility detection model, and detecting and obtaining the line facilities in the road surface image.
Step S908, determining location information of the road surface fault based on the line facilities in the road surface image.
In an example, the road surface image is input into the line facility detection model trained in the above step, so that the line facility existing in the initial road surface image can be identified, the position of the road surface image can be determined, and the position information of the disease in the road surface image can be further determined.
In the mode, the line facility detection model is constructed, so that the line facility of the road surface to be detected can be automatically identified, interference of human factors is reduced, and the accuracy of the line facility detection is improved. And by determining the position information of the pavement diseases, the follow-up accurate maintenance of the diseases of the pavement to be tested is facilitated.
According to the road detection method, the line facility detection model is built, so that the line facility of the road surface to be detected can be automatically identified, interference of human factors is reduced, and accuracy of the line facility detection is improved. And by determining the position information of the pavement diseases, the follow-up accurate maintenance of the diseases of the pavement to be tested is facilitated. The line facility detection model is trained by utilizing a large number of line facility images, the result of the line facility detected by the line facility detection model is more accurate, and the generalization and the robustness of the line facility detection model are further improved by preprocessing the line facility images and combining the structural design of the line facility detection model.
In this embodiment, a road detection method is provided, which may be used in the above terminal, such as a mobile phone, a tablet pc, etc., and fig. 10 is a flowchart of another road detection method according to an embodiment of the present invention, as shown in fig. 10, where the flowchart includes the following steps:
step S1001, obtaining a road surface image to be measured. Please refer to step S901 in the embodiment shown in fig. 9 in detail, which is not described herein.
Step S1002, constructing a pavement disease detection model. Please refer to step S902 in the embodiment shown in fig. 9 in detail, which is not described herein.
Step S1003, inputting the pavement image to be detected into a pavement damage detection model, and detecting to obtain the pavement type and the damage area of the pavement to be detected. Please refer to step S903 in the embodiment shown in fig. 9 in detail, which is not described herein.
Step S1004, collecting acceleration data and sensor attitude data of the vehicle passing through the road surface to be measured. Please refer to step S904 in the embodiment shown in fig. 9 in detail, which is not described herein.
Step S1005, calculating to obtain a road technical condition detection result of the road surface to be detected based on the acceleration data, the sensor posture data, the road surface type and the disease area of the road surface to be detected.
Specifically, the step S1005 includes:
step S10051, calculating to obtain the road surface flatness index of the road surface to be measured based on the acceleration data and the sensor posture information.
Step S10052, calculating the road surface running quality index of the road surface to be measured based on the road surface flatness index.
Step S10053, calculating the road surface damage condition index of the road surface to be measured based on the road surface type and the damaged area of the road surface to be measured.
Step S10054, calculating to obtain a road technical condition detection result of the road surface to be detected based on the road surface running quality index and the road surface damage condition index.
In one example, the road surface flatness index IRI and the road surface running quality index RQI are calculated from data collected by nine-axis sensors. The nine-axis sensor consists of a three-axis acceleration sensor, a three-axis gyroscope and a three-axis geomagnetic sensor, and the three-axis acceleration data and the sensor posture information are needed to be used in the method. The specific measurement method comprises the following steps: fixing the nine-axis sensor in a detection vehicle for initial state calibration, measuring the initial value of gravity on the sensor, and recording; measuring roads with not less than three sections and not less than thirty meters in each section by a three meter ruler method, and calculating the flatness index of each roadThen, the detecting vehicle passes through the road section at a constant speed, and nine-axis numerical values of the sensor are recorded; finally, calculating the vertical acceleration, and establishing a relation between the vertical acceleration data and the flatness index, wherein the relation is expressed as follows by a formula:
a＝0+a x i+a y j+a z k
A＝u x i+u y j+u z k
wherein a is x ，a y ，a z Representing the measured three-axis accelerations in the lateral, longitudinal and vertical directions in the nine-axis sensor detected by the sensor, a being a quaternion representation of the three-axis acceleration, g being a quaternion representation of the force of gravity,representing the conjugate matrix of q, u x ，u y ，u z The unit vectors representing three directions, a represents the unit vector representation of the rotation axis, θ is the rotation angle of the sensor relative to the initial horizontal state, and i, j, k represent imaginary units. Specifically, the relationship between i, j, k is shown in the following formula:
i 2 ＝j 2 ＝k 2 ＝-1
ij＝-ji＝k
jk＝-kj＝i
ik＝-ki＝j
Finally obtaining a 'and g' which are acceleration quaternion after rotation and gravity quaternion after rotation, and respectively taking the imaginary part coefficient a about k for the acceleration quaternion and the gravity quaternion k And g k Subtracting a from the two h ＝a k -g k Obtaining the final vertical acceleration a h . Filtering all vertical accelerations using a trapezoidal window function in combination with a high pass filter to eliminate themWhite noise and noise (such as vehicle body vibration after vehicle starting, errors of a sensor, environmental influence and the like) with medium and low frequencies are kept, acceleration information generated due to uneven road surface is reserved, and a road surface evenness index IRI and a road surface running quality index RQI are obtained through calculation, wherein the following formula is shown:
wherein f is the sampling frequency of the sensor, defaults to 100Hz, a m The M-th acceleration data of the sensor are all acceleration data collected by the sensor, S is the driving distance, and the unit is km; establishing a calibration flatness alpha IRI And manually detecting flatnessThe relationship between ω and β is a set of approximate solutions to the relationship between ω and β, thus removing the impact of the vehicle damping performance on the flatness measurement.
The process of calculating the road technical condition detection result of the road surface to be detected can comprise the following steps: calculating a line facility condition index TCI according to a line facility detection result, calculating a road surface running quality index RQI according to a nine-axis sensor, calculating a road surface technical condition index PQI according to a road surface damage condition index PCI and the road surface running quality index RQI, and then combining a manually assessed road bed technical condition index SCI and a bridge tunnel structure technical condition index BCI to finally obtain a low-grade rural road technical condition index MQI, wherein the steps comprise:
(1) And calculating the road surface running quality index according to the road surface flatness index, wherein the calculation formula is as follows:
the IRI is a road surface flatness index, and the unit is m/km; b 0 Is a coefficient, depending on the technical grade of the road surface and the road surface material, the expressway and the primary road adopt 0.026, the secondary road adopts 0.0185, the low-grade rural road asphalt pavement adopts 0.0167, and the cement concrete pavement adopts 0.0146; b 1 Is a coefficient, and depending on the technical grade of the road surface and the road surface material, the expressway and the primary road use 0.65, the secondary road use 0.58, the low-grade rural road asphalt road use 0.56 and the cement concrete road use 0.52.
(2) The road surface damage condition index PCI is calculated according to the road surface damage rate, and the calculation formula is as follows:
wherein a is 0 Is a calculation factor for high speed, primary and secondary highways: 15.00 parts of asphalt pavement and 10.66 parts of cement concrete pavement; for three-level and following rural highways: 14.03 of asphalt pavement, 10.91 of cement concrete pavement and 10.10 of gravel pavement; a, a 1 Is a calculation coefficient, for high-speed, primary and secondary roads, the asphalt pavement is 0.412, the cement concrete pavement is 0.461, for rural roads with three or below levels, the asphalt pavement is 0.37, and the cement concrete pavement is 0.392; i represents the type of road surface damage; i.e 0 Representing the total number of damage types, for primary and secondary roads, 21 asphalt pavement, 20 cement concrete pavement, for rural roads of three or less levels, 4 asphalt pavement, 5 cement concrete pavement; a is that i Represents the cumulative area (m 2 )；
A represents a road surface detection or investigation area (m 2 )；ω i The weight of the i-th road surface damage is determined according to the technical grade and the material of the road surface.
(3) The technical condition index of the road surface is calculated by combining the damage condition index of the road surface and the running quality index of the road surface, and the formula is as follows:
PQI＝w PCI PCI+w RQI RQI
table 1 is a relationship between the road surface type and the corresponding weight, as shown in table 1.
TABLE 1
(4) Calculating a line facility technical condition index according to a line facility image collected by the detection vehicle, wherein the formula is as follows:
wherein GD i Total points representing damage to type i facilities with a maximum score of 100; w (w) i A weight representing damage to the type i facility; fig. 11 is a schematic view of a line facility deduction standard according to a lightweight road detection method according to an embodiment of the present invention, and as shown in fig. 11, specific weight values of i-th type facility damage are shown in fig. 11.
(5) According to the manually measured roadbed technical condition index, the bridge tunnel structure technical condition index, the light weight platform assisted manual detection along line facility technical condition index and the road surface technical condition index generated by the light weight detection platform, obtaining a low-grade rural road technical condition index, and expressing the low-grade rural road technical condition index by a formula as follows:
MQI＝w SCI SCI+w PQI PQI+w BCI BCI+w TCI TCI
Wherein w is SCI The weight of SCI in MQI is 0.08; w (w) PQI The weight of the PQI in the MQI is 0.60; w (w) BCI Weights in MQI for BCIThe value is 0.20; w (w) TCI The weight of TCI in MQI is 0.12.
In the mode, the road surface running quality detection is carried out by collecting information through the sensor, the road technical condition detection result of the road surface to be detected is obtained through calculation, the light-weight and automatic road flatness detection is realized, the carrying and the deployment are convenient, the manpower and the material resources are greatly saved, and the working efficiency is improved.
Specifically, after the step S1005, the road detection method further includes:
step S1006, based on the road technical condition detection result of the road surface to be detected, the quality evaluation is carried out on the road surface to be detected, and the maintenance scheme of the road surface to be detected is determined.
In one example, road quality assessment and maintenance recommendations, as well as predictions of future maintenance trends, are given by analyzing known road surface condition information. First, road surface condition information is analyzed using a road surface failure detection model to obtain a road surface Running Quality Index (RQI), a road surface damage condition index (PCI), and a road surface technical condition index (PQI). Then, system rules related to road surface quality evaluation, maintenance advice and future maintenance trend are set, and system super parameters such as temperature, repeated occurrence punishment, frequency punishment and the like are set. And finally, calling a GPT interface, transmitting the rule, the super parameter and the road surface condition information in, and giving out road surface quality evaluation, maintenance decision and maintenance planning by a natural language large model.
Specifically, a road evaluation expert system is designed, the system calls a GPT4 natural language large model interface, and system rules are customized and super parameters are adjusted according to various indexes of road evaluation to generate road quality evaluation, road maintenance suggestions and future maintenance planning. The specific method comprises the following steps: the running quality, safety and structural integrity of the current road are evaluated by combining various disease information of the road surface through main index data of the road surface in nearly five years, such as a road rutting depth index RDI, a road surface damage condition index PCI, a road surface running quality index RQI, a road surface structural strength index PSSI and a road bed technical condition index SCI, then a maintenance scheme such as daily maintenance, functional repair or structural repair is determined, and finally the indexes and the evaluation are transmitted into an interface to estimate the maintenance planning and the maintenance cost in three to five years in the future.
In the method, through overall technical condition assessment according to the detection result, more comprehensive road condition information can be provided for users, more targeted suggestions are given to future maintenance methods and maintenance plans, and accurate pavement maintenance is facilitated in time.
According to the road detection method, the road running quality is detected through the information collected by the sensor, the road technical condition detection result of the road to be detected is obtained through calculation, the light-weight and automatic road flatness detection is realized, the carrying and the deployment are convenient, the manpower and the material resources are greatly saved, and the working efficiency is improved. By evaluating the overall technical condition according to the detection result, more comprehensive road condition information can be provided for users, more targeted suggestions are given to future maintenance methods and maintenance plans, and accurate pavement maintenance is facilitated in time.
In this embodiment, a road detection device is further provided, and the device is used to implement the foregoing embodiments and preferred embodiments, and will not be described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides a road detection apparatus, as shown in fig. 12, including:
the image acquisition module 1201 is configured to acquire an image of a road surface to be measured. Please refer to step S101 in the embodiment shown in fig. 1 in detail, which is not described herein.
The model construction module 1202 is configured to construct a pavement damage detection model. Please refer to step S102 in the embodiment shown in fig. 1 in detail, which is not described herein.
The defect detection module 1203 is configured to input the image of the road surface to be detected into a road surface defect detection model, and detect the road surface type and the defect area of the road surface to be detected. Please refer to step S103 in the embodiment shown in fig. 1 in detail, which is not described herein.
The sensor data acquisition module 1204 is used for acquiring acceleration data and sensor attitude data of the vehicle passing through the road surface to be detected. Please refer to step S104 in the embodiment shown in fig. 1 in detail, which is not described herein.
The technical condition calculating module 1205 is configured to calculate and obtain a road technical condition detection result of the road surface to be detected based on the acceleration data, the sensor posture data, the road surface type and the disease area of the road surface to be detected. Please refer to step S105 in the embodiment shown in fig. 1 in detail, which is not described herein.
In some alternative embodiments, model building module 1202 includes:
a first image data set acquisition unit configured to acquire a road surface image data set including: the road surface image with the defects, the real road surface type corresponding to the road surface image and the real frame coordinates of the defect positions on the road surface image.
The first data pre-training unit is used for preprocessing and enhancing the pavement image to obtain a disease training image set.
And the prediction result unit is used for inputting the disease training image set into the initial pavement disease detection model to obtain the predicted pavement type and the predicted frame coordinates of the disease training image set.
The first loss calculation unit is used for calculating the total training loss based on the predicted pavement type and predicted frame coordinates, the real pavement type and real frame coordinates of the disease training image set.
And the disease model training unit is used for training the initial pavement disease detection model based on the training total loss to obtain a target pavement disease detection model.
In some alternative embodiments, disease detection module 1203 includes:
the image input unit is used for inputting the image of the road surface to be detected into the target road surface damage detection model to obtain the road surface type and the damage coordinates of the road surface to be detected.
The thermal distribution map calculation unit is used for acquiring a calibration plate data set, calculating to obtain an area thermal distribution map of the real world based on the calibration plate data set, wherein the calibration plate data set comprises: and (3) collecting calibration plate images containing the calibration plates under different angles and positions.
And the disease area calculating unit is used for calculating the disease area of the pavement to be measured based on the area thermal distribution diagram and combining the disease coordinates.
In some alternative embodiments, the thermal profile calculation unit comprises:
and the boundary frame marking subunit is used for marking the position of the calibration plate in the calibration plate image to obtain a boundary frame corresponding to the calibration plate, wherein the boundary frame comprises the position and the size of the calibration plate in the calibration plate image.
And the relation calculating subunit is used for carrying out edge correction on the boundary frame, determining the inclination angles of the edges of the calibration plate, and calculating to obtain the relation between the unit pixel area in the calibration image and the coordinates in the real world.
And the thermodynamic distribution diagram calculating subunit is used for calculating the area values of all pixels in the calibration plate image based on the relation between the unit pixel area in the calibration image and the coordinates in the real world to obtain the real world area thermodynamic distribution diagram.
In some alternative embodiments, state of the art calculation module 1205 includes:
and the flatness calculation unit is used for calculating and obtaining the road surface flatness index of the road surface to be measured based on the acceleration data and the sensor posture information.
And the running quality calculating unit is used for calculating the road surface running quality index of the road surface to be measured based on the road surface flatness index.
The road surface damage condition calculation unit is used for calculating the road surface damage condition index of the road surface to be measured based on the road surface type and the disease area of the road surface to be measured.
The technical condition calculating unit is used for calculating and obtaining a road technical condition detection result of the road to be detected based on the road surface running quality index and the road surface damage condition index.
In some alternative embodiments, the road detection apparatus further comprises:
and the line facility detection model construction unit is used for constructing a line facility detection model.
The line facility detection unit is used for inputting the road surface image into the line facility detection model and detecting and obtaining the line facilities in the road surface image.
And the disease position determining unit is used for determining the position information of the pavement disease based on the line facilities in the pavement image.
In some alternative embodiments, the line facility detection model building unit includes:
a second image dataset acquisition subunit for acquiring a line-along facility image dataset comprising: with along-line facility images, real categories and real frame coordinates corresponding to along-line facility images.
And the second data pre-training subunit is used for preprocessing and enhancing the line facility images to obtain a line facility training image set.
And the second prediction result subunit is used for inputting the line facility training image set into the initial line facility detection model to obtain the prediction frame category and the prediction frame coordinate of the line facility training image set.
And the second loss calculation subunit is used for calculating the total loss of the facility training along the line based on the predicted frame type and the predicted frame coordinates, the real type and the real frame coordinates of the facility training image set along the line.
And the line facility model training subunit is used for training the initial line facility detection model based on the total loss of the line facility training to obtain the target line facility detection model.
In some alternative embodiments, the road detection apparatus further comprises:
the maintenance scheme determining unit is used for evaluating the quality of the road surface to be tested based on the road technical condition detection result of the road surface to be tested and determining the maintenance scheme of the road surface to be tested.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The road detection means in this embodiment are presented in the form of functional units, here referred to as ASIC (Application Specific Integrated Circuit ) circuits, processors and memories executing one or more software or fixed programs, and/or other devices that can provide the above described functionality.
The embodiment of the invention also provides computer equipment, which is provided with the highway detection device shown in the figure 12.
Referring to fig. 13, fig. 13 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. 13, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 13.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods shown in implementing the above embodiments.
The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.
The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.
Claims (11)
1. A method of highway inspection, the method comprising:
acquiring a pavement image to be measured;
constructing a pavement disease detection model;
inputting the pavement image to be detected into the pavement disease detection model, and detecting to obtain the pavement type and the disease area of the pavement to be detected;
collecting acceleration data and sensor attitude data of a vehicle passing through the road surface to be tested;
and calculating to obtain a road technical condition detection result of the road to be detected based on the acceleration data, the sensor attitude data, the road type and the disease area of the road to be detected.
2. The method of claim 1, wherein the constructing a pavement damage detection model comprises:
acquiring a road surface image dataset comprising: the method comprises the steps of providing a pavement image with defects, a real pavement type corresponding to the pavement image and real frame coordinates of the defect position on the pavement image;
Preprocessing and data enhancement are carried out on the pavement image to obtain a disease training image set;
inputting the disease training image set into an initial pavement disease detection model to obtain a predicted pavement type and predicted frame coordinates of the disease training image set;
calculating to obtain the total training loss based on the predicted road surface type and predicted frame coordinates, the real road surface type and real frame coordinates of the disease training image set;
and training the initial pavement disease detection model based on the training total loss to obtain a target pavement disease detection model.
3. The method according to claim 2, wherein the inputting the road surface image to be measured into the road surface disease detection model, detecting the road surface type and the disease area of the road surface to be measured, includes:
inputting the road surface image to be detected into the target road surface disease detection model to obtain the road surface type and the disease coordinate of the road surface to be detected;
acquiring a calibration plate data set, and calculating to obtain an area thermal distribution map of the real world based on the calibration plate data set, wherein the calibration plate data set comprises: calibration plate images containing calibration plates are collected under different angles and positions;
And calculating the disease area of the pavement to be detected based on the area thermal distribution diagram and combining the disease coordinates.
4. A method according to claim 3, wherein said obtaining a calibration plate dataset, based on which a real world area thermal profile is calculated, comprises:
marking the position of the calibration plate in the calibration plate image to obtain a boundary frame corresponding to the calibration plate, wherein the boundary frame comprises the position and the size of the calibration plate in the calibration plate image;
performing edge correction on the boundary frame, determining the inclination angles of the edges of the calibration plate, and calculating to obtain the relation between the unit pixel area in the calibration image and the coordinates in the real world;
and calculating area values of all pixels in the calibration plate image based on the relation between the unit pixel area in the calibration image and the coordinates in the real world to obtain an area thermal distribution map of the real world.
5. The method according to claim 1, wherein the calculating the road technical condition detection result of the road surface to be detected based on the acceleration data, the sensor posture data, the road surface type and the diseased area of the road surface to be detected includes:
Calculating to obtain a road surface flatness index of the road surface to be measured based on the acceleration data and the sensor attitude information;
calculating to obtain a road surface running quality index of the road surface to be measured based on the road surface flatness index;
calculating to obtain a pavement damage condition index of the pavement to be detected based on the pavement type and the disease area of the pavement to be detected;
and calculating to obtain a road technical condition detection result of the road to be detected based on the road surface running quality index and the road surface damage condition index.
6. The method according to claim 1, wherein the method further comprises:
constructing a line facility detection model;
inputting the road surface image into the line facility detection model, and detecting to obtain line facilities in the road surface image;
and determining the position information of the pavement diseases based on the line facilities in the pavement image.
7. The method of claim 6, wherein said constructing a line facility detection model comprises:
acquiring a line facility image dataset, the line facility image dataset comprising: the method comprises the steps of providing a line facility image, a real category corresponding to the line facility image and real frame coordinates;
Preprocessing and data enhancement are carried out on the along-line facility images to obtain a along-line facility training image set;
inputting the along-line facility training image set into an initial along-line facility detection model to obtain a prediction frame category and a prediction frame coordinate of the along-line facility training image set;
calculating to obtain total loss of the line facility training based on the predicted frame type and predicted frame coordinates, the real type and real frame coordinates of the line facility training image set;
and training the initial along-line facility detection model based on the total loss of the along-line facility training to obtain a target along-line facility detection model.
8. The method according to any one of claims 1 to 7, further comprising:
and carrying out quality evaluation on the road surface to be tested based on the road technical condition detection result of the road surface to be tested, and determining a maintenance scheme of the road surface to be tested.
9. A highway inspection device, the device comprising:
the image acquisition module is used for acquiring a pavement image to be detected;
the model construction module is used for constructing a pavement disease detection model;
the pavement damage detection module is used for inputting the pavement image to be detected into the pavement damage detection model and detecting the pavement type and the damage area of the pavement to be detected;
The sensor data acquisition module is used for acquiring acceleration data and sensor attitude data of the vehicle passing through the road surface to be detected;
the technical condition calculation module is used for calculating and obtaining a road technical condition detection result of the road to be detected based on the acceleration data, the sensor attitude data, the road type and the disease area of the road to be detected.
10. A computer device, comprising:
a memory and a processor in communication with each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the road detection method of any one of claims 1 to 8.
11. A computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the road detection method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311499044.9A CN117422699A (en) | 2023-11-10 | 2023-11-10 | Highway detection method, highway detection device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311499044.9A CN117422699A (en) | 2023-11-10 | 2023-11-10 | Highway detection method, highway detection device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117422699A true CN117422699A (en) | 2024-01-19 |
Family
ID=89530070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311499044.9A Pending CN117422699A (en) | 2023-11-10 | 2023-11-10 | Highway detection method, highway detection device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117422699A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111985494A (en) * | 2020-07-07 | 2020-11-24 | 深圳大学 | Road disease detection and classification method based on lightweight vehicle-mounted terminal |
| CN113609911A (en) * | 2021-07-07 | 2021-11-05 | 北京工业大学 | Pavement disease automatic detection method and system based on deep learning |
| US20220009502A1 (en) * | 2019-03-25 | 2022-01-13 | Denso Corporation | Road surface condition determination device for vehicle, driving assistance system, and road surface condition determination method |
-
2023
- 2023-11-10 CN CN202311499044.9A patent/CN117422699A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220009502A1 (en) * | 2019-03-25 | 2022-01-13 | Denso Corporation | Road surface condition determination device for vehicle, driving assistance system, and road surface condition determination method |
| CN111985494A (en) * | 2020-07-07 | 2020-11-24 | 深圳大学 | Road disease detection and classification method based on lightweight vehicle-mounted terminal |
| CN113609911A (en) * | 2021-07-07 | 2021-11-05 | 北京工业大学 | Pavement disease automatic detection method and system based on deep learning |
Non-Patent Citations (2)
| Title |
|---|
| 吴冬秀等: "《陆地生态系统生物观测数据质量保证与质量控制》", 31 August 2012, 中国环境科学出版社, pages: 93 - 96 * |
| 龙伍丹等: "基于加强特征提取的道路病害检测算法", 《计算机应用》, 27 September 2023 (2023-09-27), pages 1 - 9 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210319561A1 (en) | Image segmentation method and system for pavement disease based on deep learning | |
| KR102166458B1 (en) | Defect inspection method and apparatus using image segmentation based on artificial neural network | |
| CN106875381B (en) | Mobile phone shell defect detection method based on deep learning | |
| CN110728658A (en) | High-resolution remote sensing image weak target detection method based on deep learning | |
| CN111709416B (en) | License plate positioning method, device, system and storage medium | |
| CN114693615A (en) | Deep learning concrete bridge crack real-time detection method based on domain adaptation | |
| CN113240626A (en) | Neural network-based method for detecting and classifying concave-convex flaws of glass cover plate | |
| CN104574393A (en) | Three-dimensional pavement crack image generation system and method | |
| CN112801972A (en) | Bridge defect detection method, device, system and storage medium | |
| CN113240623B (en) | Pavement disease detection method and device | |
| CN110706224A (en) | Optical element weak scratch detection method, system and device based on dark field image | |
| CN115830004A (en) | Surface defect detection method, device, computer equipment and storage medium | |
| CN114842201A (en) | Sandstone aggregate image segmentation method based on improved Mask _ Rcnn | |
| CN113435407A (en) | Small target identification method and device for power transmission system | |
| CN115937518A (en) | Pavement disease identification method and system based on multi-source image fusion | |
| Guo et al. | Surface defect detection of civil structures using images: Review from data perspective | |
| Dipankar et al. | Pavement crack detection based on a deep learning approach and visualisation by using GIS | |
| Khoshboresh-Masouleh et al. | Robust building footprint extraction from big multi-sensor data using deep competition network | |
| CN112924037A (en) | Infrared body temperature detection system and detection method based on image registration | |
| Guan et al. | Multi-scale asphalt pavement deformation detection and measurement based on machine learning of full field-of-view digital surface data | |
| CN117058069A (en) | Automatic detection method for apparent diseases of pavement in panoramic image | |
| CN115457044B (en) | Pavement crack segmentation method based on class activation mapping | |
| CN116665153A (en) | Road scene segmentation method based on improved deep bv3+ network model | |
| US11481891B1 (en) | Systems and methods for quantifying concrete surface roughness | |
| Baduge et al. | Assessment of crack severity of asphalt pavements using deep learning algorithms and geospatial system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |