CN113780312A - Highway road surface condition detecting system - Google Patents
Highway road surface condition detecting system Download PDFInfo
- Publication number
- CN113780312A CN113780312A CN202111093054.3A CN202111093054A CN113780312A CN 113780312 A CN113780312 A CN 113780312A CN 202111093054 A CN202111093054 A CN 202111093054A CN 113780312 A CN113780312 A CN 113780312A
- Authority
- CN
- China
- Prior art keywords
- image
- visible light
- road surface
- images
- light camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 claims abstract description 52
- 238000001514 detection method Methods 0.000 claims abstract description 32
- 230000004927 fusion Effects 0.000 claims abstract description 24
- 230000000007 visual effect Effects 0.000 claims abstract description 17
- 230000004297 night vision Effects 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 26
- 230000002159 abnormal effect Effects 0.000 claims description 24
- 239000013598 vector Substances 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 15
- 238000000034 method Methods 0.000 claims description 11
- 238000002372 labelling Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000003287 optical effect Effects 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 238000005286 illumination Methods 0.000 claims description 4
- 230000007704 transition Effects 0.000 claims description 4
- 230000005856 abnormality Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 abstract description 14
- 230000004438 eyesight Effects 0.000 description 16
- 238000007689 inspection Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 201000010099 disease Diseases 0.000 description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Traffic Control Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a highway pavement condition detection system, which relates to the technical field of traffic monitoring and comprises a controller and a plurality of roadside visual monitoring base stations, wherein the roadside visual monitoring base stations are connected with the controller and are arranged on the roadside of a highway, and each roadside visual monitoring base station comprises a visible light camera and an infrared camera; the visual angles of the visible light camera and the infrared camera are kept parallel; the visible light camera is used for acquiring a first image and a second image of the highway pavement in real time and uploading the first image and the second image to the controller; the infrared camera is used for sensing the highway pavement in real time in a night vision mode and uploading a third acquired image of the highway pavement to the controller; the controller comprises a road surface abnormity detection module, a multi-section image splicing module and a multi-source image data fusion module. Compared with the prior art, the invention has the advantages of strong anti-interference capability, improvement of the timeliness of pavement maintenance and management and the like.
Description
The invention discloses a divisional application of an expressway road surface condition detection system based on novel visual sensing equipment, wherein the application number of a parent application is 201911149152.7, and the application date is 2019.11.21.
Technical Field
The invention relates to the technical field of traffic monitoring, in particular to a highway pavement condition detection system.
Background
With the continuous increase of the highway mileage in China, on one hand, the ever-increasing urban and intercity traffic trip requirements of people are met, the convenience, the high efficiency and the comfort of people in trip are improved, and the rapid development of national economy in China is driven; on the other hand, due to the influence of traffic load and environmental change, the highway surface is gradually deteriorated, so that the service performance and the safety performance of the highway are reduced, and the service life of the highway is shortened. The development mode of the expressway in China is gradually changed from a construction type to a maintenance type, and the advanced and intelligent expressway pavement condition detection technology is urgently needed to promote the maintenance development of the expressway.
At present, the intelligent inspection mode adopted by highway pavement maintenance departments mainly depends on the installation of laser, radar, vision and other equipment on vehicles, and the road pavement state inspection is carried out on corresponding road sections regularly through manual driving so as to report to the highway management departments in time to finish the pavement maintenance. However, the highway pavement inspection method based on the intelligent inspection vehicle is expensive in equipment, time-consuming and labor-consuming, and low in pavement disease detection precision due to low visual strength and serious scene interference caused by dynamic detection; meanwhile, the regular local section inspection mode is poor in overall monitoring strength, and the uploading real-time performance of a road surface condition detection result is low, so that the maintenance strength of a traffic management department is influenced.
Disclosure of Invention
The invention aims to provide a highway pavement condition detection system, which improves the timeliness of pavement maintenance and management and has strong anti-interference capability.
In order to achieve the purpose, the invention provides the following scheme:
a highway pavement condition detection system comprises a controller and a plurality of roadside visual monitoring base stations which are connected with the controller and arranged on the roadside of a highway, wherein each roadside visual monitoring base station comprises a visible light camera and an infrared camera;
the visual angles of the visible light camera and the infrared camera are kept parallel;
the visible light camera is used for acquiring a first image and a second image of the highway pavement in real time and uploading the first image and the second image to the controller; the first image is a local pavement image of the highway pavement, which is acquired by the visible light camera in a far focus mode in a normal mode; the second image is an abnormal road surface area texture image acquired after the visible light camera adjusts the holder to a short focus mode when the road surface abnormal condition is detected;
the infrared camera is used for sensing the highway pavement in real time in a night vision mode and uploading a third acquired image of the highway pavement to the controller;
the controller includes:
the road surface abnormity detection module is connected with the visible light camera and used for training and classifying the semantic features of the second image by adopting a semantic classification network to determine the classification result of the road surface abnormity condition;
the multi-road-section image splicing module is connected with the visible light camera and used for extracting a feature vector from the first image by an SIFT feature extraction method, matching the similarity of the feature vector, optimizing the matching result by using an RANSAC algorithm, eliminating the feature points which are mistakenly matched and realizing the image matching and splicing among different road sections;
and the multi-source image data fusion module is used for analyzing the difference of the image overlapping parts of different road sections, and realizing the transition fusion of the images of the multiple road sections between illumination and image form change based on a fusion algorithm of dynamic weighted average so as to establish a global image of the highway pavement.
Optionally, the calculation formula of the monitoring height from the ground to the plane where the lens of the visible light camera is located is specifically as follows:
wherein the content of the first and second substances,
dGfor monitoring height, xlFor the transverse length, y, of the monitoring field of view of a visible light cameralLongitudinal length of field of view for monitoring of visible camera, flFocal length, x, corresponding to the monitoring field range of the visible light camera1Is the transverse length, y, of the optical photosensitive surface of a visible light camera1Is the longitudinal length of the optical photosensitive surface of a visible light camera, d1And d2Is a process variable.
Optionally, the road surface abnormality detection module specifically includes:
accessing a local storage platform through a local area network to acquire the second image stored in the current time period;
semantic annotation is carried out on the second images through a labelme annotation data tool, json annotation files corresponding to the second images are generated, and then a total json file in a COCO data set format is obtained through a labelme annotation data set to COCO data set conversion tool;
constructing a Mask RCNN network on a local video processing platform;
classifying the total json files in the COCO data set format according to a Mask RCNN network training interface, and respectively establishing labeling, training and verifying data set folders;
modifying the network model, linking to a training and verification data set corresponding to the local video processing platform, and simultaneously modifying training classification data into the number of classes to be classified and the number of GPUs used for training;
and executing a network operation instruction on a local video processing platform, starting a network to start data training, and adjusting a network model by modifying iteration times to enable the average detection rate of a training result to reach an ideal value so as to obtain a road surface abnormal condition detection model.
Optionally, the road surface anomalies include road surface spills, cracks, potholes, and ruts.
Optionally, the multi-road-section image stitching module specifically includes:
grouping management is carried out on the first images, and the proximity relation among the images in different groups is recorded;
respectively capturing two frame images of two adjacent side frequency monitoring base stations, and extracting SIFT features;
determining the matching degree of any two feature vectors according to the similarity between the feature vectors corresponding to the SIFT features;
and if the matching between the two frames of images still has a plurality of mismatching conditions, optimizing the matching result by adopting a RANSAC algorithm.
Optionally, the matching constraint conditions of the feature vectors are specifically as follows:
wherein, PRIs the closest feature point to the feature point R, PiRepresenting the feature point that is the second closest to R, d representing the Euclidean distance between two features, tdRepresenting a distance threshold.
Optionally, the fusion algorithm based on the dynamic weighted average specifically includes:
F(x,y)＝λAA(x,y)+λBB(x,y)
wherein F (x, y) represents the fused image, A (x, y) and B (x, y) represent any two images participating in the fusion, and λAAnd λBRepresenting the weight occupied by any two images participating in the fusion.
Optionally, said λAAnd λBConcrete meterThe calculation is as follows:
λB＝1-λA；
wherein, x represents the coordinate of the current pixel point in the horizontal direction when the two images are fused, and xminAnd xmaxCoordinates respectively indicating the start position and the end position of the overlapped portion of the two images in the horizontal direction.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the roadside vision monitoring base station comprises the visible light camera and the infrared camera, so that the night vision monitoring capability of the expressway is improved while the day vision monitoring capability of the original vision monitoring system is ensured, and the timeliness of the pavement maintenance and management of the expressway is ensured.
The invention also adopts image processing and deep learning methods to detect the abnormal road surface condition of the highway road surface, reduces the interference of dynamic environment to image processing algorithm, meets the requirement of actual maintenance engineering of the highway on a vision-based highway road surface condition detection system, can be quickly applied to a domestic highway roadside monitoring system, and realizes efficient and stable road surface condition detection.
The invention also realizes multi-source image data fusion, establishes a global road surface coordinate system, and performs data communication with the road section monitoring platform in real time, thereby providing global image data information for highway road surface maintenance personnel, enabling the highway road surface maintenance personnel to timely process the detected road surface abnormal condition, and ensuring the service quality and the service performance of the highway.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other structural schematic diagrams according to these drawings without inventive labor.
FIG. 1 is a schematic structural view of a highway pavement condition detection system according to the present invention;
FIG. 2 is a schematic structural diagram of a roadside vision monitoring base station according to the present invention;
FIG. 3(a) is a schematic view of the roadside vision monitoring base station according to the present invention;
FIG. 3(b) is a schematic view of the operation of the roadside vision monitoring base station under night vision conditions according to the present invention.
Description of the symbols:
11-visible light camera, 12-infrared camera, 13-upright post, 14-power supply box, network cable and other power supply boxes.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a highway pavement condition detection system, which overcomes the defects of low pavement disease detection precision and low real-time performance in the prior art.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
As shown in fig. 1, the present invention provides a system for detecting a road surface condition of an expressway, comprising a controller and a plurality of roadside visual monitoring base stations connected to the controller and disposed at roadside of the expressway, wherein the roadside visual monitoring base stations comprise visible light cameras and infrared cameras. As shown in fig. 2, the visual angle of the visible light camera 11 and the infrared camera 12 are kept parallel, and the lens faces downward at 90 degrees vertically to the expressway; the visible light camera 11 and the infrared camera 12 are both arranged on a vertical column 13, and a power supply box 14 such as a power supply and a network cable is also arranged on the vertical column 13.
The visible light camera collects road surface area images in a far focus mode in a normal state, and when the road surface abnormal state is detected, the holder is adjusted to a short focus mode to collect clear images of abnormal road surface area textures.
Specifically, the calculation formula of the monitoring height from the ground to the plane where the lens of the visible light camera is located is as follows:
wherein the content of the first and second substances,
dGfor monitoring height, xlFor the transverse length, y, of the monitoring field of view of a visible light cameralLongitudinal length of field of view for monitoring of visible camera, flFocal length, x, corresponding to the monitoring field range of the visible light camera1Is the transverse length, y, of the optical photosensitive surface of a visible light camera1Is the longitudinal length of the optical photosensitive surface of a visible light camera, d1And d2Is a process variable.
As shown in fig. 3, under a day-to-day situation, the roadside vision monitoring base station acquires road surface images in the perception area in the tele mode in real time and uploads the road surface images to the controller; when the abnormal condition of the road surface is detected, the focal length of the holder is adjusted to be in the short-focus mode, and the abnormal area (transverse length x) of the road surface is capturedsLongitudinal length ys) Guarantee the acquired image (horizontal length x)2Longitudinal length y2) The local characteristics of the abnormal area can be maximized, and the image is stored to the storage platform where the base station is located in real time. Under the night vision condition, the infrared camera senses the road surface area in real time and uploads the road surface image to the controller.
The controller comprises a road surface abnormity detection module, a multi-section image splicing module and a multi-source image data fusion module.
The road surface abnormity detection module adopts a semantic classification network, captures image data of local abnormity and diseases of the road surface in a short-focus mode of a visible light camera for classification learning, trains and classifies semantic features of images under the condition of road surface abnormity, and detects a classification result of the road surface abnormity.
The image splicing module of the multiple road sections collects local road surface images captured under the far focus mode of each road side vision monitoring base station, characteristic vectors are extracted through an SIFT characteristic extraction method, similarity matching of the characteristic vectors is carried out, and finally the RANSAC algorithm is used for optimizing matching results, eliminating mismatching characteristic points and achieving image matching and splicing among different road sections.
And the multi-source image data fusion module analyzes the difference of the overlapped parts of the images of different road sections after the images of the multi-element road sections are spliced and the images of different road sections are matched, realizes the transition fusion of the images of the multi-element road sections between illumination and image form change through a fusion algorithm based on dynamic weighted average, and establishes a global image of the highway pavement.
The roadside vision monitoring base station captures road surface images under the conditions of day view and night vision in real time, obtains local clear texture image data of an expressway area where the road surface with the abnormal condition is located by adjusting the focal length of the camera under the condition that the road surface is abnormal, and uploads the local clear texture image data to the controller in real time.
The road surface abnormity detection is realized by collecting road surface abnormity images on storage platforms of different roadside visual monitoring base stations to train a deep learning network, and detecting images with different abnormal conditions; semantic annotation is carried out on the training image data set to obtain a training data set corresponding to four types of images of road surface sprinkles, cracks, pits and ruts; a deep learning network based on semantic learning is built on a video processing server platform of a road segment monitoring center to detect images of abnormal road conditions, and meanwhile, the performance of a detection classifier is improved by adjusting training parameters and optimizing a training data set. The specific implementation steps are as follows:
step S101: and the road abnormal condition image acquired by the visible light camera with the cloud deck in the roadside vision monitoring base station in the short-focus mode is stored in a local storage platform in real time, and the storage platform is accessed through a local area network to acquire the road abnormal condition image stored in the current time period.
Step S102: labeling the road condition abnormal image data sets through a labelme labeling data tool to generate a json labeling file corresponding to each image, and generating a data file which corresponds to the labeling file and can be identified by a semantic network Mask RCNN through a labelme labeling data set-COCO data set conversion tool to obtain a total json file in a COCO data set format.
Step S103: the method comprises the steps of completing the establishment of a Mask RCNN network on a local video processing platform, firstly installing installation file dependence items matched with a system platform, simultaneously installing a Caffe2 network environment, installing a COCO data set application interface after the installation of the Caffe2 network environment is completed, and finally installing a Detectron network environment to complete the integral establishment of the Mask RCNN network.
Step S104: classifying the labeled and converted data sets according to a Mask RCNN network training interface, and respectively establishing labeled, trained and verified data set folders.
Step S105: and modifying the network model, linking to a training and verification data set corresponding to the local processing platform, and simultaneously modifying training classification data into the number of classes to be classified and the number of GPUs used for training.
Step S106: executing a network operation instruction on a local video processing platform, starting a network to start data training, adjusting a network model by modifying iteration times to enable the average detection rate of a training result to reach an ideal value, and finally obtaining a pkl model file which is a road surface abnormal condition detection model to realize the detection of image types of four road surface abnormal conditions such as a spilled object, a crack, a pit slot, a rut and the like.
The multi-section image splicing realizes the accurate feature vector matching of any two frames of section images by extracting the characteristics of the local monitoring area images of each section collected by a section monitoring center video processing platform; improving the matching accuracy of the feature vectors through corresponding vector matching constraint conditions; and the RANSAC algorithm is used for optimizing and eliminating the mismatching characteristic data, so that the accuracy of the image splicing algorithm is improved. The specific implementation steps are as follows:
step S201: grouping management is carried out on roadside local image data uploaded to a road section monitoring center platform, and the proximity relation among images in different groups is recorded.
Step S202: and carrying out SIFT feature extraction on two frames of images adjacent to the space acquisition position, namely two frames of images captured from two adjacent roadside video monitoring base stations respectively.
Step S203: and judging the matching degree of any two feature points according to the similarity between the feature vectors corresponding to the SIFT features. The matching accuracy of the feature vectors is improved through corresponding vector matching constraint conditions, wherein the constraint conditions are as follows:
wherein, PRIs the closest feature point to the feature point R, PiRepresenting the feature point that is the second closest to R, d representing the Euclidean distance between two features, tdRepresenting a distance threshold.
Step S204: after the matching process is completed, the matching between the two frames of images still has more mismatching conditions, the RANSAC algorithm is adopted to optimize the matching result, the characteristic points of the mismatching are eliminated, and the matching accuracy between the two adjacent frames of images is further improved.
Furthermore, the multi-source image data fusion improves the illumination change consistency, the object form consistency and the transition stability between images after image splicing through a fusion algorithm based on dynamic weighted average, and the fusion algorithm based on dynamic weighted average specifically comprises the following steps:
F(x,y)＝λAA(x,y)+λBB(x,y)。
wherein F (x, y) represents the fused image, A (x, y) and B (x, y) represent any two images participating in the fusion, and λAAnd λBRepresenting the weight occupied by any two images participating in the fusion.
Said lambdaAAnd λBThe specific calculation method is as follows:
λB＝1-λA；
wherein, x represents the coordinate of the current pixel point in the horizontal direction when the two images are fused, and xminAnd xmaxCoordinates respectively indicating the start position and the end position of the overlapped portion of the two images in the horizontal direction.
Compared with the prior art, the invention also has the following advantages:
(1) the invention adopts image processing and deep learning methods to process the image of the highway pavement to detect the abnormal condition of the pavement, reduces the interference of dynamic environment to the image processing algorithm, meets the requirement of actual maintenance engineering of the highway on a vision-based highway pavement condition detection system, can be quickly applied to a highway roadside monitoring system in China, and realizes efficient and stable pavement condition detection.
(2) According to the invention, the existing roadside vision monitoring system is improved, the infrared sensing equipment is additionally arranged, the day vision monitoring capability of the original vision monitoring system is ensured, meanwhile, the night vision monitoring capability of different highway sections is improved, and the timeliness of pavement maintenance and management is ensured.
(3) According to the invention, by means of a multi-section image splicing algorithm, multi-source image data fusion is realized, a global road surface coordinate system is established, data communication is carried out with a road section monitoring platform in real time, global image data information is provided for highway road surface maintenance personnel, detected road surface abnormal conditions are processed in time, and the service quality and the service performance of the highway are ensured.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (8)
1. A highway pavement condition detection system comprises a controller and a plurality of roadside visual monitoring base stations which are connected with the controller and arranged on the roadside of a highway, wherein the roadside visual monitoring base stations comprise visible light cameras, and the roadside visual monitoring base stations further comprise infrared cameras;
the visual angles of the visible light camera and the infrared camera are kept parallel;
the visible light camera is used for acquiring a first image and a second image of the highway pavement in real time and uploading the first image and the second image to the controller; the first image is a local pavement image of the highway pavement, which is acquired by the visible light camera in a far focus mode in a normal mode; the second image is an abnormal road surface area texture image acquired after the visible light camera adjusts the holder to a short focus mode when the road surface abnormal condition is detected;
the infrared camera is used for sensing the highway pavement in real time in a night vision mode and uploading a third acquired image of the highway pavement to the controller;
the controller includes:
the road surface abnormity detection module is connected with the visible light camera and used for training and classifying the semantic features of the second image by adopting a semantic classification network to determine the classification result of the road surface abnormity condition;
the multi-road-section image splicing module is connected with the visible light camera and used for extracting a feature vector from the first image by an SIFT feature extraction method, matching the similarity of the feature vector, optimizing the matching result by using an RANSAC algorithm, eliminating the feature points which are mistakenly matched and realizing the image matching and splicing among different road sections;
and the multi-source image data fusion module is used for analyzing the difference of the image overlapping parts of different road sections, and realizing the transition fusion of the images of the multiple road sections between illumination and image form change based on a fusion algorithm of dynamic weighted average so as to establish a global image of the highway pavement.
2. The system for detecting the road surface condition of the expressway according to claim 1, wherein a calculation formula of the monitoring height from the ground to a plane where a lens of the visible light camera is located is as follows:
wherein the content of the first and second substances,
dGfor monitoring height, xlFor the transverse length, y, of the monitoring field of view of a visible light cameralLongitudinal length of field of view for monitoring of visible camera, flFocal length, x, corresponding to the monitoring field range of the visible light camera1Is the transverse length, y, of the optical photosensitive surface of a visible light camera1Is the longitudinal length of the optical photosensitive surface of a visible light camera, d1And d2Is a process variable.
3. The system for detecting the road surface condition of the expressway according to claim 1, wherein the road surface abnormality detecting module specifically comprises:
accessing a local storage platform through a local area network to acquire the second image stored in the current time period;
semantic annotation is carried out on the second images through a labelme annotation data tool, json annotation files corresponding to the second images are generated, and then a total json file in a COCO data set format is obtained through a labelme annotation data set to COCO data set conversion tool;
constructing a Mask RCNN network on a local video processing platform;
classifying the total json files in the COCO data set format according to a Mask RCNN network training interface, and respectively establishing labeling, training and verifying data set folders;
modifying the network model, linking to a training and verification data set corresponding to the local video processing platform, and simultaneously modifying training classification data into the number of classes to be classified and the number of GPUs used for training;
and executing a network operation instruction on a local video processing platform, starting a network to start data training, and adjusting a network model by modifying iteration times to enable the average detection rate of a training result to reach an ideal value so as to obtain a road surface abnormal condition detection model.
4. The highway pavement condition detecting system according to claim 1, wherein the pavement abnormality conditions include pavement throws, cracks, pits, and ruts.
5. The system for detecting the road surface condition of the expressway according to claim 1, wherein the multi-segment image stitching module specifically comprises:
grouping management is carried out on the first images, and the proximity relation among the images in different groups is recorded;
respectively capturing two frame images of two adjacent side frequency monitoring base stations, and extracting SIFT features;
determining the matching degree of any two feature vectors according to the similarity between the feature vectors corresponding to the SIFT features;
and if the matching between the two frames of images still has a plurality of mismatching conditions, optimizing the matching result by adopting a RANSAC algorithm.
6. The system for detecting the road surface condition of the expressway according to claim 5, wherein the matching constraint conditions of the eigenvectors are specifically as follows:
wherein, PRIs the closest feature point to the feature point R, PiRepresenting the feature point that is the second closest to R, d representing the Euclidean distance between two features, tdRepresenting a distance threshold.
7. The system according to claim 1, wherein the fusion algorithm based on the dynamic weighted average is specifically:
F(x,y)＝λAA(x,y)+λBB(x,y)；
wherein F (x, y) represents the fused image, A (x, y) and B (x, y) represent any two images participating in the fusion, and λAAnd λBRepresenting the weight occupied by any two images participating in the fusion.
8. The highway pavement condition detecting system according to claim 7, wherein λAAnd λBThe specific calculation method is as follows:
λB＝1-λA；
wherein, x represents the coordinate of the current pixel point in the horizontal direction when the two images are fused, and xminAnd xmaxCoordinates respectively indicating the start position and the end position of the overlapped portion of the two images in the horizontal direction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111093054.3A CN113780312B (en) | 2019-11-21 | 2019-11-21 | Highway road surface condition detecting system |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911149152.7A CN110991466A (en) | 2019-11-21 | 2019-11-21 | Highway road surface condition detecting system based on novel vision sensing equipment |
| CN202111093054.3A CN113780312B (en) | 2019-11-21 | 2019-11-21 | Highway road surface condition detecting system |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911149152.7A Division CN110991466A (en) | 2019-11-21 | 2019-11-21 | Highway road surface condition detecting system based on novel vision sensing equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113780312A true CN113780312A (en) | 2021-12-10 |
| CN113780312B CN113780312B (en) | 2024-04-12 |
Family
ID=70085566
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911149152.7A Pending CN110991466A (en) | 2019-11-21 | 2019-11-21 | Highway road surface condition detecting system based on novel vision sensing equipment |
| CN202111093054.3A Active CN113780312B (en) | 2019-11-21 | 2019-11-21 | Highway road surface condition detecting system |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911149152.7A Pending CN110991466A (en) | 2019-11-21 | 2019-11-21 | Highway road surface condition detecting system based on novel vision sensing equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN110991466A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114758322A (en) * | 2022-05-13 | 2022-07-15 | 安徽省路通公路工程检测有限公司 | Road quality detection system based on machine identification |
| CN114842649A (en) * | 2022-07-01 | 2022-08-02 | 无锡软美信息科技有限公司 | Traffic control method and system for expressway |
| CN116994214A (en) * | 2023-09-25 | 2023-11-03 | 南京华琨信息科技有限公司 | Highway road safety evaluation method and system |
| CN117876182A (en) * | 2023-12-25 | 2024-04-12 | 山东高速股份有限公司 | Expressway maintenance decision management method based on multi-source data analysis |
| CN118038284A (en) * | 2024-04-15 | 2024-05-14 | 贵州黔通工程技术有限公司 | Intelligent detection method and system for expressway diseases |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111523536A (en) * | 2020-04-10 | 2020-08-11 | 南京智行信息科技有限公司 | Self-adaptive road surface projectile intelligent detection method based on fast RCNN |
| CN111553902B (en) * | 2020-04-28 | 2021-08-17 | 江西方兴科技有限公司 | Highway road surface safety monitoring system based on big data |
| CN112462752A (en) * | 2020-10-12 | 2021-03-09 | 星火科技技术(深圳)有限责任公司 | Data acquisition method, equipment, storage medium and device of intelligent trolley |
| CN112488221B (en) * | 2020-12-07 | 2022-06-14 | 电子科技大学 | Road pavement abnormity detection method based on dynamic refreshing positive sample image library |
| CN112800913B (en) * | 2021-01-20 | 2022-06-28 | 同济大学 | Pavement damage data space-time analysis method based on multi-source feature fusion |
| CN112906664B (en) * | 2021-04-06 | 2023-02-24 | 昭通亮风台信息科技有限公司 | Neural network-based method and system for detecting abnormal events in tunnel |
| CN113191270B (en) * | 2021-04-30 | 2022-11-11 | 北京市商汤科技开发有限公司 | Method and device for detecting throwing event, electronic equipment and storage medium |
| CN113780259B (en) * | 2021-11-15 | 2022-03-15 | 中移(上海)信息通信科技有限公司 | Road surface defect detection method and device, electronic equipment and readable storage medium |
| CN117037105B (en) * | 2023-09-28 | 2024-01-12 | 四川蜀道新能源科技发展有限公司 | Pavement crack filling detection method, system, terminal and medium based on deep learning |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101950426A (en) * | 2010-09-29 | 2011-01-19 | 北京航空航天大学 | Vehicle relay tracking method in multi-camera scene |
| WO2016086754A1 (en) * | 2014-12-03 | 2016-06-09 | 中国矿业大学 | Large-scale scene video image stitching method |
| CN109029731A (en) * | 2018-05-24 | 2018-12-18 | 河海大学常州校区 | A kind of power equipment exception monitoring system and method based on multi-vision visual |
| WO2019109771A1 (en) * | 2017-12-05 | 2019-06-13 | 南京南瑞信息通信科技有限公司 | Power artificial-intelligence visual-analysis system on basis of multi-core heterogeneous parallel computing |
| CN110378879A (en) * | 2019-06-26 | 2019-10-25 | 杭州电子科技大学 | A kind of Bridge Crack detection method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102590883A (en) * | 2011-09-30 | 2012-07-18 | 长春奥普光电技术股份有限公司 | Airfield pavement foreign body scanning device |
| CN104219508A (en) * | 2014-09-30 | 2014-12-17 | 成都市晶林科技有限公司 | Urban intelligent transport scheduling system and method |
| CN107328783A (en) * | 2017-07-31 | 2017-11-07 | 广东容祺智能科技有限公司 | A kind of bridge intelligent checking system based on unmanned plane |
| CN110415233A (en) * | 2019-07-26 | 2019-11-05 | 东南大学 | Pavement crack rapid extracting method based on two step convolutional neural networks |
-
2019
- 2019-11-21 CN CN201911149152.7A patent/CN110991466A/en active Pending
- 2019-11-21 CN CN202111093054.3A patent/CN113780312B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101950426A (en) * | 2010-09-29 | 2011-01-19 | 北京航空航天大学 | Vehicle relay tracking method in multi-camera scene |
| WO2016086754A1 (en) * | 2014-12-03 | 2016-06-09 | 中国矿业大学 | Large-scale scene video image stitching method |
| WO2019109771A1 (en) * | 2017-12-05 | 2019-06-13 | 南京南瑞信息通信科技有限公司 | Power artificial-intelligence visual-analysis system on basis of multi-core heterogeneous parallel computing |
| CN109029731A (en) * | 2018-05-24 | 2018-12-18 | 河海大学常州校区 | A kind of power equipment exception monitoring system and method based on multi-vision visual |
| CN110378879A (en) * | 2019-06-26 | 2019-10-25 | 杭州电子科技大学 | A kind of Bridge Crack detection method |
Non-Patent Citations (2)
| Title |
|---|
| 安明伟;陈启美;郭宗良;: "基于路况视频的气象能见度检测方法与系统设计", 仪器仪表学报, no. 05 * |
| 李丽: "复杂背景下的路面裂缝检测算法研究", 中国优秀硕士学位论文全文数据库工程科技Ⅱ辑, pages 40 - 41 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114758322A (en) * | 2022-05-13 | 2022-07-15 | 安徽省路通公路工程检测有限公司 | Road quality detection system based on machine identification |
| CN114842649A (en) * | 2022-07-01 | 2022-08-02 | 无锡软美信息科技有限公司 | Traffic control method and system for expressway |
| CN114842649B (en) * | 2022-07-01 | 2022-09-06 | 无锡软美信息科技有限公司 | Traffic control method and system for expressway |
| CN116994214A (en) * | 2023-09-25 | 2023-11-03 | 南京华琨信息科技有限公司 | Highway road safety evaluation method and system |
| CN116994214B (en) * | 2023-09-25 | 2023-12-08 | 南京华琨信息科技有限公司 | Highway road safety evaluation method and system |
| CN117876182A (en) * | 2023-12-25 | 2024-04-12 | 山东高速股份有限公司 | Expressway maintenance decision management method based on multi-source data analysis |
| CN117876182B (en) * | 2023-12-25 | 2024-06-21 | 山东高速股份有限公司 | Expressway maintenance decision management method based on multi-source data analysis |
| CN118038284A (en) * | 2024-04-15 | 2024-05-14 | 贵州黔通工程技术有限公司 | Intelligent detection method and system for expressway diseases |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110991466A (en) | 2020-04-10 |
| CN113780312B (en) | 2024-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113780312A (en) | Highway road surface condition detecting system | |
| Arya et al. | Transfer learning-based road damage detection for multiple countries | |
| CN110175576B (en) | Driving vehicle visual detection method combining laser point cloud data | |
| Rateke et al. | Road surface classification with images captured from low-cost camera-road traversing knowledge (rtk) dataset | |
| CN112800913B (en) | Pavement damage data space-time analysis method based on multi-source feature fusion | |
| Garilli et al. | Automatic detection of stone pavement's pattern based on UAV photogrammetry | |
| CN111027430B (en) | Traffic scene complexity calculation method for intelligent evaluation of unmanned vehicles | |
| CN105654732A (en) | Road monitoring system and method based on depth image | |
| CN110910354A (en) | Road detection vehicle and road detection method and device | |
| CN102867417A (en) | Taxi anti-forgery system and taxi anti-forgery method | |
| CN102346844B (en) | Device and method for identifying fault of losing screw bolts for truck center plates | |
| CN112528979B (en) | Transformer substation inspection robot obstacle distinguishing method and system | |
| CN112508911A (en) | Rail joint touch net suspension support component crack detection system based on inspection robot and detection method thereof | |
| CN105303162A (en) | Target proposed algorithm-based insulator recognition algorithm for aerial images | |
| WO2022104798A1 (en) | 5g-based unmanned electronic traffic police duty system | |
| CN116901089B (en) | Multi-angle vision distance robot control method and system | |
| CN104331708B (en) | A kind of zebra crossing automatic detection analysis method and system | |
| Zhang et al. | Pavement crack detection based on deep learning | |
| CN202887450U (en) | Taxi anti-fake system | |
| CN104572792B (en) | Image indexing method, image indexing device and computer readable medium | |
| Sun et al. | UAV photogrammetry-based accident assessment road condition analysis using image classification | |
| CN112488026A (en) | Lane damage detection method based on video analysis | |
| Serafim et al. | Automated Detection of Defects and Vertical Signs on Roadways Using Images Produced by Drivers | |
| Zhu et al. | Automatic utility pole inclination angle measurement using unmanned aerial vehicle and deep learning | |
| CN111595859A (en) | Bridge and culvert damage detection method and damage detection and management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |