CN114973694B - Tunnel traffic flow monitoring system and method based on inspection robot - Google Patents
Tunnel traffic flow monitoring system and method based on inspection robot Download PDFInfo
- Publication number
- CN114973694B CN114973694B CN202210545940.3A CN202210545940A CN114973694B CN 114973694 B CN114973694 B CN 114973694B CN 202210545940 A CN202210545940 A CN 202210545940A CN 114973694 B CN114973694 B CN 114973694B
- Authority
- CN
- China
- Prior art keywords
- module
- vehicle
- inspection
- track
- fault detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 130
- 238000012544 monitoring process Methods 0.000 title claims abstract description 36
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000001514 detection method Methods 0.000 claims abstract description 62
- 238000007781 pre-processing Methods 0.000 claims abstract description 14
- 238000005070 sampling Methods 0.000 claims abstract description 8
- 238000004891 communication Methods 0.000 claims description 36
- 238000004364 calculation method Methods 0.000 claims description 21
- 238000012549 training Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 6
- 230000009526 moderate injury Effects 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 abstract description 10
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/065—Traffic control systems for road vehicles by counting the vehicles in a section of the road or in a parking area, i.e. comparing incoming count with outgoing count
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
- G07C1/20—Checking timed patrols, e.g. of watchman
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention belongs to the technical field of inspection robots, and discloses a tunnel traffic flow monitoring system and a tunnel traffic flow monitoring method based on the inspection robots, wherein the tunnel traffic flow monitoring method comprises the following steps: establishing a vehicle target identification model and a patrol track fault detection model; performing fault detection on the inspection track according to the inspection track video data by using an inspection track fault detection model; collecting real-time vehicle passing video data and real-time inspection track video data in a tunnel; preprocessing real-time vehicle passing video data to obtain preprocessed vehicle passing image data of continuous frames; sequentially inputting the preprocessed vehicle passing image data of the continuous frames into a vehicle target recognition model to recognize and track the vehicle target; and calculating the vehicle flow according to the vehicle target number of the sampling period. The invention solves the problems of difficult monitoring of traffic flow in a tunnel, low data transmission efficiency, low safety, large labor cost investment and low accuracy in the prior art.
Description
Technical Field
The invention belongs to the technical field of inspection robots, and particularly relates to a tunnel traffic flow monitoring system and method based on an inspection robot.
Background
Along with the development of traffic construction, a large amount of mountain-penetrating or sea-penetrating tunnels are arranged, so that the living quality and the traveling quality of citizens are improved, meanwhile, the monitoring and management difficulty of traffic flow in the tunnels is increased, and due to narrow space and poor signals in the tunnels, the data transmission efficiency is low, a manual monitoring mode is generally adopted, the safety is low, the labor cost is high, the vehicle passing speed is high, and the accuracy is low only in a mode of carrying out on-site statistics or video statistics by naked eyes.
Disclosure of Invention
The invention aims to solve the problems of difficult monitoring of traffic flow in a tunnel, low data transmission efficiency, low safety, large labor cost investment and low accuracy in the prior art, and provides a system and a method for monitoring the traffic flow of the tunnel based on a patrol robot.
The technical scheme adopted by the invention is as follows:
The utility model provides a tunnel traffic flow monitoring system based on patrol robot, including patrol robot, patrol track, edge calculation gateway and monitoring center, patrol robot sets up on patrol track top, patrol robot is provided with vehicle video acquisition unit towards the current one side of vehicle, and patrol robot and edge calculation gateway communication connection, patrol track sets up in the inside one side of tunnel, edge calculation gateway sets up on the inside top of tunnel, and edge calculation gateway and monitoring center communication connection.
Further, the inspection robot comprises a body, a mobile unit arranged at the bottom end of the body, a vehicle video acquisition unit arranged at one side of the body facing the vehicle passing, a track video acquisition unit arranged at one end of the body, and a robot main control unit and a rechargeable battery which are arranged in the body, wherein the robot main control unit is respectively and electrically connected with the mobile unit, the vehicle video acquisition unit and the track video acquisition unit, the robot main control unit is in communication connection with an edge computing gateway, the mobile unit is arranged at the top end of the inspection track, and the rechargeable battery is respectively and electrically connected with the robot main control unit, the mobile unit, the vehicle video acquisition unit and the track video acquisition unit;
The robot main control unit comprises a robot main control module, a positioning module, a first storage module, a motor driving module and a wireless communication module, wherein the robot main control module is respectively and electrically connected with the positioning module, the first storage module, the motor driving module and the wireless communication module, the motor driving module is electrically connected with the mobile unit, and the wireless communication module is in communication connection with the edge computing gateway;
The vehicle video acquisition unit is a first high-speed infrared camera, and the first high-speed infrared camera is arranged facing one side where vehicles pass through, the track video acquisition unit is a second high-speed infrared camera arranged at two ends of the body, and the second high-speed infrared camera is arranged facing the inspection track.
Further, the edge computing gateway comprises an edge computing unit and a network unit, wherein the edge computing unit is electrically connected with the network unit, and the network unit is respectively in communication connection with the wireless communication module of the inspection robot and the monitoring center;
The edge computing unit comprises an edge computing main control module, a second storage module, an image preprocessing module and an encryption module, and the edge computing main control module is respectively connected with the second storage module, the image preprocessing module, the encryption module and the network unit.
Further, the monitoring center is provided with a data server, the data server is in communication connection with the edge computing gateway, and the data server is in communication connection with an external cloud data center;
The data server comprises a data parallel receiving module, a decryption module, a vehicle target identification module, a vehicle flow calculation module, a routing inspection track fault detection module, a cache database module and a data parallel uploading module, wherein the data parallel receiving module is respectively connected with the edge calculation gateway and the decryption module, the decryption module is respectively connected with the vehicle target identification module and the routing inspection track fault detection module, the vehicle target identification module is connected with the vehicle flow calculation module, the cache database module is respectively connected with the data parallel receiving module, the decryption module, the vehicle target identification module, the vehicle flow calculation module, the routing inspection track fault detection module and the data parallel uploading module, and the data parallel uploading module is in communication connection with an external cloud data center;
the vehicle target recognition module is provided with a vehicle target recognition model, and the inspection track fault detection module is provided with an inspection track fault detection model.
A tunnel traffic flow monitoring method based on a patrol robot and a tunnel traffic flow monitoring system, comprising the following steps:
establishing a vehicle target identification model and a patrol track fault detection model;
Performing fault detection on the inspection track according to the last inspection track video data by using an inspection track fault detection model, sending an alarm signal if serious faults exist, stopping the tunnel traffic flow monitoring method, and otherwise, entering the next step;
Collecting real-time vehicle passing video data and real-time inspection track video data in a tunnel;
preprocessing real-time vehicle passing video data to obtain preprocessed vehicle passing image data of continuous frames;
sequentially inputting the preprocessed vehicle passing image data of the continuous frames into a vehicle target recognition model to recognize and track the vehicle targets, so as to obtain the number of the vehicle targets;
and calculating the traffic flow according to the number of the vehicle targets in the sampling period to obtain a tunnel traffic flow result.
Further, a vehicle target recognition model and a patrol track fault detection model are established, and the method comprises the following steps:
Acquiring a historical vehicle passing image data set and a historical inspection track image data set, and carrying out data set expansion and preprocessing on the historical vehicle passing image data set and the historical inspection track image data set to obtain an expanded vehicle passing training data set and an expanded inspection track training data set;
Establishing an initial vehicle target recognition model based on YOLOv algorithm, and establishing an initial inspection orbit fault detection model based on PP-YOLO-Tiny algorithm;
inputting the vehicle passing training data set into an initial vehicle target recognition model for optimization training to obtain an optimal vehicle target recognition model;
inputting the inspection track training data set into an initial inspection track fault detection model to obtain an optimal inspection track fault detection model.
Further, the network structure of the vehicle target recognition model comprises a first input end, a backstone module, a first DetectionNeck module and a first Prediction module which are connected in sequence;
The network structure of the inspection track fault detection model comprises a second input end, a MobileNetV module, a second DetectionNeck module, a DetectionHead module and a second Prediction module which are sequentially connected.
Further, the inspection track fault detection model is used for carrying out fault detection on the inspection track according to the last inspection track video data, and the method comprises the following steps:
performing frame interception and pretreatment on the last inspection track video data to obtain continuous frame pretreatment post-inspection track video data;
sequentially inputting the video data of the tour-inspection track after the pretreatment of the continuous frames into a tour-inspection track fault detection model to carry out target detection, so as to obtain a plurality of tour-inspection track fault target images;
performing fault detection on a plurality of inspection track fault target images to obtain a fault detection result;
fault detection results include severe faults, moderate damage, and mild scratches.
Further, the method for preprocessing the real-time vehicle passing video data comprises the following steps:
frame interception is carried out on the real-time vehicle passing video data to obtain initial vehicle passing image data of continuous frames;
And carrying out denoising, gray level processing and normalization processing on the initial vehicle passing image data of the continuous frames to obtain the preprocessed vehicle passing image data of the continuous frames.
Further, the pre-processed vehicle passing image data of the continuous frames are sequentially input into a vehicle target recognition model for vehicle target recognition and tracking, and the method comprises the following steps:
Performing grid division on the preprocessed vehicle passing image data of all frames, and acquiring prior frames of each grid;
Acquiring an initial prediction frame corresponding to each prior frame according to the offset of the prior frame and a preset prediction frame;
Performing non-maximum suppression screening on the initial prediction frames according to the preset intersection ratio and the preset confidence coefficient to obtain final prediction frames of the preprocessed vehicle passing image data of all frames;
Carrying out vehicle target recognition on the image area in the final prediction frame to obtain a plurality of vehicle targets;
And tracking and counting the same vehicle target by using a multi-target tracking algorithm to obtain the number of the vehicle targets.
The beneficial effects of the invention are as follows:
1) According to the tunnel traffic flow monitoring system based on the inspection robot, the inspection robot and the inspection track are adopted to monitor traffic flow in the tunnel, so that a manual mode is avoided, safety and accuracy are improved, tunnel traffic flow monitoring difficulty is reduced, labor cost investment is reduced, meanwhile, an edge computing gateway is adopted to realize data transmission under the condition of no network, and data transmission efficiency is improved.
2) According to the tunnel traffic flow monitoring method based on the inspection robot, the vehicle target and the inspection track fault target are automatically detected and identified based on the deep learning algorithm, so that the manual identification and statistics modes are avoided, the monitoring accuracy is improved, the inspection track fault detection is firstly carried out between each tunnel traffic flow monitoring, the normal work of the inspection robot is ensured, and the practicability and the safety are improved.
Other advantageous effects of the present invention will be further described in the detailed description.
Drawings
Fig. 1 is a block diagram of a tunnel traffic flow monitoring system based on a patrol robot in the present invention.
Fig. 2 is a flow chart of a method for monitoring tunnel traffic flow based on a patrol robot in the present invention.
Detailed Description
The invention is further illustrated by the following description of specific embodiments in conjunction with the accompanying drawings.
Example 1:
As shown in fig. 1, this embodiment provides a tunnel traffic flow monitoring system based on inspection robot, including setting up the inspection robot in different tunnels, the track of inspecting, edge calculation gateway and unique monitoring center, the inspection robot sets up on the track top of inspecting, inspection robot is provided with vehicle video acquisition unit towards the current one side of vehicle, and inspection robot and edge calculation gateway communication connection, the track of inspecting sets up in the inside one side of tunnel, edge calculation gateway sets up on the inside top of tunnel, and edge calculation gateway and monitoring center communication connection.
The inspection track ensures the normal action of the inspection robot, the vehicle video acquisition unit is used for acquiring vehicle traffic video data in the tunnel, the edge computing gateway is used for data transmission, the monitoring center monitors the vehicle flow according to the vehicle traffic video data, and the edge computing gateway can ensure the data transmission under the condition of network difference or network loss in the tunnel and has certain edge computing capability.
Preferably, the inspection robot comprises a body, a mobile unit arranged at the bottom end of the body, a vehicle video acquisition unit arranged at one side of the body facing the vehicle passing, a track video acquisition unit arranged at one end of the body, and a robot main control unit and a rechargeable battery which are arranged in the body, wherein the robot main control unit is respectively and electrically connected with the mobile unit, the vehicle video acquisition unit and the track video acquisition unit, the robot main control unit is in communication connection with an edge computing gateway, the mobile unit is arranged at the top end of the inspection track, and the rechargeable battery is respectively and electrically connected with the robot main control unit, the mobile unit, the vehicle video acquisition unit and the track video acquisition unit;
The robot main control unit comprises a robot main control module, a positioning module, a first storage module, a motor driving module and a wireless communication module, wherein the robot main control module is respectively and electrically connected with the positioning module, the first storage module, the motor driving module and the wireless communication module, the motor driving module is electrically connected with the mobile unit, and the wireless communication module is in communication connection with the edge computing gateway;
The vehicle video acquisition unit is a first high-speed infrared camera, and the first high-speed infrared camera is arranged facing one side where vehicles pass through, the track video acquisition unit is a second high-speed infrared camera arranged at two ends of the body, and the second high-speed infrared camera is arranged facing the inspection track.
Preferably, the edge computing gateway comprises an edge computing unit and a network unit, wherein the edge computing unit is electrically connected with the network unit, and the network unit is respectively connected with the wireless communication module of the inspection robot and the monitoring center in a communication way;
The edge computing unit comprises an edge computing main control module, a second storage module, an image preprocessing module and an encryption module, and the edge computing main control module is respectively connected with the second storage module, the image preprocessing module, the encryption module and the network unit; the video data is preprocessed in the edge computing units, so that the data processing pressure of the monitoring center can be reduced, the storage space of the second storage module can be saved, the monitoring center broadcasts the public key to all the edge computing units, the corresponding private key is kept locally, the encryption module encrypts and uploads the vehicle passing image data and the like acquired by the inspection robot according to the public key, the monitoring center decrypts the encrypted data by using the private key, and the safety of data transmission is guaranteed.
Preferably, the monitoring center is provided with a data server, the data server is in communication connection with the edge computing gateway, and the data server is in communication connection with an external cloud data center;
The data server comprises a data parallel receiving module, a decryption module, a vehicle target identification module, a vehicle flow calculation module, a routing inspection track fault detection module, a cache database module and a data parallel uploading module, wherein the data parallel receiving module is respectively connected with the edge calculation gateway and the decryption module, the decryption module is respectively connected with the vehicle target identification module and the routing inspection track fault detection module, the vehicle target identification module is connected with the vehicle flow calculation module, the cache database module is respectively connected with the data parallel receiving module, the decryption module, the vehicle target identification module, the vehicle flow calculation module, the routing inspection track fault detection module and the data parallel uploading module, and the data parallel uploading module is in communication connection with an external cloud data center;
The vehicle target recognition module is provided with a vehicle target recognition model, and the inspection track fault detection module is provided with an inspection track fault detection model; because the light PP-YOLO-Tiny algorithm is adopted in the follow-up method to establish the inspection track fault detection model, the inspection track fault detection model can be placed on the edge computing gateway, and the data processing pressure of the data server is further reduced.
According to the tunnel traffic flow monitoring system based on the inspection robot, the inspection robot and the inspection track are adopted to monitor traffic flow in the tunnel, so that a manual mode is avoided, safety and accuracy are improved, tunnel traffic flow monitoring difficulty is reduced, labor cost investment is reduced, meanwhile, an edge computing gateway is adopted to realize data transmission under the condition of no network, and data transmission efficiency is improved.
Example 2:
as shown in fig. 2, the present embodiment provides a method for monitoring tunnel traffic flow based on a patrol robot, and a system for monitoring tunnel traffic flow, which includes the following steps:
the method for establishing the vehicle target recognition model and the inspection track fault detection model comprises the following steps:
Acquiring a historical vehicle passing image data set and a historical inspection track image data set, and carrying out data set expansion and preprocessing on the historical vehicle passing image data set and the historical inspection track image data set to obtain an expanded vehicle passing training data set and an expanded inspection track training data set;
Performing data set expansion including overturning, cutting, translating, contrast enhancement and the like on the image data;
Establishing an initial vehicle target recognition model based on YOLOv algorithm, and establishing an initial inspection orbit fault detection model based on PP-YOLO-Tiny algorithm;
inputting the vehicle passing training data set into an initial vehicle target recognition model for optimization training to obtain an optimal vehicle target recognition model;
Inputting the inspection track training data set into an initial inspection track fault detection model to obtain an optimal inspection track fault detection model;
the network structure of the vehicle target recognition model comprises a first input end, a backlight module, a first DetectionNeck module and a first Prediction module which are connected in sequence;
the first input end processes an input image by using a mosaicdata enhancement method;
The back bone module comprises a Focus structure and a CSP structure;
The structure of the first DetectionNeck module is an FPN+PAN structure;
The first Prediction module uses GIOU _loss function to calculate Loss;
The network structure of the inspection track fault detection model comprises a second input end, a MobileNetV module, a second DetectionNeck module, a DetectionHead module and a second Prediction module which are connected in sequence;
MobileNetV3 module is a lightweight network structure combining depth separable convolution, inverted Residuals and Linear Bottleneck and SE module, searching configuration and parameters of the network using NAS neural structure search;
The second DetectionNeck module aggregates the feature information from top to bottom using a PAN structure and applies mish activation functions;
The DetectionHead module adopts the depth separable convolution which is more suitable for the mobile terminal, and compared with the conventional convolution operation, the method has fewer parameters and operation cost, and is more suitable for the memory space and calculation force of the mobile terminal;
the inspection track video data are infrared video data, and the inspection track is subjected to fault detection according to the last inspection track video data by using an inspection track fault detection model, and the method comprises the following steps:
performing frame interception and pretreatment on the last inspection track video data to obtain continuous frame pretreatment post-inspection track video data;
sequentially inputting the video data of the tour-inspection track after the pretreatment of the continuous frames into a tour-inspection track fault detection model to carry out target detection, so as to obtain a plurality of tour-inspection track fault target images;
performing fault detection on a plurality of inspection track fault target images to obtain a fault detection result;
The fault detection result comprises serious faults, moderate damage and light scratch, wherein the serious faults are breakage or serious damage of the inspection track caused by long-term running or vehicle collision, so that the inspection robot cannot normally run, the moderate damage is defect or scratch of the inspection track, the inspection robot is not influenced to pass, but an alarm signal is required to be sent to inform inspection personnel to repair, the light scratch is a shallower scratch, and the deep scratch or defect possibly develops in the future, but is not treated at present;
if serious faults exist, an alarm signal is sent out, the tunnel traffic flow monitoring method is stopped, and if not, the next step is carried out;
Collecting real-time vehicle passing video data and real-time inspection track video data in a tunnel;
The vehicle passing video data is provided with a time tag, is infrared video data, and is preprocessed to obtain preprocessed vehicle passing image data of continuous frames, and the method comprises the following steps:
frame interception is carried out on the real-time vehicle passing video data to obtain initial vehicle passing image data of continuous frames;
Denoising, gray-scale treatment and normalization treatment are carried out on the initial vehicle passing image data of the continuous frames, so as to obtain the preprocessed vehicle passing image data of the continuous frames;
Sequentially inputting the preprocessed vehicle passing image data of the continuous frames into a vehicle target recognition model to recognize and track the vehicle targets to obtain the number of the vehicle targets, wherein the method comprises the following steps of:
Performing grid division on the preprocessed vehicle passing image data of all frames, and acquiring prior frames of each grid;
Acquiring an initial prediction frame corresponding to each prior frame according to the offset of the prior frame and a preset prediction frame;
Performing non-maximum suppression screening on the initial prediction frames according to the preset intersection ratio and the preset confidence coefficient to obtain final prediction frames of the preprocessed vehicle passing image data of all frames;
Carrying out vehicle target recognition on the image area in the final prediction frame to obtain a plurality of vehicle targets;
Tracking and counting the same vehicle target by using a multi-target tracking algorithm to obtain the number of the vehicle targets;
Calculating the traffic flow according to the number of the vehicles in the sampling period to obtain a tunnel traffic flow result;
because the vehicle passing video data is provided with the time tag, the number of the vehicles passing through in the sampling period can be obtained, the number of the vehicles passing through in the sampling period is divided by the sampling period to obtain the tunnel traffic flow result, and the sampling period is usually not more than 48 hours.
According to the tunnel traffic flow monitoring method based on the inspection robot, the vehicle target and the inspection track fault target are automatically detected and identified based on the deep learning algorithm, so that the manual identification and statistics modes are avoided, the monitoring accuracy is improved, the inspection track fault detection is firstly carried out between each tunnel traffic flow monitoring, the normal work of the inspection robot is ensured, and the practicability and the safety are improved.
The invention is not limited to the alternative embodiments described above, but any person may derive other various forms of products in the light of the present invention. The above detailed description should not be construed as limiting the scope of the invention, which is defined in the claims and the description may be used to interpret the claims.
Claims (4)
1. A tunnel traffic flow monitoring method based on a patrol robot is characterized in that: the method comprises the following steps:
the method for establishing the vehicle target recognition model and the inspection track fault detection model comprises the following steps:
Acquiring a historical vehicle passing image data set and a historical inspection track image data set, and carrying out data set expansion and preprocessing on the historical vehicle passing image data set and the historical inspection track image data set to obtain an expanded vehicle passing training data set and an expanded inspection track training data set;
Establishing an initial vehicle target recognition model based on YOLOv algorithm, and establishing an initial inspection orbit fault detection model based on PP-YOLO-Tiny algorithm;
inputting the vehicle passing training data set into an initial vehicle target recognition model for optimization training to obtain an optimal vehicle target recognition model;
Inputting the inspection track training data set into an initial inspection track fault detection model to obtain an optimal inspection track fault detection model;
and performing fault detection on the inspection track according to the last inspection track video data by using an inspection track fault detection model, wherein the method comprises the following steps of:
performing frame interception and pretreatment on the last inspection track video data to obtain continuous frame pretreatment post-inspection track video data;
sequentially inputting the video data of the tour-inspection track after the pretreatment of the continuous frames into a tour-inspection track fault detection model to carry out target detection, so as to obtain a plurality of tour-inspection track fault target images;
performing fault detection on a plurality of inspection track fault target images to obtain a fault detection result;
The fault detection result comprises serious faults, moderate damages and slight scratches;
if serious faults exist, an alarm signal is sent out, the tunnel traffic flow monitoring method is stopped, and if not, the next step is carried out;
Collecting real-time vehicle passing video data and real-time inspection track video data in a tunnel;
preprocessing real-time vehicle passing video data to obtain preprocessed vehicle passing image data of continuous frames;
sequentially inputting the preprocessed vehicle passing image data of the continuous frames into a vehicle target recognition model to recognize and track the vehicle targets, so as to obtain the number of the vehicle targets;
Calculating the traffic flow according to the number of the vehicles in the sampling period to obtain a tunnel traffic flow result;
The system comprises a patrol robot, a patrol track, an edge computing gateway and a monitoring center, wherein the patrol robot is arranged at the top end of the patrol track, a vehicle video acquisition unit is arranged at one side of the patrol robot, which faces to the passing of vehicles, the patrol robot is in communication connection with the edge computing gateway, the patrol track is arranged at one side of the inside of the tunnel, the edge computing gateway is arranged at the top end of the inside of the tunnel, and the edge computing gateway is in communication connection with the monitoring center;
The inspection robot comprises a body, a moving unit arranged at the bottom end of the body, a vehicle video acquisition unit arranged at one side of the body facing the vehicle passing, a track video acquisition unit arranged at one end of the body, and a robot main control unit and a rechargeable battery which are arranged in the body, wherein the robot main control unit is respectively and electrically connected with the moving unit, the vehicle video acquisition unit and the track video acquisition unit, the robot main control unit is in communication connection with an edge computing gateway, the moving unit is arranged at the top end of the inspection track, and the rechargeable battery is respectively and electrically connected with the robot main control unit, the moving unit, the vehicle video acquisition unit and the track video acquisition unit;
the robot main control unit comprises a robot main control module, a positioning module, a first storage module, a motor driving module and a wireless communication module, wherein the robot main control module is respectively and electrically connected with the positioning module, the first storage module, the motor driving module and the wireless communication module, the motor driving module is electrically connected with the mobile unit, and the wireless communication module is in communication connection with the edge computing gateway;
The vehicle video acquisition unit is a first high-speed infrared camera, the first high-speed infrared camera is arranged on one side facing the vehicle passing, the track video acquisition unit is a second high-speed infrared camera arranged at two ends of the body, and the second high-speed infrared camera is arranged towards the inspection track;
The edge computing gateway comprises an edge computing unit and a network unit, wherein the edge computing unit is electrically connected with the network unit, and the network unit is respectively in communication connection with a wireless communication module and a monitoring center of the inspection robot;
The edge computing unit comprises an edge computing main control module, a second storage module, an image preprocessing module and an encryption module, wherein the edge computing main control module is respectively connected with the second storage module, the image preprocessing module, the encryption module and the network unit;
The monitoring center is provided with a data server which is in communication connection with the edge computing gateway, and the data server is in communication connection with an external cloud data center;
The data server comprises a data parallel receiving module, a decryption module, a vehicle target identification module, a vehicle flow calculation module, a routing inspection track fault detection module, a cache database module and a data parallel uploading module, wherein the data parallel receiving module is respectively connected with the edge calculation gateway and the decryption module, the decryption module is respectively connected with the vehicle target identification module and the routing inspection track fault detection module, the vehicle target identification module is connected with the vehicle flow calculation module, the cache database module is respectively connected with the data parallel receiving module, the decryption module, the vehicle target identification module, the vehicle flow calculation module, the routing inspection track fault detection module and the data parallel uploading module, and the data parallel uploading module is in communication connection with an external cloud data center;
The vehicle target recognition module is provided with a vehicle target recognition model, and the inspection track fault detection module is provided with an inspection track fault detection model; the vehicle target recognition model is built based on YOLOv algorithm, and the inspection orbit fault detection model is built based on PP-YOLO-Tiny algorithm.
2. The inspection robot-based tunnel traffic monitoring method according to claim 1, wherein: the network structure of the vehicle target recognition model comprises a first input end, a backstone module, a first DetectionNeck module and a first Prediction module which are connected in sequence;
The network structure of the inspection track fault detection model comprises a second input end, a MobileNetV module, a second DetectionNeck module, a DetectionHead module and a second Prediction module which are sequentially connected.
3. The inspection robot-based tunnel traffic monitoring method according to claim 2, wherein: the method for preprocessing the real-time vehicle passing video data comprises the following steps:
frame interception is carried out on the real-time vehicle passing video data to obtain initial vehicle passing image data of continuous frames;
And carrying out denoising, gray level processing and normalization processing on the initial vehicle passing image data of the continuous frames to obtain the preprocessed vehicle passing image data of the continuous frames.
4. A method for monitoring tunnel traffic flow based on inspection robot according to claim 3, characterized in that: the method sequentially inputs the preprocessed vehicle passing image data of the continuous frames into a vehicle target recognition model to recognize and track the vehicle target, and comprises the following steps:
Performing grid division on the preprocessed vehicle passing image data of all frames, and acquiring prior frames of each grid;
Acquiring an initial prediction frame corresponding to each prior frame according to the offset of the prior frame and a preset prediction frame;
Performing non-maximum suppression screening on the initial prediction frames according to the preset intersection ratio and the preset confidence coefficient to obtain final prediction frames of the preprocessed vehicle passing image data of all frames;
Carrying out vehicle target recognition on the image area in the final prediction frame to obtain a plurality of vehicle targets;
And tracking and counting the same vehicle target by using a multi-target tracking algorithm to obtain the number of the vehicle targets.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210545940.3A CN114973694B (en) | 2022-05-19 | 2022-05-19 | Tunnel traffic flow monitoring system and method based on inspection robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210545940.3A CN114973694B (en) | 2022-05-19 | 2022-05-19 | Tunnel traffic flow monitoring system and method based on inspection robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114973694A CN114973694A (en) | 2022-08-30 |
| CN114973694B true CN114973694B (en) | 2024-05-24 |
Family
ID=82985284
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210545940.3A Active CN114973694B (en) | 2022-05-19 | 2022-05-19 | Tunnel traffic flow monitoring system and method based on inspection robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114973694B (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108491758A (en) * | 2018-02-08 | 2018-09-04 | 深圳市睿灵创新科技开发有限公司 | A kind of track detection method and robot |
| CN109688388A (en) * | 2019-01-31 | 2019-04-26 | 宁波诠航机械科技有限公司 | A method of using the comprehensive real time monitoring of tunnel crusing robot |
| CN109935080A (en) * | 2019-04-10 | 2019-06-25 | 武汉大学 | The monitoring system and method that a kind of vehicle flowrate on traffic route calculates in real time |
| CN110085029A (en) * | 2019-05-31 | 2019-08-02 | 重庆交通大学 | Highway cruising inspection system and method based on rail mounted crusing robot |
| CN210850282U (en) * | 2019-08-02 | 2020-06-26 | 武庆珍 | Highway tunnel inspection robot |
| CN111815951A (en) * | 2020-07-17 | 2020-10-23 | 山东科技大学 | Road vehicle monitoring system and method based on intelligent vision Internet of things |
| CN112200483A (en) * | 2020-10-22 | 2021-01-08 | 卡斯柯信号有限公司 | Automatic inspection system and automatic inspection method for subway trackside equipment |
| CN112232237A (en) * | 2020-10-20 | 2021-01-15 | 城云科技（中国）有限公司 | Vehicle flow monitoring method, system, computer device and storage medium |
| CN112348201A (en) * | 2020-11-11 | 2021-02-09 | 扬州大学 | Intelligent decision implementation method for automatic driving group vehicle based on federal deep reinforcement learning |
| CN112665578A (en) * | 2021-03-16 | 2021-04-16 | 湖南承希科技有限公司 | Inspection positioning method and system based on tunnel inspection robot |
| CN112743559A (en) * | 2020-12-29 | 2021-05-04 | 上海市东方海事工程技术有限公司 | Suspension type tunnel inspection robot, system and method |
| CN113139495A (en) * | 2021-04-29 | 2021-07-20 | 姜冬阳 | Tunnel side-mounted video traffic flow detection method and system based on deep learning |
| CN113327248A (en) * | 2021-08-03 | 2021-08-31 | 四川九通智路科技有限公司 | Tunnel traffic flow statistical method based on video |
| CN113961222A (en) * | 2021-11-10 | 2022-01-21 | 交控科技股份有限公司 | Rail transit software remote maintenance method and device and storage medium |
| CN114023062A (en) * | 2021-10-27 | 2022-02-08 | 河海大学 | Traffic flow information monitoring method based on deep learning and edge calculation |
| CN114139570A (en) * | 2021-10-28 | 2022-03-04 | 中国科学院自动化研究所 | Distributed detection system based on active learning |
-
2022
- 2022-05-19 CN CN202210545940.3A patent/CN114973694B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108491758A (en) * | 2018-02-08 | 2018-09-04 | 深圳市睿灵创新科技开发有限公司 | A kind of track detection method and robot |
| CN109688388A (en) * | 2019-01-31 | 2019-04-26 | 宁波诠航机械科技有限公司 | A method of using the comprehensive real time monitoring of tunnel crusing robot |
| CN109935080A (en) * | 2019-04-10 | 2019-06-25 | 武汉大学 | The monitoring system and method that a kind of vehicle flowrate on traffic route calculates in real time |
| CN110085029A (en) * | 2019-05-31 | 2019-08-02 | 重庆交通大学 | Highway cruising inspection system and method based on rail mounted crusing robot |
| CN210850282U (en) * | 2019-08-02 | 2020-06-26 | 武庆珍 | Highway tunnel inspection robot |
| CN111815951A (en) * | 2020-07-17 | 2020-10-23 | 山东科技大学 | Road vehicle monitoring system and method based on intelligent vision Internet of things |
| CN112232237A (en) * | 2020-10-20 | 2021-01-15 | 城云科技（中国）有限公司 | Vehicle flow monitoring method, system, computer device and storage medium |
| CN112200483A (en) * | 2020-10-22 | 2021-01-08 | 卡斯柯信号有限公司 | Automatic inspection system and automatic inspection method for subway trackside equipment |
| CN112348201A (en) * | 2020-11-11 | 2021-02-09 | 扬州大学 | Intelligent decision implementation method for automatic driving group vehicle based on federal deep reinforcement learning |
| CN112743559A (en) * | 2020-12-29 | 2021-05-04 | 上海市东方海事工程技术有限公司 | Suspension type tunnel inspection robot, system and method |
| CN112665578A (en) * | 2021-03-16 | 2021-04-16 | 湖南承希科技有限公司 | Inspection positioning method and system based on tunnel inspection robot |
| CN113139495A (en) * | 2021-04-29 | 2021-07-20 | 姜冬阳 | Tunnel side-mounted video traffic flow detection method and system based on deep learning |
| CN113327248A (en) * | 2021-08-03 | 2021-08-31 | 四川九通智路科技有限公司 | Tunnel traffic flow statistical method based on video |
| CN114023062A (en) * | 2021-10-27 | 2022-02-08 | 河海大学 | Traffic flow information monitoring method based on deep learning and edge calculation |
| CN114139570A (en) * | 2021-10-28 | 2022-03-04 | 中国科学院自动化研究所 | Distributed detection system based on active learning |
| CN113961222A (en) * | 2021-11-10 | 2022-01-21 | 交控科技股份有限公司 | Rail transit software remote maintenance method and device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114973694A (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101942491B1 (en) | Hybrid ai cctv mediation module device consisting of road traffic situation monitoring and real time traffic information analysis | |
| WO2020199538A1 (en) | Bridge key component disease early-warning system and method based on image monitoring data | |
| CN108051450B (en) | Bridge health detection system and method based on unmanned aerial vehicle | |
| CN110059631A (en) | The contactless monitoring defect identification method of contact net | |
| CN102353355A (en) | Method for measuring power transmission line pole and tower inclination based on video differences | |
| CN110648319B (en) | Equipment image acquisition and diagnosis system and method based on double cameras | |
| CN101739692B (en) | Fast correlation tracking method for real-time video target | |
| CN113284144B (en) | Tunnel detection method and device based on unmanned aerial vehicle | |
| CN112530144B (en) | Method and system for warning violation behaviors of thermal power plant based on neural network | |
| CN114565845A (en) | Intelligent inspection system for underground tunnel | |
| CN106686341A (en) | Interactive vehicle-mounted intelligent terminal system and monitoring method | |
| CN114550334A (en) | Bridge robot inspection teaching training system, method and storage medium | |
| CN114973694B (en) | Tunnel traffic flow monitoring system and method based on inspection robot | |
| CN205232353U (en) | Interactive intelligent terminal system | |
| CN114037834A (en) | Semantic segmentation method and device based on fusion of vibration signal and RGB image | |
| CN114935451A (en) | Coal conveying belt carrier roller fault detection system and method based on inspection robot | |
| CN112926415A (en) | Pedestrian avoiding system and pedestrian monitoring method | |
| CN114979214B (en) | Intelligent collaborative alarm system, method and device for power transmission line | |
| CN116703864A (en) | Automatic pavement disease detection system and method based on polarization imaging and integrated learning | |
| CN110427504B (en) | System and method for acquiring and processing automobile data in real time based on button | |
| CN114549444A (en) | Cross-camera target detection and tracking system and method | |
| CN202650248U (en) | Intelligent transportation control device based on image processing | |
| CN112985852A (en) | Urban rail vehicle detection system | |
| CN110989666A (en) | Unmanned aerial vehicle system for line inspection and control method thereof | |
| CN212675424U (en) | Smart city facility operation and maintenance guard |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |