CN117237692A - Multi-feature fusion system and method for automatically identifying working state of special working vehicle - Google Patents
Multi-feature fusion system and method for automatically identifying working state of special working vehicle Download PDFInfo
- Publication number
- CN117237692A CN117237692A CN202310859355.5A CN202310859355A CN117237692A CN 117237692 A CN117237692 A CN 117237692A CN 202310859355 A CN202310859355 A CN 202310859355A CN 117237692 A CN117237692 A CN 117237692A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- passing
- special
- card
- working
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 239000003550 marker Substances 0.000 claims abstract description 16
- 238000010276 construction Methods 0.000 claims description 26
- 230000015654 memory Effects 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 241000512668 Eunectes Species 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000006386 memory function Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Landscapes
- Traffic Control Systems (AREA)
Abstract
The application discloses a multi-feature fusion system and a method for automatically identifying the working state of a special working vehicle, wherein a passing IC card is issued to each approaching vehicle at an entrance; shooting a vehicle video, identifying and obtaining vehicle entrance information, and writing the vehicle entrance information into a passing IC card; the vehicle carries the passing IC card to interact with the marker in the road side sensing device, and after the marker reads the signal of the vehicle, the characteristic information of the special vehicle is extracted from the real-time state video of the nearby vehicle shot by the camera; after the time fusion of the radar track data and the video track data, a unified coordinate system is established to convert the measured values of different sensors to the same coordinate system; inputting the extracted fusion information into a Yolov5 target detection model, detecting and tracking a target vehicle in real time, and identifying the working state of the special working vehicle by combining the vehicle motion path; uploading the working state of the vehicle to the host computer. The application overcomes the influence of external environments such as obstacles on the path recognition precision of the special working vehicle, and greatly increases the path recognition precision.
Description
Technical Field
The application relates to the technical field of special vehicle identification, in particular to an automatic identification system and an automatic identification method for the working state of a multi-feature fusion special working vehicle.
Background
In the field of vehicle identification, electronic monitoring systems have greatly progressed to the identification of special vehicles and exhibit excellent performance. In the working process of the special working vehicle, the appearance of the vehicle and dirt of license plates can be generated, so that the accuracy of license plate identification can be affected by dirt on the license plates of the vehicle in the process of identifying the license plates. Under certain conditions, the recognition of the special vehicle can have errors due to factors such as angles, light rays, resolution and the like shot by different cameras, motion blur generated by unfixed vehicle speed and the like, and meanwhile, the problem of low recognition accuracy exists in a mode of recognizing the vehicle only by recognizing license plate characters.
Therefore, if only a single recognition technology is used, a perfect effect is not generated, and in the path recognition of a special work vehicle, the problem of low recognition accuracy is generated due to insufficient environmental transparency or brightness, equipment failure and the like.
Disclosure of Invention
The application aims to provide a system and a method for automatically identifying the working state of a special work vehicle with multi-feature fusion.
The technical scheme adopted by the application is as follows:
the automatic recognition system for the working state of the special work vehicle with the fused characteristics is characterized in that: the intelligent traffic control system comprises a control host, a traffic IC card, a Mifare reader-writer, an entrance camera and a road side sensing device; the passing IC card is respectively and correspondingly distributed when each vehicle enters the construction site; the road side sensing devices are arranged at intervals on two sides of a road on which the vehicle runs, and millimeter wave radar, a camera and a marker are arranged in the road side sensing devices; the Mifare reader-writer and the entrance camera are arranged at the entrance and the exit of the construction site, and the entrance camera shoots the vehicle video at the entrance and the exit and is identified by the control host to obtain the entrance information of the vehicle; the Mifare reader is used for communicating with the passing IC card and writing in entrance information when the vehicle enters, and reading the entrance information and the path information on the passing IC card when the vehicle leaves; when the vehicle runs to the road side sensing device accessory, the millimeter wave radar starts to work, radar track data of a target vehicle are obtained, and a camera shoots a video sequence to extract video track data of a special vehicle; the identifier is provided with corresponding path information and is communicated with the passing IC card so as to write the path information into the passing IC card; fusing data from the radar sensor and the camera; and finally, inputting the acquired fusion information of the millimeter wave and the image into a Yolov5 algorithm by a control host to detect and track the target vehicle, and identifying the working state of the special working vehicle by combining the vehicle motion path, wherein the working state is divided into two states, namely a working state and a stopping state, so as to judge whether the vehicle is working.
Further, the control host is also connected with a key unit, a storage unit, a display unit and an alarm unit; the control host adopts a singlechip as a control core, and the singlechip is internally provided with single-cycle multiplication and hardware division functions.
The alarm unit comprises a buzzer and a led lamp, adopts a mode of combining the buzzer and the led lamp, realizes sound and light alarm at the same time, and improves alarm efficiency.
Further, the singlechip is an STM32 singlechip. The Mifare reader is a JT-M2320 (M100) read-write unit.
Further, the entry information includes entry time, license plate, and vehicle type.
Further, the passing IC card is a double-frequency passing IC card, the double-frequency passing IC card comprises a passing card CPU, a memory, an RF433M interface and a RF16.56M interface, wherein the memory, the RF433M interface and the RF16.56M interface are connected with the passing card CPU, and the passing card CPU is used for format conversion and security verification when the RF433M interface and the RF16.56M interface write data; the RF433M interface is used for receiving the path information written by the identifiers on the two sides of the vehicle path; RF16.56M interface is used to communicate with Mifare readers at the site entry.
Further, 433MHz and 16.56MHz dual-frequency interfaces of the dual-frequency passing IC card share a memory.
Further, a binocular camera is used, and the sampling interval of the camera is about 40ms, and 25 frames per second. The binocular camera can simultaneously generate image data under two different visual angles, and combine the image data into an image with depth and color information, so that the imaging effect is more vivid and clear.
Further, the frequency band of the millimeter wave radar is 24GHz, the millimeter wave radar is used for short-medium distance detection, and the detection distance range is 15-30 m.
The automatic recognition method for the working state of the special work vehicle with the multi-feature fusion comprises the following steps:
step 1, issuing a passing IC card to each approaching vehicle at an entrance;
step 2, the entrance camera shoots a vehicle video, and the control host computer obtains entrance information of the vehicle based on the vehicle video identification and then writes the entrance information into a corresponding passing IC card; the entrance information comprises entrance time, license plates and vehicle types;
step 3, the vehicle carries a corresponding passing IC card to interact with a marker in the road side sensing device during running, after the marker reads a signal of the vehicle, a real-time state video of a nearby vehicle is shot through a camera, and characteristic information of the special vehicle, including a target ID, a vehicle running speed V, a track image u coordinate and a track image V coordinate, is extracted from a video sequence; extracting radar track data of the vehicle through a millimeter wave radar, wherein the radar track data comprises a time stamp t, a target ID, a track x coordinate, a track y coordinate and a vehicle running speed V;
and 4, realizing time fusion of radar track data and video track data by adopting a sampling period of a radar sensor and a video sensor which have long sampling periods as a sampling period of a system, and ensuring the synchronization of millimeter wave radar and camera data in time.
Step 5, for spatial fusion of the radar sensor and the video sensor, a unified coordinate system is established, and measured values of different sensors are converted to the same coordinate system;
step 6, inputting the extracted fusion information into a Yolov5 target detection model, detecting and tracking a target vehicle in real time, and identifying the working state of the special working vehicle by combining a vehicle motion path, wherein the working state is divided into two states, one is in working and the other is in a stopping state, so as to judge whether the vehicle is working;
and 7, uploading the working state of the vehicle to an upper host.
Further, the entry information includes entry time, license plate, and vehicle type.
Further, in the step 6, the completion calculation is carried out on the motion path of the vehicle based on a historical data path probability recognition algorithm and a greedy algorithm;
further, the specific steps of the historical data path probability recognition algorithm are as follows:
step 6-1, firstly, reading historical track data, traversing the data of the historical roadside identification points, and searching the track data of the identification points of the special working vehicle within the time range of entering and exiting the construction site;
step 6-2, splicing the identification point numbers according to the passing time sequence of the vehicle, and finally storing spliced track data into a database;
step 6-3, when traversing each piece of track data, firstly judging that a plurality of road sections are included in the path, then adding 1 to the number of passes of the corresponding road sections, and finally calculating the ratio of the number of passes of each road section to the number of passes of the historical total road section to be used as the passing probability of each road section;
and 6-4, obtaining a complete driving path of the special working vehicle by using a greedy algorithm based on the traffic probability of each road section.
Further, the greedy algorithm in step 6-4 comprises the following specific steps:
step 6-4-1, forming a road network by road side sensing devices arranged on two sides of a road of a construction site, and searching a next passable passing point in the road network from the entrance of the construction site by sequentially searching paths from the entrance to the exit of the construction site in the road network;
step 6-4-2, judging whether a plurality of passpoints can be passed or not; if yes, selecting the road section with the highest passing probability value in the parallel road sections to be added as a part of the path; otherwise, only one passing point is determined and the corresponding road section is directly selected as a part of the path.
Step 6-4-3, searching whether a next passable passing point exists in the road network by taking the end point of the latest path as a starting point; if yes, executing the step 6-4-2; otherwise, the current path is output as a complete driving path of the special working vehicle.
Further, in step 6, whether the vehicle is working or not is judged based on the YOLOv5 target detection model; the method comprises the following specific steps:
s1, constructing a YOLOv5 training model and a radar data fusion algorithm;
s2, utilizing the public data set, inputting the data set into a Yolov5 training model after using a fusion algorithm to obtain an algorithm model for detecting the working state of the vehicle;
and S3, acquiring data of a construction site video sensor and a radar sensor, fusing by an algorithm, and inputting fused information into a YOLOv5 target detection model to identify and obtain a vehicle working state.
According to the technical scheme, when a driver enters a building site gate, a security guard issues a double-frequency card, vehicle information is identified through a gate camera and recorded into the double-frequency card, after a special working vehicle carrying the double-frequency card enters the building site, the vehicle is sensed through a marker on a road side sensing device, so that a radar sensor and the camera acquire radar data of a target vehicle and video track data of the special vehicle are extracted from a shooting video sequence, and time track information of the vehicle is recorded and copied and written into the card. After the operation is finished, the double-frequency card is returned, and the information in the card, the identifier information and the like are read by the reader-writer, so that the complete and accurate path of the vehicle running at the time can be determined, and the effective monitoring and restoration of the vehicle movement path are realized. The radar data and the video data are fused, a Yolov5 algorithm is input to detect and track the target vehicle, and the working state of the special working vehicle is identified by combining the vehicle motion path, and the radar data and the video data are divided into two states, wherein one is in working and the other is in a stopping state, so that whether the vehicle is working or not is judged in real time.
Drawings
The application is described in further detail below with reference to the drawings and detailed description;
FIG. 1 is a schematic diagram of a method for automatically identifying the working state of a special work vehicle by multi-feature fusion according to the application;
description of the embodiments
In order to make the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.
As shown in fig. 1, the application discloses an automatic recognition system for the working state of a special working vehicle with multi-feature fusion, which is characterized in that: the intelligent traffic control system comprises a control host, a traffic IC card, a Mifare reader-writer, an entrance camera and a road side sensing device; the passing IC card is respectively and correspondingly distributed when each vehicle enters the construction site; the road side sensing devices are arranged at intervals on two sides of a road on which the vehicle runs, and millimeter wave radar, a camera and a marker are arranged in the road side sensing devices; the Mifare reader-writer and the entrance camera are arranged at the entrance and the exit of the construction site, and the entrance camera shoots the vehicle video at the entrance and the exit and is identified by the control host to obtain the entrance information of the vehicle; the Mifare reader is used for communicating with the passing IC card and writing in entrance information when the vehicle enters, and reading the entrance information and the path information on the passing IC card when the vehicle leaves; when the vehicle runs to the road side sensing device accessory, the millimeter wave radar starts to work, radar track data of a target vehicle are obtained, and a camera shoots a video sequence to extract video track data of a special vehicle; the identifier is provided with corresponding path information and is communicated with the passing IC card so as to write the path information into the passing IC card; fusing data from the radar sensor and the camera; and finally, inputting the acquired fusion information of the millimeter wave and the image into a Yolov5 algorithm by the control host to detect and track the target vehicle, and identifying the working state of the special working vehicle by combining the vehicle path, wherein the working state is divided into two states, namely a working state and a stopping state, so as to judge whether the vehicle is working in real time.
Further, the control host is also connected with a key unit, a storage unit, a display unit and an alarm unit; the control host adopts a singlechip as a control core, and the singlechip is internally provided with single-cycle multiplication and hardware division functions.
The alarm unit comprises a buzzer and a led lamp, adopts a mode of combining the buzzer and the led lamp, realizes sound and light alarm at the same time, and improves alarm efficiency.
Further, the singlechip is an STM32 singlechip. The Mifare reader is a JT-M2320 (M100) read-write unit.
Further, the entry information includes entry time, license plate, and vehicle type.
Further, the passing IC card is a double-frequency passing IC card, the double-frequency passing IC card comprises a passing card CPU, a memory, an RF433M interface and a RF16.56M interface, wherein the memory, the RF433M interface and the RF16.56M interface are connected with the passing card CPU, and the passing card CPU is used for format conversion and security verification when the RF433M interface and the RF16.56M interface write data; the RF433M interface is used for receiving the path information written by the identifiers on the two sides of the vehicle path; RF16.56M interface is used to communicate with Mifare readers at the site entry.
Further, 433MHz and 16.56MHz dual-frequency interfaces of the dual-frequency passing IC card share a memory.
Further, a binocular camera is used, and the sampling interval of the camera is about 40ms, and 25 frames per second. The binocular camera can simultaneously generate image data under two different visual angles, and combine the image data into an image with depth and color information, so that the imaging effect is more vivid and clear.
Further, the frequency band of the millimeter wave radar is 24GHz, the millimeter wave radar is used for short-medium distance detection, and the detection distance range is 15-30 m.
The automatic recognition method for the working state of the special work vehicle with the multi-feature fusion comprises the following steps:
step 1, issuing a passing IC card to each approaching vehicle at an entrance;
step 2, the entrance camera shoots a vehicle video, and the control host computer obtains entrance information of the vehicle based on the vehicle video identification and then writes the entrance information into a corresponding passing IC card; the entrance information comprises entrance time, license plates and vehicle types;
step 3, the vehicle carries a corresponding passing IC card to interact with a marker in the drive test sensing device in the driving way, after the marker reads a signal of the vehicle, a real-time state video of a nearby vehicle is shot through a camera, and video track data of the special vehicle, including a target ID, a vehicle driving speed V, a track image u coordinate and a track image V coordinate, are extracted from a video sequence; extracting radar track data of the vehicle through a millimeter wave radar, wherein the radar track data comprises a time stamp t, a target ID, a track x coordinate, a track y coordinate and a vehicle running speed V;
and 4, realizing time fusion of radar track data and video track data by adopting a sampling period of a radar sensor and a video sensor which have long sampling periods as a sampling period of a system, and ensuring the synchronization of millimeter wave radar and camera data in time.
Step 5, for spatial fusion of the radar sensor and the video sensor, a unified coordinate system is established, and measured values of different sensors are converted to the same coordinate system;
and 6, inputting the extracted fusion information into a Yolov5 target detection model, and detecting and tracking the target vehicle in real time. The working state of the special working vehicle is identified by combining the vehicle motion path, and the special working vehicle is divided into two states, wherein one is in working and the other is in a stopping state, so that whether the vehicle is working or not is judged, and the working state mark is updated in real time;
and 7, uploading the working state of the vehicle to an upper host.
Further, the entry information includes entry time, license plate, and vehicle type.
Further, in the step 6, the completion calculation is carried out on the motion path of the vehicle based on a historical data path probability recognition algorithm and a greedy algorithm;
further, the specific steps of the historical data path probability recognition algorithm are as follows:
step 6-1, firstly, reading historical track data, traversing the data of the historical roadside identification points, and searching the track data of the identification points of the special working vehicle within the time range of entering and exiting the construction site;
step 6-2, splicing the identification point numbers according to the passing time sequence of the vehicle, and finally storing spliced track data into a database;
step 6-3, when traversing each piece of track data, firstly judging that a plurality of road sections are included in the path, then adding 1 to the number of passes of the corresponding road sections, and finally calculating the ratio of the number of passes of each road section to the number of passes of the historical total road section to be used as the passing probability of each road section;
and 6-4, obtaining a complete driving path of the special working vehicle by using a greedy algorithm based on the traffic probability of each road section.
Further, the greedy algorithm in step 6-4 comprises the following specific steps:
step 6-4-1, forming a road network by road test sensing devices arranged on two sides of a road of a construction site, and searching a next passable passing point in the road network from the entrance of the construction site by sequentially searching paths from the entrance to the exit of the construction site in the road network;
step 6-4-2, judging whether a plurality of passpoints can be passed or not; if yes, selecting the road section with the highest passing probability value in the parallel road sections to be added as a part of the path; otherwise, only one passing point is determined and the corresponding road section is directly selected as a part of the path.
Step 6-4-3, searching whether a next passable passing point exists in the road network by taking the end point of the latest path as a starting point; if yes, executing the step 6-4-2; otherwise, the current path is output as a complete driving path of the special working vehicle.
Further, in step 6, whether the vehicle is working or not is judged based on the YOLOv5 target detection model; the method comprises the following specific steps:
s1, constructing a YOLOv5 training model and a radar data fusion algorithm;
s2, utilizing the public data set, inputting the data set into a Yolov5 training model after using a fusion algorithm to obtain an algorithm model for detecting the working state of the vehicle;
and S3, acquiring data of a construction site video sensor and a radar sensor, fusing by an algorithm, and inputting fused information into a YOLOv5 target detection model to identify and obtain a vehicle working state.
The following is a detailed description of the specific principles of the present application:
the application discloses a multi-feature fusion automatic identification system for the working state of a special working vehicle, which comprises a main control host, a passing IC card, a Mifare reader-writer, a road side sensing device, a key unit, a storage unit, a display unit and an alarm unit. The functions and types of each unit are as follows:
and (3) a main control host: STM32 is adopted as a control core, the chip is 32 bits, 144I/O ports, the stability is high, the power consumption is low, the cost is low, and the single-period multiplication and hardware division functions are built in.
Mifare reader: the JT-M2320 (M100) read-write module is a small-sized read-write module with ultra-low power consumption, the read distance is 0-20M, the read rate can reach more than 300 times per second, the ISO18000-6C (EPC C1 GEN 2) protocol is supported, the sensitivity is-10 dBm, the test false alarm rate is-65 dBm under the local blocking condition, and the test false alarm rate is 1%, so that the read-write module can be easily embedded into a handheld terminal such as a tablet personal computer and a PDA, can realize the RFID expansion function of the device, and is widely applied to the fields of product quality inspection, logistics management and the like.
A key unit: and an independent key is adopted for selecting, adding and deleting labels and also for releasing the alarm.
And a storage unit: the final data is stored in the W25Q128 memory. The 25Q series power-down data is not lost, has a memory function, directly executes codes from double/four channel SPI (XIP) and stores voice text and data, and has strong performance.
And a display unit: the display module of the application adopts an OLED display screen with wide viewing angle, uniform image quality, high response speed, stable image and high resolution, and is used for displaying a label menu and alarm information.
And an alarm unit: for the special vehicle early warning system part in the intelligent building site, the mode that the buzzer is combined with the led lamp is adopted, and meanwhile, sound and light alarm is realized, so that the alarm efficiency is improved.
Road side perception device: the device comprises a camera, a millimeter wave radar and a marker. With a binocular camera, the sample interval of the camera is about 40ms, 25 frames per second. The binocular camera can simultaneously generate image data under two different visual angles, and combine the image data into an image with depth and color information, so that the imaging effect is more vivid and clear. The frequency band of the millimeter wave radar is 24GHz, the millimeter wave radar is used for short-medium distance detection, and the detection distance range is 15-30 m.
Specifically, the dual-frequency access card is divided into four parts, namely a CPU, an RF433M interface, a RF13.56M interface and a storage unit, and is actually an integrated access card with 433MHz and 13.56MHz dual-frequency interface shared memories, so that the function of one card dual-interface is realized. The CPU in the card is mainly responsible for the conversion of the format and the security verification when the F433M interface and the RF13.56M interface write data, the RF433MHz interface is mainly responsible for receiving the path information written by the identifier, and the RF13.56MHz interface is responsible for communicating with the Mifare card reader at the site exit.
The method comprises the steps that a double-frequency pass card is issued at a site entrance, a Mifare card reader wakes up an RF13.56MHz part, a camera is used for identifying information such as license plates, vehicle types and time of a special work vehicle, the information is written into a storage unit of the double-frequency pass card, when the special work vehicle enters the site, a marker in a road side sensing device receives a double-frequency card transmitting RFID identification code in the special work vehicle which is passed through by an antenna system, and a double-frequency card RF433M part accurately writes track identification information into the same storage unit in real time. When the security personnel go out of the construction site, the security personnel recover the double-frequency pass card, and the Mifare reader-writer extracts the entrance information and the path information of the RF433M interface and the RF13.56M interface in the double-frequency pass IC card.
In order to reduce power consumption, the dual-frequency passing IC card can optimize RF13.56M and RF433M antennas on hardware to improve sensitivity and reliability of the antennas, reduce power consumption of the dual-frequency passing card and transmitting power of the identification station, and set a sleep mode according to layout and working principle of the road side sensing device. However, the "sleep state" does not represent a shutdown, and when the dual-frequency IC card detects the radio frequency signal transmitted from the tag, it immediately resumes the normal operation mode, and in addition, the RF433MHz interface is set to be inactive, so as to reduce power consumption generated by the RF433MHz interface.
The method of RFID technology and site camera discloses an automatic recognition system for the working state of a special work vehicle with multi-feature fusion, and the problem of the path recognition of the special work vehicle is solved through hardware construction. The working state of the special working vehicle is judged in a manner of combining a thunder fusion algorithm with vehicle path recognition, so that a new research angle and a higher-precision solution are obtained in the research of automatic recognition of the working state of the special working vehicle. And (3) monitoring and researching the motion trail of the special vehicle based on the ultra-low power consumption double-frequency card. In order to reduce the false detection rate of video detection and tracking, the motion trail monitoring of the special vehicle is realized by the ultra-low power consumption double-frequency card. When a driver enters a site gate, a security guard issues a double-frequency card, vehicle information is identified through a gate camera, the vehicle information is recorded into the double-frequency card, after a special working vehicle carrying the double-frequency card enters the site, the time track information of the vehicle is recorded and copied and written into the card by using a marker in a road side sensing device, the double-frequency card is returned after the operation is finished, and the information, the marker information and the like in the card are read out by a reader-writer, so that the complete accurate path of the vehicle running at the time can be determined, and the effective monitoring and restoration of the vehicle movement path are realized. Meanwhile, the radar and the camera of the road side sensing device acquire vehicle radar data and capture video track data of a special vehicle extracted from a video sequence, and whether the vehicle is working or not can be judged by fusing the data of the two sensors and inputting the data into a trained yolov5 model. Due to insufficient ambient brightness or transparency, equipment faults and other reasons, the identification point data uploaded by the identifier are missing, so that the driving path of the vehicle is difficult to accurately identify in some time periods, and an auxiliary path identification method (namely a historical data path probability identification algorithm and a greedy algorithm) is required to be designed aiming at the situation that the identification point data is missing. The method for identifying the path probability based on the historical data comprises the steps of calculating the passing probability of each road section in a road network by using the original in-out track information and the identification point data of the vehicle, wherein the historical data is massive original data which is not processed, and the problems of missing, redundancy, errors and the like of data attributes are caused, so that the subsequent calculation processing is difficult. After accurate and complete path information is obtained, the working state of the special working vehicle is identified, and the special working vehicle is divided into two states, wherein one is in work and the other is in a stop state.
Specifically, the road segment probability data calculation process based on the road segment clusters can construct a historical driving track of the special work vehicle according to the cleaned special work vehicle historical traffic data. Firstly, reading historical track data, then traversing the data of the roadside identification points, searching the track data of the identification points of the special working vehicle in the time range of entering and exiting the construction site, splicing the identification point numbers according to the passing time sequence of the vehicle, and finally storing the spliced track data into a database. The number of passes for each road segment may be calculated in the historical trajectory data. Dividing a track path into different road sections, calculating the passing times of each road section, adopting a clustering method based on the road sections to calculate the passing times of the road sections, considering the history track of each special working vehicle as a plurality of road section combinations connected with each other, traversing history track data, judging which road sections are included in the path when traversing each track data, adding 1 to the passing times of the corresponding road sections, and finally calculating the ratio of the passing times of each road section to the passing times of the history total road sections to be used as the passing probability of each road section. The path recognition process of the special working vehicle is basically the optimal selection process of each road section, and the vehicle path recognition problem has optimal substructure characteristics and is suitable for being solved by using a greedy algorithm. Aiming at the condition that the data of the identification point is missing, a path identification method combining a greedy algorithm and probability data of each road section is adopted, the greedy algorithm is utilized to search a path from an entrance to an exit of a construction site, the next point which can pass through in a road network is searched from the entrance of the construction site, and if one point exists, the road section is directly selected as a part of the path. If there are a plurality of the sections, selecting the section with the highest probability value in the parallel sections as a part of the path, and then continuing to search for the next point, and simultaneously performing the same identification process until reaching the site exit. And iterating for a plurality of times, selecting a road section with the highest probability to construct a path, and finally obtaining a complete running path of the special working vehicle.
Vehicle state recognition based on deep learning is performed by first installing a pytorch deep learning environment through anaconda, so that a YOLOv5 target detection model is constructed. And then constructing a data fusion algorithm from the obtained radar data and video data, and inputting the fused data into a YOLOv5 target detection model to obtain a corresponding YOLOv5 training model.
The application uses RFID technology and adopts dynamic frame time slot ALOHA algorithm to solve the vehicle identification problem, breaks through the traditional identification from the angles of Bluetooth, GPS, WIFI, infrared, ultrasonic and the like, so that a new research angle and a higher-precision solution method are obtained by research; meanwhile, in order to reduce the problem of low recognition accuracy caused by insufficient environmental transparency or insufficient brightness, equipment faults and the like, a mode of combining a historical data path probability recognition algorithm and a greedy algorithm based on big data is introduced. The combined use of the two methods breaks through the traditional mode of identifying the vehicle path from the single-sided angle of video capture of the camera in the field of computer vision, overcomes the influence of external environments such as obstacles on the path identification precision of the special working vehicle, greatly increases the path identification precision, greatly improves the data accuracy and the real-time performance, and greatly reduces the loss rate of historical data.
Compared with the prior art, the application has the following technical advantages: the millimeter wave radar and the target vehicle detection tracking scheme are adopted in the vehicle identification process, so that the video information and radar advantages of the monitoring camera can be fully utilized, and the target vehicle detection tracking problem in the monitoring scene can be solved. Is widely applied, and can seriously influence the performance of the vision sensor in severe weather (rain, fog, strong light and the like). The millimeter wave radar sensor can not be greatly influenced in performance under severe light and weather conditions.
It will be apparent that the described embodiments are some, but not all, embodiments of the application. Embodiments of the application and features of the embodiments may be combined with each other without conflict. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the application is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Claims (10)
1. The utility model provides a special type operation vehicle operating condition automatic identification system of many characteristics fusion which characterized in that: the intelligent traffic control system comprises a control host, a traffic IC card, a Mifare reader-writer, an entrance camera and a road side sensing device; the passing IC card is respectively and correspondingly distributed when each vehicle enters the construction site; the road side sensing devices are arranged at intervals on two sides of a road on which the vehicle runs, and millimeter wave radar, a camera and a marker are arranged in the road side sensing devices; the Mifare reader-writer and the entrance camera are arranged at the entrance and the exit of the construction site, and the entrance camera shoots the vehicle video at the entrance and the exit and is identified by the control host to obtain the entrance information of the vehicle; the Mifare reader is used for communicating with the passing IC card and writing in entrance information when the vehicle enters, and reading the entrance information and the path information on the passing IC card when the vehicle leaves; when the vehicle runs to the road side sensing device accessory, the millimeter wave radar starts to work, radar track data of a target vehicle are obtained, and a camera shoots a video sequence to extract video track data of a special vehicle; the identifier is provided with corresponding path information and is communicated with the passing IC card so as to write the path information into the passing IC card; fusing data from the radar sensor and the camera; and finally, inputting the acquired fusion information of the millimeter wave and the video into a Yolov5 algorithm by the control host to detect and track the target vehicle, and identifying the working state of the special working vehicle by combining the vehicle path, wherein the working state is divided into two states, namely a working state and a stopping state, so as to judge whether the vehicle is working in real time.
2. The multi-feature fusion automatic identification system for the operating state of a special work vehicle according to claim 1, wherein: the control host is also connected with a key unit, a storage unit, a display unit and an alarm unit; the control host adopts a singlechip as a control core, and the singlechip is internally provided with a single-cycle multiplication function and a hardware division function; the alarm unit comprises a buzzer and an led lamp, and realizes sound and light alarm.
3. The multi-feature fusion automatic identification system for the operating state of a special work vehicle according to claim 1, wherein: the entry information includes entry time, license plate, and vehicle type.
4. The multi-feature fusion automatic identification system for the operating state of a special work vehicle according to claim 1, wherein:
the millimeter wave radar measures radar track data of the special vehicle; the camera obtains the characteristic information of the special vehicle.
5. The multi-feature fusion automatic identification system for the operating state of a special work vehicle according to claim 1, wherein: the passing IC card is a double-frequency passing IC card, the double-frequency passing IC card comprises a passing card CPU, a memory, an RF433M interface and a RF16.56M interface, wherein the memory, the RF433M interface and the RF16.56M interface are connected with the passing card CPU, and the passing card CPU is used for format conversion and security verification when the RF433M interface and the RF16.56M interface write data; the RF433M interface is used for receiving the path information written by the identifiers on the two sides of the vehicle path; the RF16.56M interface is used for communicating with a Mifare reader at the entrance of the worksite; the 433MHz and 16.56MHz dual-frequency interfaces of the dual-frequency passing IC card share a memory.
6. The automatic recognition method for the working state of the multi-feature fusion special working vehicle is characterized in that the automatic recognition system for the working state of the multi-feature fusion special working vehicle is adopted, and is characterized in that: the method comprises the following steps:
step 1, issuing a passing IC card to each approaching vehicle at an entrance;
step 2, the entrance camera shoots a vehicle video, and the control host computer obtains entrance information of the vehicle based on the vehicle video identification and then writes the entrance information into a corresponding passing IC card; the entrance information comprises entrance time, license plates and vehicle types;
step 3, the vehicle carries a corresponding passing IC card to interact with a marker in the road side sensing device in the middle of running, after the marker reads a signal of the vehicle, a real-time state video of a nearby vehicle is shot through a camera, video track data of the special vehicle is extracted from a video sequence, and the video track data of the special vehicle comprises a target ID, a vehicle running speed V, a track image u coordinate and a track image V coordinate; extracting radar track data of the vehicle through a millimeter wave radar, wherein the radar track data comprises a time stamp t, a target ID, a track x coordinate, a track y coordinate and a vehicle running speed V;
step 4, the sampling period of the radar sensor and the video sensor is long and is used as the sampling period of a system to realize the time fusion of radar track data and video track data, so that the synchronization of millimeter wave radar and camera data in time is ensured;
step 5, for spatial fusion of the radar sensor and the video sensor, a unified coordinate system is established, and measured values of different sensors are converted to the same coordinate system;
step 6, inputting the extracted fusion information into a Yolov5 target detection model, and detecting and tracking a target vehicle in real time; the working state of the special working vehicle is identified by combining the vehicle motion path, and the special working vehicle is divided into two states, wherein one state is in work and the other state is in a stop state, so that whether the vehicle is working or not is judged, and the working state mark is updated;
and 7, uploading the working state of the vehicle to an upper host.
7. The multi-feature fusion automatic recognition method for the working state of the special working vehicle according to claim 6, wherein the method comprises the following steps: and 6, performing completion calculation on the motion path of the vehicle based on the historical data path probability recognition algorithm and the greedy algorithm.
8. The multi-feature fusion automatic recognition method for the working state of the special working vehicle according to claim 7, wherein the method comprises the following steps: the specific steps of the historical data path probability recognition algorithm are as follows:
step 6-1, firstly, reading historical track data, traversing the data of the historical roadside identification points, and searching the track data of the identification points of the special working vehicle within the time range of entering and exiting the construction site;
step 6-2, splicing the identification point numbers according to the passing time sequence of the vehicle, and finally storing spliced track data into a database;
step 6-3, when traversing each piece of track data, firstly judging that a plurality of road sections are included in the path, then adding 1 to the number of passes of the corresponding road sections, and finally calculating the ratio of the number of passes of each road section to the number of passes of the historical total road section to be used as the passing probability of each road section;
and 6-4, obtaining a complete driving path of the special working vehicle by using a greedy algorithm based on the traffic probability of each road section.
9. The multi-feature fusion automatic identification method for the working state of the special working vehicle according to claim 8, wherein the method comprises the following steps: the greedy algorithm in step 6-4 comprises the following specific steps:
step 6-4-1, forming a road network by road side sensing devices arranged on two sides of a road of a construction site, and searching a next passable passing point in the road network from the entrance of the construction site by sequentially searching paths from the entrance to the exit of the construction site in the road network;
step 6-4-2, judging whether a plurality of passpoints can be passed or not; if yes, selecting the road section with the highest passing probability value in the parallel road sections to be added as a part of the path; otherwise, judging that only one passing point exists and directly selecting a corresponding road section as a part of the path;
step 6-4-3, searching whether a next passable passing point exists in the road network by taking the end point of the latest path as a starting point; if yes, executing the step 6-4-2; otherwise, the current path is output as a complete driving path of the special working vehicle.
10. The multi-feature fusion automatic recognition method for the working state of the special working vehicle according to claim 1, wherein the method comprises the following steps: step 6, judging whether the vehicle is working or not based on the YOLOv5 target detection model; the method comprises the following specific steps:
s1, constructing a YOLOv5 training model and a radar data fusion algorithm;
s2, inputting the public data set into a Yolov5 training model to obtain an algorithm model for detecting the working state of the vehicle after using a fusion algorithm;
and S3, acquiring data of a construction site video sensor and a radar sensor, fusing the data by an algorithm, and inputting fused information into a YOLOv5 target detection model to identify and obtain a vehicle working state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310859355.5A CN117237692A (en) | 2023-07-13 | 2023-07-13 | Multi-feature fusion system and method for automatically identifying working state of special working vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310859355.5A CN117237692A (en) | 2023-07-13 | 2023-07-13 | Multi-feature fusion system and method for automatically identifying working state of special working vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117237692A true CN117237692A (en) | 2023-12-15 |
Family
ID=89088705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310859355.5A Pending CN117237692A (en) | 2023-07-13 | 2023-07-13 | Multi-feature fusion system and method for automatically identifying working state of special working vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117237692A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117434531A (en) * | 2023-12-21 | 2024-01-23 | 中交第一公路勘察设计研究院有限公司 | Method and equipment for fusing detection target characteristics of millimeter wave radar and camera |
-
2023
- 2023-07-13 CN CN202310859355.5A patent/CN117237692A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117434531A (en) * | 2023-12-21 | 2024-01-23 | 中交第一公路勘察设计研究院有限公司 | Method and equipment for fusing detection target characteristics of millimeter wave radar and camera |
| CN117434531B (en) * | 2023-12-21 | 2024-03-12 | 中交第一公路勘察设计研究院有限公司 | Method and equipment for fusing detection target characteristics of millimeter wave radar and camera |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9013282B2 (en) | Object tracking apparatus and method, and sensor position designating method | |
| CN101758836B (en) | Proofreading method and system for mileage of track inspection car | |
| CN117237692A (en) | Multi-feature fusion system and method for automatically identifying working state of special working vehicle | |
| CN103310513A (en) | Door-access control method and device | |
| CN103400500A (en) | Vehicle information data acquisition device and method | |
| CN103440687A (en) | RFID (Radio Frequency Identification) technology-based attendance checking method and system | |
| CN105528912A (en) | Indoor parking stall recommendation method and system | |
| CN103942518A (en) | Three-dimensional surrounding tracing type radio frequency identification system and goods detection method thereof | |
| CN111311957A (en) | Parking guidance system based on RFID and unmanned aerial vehicle and working method | |
| CN106815613A (en) | A kind of information of vehicles detection method based on RFID | |
| Qu et al. | Improving maritime traffic surveillance in inland waterways using the robust fusion of AIS and visual data | |
| CN111796287A (en) | Automatic drive car road data acquisition and analysis device | |
| CN116659524A (en) | Vehicle positioning method, device, equipment and storage medium | |
| CN114298163A (en) | Online road condition detection system and method based on multi-source information fusion | |
| CN109977853B (en) | Underground worker panoramic monitoring method based on multiple identification devices | |
| Yuan et al. | A novel approach to image-sequence-based mobile robot place recognition | |
| CN113450575A (en) | Management method and device for roadside parking | |
| CN103679859A (en) | Fixed asset inspection system and corresponding fixed asset | |
| CN112268554A (en) | Regional range loitering detection method and system based on path trajectory analysis | |
| CN115311867B (en) | Tunnel scene positioning method and device, computer equipment and storage medium | |
| CN210000341U (en) | Rail vehicle positioning device and system based on offset calculation | |
| CN112614156A (en) | Training method and device for multi-target tracking network model and related equipment | |
| CN112347953A (en) | Recognition device for road condition irregular obstacles of unmanned vehicle | |
| KR100810898B1 (en) | Apparatus of sensing vehicle speed using RFID tag and doppler radar | |
| KR20210007812A (en) | Server, system, method, computer program, computer-readable recording medium for managing parking of vehicle in parking lot by using radar |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |