CN112289036A - Scene type violation attribute identification system and method based on traffic semantics - Google Patents
Scene type violation attribute identification system and method based on traffic semantics Download PDFInfo
- Publication number
- CN112289036A CN112289036A CN202011056632.1A CN202011056632A CN112289036A CN 112289036 A CN112289036 A CN 112289036A CN 202011056632 A CN202011056632 A CN 202011056632A CN 112289036 A CN112289036 A CN 112289036A
- Authority
- CN
- China
- Prior art keywords
- violation
- traffic
- management terminal
- edge computing
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 54
- 238000012806 monitoring device Methods 0.000 claims abstract description 41
- 230000005540 biological transmission Effects 0.000 claims abstract description 17
- 230000006399 behavior Effects 0.000 claims abstract description 13
- 238000001514 detection method Methods 0.000 claims description 14
- 230000009471 action Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 4
- 230000003993 interaction Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 8
- 238000007689 inspection Methods 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 230000009545 invasion Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000012015 optical character recognition Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000001568 sexual effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
- G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y20/00—Information sensed or collected by the things
- G16Y20/10—Information sensed or collected by the things relating to the environment, e.g. temperature; relating to location
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y20/00—Information sensed or collected by the things
- G16Y20/20—Information sensed or collected by the things relating to the thing itself
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y20/00—Information sensed or collected by the things
- G16Y20/40—Information sensed or collected by the things relating to personal data, e.g. biometric data, records or preferences
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y40/00—IoT characterised by the purpose of the information processing
- G16Y40/10—Detection; Monitoring
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y40/00—IoT characterised by the purpose of the information processing
- G16Y40/20—Analytics; Diagnosis
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y40/00—IoT characterised by the purpose of the information processing
- G16Y40/60—Positioning; Navigation
Abstract
The invention discloses a scene type violation attribute identification system and method based on traffic semantics, wherein the system comprises the following steps: the system comprises a monitoring device, edge computing equipment, a mobile device, a comprehensive management terminal and a transmission device, wherein the monitoring device and the edge computing equipment are loaded at the mobile device, and the edge computing equipment is provided with a traffic semantic scene system; the monitoring device is used for acquiring video data of the environment on the moving route of the mobile device when the mobile device moves and transmitting the video data to the edge computing equipment; the edge computing equipment is used for receiving the video data, inputting the video data into the traffic semantic scene system through the transmission device to obtain a semantic processing result, and transmitting the semantic processing result to the comprehensive management terminal; the comprehensive management terminal is used for carrying out video viewing and historical violation information viewing on the monitoring device; the comprehensive management terminal is also used for controlling the start and the end of the violation identification operation and broadcasting the violation operation behaviors in real time.
Description
Technical Field
The invention relates to the technical field of violation attribute identification, in particular to a traffic semantic-based scene type violation attribute identification system and method.
Background
With the continuous improvement of the living standard of people, the reserved quantity of vehicles is rapidly improved, and the road traffic condition is increasingly complex.
In the related technology, the more adopted violation monitoring method is to dispatch police officers to monitor roads, or realize non-manual monitoring in a vehicle follow-up manner at a larger intersection, that is, the current traffic violation management and control is only limited to pass through a fixed gate or detect the static motor vehicle violation behaviors, but the non-manual monitoring is limited to simple behavior judgment, such as whether a pedestrian runs a red light, and the like.
Therefore, in order to further improve the current traffic violation management and control efficiency, the invention provides a scene type violation attribute identification system and method based on traffic semantics.
Disclosure of Invention
The invention aims to provide a scene type violation attribute identification system and method based on traffic semantics, and aims to solve the problem that traffic violation management and control in the related technology is limited to pass through a fixed gate or the detection of static motor vehicle violation behaviors is provided in the background technology. The invention provides the following technical scheme:
in one aspect, a traffic semantics-based scene type violation attribute recognition system is provided, which includes: the system comprises a monitoring device, edge computing equipment, a mobile device, a comprehensive management terminal and a transmission device, wherein the monitoring device and the edge computing equipment are loaded at the mobile device, and the edge computing equipment is provided with a traffic semantic scene system;
the monitoring device is used for acquiring video data of an environment on a moving route of the mobile device when the mobile device moves and transmitting the video data to the edge computing equipment, wherein the mobile device is a driving device with a traveling function;
the edge computing equipment is used for receiving the video data, inputting the video data into the traffic semantic scene system through the transmission device to obtain a semantic processing result, and transmitting the semantic processing result to the comprehensive management terminal, wherein the semantic processing result comprises the traffic identification information and the surrounding vehicle information;
the comprehensive management terminal is used for carrying out on-off control on the monitoring device and carrying out video check and historical violation information check on the monitoring device; the comprehensive management terminal is also used for controlling the start and the end of the violation identification operation and broadcasting the violation operation behaviors in real time; the integrated management terminal is also used for controlling the operation of the monitoring device and the edge computing equipment; and the comprehensive management terminal performs data interaction with the processing background.
Preferably, the traffic semantic scene system comprises a road scene recognition module, a traffic light state recognition module, a human body multi-label module, a pedestrian action recognition module, a target detection module, an application program module, a data acquisition module, a processing module, an object tracking module, lane line detection, a human face recognition module, a license plate recognition module, a violation judgment module and a evidence obtaining image generation module.
Preferably, the road scene identification is used for positioning the mobile device by a GPS and an inertial measurement unit, IMU, device, wherein the IMU device is disposed within the mobile device.
Preferably, the video data is in at least one of a multi-channel video stream format and a panoramic video format, and the edge computing device is further configured to perform synchronous processing on the multi-channel video stream and process the panoramic video data after receiving the video data.
Preferably, the integrated management terminal is a portable terminal device, and a central processing unit CPU, a storage module, a display module, and an application control module are simultaneously disposed in the portable terminal device.
In another aspect, a traffic semantics-based scene type violation attribute identification method is provided, and the method includes:
the master control end determines the target route according to the large-scale crowd sexual activity and sends the target route to the at least one mobile acquisition end, and the at least one mobile acquisition end is used for acquiring the environmental information of the target route and sending the acquired environmental information to the analysis end;
the master control end receives the police deployment report sent by the analysis end, and the police deployment report is generated by the analysis end according to the environment information;
in response to receiving a security confirmation instruction, the master control end refers to the police deployment report and sends the verification instruction to the at least one mobile acquisition end;
responding to the abnormal deployment of the point to be deployed, and receiving the abnormal deployment signal sent by the mobile acquisition end by the master control end;
and the master control end determines the geographical position with abnormal deployment according to the deployment abnormal signal and performs police force reconfiguration by referring to the police force deployment report.
Compared with the prior art, the invention has the beneficial effects that: the invention relates to a scene type violation attribute recognition system and a scene type violation attribute recognition method based on traffic semantics.A monitoring device acquires video data in a moving route when a mobile device runs through the mobile device when the mobile device moves, and the video data in the moving route acquired by the monitoring device is transmitted to edge computing equipment through a transmission device for processing; the method comprises the steps of identifying traffic identification and surrounding vehicles in a moving route according to video data to obtain corresponding traffic identification information and surrounding vehicle information, carrying out road scene identification, traffic light state identification, human body identification and pedestrian action identification on collected videos by edge computing equipment, carrying out data fusion, matching with a rule base, generating aggregate violation information and uploading the aggregate violation information to a processing background, and effectively solving the problems that the current traffic violation management and control can only detect the violation behaviors of static motor vehicles through a fixed bayonet, such as red light running, illegal parking and copying and the like.
Drawings
FIG. 1 is a flow chart of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the present invention provides a traffic semantics-based scene type violation attribute identification system: the traffic semantic based scene type violation attribute recognition system comprises a monitoring device, edge computing equipment, a mobile device, a comprehensive management terminal and a transmission device, wherein the monitoring device and the edge computing equipment are loaded at the mobile device, and the edge computing equipment is provided with a traffic semantic scene system; the monitoring device is used for acquiring video data of the environment on the moving route of the mobile device when the mobile device moves and transmitting the video data to the edge computing equipment, wherein the mobile device is a driving device with a traveling function; the edge computing equipment is used for inputting the video data into a traffic semantic scene system through a transmission device after receiving the video data to obtain a semantic processing result, and transmitting the semantic processing result to the comprehensive management terminal, wherein the semantic processing result comprises traffic identification information and surrounding vehicle information; the comprehensive management terminal is used for carrying out on-off control on the monitoring device and carrying out video check and historical violation information check on the monitoring device; the comprehensive management terminal is also used for controlling the start and the end of the violation identification operation and broadcasting the violation operation behaviors in real time; the integrated management terminal is also used for controlling the operation of the monitoring device and the edge computing equipment; and the comprehensive management terminal performs data interaction with the processing background.
In various embodiments of the present application, "traffic semantics" indicate relationships between various types of traffic elements between people, vehicles, and roads. For example, traffic semantics such as "a pedestrian passes through a sidewalk" and "a pedestrian runs a red light" can be formed between the pedestrian and a road.
In addition, mobile device not only can carry out the collection of portable making a video recording, can also carry out the fixed point of fixed position and make a video recording under the control of integrated management terminal and gather, and this application embodiment does not limit to this.
Specifically, the traffic semantic scene system comprises a road scene recognition module, a traffic light state recognition module, a human body multi-label module, a pedestrian action recognition module, a target detection module, an application program module, a data acquisition module, a processing module, an object tracking module, lane line detection module, a face recognition module, a license plate recognition module, a violation judgment module and a evidence obtaining image generation module.
The human body multi-label module judges whether a person rides a non-motor vehicle or not by using a multi-label algorithm, and wears a helmet mask and the like.
The target detection module uses a target detection algorithm and an instance segmentation algorithm, or other algorithms for target detection, which is not limited in the embodiments of the present application. Taking the target detection algorithm as an example, the contents are as follows: and detecting objects such as people, automobiles, traffic signs and the like in the camera data.
The pedestrian action recognition and application program module is mainly used for communicating with the edge device through an application program installed on the comprehensive management terminal, and the comprehensive management terminal controls the edge device to execute specific instructions and tasks and can switch data sources.
The data acquisition module acquires video data in the monitoring device and inputs the video data into the edge equipment.
The object tracking module extracts the characteristics of people and vehicles identified in each frame of video data by using a neural network, and then the characteristic values are matched with each other to match the action track of each vehicle of each person.
And detecting the lane line, wherein the lane line of the video data is detected by using a lane line detection algorithm.
The face recognition module detects faces in each frame of picture and matches the faces with a dangerous person face library to realize face inspection.
The license plate recognition module detects the position of a license plate and recognizes the content of the license plate by using an OCR (optical character recognition) technology.
The violation judging module can detect violation behaviors nearby the mobile device by performing data fusion and rule judgment on data among the target detecting module, the lane line detecting module, the face identifying module, the license plate identifying module and the like. The detectable illegal behaviors comprise illegal parking, illegal line pressing and lane changing, public traffic lane invasion, non-motor lane invasion, pedestrian courtesy avoidance, driving according to no guide lane, reverse driving, forbidden zone violation and the like, and meanwhile, face inspection and vehicle inspection can be carried out to find some suspicious persons and suspicious vehicles.
The system comprises a rule-breaking evidence obtaining module, a comprehensive management terminal and a rule-breaking evidence obtaining module, wherein the evidence obtaining module is used for splicing the detected violation behaviors into corresponding violation evidence obtaining images and videos according to evidence obtaining rules of a public security department and uploading the violation evidence obtaining images and videos to the comprehensive management terminal.
The application program control module is mainly used for realizing the application program control function of the module through the communication between an application program installed on the integrated management terminal and the edge computing equipment, controlling the edge equipment to execute specific instructions and tasks, and simultaneously switching the data source of the application program.
The data acquisition module acquires data streams in the camera and inputs the data streams into the specific processing module of the edge device. The module also provides a camera video recording function.
The detectable illegal behaviors comprise illegal parking, illegal line pressing and lane changing, public traffic lane invasion, non-motor lane invasion, pedestrian courtesy avoidance, driving according to no guide lane, reverse driving, forbidden zone violation and the like, and meanwhile, face inspection and vehicle inspection can be carried out to find some suspicious persons and suspicious vehicles.
The system comprises a rule-breaking evidence obtaining module, a rule-breaking evidence obtaining module and a management platform, wherein the rule-breaking evidence obtaining module is used for splicing the detected rule-breaking behaviors into corresponding rule-breaking evidence obtaining images and videos according to the evidence obtaining.
In the invention, the traffic semantic scene system adopts the latest machine vision optimization algorithm, improves the generalization capability and optimization speed of the algorithm, can realize quick self-learning on a new scene, continuously improves the accuracy of the model, and enriches the knowledge base in the traffic management field.
In particular, road scene identification is used to locate a mobile device via a GPS and an inertial measurement unit IMU apparatus, wherein the IMU apparatus is disposed within the mobile device.
In one possible implementation, the edge computing device obtains GPS coordinates via the IMU device, the GPS coordinates being used primarily to locate the geographic position of the scene, and as a basis for the mobile device trajectory, the position of the vehicle can be more accurately positioned through the GPS and the IMU equipment, for example, whether the vehicle runs on an overhead or runs under the overhead is judged, the limitation that the GPS equipment can not judge different roads at the same position is compensated, the IMU is a device for measuring the three-axis attitude angle or angular rate and acceleration of an object, a general IMU comprises three single-axis accelerometers and three single-axis gyroscopes, the accelerometers detect the acceleration signals of the object in independent three axes of a carrier coordinate system, the gyroscope detects angular velocity signals of the carrier relative to a navigation coordinate system, measures the angular velocity and the acceleration of the object in a three-dimensional space, and calculates the posture of the object according to the angular velocity signals; the GPS is a satellite navigation system which is developed and established by the American national defense department and has all-round, all-weather, all-time and high precision, can provide navigation information such as low-cost and high-precision three-dimensional position, speed, accurate timing and the like for global users, is an application model of a satellite communication technology in the navigation field, greatly improves the informatization level of the earth society, and powerfully promotes the development of digital economy, and can provide functions of vehicle positioning, theft prevention, anti-hijacking, driving route monitoring, call command and the like; three elements of a GPS terminal, a transmission network and a monitoring platform are required to realize all the functions.
Specifically, the video data is in at least one of a multi-channel video stream format and a panoramic video format, and the edge computing device is further configured to perform synchronous processing on the multi-channel video stream after receiving the video data, and perform processing on the panoramic video data.
Specifically, the integrated management terminal is a portable terminal device, a Central Processing Unit (CPU), a storage module, a display module and an application program control module are simultaneously arranged in the portable terminal device, data processed by the edge computing device are transmitted into the integrated management terminal, processed confidence is stored in the storage module, and the monitoring device is controlled through the application program module.
It should be noted that, in the related art, there is no technical scheme for performing scene recognition in combination with traffic semantics, and the more common operation is: license plate information is recognized through the camera shooting equipment, and the camera shooting equipment is set for a fixed place. For example, a camera device is fixedly arranged on a bus lane, but all vehicles entering the bus lane are identified as an event of encroaching on the bus lane (the vehicles are excluded through license plate information), and the like is performed on a non-motor lane. Therefore, the related technology only relates to the identification of the license plate information for judging the road violation.
The scene type violation attribute recognition system based on the traffic semantics not only recognizes license plate information based on the camera information, but also acquires the traffic semantics based on the camera information; furthermore, after the system automatically identifies the current traffic scene and elements according to the traffic semantics, the system performs inference on violation information by combining the violation rule base, so that compared with the related technology, the accuracy of violation identification is greatly improved, and the intelligent processing of violation judgment is realized.
The mode of acquiring the camera information is a mobile acquisition mode, and compared with the mode of setting the camera equipment at a fixed place in the related art, the mode of acquiring the camera information is more flexible.
The invention also provides a scene type violation attribute identification method based on traffic semantics, which comprises the following steps:
and S1, responding to the mobile device receiving the violation identification starting signal, starting the mobile device and the monitoring device, acquiring video data of the environment of the mobile device when the monitoring device runs, and sending the violation identification starting signal by the comprehensive management terminal.
And S2, transmitting the video data to the edge computing equipment in real time by the monitoring device.
And S3, the edge computing equipment inputs the video data into the traffic semantic scene system through the transmission device to obtain a semantic processing result, and transmits the semantic processing result to the comprehensive management terminal, wherein the semantic processing result comprises traffic identification information and surrounding vehicle information.
And S4, the edge computing equipment further positions the mobile device through road scene recognition of the traffic semantic scene system, wherein the road scene recognition is realized based on GPS and IMU equipment.
S5, the edge computing equipment sends the semantic processing result and/or the positioning data of the mobile device to the comprehensive management terminal.
And S6, the comprehensive management terminal judges whether the violation operation exists according to the semantic processing result and/or the positioning data of the mobile device, and the comprehensive management terminal broadcasts in real time in response to the violation operation.
Wherein, under the condition that only needing to obtain whether there is the operation of violating regulations, can not obtain the location data of the mobile device.
Preferably, in step S3, the content of the "the edge computing device inputs the video data to the traffic semantic scene system through the transmission device to obtain the semantic processing result" is specifically the following content:
the method comprises the steps that firstly, edge computing equipment synchronously receives at least two paths of video stream formats and/or video data in a panoramic video format, and inputs the video data to a traffic semantic scene system through a transmission device to obtain a semantic processing result;
and secondly, in response to receiving a video stream stop receiving signal, the edge computing device stops receiving and transmitting the target video stream, wherein the video stream stop receiving signal is sent by the integrated management terminal and is used for indicating the target video stream.
Preferably, the method further comprises:
s7, the comprehensive management terminal controls the on-off of the monitoring device, and performs video check and historical violation information check on the monitoring device.
Preferably, the method further comprises:
and S8, the integrated management terminal controls the start and the end of the violation identification operation.
Preferably, the method further comprises:
s9, the comprehensive management terminal matches the semantic processing result with the violation judgment module to detect the violation event and generate violation evidence obtaining information;
and S10, the comprehensive management terminal sends the violation evidence obtaining information to a processing background.
It should be noted that the process of detecting the violation event and generating violation forensic information by the semantic processing result matching violation determination module can also be executed by the edge computing device, that is, after the edge computing device obtains the semantic processing result through the traffic semantic scene system, the semantic processing result matching violation determination module continues to detect the violation event and generate violation forensic information, and the violation forensic information is sent to the comprehensive management terminal, and the comprehensive management terminal sends the violation forensic information to the processing background.
The traffic semantics-based scene type violation attribute identification method can also be summarized as follows: firstly, when the mobile device moves, the monitoring device collects video data in a moving route when the mobile device runs, and the video data in the moving route collected by the monitoring device is transmitted to the edge computing equipment for processing; identifying traffic marks and surrounding vehicles in a moving route according to video data to obtain corresponding traffic mark information and surrounding vehicle information, collecting vehicle tracks of a moving device through GPS and IMU equipment, positioning, transmitting the collected data into edge computing equipment through wireless signals for processing, acquiring video streams of a detection device by the edge computing equipment, simultaneously processing multiple video streams and panoramic videos by the edge computing equipment, transmitting the video streams monitored by a monitoring device through a transmission device, randomly starting and stopping any one video stream through the edge computing equipment, fusing calculation, networking and storing into a whole by the edge computing equipment by adopting an edge computing technology, intelligently analyzing and processing video images collected by all monitoring devices through an AI algorithm, and transmitting the processed information to a comprehensive management terminal through the monitoring devices, the comprehensive management terminal device conducts angle and focal length adjustment, illegal detection starting or closing, video viewing and historical illegal information viewing, conducts illegal identification, face inspection, vehicle inspection, panoramic live broadcast, road scene identification, traffic light state identification, human body identification and pedestrian action identification on a camera video through edge computing equipment, conducts data fusion, matches an illegal rule base, generates aggregate illegal information and uploads the aggregate illegal information to a processing background.
In the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "fixed" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral part; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
The standard parts used in the invention can be purchased from the market, the special-shaped parts can be customized according to the description of the specification and the accompanying drawings, the specific connection mode of each part adopts conventional means such as mature bolts, rivets, welding and the like in the prior art, the machines, the parts and equipment adopt conventional models in the prior art, and the circuit connection adopts the conventional connection mode in the prior art, so that the detailed description is omitted.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. The traffic semantics-based scene type violation attribute identification system is characterized by comprising the following steps: the system comprises a monitoring device, edge computing equipment, a mobile device, a comprehensive management terminal and a transmission device, wherein the monitoring device and the edge computing equipment are loaded at the mobile device, and the edge computing equipment is provided with a traffic semantic scene system;
the monitoring device is used for acquiring video data of an environment on a moving route of the mobile device when the mobile device moves and transmitting the video data to the edge computing equipment, wherein the mobile device is a driving device with a traveling function;
the edge computing equipment is used for receiving the video data, inputting the video data into the traffic semantic scene system through the transmission device to obtain a semantic processing result, and transmitting the semantic processing result to the comprehensive management terminal, wherein the semantic processing result comprises the traffic identification information and the surrounding vehicle information;
the comprehensive management terminal is used for carrying out on-off control on the monitoring device and carrying out video check and historical violation information check on the monitoring device; the comprehensive management terminal is also used for controlling the start and the end of the violation identification operation and broadcasting the violation operation behaviors in real time; the integrated management terminal is also used for controlling the operation of the monitoring device and the edge computing equipment; and the comprehensive management terminal performs data interaction with the processing background.
2. The traffic semantics based scenic violation attribute recognition system of claim 1, wherein the traffic semantics scene system comprises road scene recognition, traffic light state recognition, human body recognition, a human body multi-label module, pedestrian action recognition, an object detection module, an application module, a data acquisition module, a processing module, an object tracking module, lane line detection, a face recognition module, a license plate recognition module, a violation determination module, and a generate forensics map module.
3. The traffic semantics based scenic violation attribute identification system of claim 2 wherein the road scene identification is used to locate the mobile device via a GPS and inertial measurement unit, IMU, device, wherein the IMU device is disposed within the mobile device.
4. The traffic semantics based scenic violation attribute identification system of claim 1 wherein the video data is in at least one of a multi-channel video stream format and a panoramic video format, and the edge computing device is further configured to receive the video data and then synchronize the multi-channel video stream and process the panoramic video data.
5. The traffic semantics based scene-based violation attribute recognition system of claim 1 wherein the integrated management terminal is a portable terminal device, and the portable terminal device is internally provided with a Central Processing Unit (CPU), a storage module, a display module and an application control module.
6. A traffic semantics based scenic violation attribute identification method, which is used for the traffic semantics based scenic violation attribute identification system of any one of claims 1 to 5, and comprises the following steps:
responding to the mobile device receiving a violation identification starting signal, starting the mobile device and the monitoring device, acquiring the video data of the environment of the mobile device when the monitoring device runs, and sending the violation identification starting signal by the comprehensive management terminal;
the monitoring device transmits the video data to the edge computing equipment in real time;
the edge computing equipment inputs the video data into the traffic semantic scene system through the transmission device to obtain the semantic processing result, and transmits the semantic processing result to the comprehensive management terminal, wherein the semantic processing result comprises the traffic identification information and the surrounding vehicle information;
the edge computing device further locates the mobile device by road scene recognition of the traffic semantic scene system, the road scene recognition being implemented based on the GPS and the IMU device;
the edge computing equipment sends the semantic processing result and/or the positioning data of the mobile device to the comprehensive management terminal;
and the comprehensive management terminal judges whether the violation operation exists according to the semantic processing result and/or the positioning data of the mobile device, and responds to the violation operation, and the comprehensive management terminal broadcasts in real time.
7. The traffic semantics based scenic violation attribute recognition method of claim 6, wherein the edge computing device inputs the video data to the traffic semantics scene system via the transmission device to obtain the semantic processing result, comprising:
the edge computing equipment synchronously receives at least two paths of video stream formats and/or video data in a panoramic video format, and inputs the video data to the traffic semantic scene system through the transmission device to obtain the semantic processing result;
in response to receiving a video stream stop receiving signal, the edge computing device stops receiving and transmitting a target video stream, wherein the video stream stop receiving signal is sent by the integrated management terminal, and the video stream stop receiving signal is used for indicating the target video stream.
8. The traffic semantics based scenic violation attribute identification method of claim 6 further comprising:
and the comprehensive management terminal controls the monitoring device to be switched on and off, and performs video check and historical violation information check on the monitoring device.
9. The traffic semantics based scenic violation attribute identification method of claim 6 further comprising:
and the comprehensive management terminal controls the starting and the ending of the violation identification operation.
10. The traffic semantics based scenic violation attribute identification method of claim 6 further comprising:
the comprehensive management terminal matches the semantic processing result with the violation judging module to detect a violation event and generate violation evidence obtaining information;
and the comprehensive management terminal sends violation evidence obtaining information to the processing background.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010907113 | 2020-09-02 | ||
| CN2020109071135 | 2020-09-02 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112289036A true CN112289036A (en) | 2021-01-29 |
Family
ID=74421200
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011056632.1A Pending CN112289036A (en) | 2020-09-02 | 2020-09-29 | Scene type violation attribute identification system and method based on traffic semantics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112289036A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113870573A (en) * | 2021-09-17 | 2021-12-31 | 延边国泰新能源汽车有限公司 | Bus management service system and method based on operation of buses in alpine regions |
| CN114371693A (en) * | 2021-11-26 | 2022-04-19 | 江苏金晓电子信息股份有限公司 | Traffic cruise controller control method based on multi-sensor data fusion under multiple scenes |
| CN116129647A (en) * | 2023-02-28 | 2023-05-16 | 禾多科技（北京）有限公司 | Full-closed-loop scene reconstruction method based on dangerous points |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140086010A (en) * | 2012-12-28 | 2014-07-08 | (주)오엔씨엑스 | Intelligent real time traffic violation enforcement information system and method thereof |
| CN105303827A (en) * | 2015-09-07 | 2016-02-03 | 北京信路威科技股份有限公司 | Traffic violation image obtaining device and method |
| CN107886595A (en) * | 2017-11-29 | 2018-04-06 | 深圳市易成自动驾驶技术有限公司 | Recognition methods violating the regulations, drive recorder and storage medium based on drive recorder |
| CN108154686A (en) * | 2018-02-12 | 2018-06-12 | 苏州清研微视电子科技有限公司 | A kind of vehicle-mounted act of violating regulations capturing system |
| CN207650996U (en) * | 2018-01-05 | 2018-07-24 | 南京邮电大学 | A kind of autonomous type inspection based on unmanned plane is violating the regulations to penalize stop system |
| CN108958264A (en) * | 2018-08-03 | 2018-12-07 | 北京智行者科技有限公司 | Road traffic checking method and vehicle based on automatic Pilot technology |
| CN108961768A (en) * | 2018-07-30 | 2018-12-07 | 鄂尔多斯市普渡科技有限公司 | The unmanned police cruiser of one kind and patrol method |
| CN109686103A (en) * | 2018-12-29 | 2019-04-26 | 中南大学 | A kind of intelligent transportation based on image recognition is separated to stop monitoring management system and method |
| CN110536121A (en) * | 2019-09-20 | 2019-12-03 | 上海风骑信息科技有限公司 | A kind of Intelligent Dynamic video analysis patrol prevention and control system |
-
2020
- 2020-09-29 CN CN202011056632.1A patent/CN112289036A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140086010A (en) * | 2012-12-28 | 2014-07-08 | (주)오엔씨엑스 | Intelligent real time traffic violation enforcement information system and method thereof |
| CN105303827A (en) * | 2015-09-07 | 2016-02-03 | 北京信路威科技股份有限公司 | Traffic violation image obtaining device and method |
| CN107886595A (en) * | 2017-11-29 | 2018-04-06 | 深圳市易成自动驾驶技术有限公司 | Recognition methods violating the regulations, drive recorder and storage medium based on drive recorder |
| CN207650996U (en) * | 2018-01-05 | 2018-07-24 | 南京邮电大学 | A kind of autonomous type inspection based on unmanned plane is violating the regulations to penalize stop system |
| CN108154686A (en) * | 2018-02-12 | 2018-06-12 | 苏州清研微视电子科技有限公司 | A kind of vehicle-mounted act of violating regulations capturing system |
| CN108961768A (en) * | 2018-07-30 | 2018-12-07 | 鄂尔多斯市普渡科技有限公司 | The unmanned police cruiser of one kind and patrol method |
| CN108958264A (en) * | 2018-08-03 | 2018-12-07 | 北京智行者科技有限公司 | Road traffic checking method and vehicle based on automatic Pilot technology |
| CN109686103A (en) * | 2018-12-29 | 2019-04-26 | 中南大学 | A kind of intelligent transportation based on image recognition is separated to stop monitoring management system and method |
| CN110536121A (en) * | 2019-09-20 | 2019-12-03 | 上海风骑信息科技有限公司 | A kind of Intelligent Dynamic video analysis patrol prevention and control system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113870573A (en) * | 2021-09-17 | 2021-12-31 | 延边国泰新能源汽车有限公司 | Bus management service system and method based on operation of buses in alpine regions |
| CN114371693A (en) * | 2021-11-26 | 2022-04-19 | 江苏金晓电子信息股份有限公司 | Traffic cruise controller control method based on multi-sensor data fusion under multiple scenes |
| CN116129647A (en) * | 2023-02-28 | 2023-05-16 | 禾多科技（北京）有限公司 | Full-closed-loop scene reconstruction method based on dangerous points |
| CN116129647B (en) * | 2023-02-28 | 2023-09-05 | 禾多科技（北京）有限公司 | Full-closed-loop scene reconstruction method based on dangerous points |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110349405B (en) | Real-time traffic monitoring using networked automobiles | |
| US11276193B2 (en) | Information processing device | |
| CN107608388B (en) | Autonomous police vehicle | |
| JP7188394B2 (en) | Image processing device and image processing method | |
| CN112289036A (en) | Scene type violation attribute identification system and method based on traffic semantics | |
| US20240046653A1 (en) | Identifying suspicious entities using autonomous vehicles | |
| JP4752836B2 (en) | Road environment information notification device and road environment information notification program | |
| JP4093026B2 (en) | Road environment information notification device, in-vehicle notification device, information center device, and road environment information notification program | |
| WO2021024798A1 (en) | Travel assistance method, road captured image collection method, and roadside device | |
| WO2019188391A1 (en) | Control device, control method, and program | |
| KR20200043252A (en) | Overlooking image generation system of vehicle and method thereof | |
| CN111477011A (en) | Detection device and detection method for road intersection early warning | |
| KR101998834B1 (en) | Traffic Information Providing System Enabling Traffic Accident Risk Prediction | |
| KR101281629B1 (en) | Driving guidance system using sensors | |
| KR101210615B1 (en) | Regulation system of u-turn violation vehicle | |
| KR101525151B1 (en) | System and method for recognizing license plate | |
| WO2020241971A1 (en) | Traffic accident management device and traffic accident management method | |
| CN111746789A (en) | Imaging system, server, control method, and storage medium storing program | |
| CN107731005B (en) | Cloud computing-based vehicle-mounted warning system utilizing phantom imaging | |
| CN112602124A (en) | Communication method for vehicle dispatching system, vehicle dispatching system and communication device | |
| KR20220052679A (en) | Personal mobility having local monitogring function | |
| KR20030034770A (en) | Moving car recognition system | |
| JP2022056153A (en) | Temporary stop detection device, temporary stop detection system, and temporary stop detection program | |
| CN114425991A (en) | Image processing method, medium, device and image processing system | |
| KR102600313B1 (en) | Method for analyzing information of vehicle path of intersection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210129 |