CN117576859A - Vehicle-mounted monitoring-based passenger management method and device - Google Patents
Vehicle-mounted monitoring-based passenger management method and device Download PDFInfo
- Publication number
- CN117576859A CN117576859A CN202311615489.9A CN202311615489A CN117576859A CN 117576859 A CN117576859 A CN 117576859A CN 202311615489 A CN202311615489 A CN 202311615489A CN 117576859 A CN117576859 A CN 117576859A
- Authority
- CN
- China
- Prior art keywords
- data
- vehicle
- track
- driver
- driving track
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 51
- 238000007726 management method Methods 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000004458 analytical method Methods 0.000 claims description 19
- 230000008921 facial expression Effects 0.000 claims description 19
- 230000036651 mood Effects 0.000 claims description 18
- 238000001514 detection method Methods 0.000 claims description 12
- 230000001815 facial effect Effects 0.000 claims description 11
- 238000013528 artificial neural network Methods 0.000 claims description 9
- 230000001133 acceleration Effects 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000012502 risk assessment Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 5
- 238000004140 cleaning Methods 0.000 claims description 5
- 238000013135 deep learning Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 5
- 238000003058 natural language processing Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 5
- 230000007547 defect Effects 0.000 abstract description 2
- 230000006399 behavior Effects 0.000 description 12
- 238000007405 data analysis Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
- G08G1/0129—Traffic data processing for creating historical data or processing based on historical data
Abstract
The invention provides a vehicle-mounted monitoring-based passenger management method and device, wherein the method comprises the following steps: acquiring vehicle track data and road environment data; analyzing a normal driving track according to historical track data in the vehicle track data, and analyzing and predicting a driving track according to current track data in the vehicle track data; comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track is separated from the normal driving track; analyzing the risk level of the potential violation according to the road environment data; and sending early warning to a driver according to the risk level. The vehicle-mounted driver behavior monitoring system is used for solving the defect of poor warning effect of potential violations of the vehicle-mounted driver in the prior art and realizing fine supervision of the vehicle-mounted driver behavior.
Description
Technical Field
The invention relates to the technical field of vehicle driving monitoring, in particular to a vehicle-mounted monitoring-based passenger management method and device.
Background
Along with the development of railway vehicles and the improvement of digital information technology, informationized construction becomes an important component for the development of railway vehicles. At present, concepts such as DCC (vehicle field section control center) full scene monitoring and multi-professional data fusion application technical research, intelligent passenger management and the like and related technologies thereof are widely accepted and applied.
Based on the existing vehicle-mounted full scene monitoring system, the functions of fatigue detection, potential violation early warning, video real-time adjustment and the like of a vehicle-mounted driver are realized by utilizing real-time vehicle monitoring data, intelligent riding management of the driver is realized, behavior trails of the driver are analyzed, effect evaluation is carried out, and the vehicle-mounted full scene monitoring system has important significance in improving the safety operation level of the vehicle. Therefore, how to realize the fine supervision of the behavior of a vehicle-mounted driver based on the vehicle-mounted monitoring and the management of the passenger service, realize the monitoring and auxiliary decision-making functions of the dispatching production process, ensure the production operation safety, improve the production dispatching efficiency, ensure the dispatching command reliability and reduce the accident occurrence rate is a problem to be solved urgently at present.
Disclosure of Invention
The invention provides a passenger management method and device based on vehicle-mounted monitoring, which are used for solving the defect of poor early warning effect of potential violations of a vehicle-mounted driver in the prior art and realizing fine supervision of behaviors of the vehicle-mounted driver.
The invention provides a vehicle-mounted monitoring-based passenger management method, which comprises the following steps:
acquiring vehicle track data and road environment data;
analyzing a normal driving track according to historical track data in the vehicle track data, and analyzing and predicting a driving track according to current track data in the vehicle track data;
comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track is separated from the normal driving track;
analyzing the risk level of the potential violation according to the road environment data;
and sending early warning to a driver according to the risk level.
According to the vehicle-mounted monitoring-based passenger management method provided by the invention, the road environment data comprises the following steps: pedestrian data, traffic light data, and traffic flow data, said analyzing a risk level of potential violations from said road environment data comprising:
calculating collision risk probability of the predicted driving track according to the road environment data;
and determining the risk level of the potential violation according to the collision risk probability.
According to the vehicle-mounted monitoring-based passenger management method provided by the invention, the early warning is sent to the driver according to the risk level, and the method comprises the following steps:
sending corresponding early warning to the driver according to the risk level, and recording response time of the driver to the early warning;
and under the condition that the potential violation is not corrected within the preset time, controlling the vehicle to trigger automatic braking intervention.
According to the vehicle-mounted monitoring-based passenger management method provided by the invention, after the risk level of potential violations is analyzed according to the road environment data, the method further comprises the following steps:
acquiring facial image data of a driver;
acquiring eye image data according to the face image data;
detecting the opening and closing time duty ratio of eyes according to the eye image data;
judging the fatigue degree of a driver according to the eye opening and closing time duty ratio;
and sending fatigue early warning to a driver according to the fatigue degree.
According to the vehicle-mounted monitoring-based passenger management method provided by the invention, the eye image data is acquired according to the face image data, and the method comprises the following steps:
detecting eye key points of the face image data through a face detection model based on a computer vision library, and acquiring eye image data;
the detecting the eye opening and closing time duty ratio according to the eye image data comprises the following steps:
classifying an eye opening and closing state of the eye image data by a feature descriptor;
and calculating the ratio of the number of frames of the closed eyes in unit time according to the opening and closing states of the eyes to obtain the opening and closing time duty ratio of the eyes.
According to the vehicle-mounted monitoring-based passenger management method provided by the invention, after the early warning is sent to the driver according to the risk level, the method further comprises the following steps:
acquiring vehicle running data and preprocessing the vehicle running data;
analyzing the vehicle running data to obtain vehicle speed characteristics;
acquiring and analyzing facial image data of a driver to obtain facial expression characteristics;
the method comprises the steps of acquiring and analyzing voice dialogue data of a driver to obtain sentence and mood characteristics;
inputting the vehicle speed characteristics, the facial expression characteristics and the sentence mood characteristics into a driver portrait model to obtain each dimension score of the driver portrait;
and establishing a driver portrait radar chart according to the dimension scores of the driver portrait.
The driver portrait model is trained based on multi-source feature samples comprising vehicle speed features, facial expression features and sentence mood features, and each dimension score of the driver portrait corresponding to the multi-source feature samples serving as a label.
According to the vehicle-mounted monitoring-based passenger management method provided by the invention, the vehicle driving data comprise: vehicle-mounted tachograph data and CAN bus data, the vehicle-mounted tachograph data comprises: position coordinates, speed, oil consumption and steering angle, the CAN bus data comprises: a braking signal and an acceleration signal, the vehicle speed characteristics comprising: average velocity characteristics and acceleration characteristics;
the acquiring the vehicle running data, preprocessing the vehicle running data, includes:
acquiring vehicle driving data;
cleaning abnormal data in the vehicle running data, and smoothing a filtering running track;
detecting and filling data missing points in vehicle driving data;
and normalizing the source data with different formats in the vehicle driving data to a unified sample.
According to the vehicle-mounted monitoring-based passenger management method provided by the invention, the driver portrait model is a deep learning neural network comprising a convolution layer and a pooling layer, the facial expression characteristics are obtained through analysis of the convolution neural network, and the sentence language characteristics are obtained through natural language processing analysis.
According to the vehicle-mounted monitoring-based passenger management method provided by the invention, the normal driving track is analyzed through the LSTM network, and the predicted driving track is the vehicle track of 5 seconds in the future.
The invention also provides a vehicle-mounted monitoring-based passenger management device, which comprises:
the data acquisition module is used for acquiring vehicle track data and road environment data;
the track analysis module is used for analyzing a normal driving track according to historical track data in the vehicle track data and analyzing and predicting a driving track according to current track data in the vehicle track data;
the violation judging module is used for comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track is separated from the normal driving track;
the risk analysis module is used for analyzing the risk level of the potential violation according to the road environment data;
and the early warning sending module is used for sending early warning to the driver according to the risk level.
According to the vehicle-mounted monitoring-based passenger management method and device, vehicle track data and road environment data in vehicle-mounted monitoring are obtained, a normal driving track, namely a set track for driving a vehicle under normal conditions, is analyzed according to historical track data in the vehicle track data, and then a predicted driving track, namely a predicted track of the vehicle under the current driving trend, is analyzed and predicted according to current track data in the vehicle track data. And comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track deviates from the normal driving track and the predicted driving track deviates from the normal track range. After determining that the potential violation exists, the risk level of the potential violation needs to be analyzed in combination with the current road environment data, i.e. the severity of the possible consequences under the current potential violation operation. And sending early warning to the driver according to the analyzed risk level, so that the driver pays attention to the corresponding situation, and correcting the potential illegal operation in time. Through the process, the method and the system realize analysis of the behavior trails of the drivers, discover potential illegal behaviors in time and early warn, realize fine supervision and intelligent passenger management of the behaviors of the vehicle-mounted drivers, and have important significance for improving the safety operation level of the vehicles.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a vehicle-mounted monitoring-based passenger management method provided by the invention;
fig. 2 is a schematic structural diagram of a vehicle-mounted monitoring-based passenger management device provided by the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
A vehicle-mounted monitoring-based passenger management method according to a first embodiment of the present invention is described below with reference to fig. 1.
As shown in fig. 1, a first embodiment of the present invention provides a vehicle-mounted monitoring-based passenger management method, which specifically includes the following steps (the number of each step in this embodiment only performs step distinguishing function, and the specific execution sequence of each step is not limited):
step S1: vehicle track data and road environment data are acquired.
Vehicle track data and road environment data in vehicle-mounted monitoring are acquired, wherein the vehicle track data comprise: the historical track data and the current real-time track data are respectively used for predicting different driving tracks. The road environment data includes: pedestrian data, traffic light data, and traffic flow data for use in combination with analyzing risk levels of potential violations.
Step S2: and analyzing a normal driving track according to historical track data in the vehicle track data, and analyzing and predicting a driving track according to current track data in the vehicle track data.
Firstly, a normal driving track, namely a set track for driving a vehicle under normal conditions, is analyzed according to historical track data in vehicle track data, and then a predicted driving track, namely a predicted track of the vehicle under the current driving trend, is analyzed according to current track data in the vehicle track data, so that a basis is provided for subsequent track analysis and comparison.
Step S3: and comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track is separated from the normal driving track.
And comparing the predicted driving track with the normal driving track, and if the predicted driving track deviates from the normal driving track, indicating that the predicted driving track deviates from the normal track range, judging that the potential violation exists in the driver, and timely finding out the potential violation operation of the driver.
Step S4: and analyzing the risk level of the potential violation according to the road environment data.
After determining that the potential violations exist, the risk level of the potential violations needs to be analyzed in combination with the current road environment data, and then the risk assessment of the violations is carried out, namely the severity of the consequences possibly caused by the current potential violations is carried out, so that corresponding early warning is sent out. The risk level includes: low risk, medium risk and high risk.
Step S5: and sending early warning to a driver according to the risk level.
And sending early warning to the driver according to the analyzed risk level, so that the driver pays attention to the corresponding situation, and correcting the potential illegal operation in time. The early warning information sent in the embodiment includes multi-level early warning, specifically, a low risk voice prompt "notice safety", a medium risk screen displays a yellow font "please notice driving safety", and a high risk screen pops up a red warning "immediately corrects the driving direction.
According to the passenger management method based on the vehicle-mounted monitoring, the vehicle track data and the road environment data in the vehicle-mounted monitoring are obtained, the normal driving track, namely the set track of the vehicle driving under normal conditions, is analyzed according to the historical track data in the vehicle track data, and then the predicted driving track, namely the predicted track of the vehicle under the current driving trend, is analyzed according to the current track data in the vehicle track data. And comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track deviates from the normal driving track and the predicted driving track deviates from the normal track range. After determining that the potential violation exists, the risk level of the potential violation needs to be analyzed in combination with the current road environment data, i.e. the severity of the possible consequences under the current potential violation operation. And sending early warning to the driver according to the analyzed risk level, so that the driver pays attention to the corresponding situation, and correcting the potential illegal operation in time. Through the process, the method and the system realize analysis of the behavior trails of the drivers, discover potential illegal behaviors in time and early warn, realize fine supervision and intelligent passenger management of the behaviors of the vehicle-mounted drivers, and have important significance for improving the safety operation level of the vehicles.
In this embodiment, the road environment data includes: pedestrian data, traffic light data, and traffic flow data, said analyzing a risk level of potential violations from said road environment data comprising:
calculating collision risk probability of the predicted driving track according to the road environment data;
and determining the risk level of the potential violation according to the collision risk probability.
The collision risk probability of the driving track under the current road environment is predicted by comprehensively considering the road environment data such as pedestrian data, traffic signal lamp data and traffic flow data, so that the risk level of potential violations is determined, the severity of the possible consequences caused by the potential violating operation can be timely found and reflected, and the real-time running safety of the vehicle is ensured.
In this embodiment, the sending an early warning to the driver according to the risk level includes:
sending corresponding early warning to the driver according to the risk level, and recording response time of the driver to the early warning;
and under the condition that the potential violation is not corrected within the preset time, controlling the vehicle to trigger automatic braking intervention.
After the corresponding early warning is sent according to the risk level of the potential violation, the step of interaction with the driver is also introduced, namely after the corresponding early warning is sent to the driver according to the risk level, the response time of the driver to the early warning is recorded, so that the feedback adjustment speed of the driver to the potential violation early warning is known, and the later safety assessment and summarization are facilitated. Under the condition that the potential violation is not corrected within the preset time, the vehicle is controlled to trigger automatic braking intervention, and absolute safety of vehicle running is ensured.
In this embodiment, after the analyzing the risk level of the potential violation according to the road environment data, the method further includes:
acquiring facial image data of a driver;
acquiring eye image data according to the face image data;
detecting the opening and closing time duty ratio of eyes according to the eye image data;
judging the fatigue degree of a driver according to the eye opening and closing time duty ratio;
and sending fatigue early warning to a driver according to the fatigue degree.
Meanwhile, facial image data of a driver in vehicle-mounted monitoring is also acquired, eye image data is acquired through the facial image data, the fatigue degree of the driver is judged through detecting the opening and closing time ratio of eyes, the fatigue detection of the vehicle-mounted driver is realized, and fatigue early warning is sent to the driver according to the fatigue degree. In this embodiment, different levels of fatigue thresholds are set, if the eye-closing time is more than 60%, the high fatigue degree is determined, different acousto-optic early warning is adopted according to different fatigue degrees, for example, voice prompt is carried out on the high fatigue degree, and the medium and low fatigue degrees are directly displayed for reminding so as to ensure driving safety.
In this embodiment, the acquiring the eye image data according to the face image data includes:
detecting eye key points of the face image data through a face detection model based on a computer vision library, and acquiring eye image data;
the detecting the eye opening and closing time duty ratio according to the eye image data comprises the following steps:
classifying an eye opening and closing state of the eye image data by a feature descriptor;
and calculating the ratio of the number of frames of the closed eyes in unit time according to the opening and closing states of the eyes to obtain the opening and closing time duty ratio of the eyes.
Eye image data is obtained by detecting eye key points of face image data based on a face detection model of a computer vision library, and the face detection model based on the computer vision library in the embodiment is specifically an OpenCV face detection model. The OpenCV face detection model has accurate face image recognition and segmentation effects, and is suitable for real-time face image data in vehicle-mounted monitoring. The open-eye state of the eye image data is classified by the feature description sub-classification, specifically by using HOG (Histogram of Oriented Gradient, directional gradient histogram) features. The HOG feature is a feature descriptor for object detection in computer vision and image processing, and features are formed by calculating and counting a gradient direction histogram of a local area of an image, so that the HOG feature has a better recognition effect on an image of a subtle limb motion such as eye opening and closing.
In this embodiment, after the sending of the early warning to the driver according to the risk level, the method further includes:
acquiring vehicle running data and preprocessing the vehicle running data;
analyzing the vehicle running data to obtain vehicle speed characteristics;
acquiring and analyzing facial image data of a driver to obtain facial expression characteristics;
the method comprises the steps of acquiring and analyzing voice dialogue data of a driver to obtain sentence and mood characteristics;
inputting the vehicle speed characteristics, the facial expression characteristics and the sentence mood characteristics into a driver portrait model to obtain each dimension score of the driver portrait;
and establishing a driver portrait radar chart according to the dimension scores of the driver portrait.
The driver portrait model is trained based on multi-source feature samples comprising vehicle speed features, facial expression features and sentence mood features, and each dimension score of the driver portrait corresponding to the multi-source feature samples serving as a label.
And the comprehensive analysis of various data according to vehicle-mounted monitoring is supported, wherein the comprehensive analysis comprises the steps of obtaining vehicle speed characteristics according to vehicle driving data analysis, obtaining facial expression characteristics according to facial image data analysis and obtaining sentence and mood characteristics according to voice dialogue data analysis. The multi-source features of the vehicle speed feature, the facial expression feature and the sentence mood feature are input into the driver portrait model to obtain the dimension scores of the driver portrait. A driver portrait radar map is created based on the dimension scores of the driver portrait to generate a driver portrait. Therefore, the operation habit and operation weak items of individuals or groups of drivers can be analyzed, the operation weak items and the missing items can be found, the strong grasping weak points can be started, and the occurrence of potential safety hazards can be restrained.
In this embodiment, the vehicle running data includes: vehicle-mounted tachograph data and CAN bus data, the vehicle-mounted tachograph data comprises: position coordinates, speed, oil consumption and steering angle, the CAN bus data comprises: a braking signal and an acceleration signal, the vehicle speed characteristics comprising: average velocity characteristics and acceleration characteristics;
the acquiring the vehicle running data, preprocessing the vehicle running data, includes:
acquiring vehicle driving data;
cleaning abnormal data in the vehicle running data, and smoothing a filtering running track;
detecting and filling data missing points in vehicle driving data;
and normalizing the source data with different formats in the vehicle driving data to a unified sample.
The vehicle-mounted traveling data recorder and the CAN bus data CAN accurately reflect the driving habit and the characteristics of a driver, and the vehicle-mounted traveling data is preprocessed before being analyzed, and the vehicle-mounted traveling data recorder comprises: cleaning abnormal data, smoothing a filtering running track, filling data missing points, unifying format source data and the like, and ensuring the accuracy of subsequent data analysis.
In this embodiment, the driver portrait model is a deep learning neural network including a convolution layer and a pooling layer, the facial expression features are obtained by analysis of the convolution neural network, and the sentence mood features are obtained by natural language processing analysis.
The driver portrait model is a deep learning neural network comprising a convolution layer and a pooling layer, and the model prediction accuracy is ensured by training the network model through a multisource feature sample and corresponding dimension score labels of the driver portrait and using a labeling sample data set. Facial expression features and sentence mood features are obtained through convolutional neural network and natural language processing analysis respectively, and recognition accuracy and recognition efficiency are considered.
In this embodiment, the normal driving track is analyzed through an LSTM network, and the predicted driving track is a vehicle track of 5 seconds in the future.
The normal driving track is analyzed through an LSTM (Long Short-Term Memory) network, the LSTM is suitable for processing and predicting important events with longer intervals in a time sequence, and the track analysis of the vehicle is high in accuracy. The predicted driving track is a vehicle track of 5 seconds in the future, and whether the predicted driving track deviates from a normal track range is judged by predicting the vehicle track of 5 seconds in the future.
The following describes the vehicle-mounted monitoring service management device provided by the invention, and the vehicle-mounted monitoring service management device described below and the vehicle-mounted monitoring service management method described above can be correspondingly referred to each other.
As shown in fig. 2, the second embodiment of the present invention further provides a vehicle-mounted monitoring service management device, including:
the data acquisition module 210 is configured to acquire vehicle track data and road environment data.
The track analysis module 220 is configured to analyze a normal driving track according to historical track data in the vehicle track data, and analyze and predict a driving track according to current track data in the vehicle track data.
And the violation judging module 230 is configured to compare the predicted driving track with the normal driving track, and judge that the driver has a potential violation if the predicted driving track deviates from the normal driving track.
The risk analysis module 240 is configured to analyze a risk level of the potential violation according to the road environment data.
And the early warning sending module 250 is used for sending early warning to the driver according to the risk level.
According to the passenger management device based on the vehicle-mounted monitoring, the vehicle track data and the road environment data in the vehicle-mounted monitoring are obtained, the normal driving track, namely the set track of the vehicle driving under normal conditions, is analyzed according to the historical track data in the vehicle track data, and then the predicted driving track, namely the predicted track of the vehicle under the current driving trend, is analyzed according to the current track data in the vehicle track data. And comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track deviates from the normal driving track and the predicted driving track deviates from the normal track range. After determining that the potential violation exists, the risk level of the potential violation needs to be analyzed in combination with the current road environment data, i.e. the severity of the possible consequences under the current potential violation operation. And sending early warning to the driver according to the analyzed risk level, so that the driver pays attention to the corresponding situation, and correcting the potential illegal operation in time. Through the process, the method and the system realize analysis of the behavior trails of the drivers, discover potential illegal behaviors in time and early warn, realize fine supervision and intelligent passenger management of the behaviors of the vehicle-mounted drivers, and have important significance for improving the safety operation level of the vehicles.
Optionally, the road environment data includes: pedestrian data, traffic light data, and traffic flow data, the risk analysis module 240 specifically includes:
calculating collision risk probability of the predicted driving track according to the road environment data;
and determining the risk level of the potential violation according to the collision risk probability.
Optionally, the early warning sending module 250 specifically includes:
sending corresponding early warning to the driver according to the risk level, and recording response time of the driver to the early warning;
and under the condition that the potential violation is not corrected within the preset time, controlling the vehicle to trigger automatic braking intervention.
Optionally, after the risk analysis module 240, further includes: the fatigue detection module specifically comprises:
acquiring facial image data of a driver;
acquiring eye image data according to the face image data;
detecting the opening and closing time duty ratio of eyes according to the eye image data;
judging the fatigue degree of a driver according to the eye opening and closing time duty ratio;
and sending fatigue early warning to a driver according to the fatigue degree.
Optionally, the acquiring eye image data according to the face image data includes:
detecting eye key points of the face image data through a face detection model based on a computer vision library, and acquiring eye image data;
the detecting the eye opening and closing time duty ratio according to the eye image data comprises the following steps:
classifying an eye opening and closing state of the eye image data by a feature descriptor;
and calculating the ratio of the number of frames of the closed eyes in unit time according to the opening and closing states of the eyes to obtain the opening and closing time duty ratio of the eyes.
Optionally, after the early warning sending module 250, the method further includes: the portrait creation module specifically comprises:
acquiring vehicle running data and preprocessing the vehicle running data;
analyzing the vehicle running data to obtain vehicle speed characteristics;
acquiring and analyzing facial image data of a driver to obtain facial expression characteristics;
the method comprises the steps of acquiring and analyzing voice dialogue data of a driver to obtain sentence and mood characteristics;
inputting the vehicle speed characteristics, the facial expression characteristics and the sentence mood characteristics into a driver portrait model to obtain each dimension score of the driver portrait;
and establishing a driver portrait radar chart according to the dimension scores of the driver portrait.
The driver portrait model is trained based on multi-source feature samples comprising vehicle speed features, facial expression features and sentence mood features, and each dimension score of the driver portrait corresponding to the multi-source feature samples serving as a label.
Optionally, the vehicle driving data includes: vehicle-mounted tachograph data and CAN bus data, the vehicle-mounted tachograph data comprises: position coordinates, speed, oil consumption and steering angle, the CAN bus data comprises: a braking signal and an acceleration signal, the vehicle speed characteristics comprising: average velocity characteristics and acceleration characteristics;
the acquiring the vehicle running data, preprocessing the vehicle running data, includes:
acquiring vehicle driving data;
cleaning abnormal data in the vehicle running data, and smoothing a filtering running track;
detecting and filling data missing points in vehicle driving data;
and normalizing the source data with different formats in the vehicle driving data to a unified sample.
Optionally, the driver portrait model is a deep learning neural network comprising a convolution layer and a pooling layer, the facial expression features are obtained through analysis of the convolution neural network, and the sentence mood features are obtained through natural language processing analysis.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. The vehicle-mounted monitoring-based passenger management method is characterized by comprising the following steps of:
acquiring vehicle track data and road environment data;
analyzing a normal driving track according to historical track data in the vehicle track data, and analyzing and predicting a driving track according to current track data in the vehicle track data;
comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track is separated from the normal driving track;
analyzing the risk level of the potential violation according to the road environment data;
and sending early warning to a driver according to the risk level.
2. The vehicle-mounted monitoring-based passenger management method according to claim 1, wherein the road environment data comprises: pedestrian data, traffic light data, and traffic flow data, said analyzing a risk level of potential violations from said road environment data comprising:
calculating collision risk probability of the predicted driving track according to the road environment data;
and determining the risk level of the potential violation according to the collision risk probability.
3. The vehicle-mounted monitoring-based passenger management method according to claim 1, wherein the sending an early warning to the driver according to the risk level comprises:
sending corresponding early warning to the driver according to the risk level, and recording response time of the driver to the early warning;
and under the condition that the potential violation is not corrected within the preset time, controlling the vehicle to trigger automatic braking intervention.
4. The vehicle-mounted monitoring-based passenger management method according to claim 1, further comprising, after the analyzing the risk level of the potential violation according to the road environment data:
acquiring facial image data of a driver;
acquiring eye image data according to the face image data;
detecting the opening and closing time duty ratio of eyes according to the eye image data;
judging the fatigue degree of a driver according to the eye opening and closing time duty ratio;
and sending fatigue early warning to a driver according to the fatigue degree.
5. The vehicle-mounted monitoring-based passenger management method according to claim 4, wherein the acquiring eye image data from the face image data comprises:
detecting eye key points of the face image data through a face detection model based on a computer vision library, and acquiring eye image data;
the detecting the eye opening and closing time duty ratio according to the eye image data comprises the following steps:
classifying an eye opening and closing state of the eye image data by a feature descriptor;
and calculating the ratio of the number of frames of the closed eyes in unit time according to the opening and closing states of the eyes to obtain the opening and closing time duty ratio of the eyes.
6. The vehicle-mounted monitoring-based passenger management method according to claim 1, further comprising, after the sending of the early warning to the driver according to the risk level:
acquiring vehicle running data and preprocessing the vehicle running data;
analyzing the vehicle running data to obtain vehicle speed characteristics;
acquiring and analyzing facial image data of a driver to obtain facial expression characteristics;
the method comprises the steps of acquiring and analyzing voice dialogue data of a driver to obtain sentence and mood characteristics;
inputting the vehicle speed characteristics, the facial expression characteristics and the sentence mood characteristics into a driver portrait model to obtain each dimension score of the driver portrait;
and establishing a driver portrait radar chart according to the dimension scores of the driver portrait.
The driver portrait model is trained based on multi-source feature samples comprising vehicle speed features, facial expression features and sentence mood features, and each dimension score of the driver portrait corresponding to the multi-source feature samples serving as a label.
7. The vehicle-mounted monitoring-based passenger management method according to claim 6, wherein the vehicle travel data includes: vehicle-mounted tachograph data and CAN bus data, the vehicle-mounted tachograph data comprises: position coordinates, speed, oil consumption and steering angle, the CAN bus data comprises: a braking signal and an acceleration signal, the vehicle speed characteristics comprising: average velocity characteristics and acceleration characteristics;
the acquiring the vehicle running data, preprocessing the vehicle running data, includes:
acquiring vehicle driving data;
cleaning abnormal data in the vehicle running data, and smoothing a filtering running track;
detecting and filling data missing points in vehicle driving data;
and normalizing the source data with different formats in the vehicle driving data to a unified sample.
8. The vehicle-mounted monitoring-based passenger management method according to claim 6, wherein the driver portrait model is a deep learning neural network comprising a convolution layer and a pooling layer, the facial expression features are obtained through analysis of the convolution neural network, and the sentence mood features are obtained through natural language processing analysis.
9. The vehicle-mounted monitoring-based passenger management method according to any one of claims 1 to 8, wherein the normal driving track is analyzed through an LSTM network, and the predicted driving track is a vehicle track of 5 seconds in the future.
10. A vehicle-mounted monitoring-based passenger management device, comprising:
the data acquisition module is used for acquiring vehicle track data and road environment data;
the track analysis module is used for analyzing a normal driving track according to historical track data in the vehicle track data and analyzing and predicting a driving track according to current track data in the vehicle track data;
the violation judging module is used for comparing the predicted driving track with the normal driving track, and judging that the potential violation exists in the driver if the predicted driving track is separated from the normal driving track;
the risk analysis module is used for analyzing the risk level of the potential violation according to the road environment data;
and the early warning sending module is used for sending early warning to the driver according to the risk level.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311615489.9A CN117576859A (en) | 2023-11-29 | 2023-11-29 | Vehicle-mounted monitoring-based passenger management method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311615489.9A CN117576859A (en) | 2023-11-29 | 2023-11-29 | Vehicle-mounted monitoring-based passenger management method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117576859A true CN117576859A (en) | 2024-02-20 |
Family
ID=89860612
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311615489.9A Pending CN117576859A (en) | 2023-11-29 | 2023-11-29 | Vehicle-mounted monitoring-based passenger management method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117576859A (en) |
-
2023
- 2023-11-29 CN CN202311615489.9A patent/CN117576859A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220262239A1 (en) | Determining causation of traffic events and encouraging good driving behavior | |
| US20240149882A1 (en) | Multiple exposure event determination | |
| CN112700470B (en) | Target detection and track extraction method based on traffic video stream | |
| US11836985B2 (en) | Identifying suspicious entities using autonomous vehicles | |
| CN109345829B (en) | Unmanned vehicle monitoring method, device, equipment and storage medium | |
| CN110077398B (en) | Risk handling method for intelligent driving | |
| Lyu et al. | Using naturalistic driving data to identify driving style based on longitudinal driving operation conditions | |
| CN109572550A (en) | A kind of wheelpath prediction technique, system, computer equipment and storage medium | |
| CN112793576B (en) | Lane change decision method and system based on rule and machine learning fusion | |
| CN113055649A (en) | Tunnel intelligent video monitoring method and device, intelligent terminal and storage medium | |
| Bouhsissin et al. | Driver behavior classification: a systematic literature review | |
| Xue et al. | A context-aware framework for risky driving behavior evaluation based on trajectory data | |
| CN112528759A (en) | Traffic violation behavior detection method based on computer vision | |
| CN116824840A (en) | Campus vehicle driving management method and system based on Internet of Things | |
| CN111666859A (en) | Dangerous driving behavior identification method | |
| CN112435466B (en) | Method and system for predicting take-over time of CACC vehicle changing into traditional vehicle under mixed traffic flow environment | |
| CN114475656A (en) | Travel track prediction method, travel track prediction device, electronic device, and storage medium | |
| Harkous et al. | Application of hidden Markov model on car sensors for detecting drunk drivers | |
| CN115279643A (en) | On-board active learning method and apparatus for training a perception network of an autonomous vehicle | |
| Yarlagadda et al. | Heterogeneity in the driver behavior: an exploratory study using real-time driving data | |
| CN111027859A (en) | Driving risk prevention method and system based on motor vehicle state monitoring data mining | |
| CN117576859A (en) | Vehicle-mounted monitoring-based passenger management method and device | |
| Detzer et al. | Analysis of traffic safety for cyclists: The automatic detection of critical traffic situations for cyclists | |
| Bravi et al. | Detection of stop sign violations from dashcam data | |
| Yuan et al. | Predicting drivers’ eyes-off-road duration in different driving scenarios |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |