CN117574088A - Bus driver driving behavior early warning evaluation method, system and device - Google Patents
Bus driver driving behavior early warning evaluation method, system and device Download PDFInfo
- Publication number
- CN117574088A CN117574088A CN202311646112.XA CN202311646112A CN117574088A CN 117574088 A CN117574088 A CN 117574088A CN 202311646112 A CN202311646112 A CN 202311646112A CN 117574088 A CN117574088 A CN 117574088A
- Authority
- CN
- China
- Prior art keywords
- data
- behavior
- early warning
- driver
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 49
- 230000006399 behavior Effects 0.000 claims abstract description 100
- 238000000034 method Methods 0.000 claims abstract description 43
- 230000000295 complement effect Effects 0.000 claims abstract description 34
- 238000013135 deep learning Methods 0.000 claims abstract description 34
- 238000012549 training Methods 0.000 claims abstract description 28
- 230000002159 abnormal effect Effects 0.000 claims abstract description 24
- 238000007781 pre-processing Methods 0.000 claims abstract description 20
- 238000004140 cleaning Methods 0.000 claims abstract description 17
- 239000011159 matrix material Substances 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 22
- 230000006870 function Effects 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 17
- 230000015654 memory Effects 0.000 claims description 14
- 206010000117 Abnormal behaviour Diseases 0.000 claims description 11
- 230000001133 acceleration Effects 0.000 claims description 8
- 230000008859 change Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000013136 deep learning model Methods 0.000 claims description 6
- 230000014509 gene expression Effects 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000013178 mathematical model Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 description 19
- 238000005265 energy consumption Methods 0.000 description 13
- 238000004458 analytical method Methods 0.000 description 8
- 238000012217 deletion Methods 0.000 description 8
- 230000037430 deletion Effects 0.000 description 8
- 238000007726 management method Methods 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 6
- 230000006872 improvement Effects 0.000 description 5
- 230000002996 emotional effect Effects 0.000 description 4
- 239000000446 fuel Substances 0.000 description 4
- 206010039203 Road traffic accident Diseases 0.000 description 3
- 230000008921 facial expression Effects 0.000 description 3
- 230000007787 long-term memory Effects 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 2
- 230000000994 depressogenic effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000006403 short-term memory Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0809—Driver authorisation; Driver identical check
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to the technical field of traffic management, in particular to a method, a system and a device for early warning and evaluating driving behaviors of a bus driver. In the method, the data acquisition step mainly acquires vehicle running state data and driver state data. The data preprocessing step comprises data cleaning, data complement, map matching and travel division so as to ensure the quality and accuracy of the data. In the deep learning training step, a deep learning algorithm model consisting of LSTM, CNN and YOLOv5 is used for training the collected and preprocessed data, so that the model can learn and recognize the behaviors of a driver from the data. And the behavior recognition and early warning step is used for early warning the driver in time when the abnormal driving behavior is recognized. The behavior evaluation step may comprehensively evaluate the behavior of the driver. The method can early warn and evaluate the behavior of the driver accurately in real time, thereby improving the driving safety of the bus.
Description
Technical Field
The invention relates to the technical field of traffic management, in particular to a method, a system and a device for early warning and evaluating driving behaviors of a bus driver.
Background
Currently, in traffic accidents, the proportion of accidents caused by irregular behavior of bus drivers is much higher than that caused by vehicle faults. With the development of intelligent traffic, the use of digital technology to improve the operation safety and efficiency of traditional public transportation systems has become an important trend.
Although the existing systems can store abnormal behavior data for later playback and analysis, they fail to realize real-time extraction and early warning of potential safety hazards in vehicle driving, so that limitations exist in effectively improving the driving safety of buses. For example, the system disclosed in patent publication No. CN105898239a only transmits the abnormal behavior picture to the background and stores the number, and is only used for post-accident study and judgment, and its effect and practicability are limited. While the system disclosed in the patent publication number CN14943403a performs scoring evaluation on bad behaviors of a bus driver and uploads data, so that the bus driver is convenient to manage, the system cannot combine the expression of the driver to perform real-time early warning, so that the system is in a time stagnation in the aspect of correcting the error behaviors of the driver, and the evaluation is not comprehensive. Similarly, the system disclosed in patent publication CN110053627a only analyzes the driving technique and the surrounding environment during driving, does not consider the analysis of the facial expression of the driver, and also shows the limitations of the method.
Disclosure of Invention
The invention aims to provide a bus driver driving behavior early warning evaluation method, which can improve the driving safety performance of a bus and early warn dangerous driving behaviors in time.
The basic scheme provided by the invention is as follows: a bus driver driving behavior early warning evaluation method comprises the following steps: the method comprises a data acquisition step, a data preprocessing step, a deep learning training step, a behavior recognition and early warning step and a behavior evaluation step; the data acquisition step is used for acquiring vehicle running state data and driver state data; the data preprocessing step comprises data cleaning, data complement, map matching and travel division, wherein the data cleaning comprises deleting of abnormal data values, the data complement is used for processing missing data segments, and the map matching is used for identifying a normal operation route of a bus; the deep learning training step comprises training the collected and preprocessed data by adopting a deep learning algorithm model consisting of a long-term short-term memory network (LSTM), a Convolutional Neural Network (CNN) and a YOLOv 5; the behavior recognition and early warning step is based on a trained deep learning model, and vehicle driving data and vehicle-mounted images are analyzed in real time to recognize abnormal driving behaviors; the behavior recognition and early warning step further comprises timely early warning of a driver when abnormal behaviors are recognized; the behavior evaluation step is used for evaluating the behavior of the driver and sending the evaluation result to the management platform.
The invention has the realization principle and beneficial effects that: the data acquisition step is used for collecting vehicle running state data and providing a basis for subsequent behavior analysis. The data preprocessing step ensures the quality and the integrity of data through the steps of data cleaning, data complement, map matching, travel division and the like. The data cleaning can delete abnormal data, so that the accuracy of the data is ensured; the data complement is the problem of continuity of processing data, and ensures the integrity of the data; through map matching, it can be determined whether the driver is traveling along a predetermined route; the journey division sub-module divides the collected data into journey, and decomposes the whole driving process into a section of independent journey, thereby being more beneficial to fine management and evaluation of driving behaviors. The deep learning training step uses a deep learning algorithm model consisting of a long-term memory network (LSTM), a Convolutional Neural Network (CNN) and YOLOv5 to train the collected and preprocessed data. LSTM is able to handle time-series patterns of driving behavior; the CNN can extract visual characteristics in the driving process; YOLOv5 can detect key targets in the driving process in real time. Through the combined training of the three models, the driving behavior can be more comprehensively and accurately understood. The behavior recognition and early warning step and the behavior evaluation step can feed back the behavior evaluation of the driver in real time, and provide timely early warning for the driver.
Further, the data complement comprises a first complement strategy, a second complement strategy and a third complement strategy; in the first completion strategy, if the data missing time exceeds a set threshold value for missing data, deleting the data segment; if the data is lower than the threshold value, adopting a linear interpolation or cubic spline interpolation method to fill the data; in the second completion strategy, when the data segment accords with the sampling frequency and part of information is missing, the average speed calculation based on the front data point and the rear data point is used for completing speed data; if the front and rear data points show the vehicle state change, the data in the flameout state of the vehicle is adopted for complement; in the third completion strategy, the data is completed by using a lever method for the missing data of the states of the accelerator pedal and the brake pedal.
The beneficial effect of this scheme is: in the data complement stage, three different strategies are adopted to process different types of data missing, so that the data integrity is further ensured, uncertainty caused by the data missing is effectively processed, and the accuracy of an evaluation result is improved.
Further, LSTM in the deep learning algorithm model is used for processing time series data, and CNN is used for extracting features from the LSTM processed data.
The LSTM of the deep learning training step may be represented by the following formula:
wherein,for the input data at time t, < >>Representing a memory of information from a previous time, < >>Is composed of input data and unit state, wherein, < ->Input representing input gate,/->Input representing forget gate, < >>Input representing output gate and +.>An input representing a memory cell; />The outputs of the parts after different activation functions are respectively, wherein +.>Representing the output of the input gate,/->Output representing forgetting gate, < >>Output representing output gate and +.>Representing an output of the memory cell; />Respectively a weight matrix of the network, wherein +.>Weight matrix representing input gate relative to current input, +.>Weight matrix representing forgetting gate relative to current input, +.>Weight matrix and +.>Representing a weight matrix of the memory unit relative to the current input; />Weight matrix representing the state of the input gate relative to the cell at the previous time, < >>Weight matrix representing the state of the forgetting gate relative to the cell at the previous moment, < >>A weight matrix representing the state of the output gate relative to the cell at the previous time; />The final output of the module is expressed.
The deep learning training step further comprises importing the LSTM processed data into a CNN component and extracting features from the LSTM processed data, wherein a mathematical model of the CNN is expressed by the following functions:
wherein,for convolution result +.>And->Representing convolution kernels and offsets as parameters in the convolution process; />And->Is a simple expression of the pooling process.
The Yolov5 algorithm in the deep learning training step adopts a CIOU_Loss function, and the sum calculation method of the CIOU_Loss function is as follows:
CIOU_Loss=1-CIOU=1-(IOU-)
v=
wherein IOU (Intersection over Union) is used to measure the degree of overlap between two bounding boxes (typically rectangular boxes). It determines the degree of overlap by calculating the area of the intersection area of the two bounding boxes divided by the area of the union of the two bounding boxes. CIOU (Complete IOU) is a modified version of the IOU (Intersection over Union, cross-over). It takes into account not only the degree of overlap between the target frames, but also their differences in center point distance and size. The IOU has a value ranging from 0 to 1, where 0 indicates that the two bounding boxes do not overlap and 1 indicates that the two bounding boxes completely overlap. When the IOU is greater than a set threshold, it is generally considered a proper detection or match. Distance_c represents a coefficient for punishing the Distance of the center point of the box. Distance_2 represents the Euclidean Distance between two center points of the prediction box and the real box.A term for the aspect ratio of the penalty box is represented. />、/>、/>、/>Representing the width and height of the prediction and real frames, respectively.
The beneficial effect of this scheme is: LSTM and CNN are used to process time series data such as speed, acceleration, trajectory, steering angle, etc. during vehicle travel. YOLOv5 is mainly used for processing image data, such as driver images, pedestrian images and the like, and can accurately identify and locate key targets in the driving process. By using LSTM, CNN, and YOLOv5 in combination, the behavior of the driver can be accurately estimated.
And in the behavior evaluation step, the behavior of the driver is comprehensively evaluated according to different scoring indexes and weights.
The beneficial effect of this scheme is: in the behavior evaluation step, through a multi-index and multi-dimensional evaluation mode, the driving behavior and skill level of a driver can be reflected more comprehensively and accurately, key driving behaviors can be highlighted according to different scoring indexes and weights, driving problems can be found timely, and traffic accidents are prevented.
Drawings
FIG. 1 is a flow chart of a method for early warning and evaluating driving behavior of a bus driver;
FIG. 2 is a flow chart of vehicle operation data acquisition for a method for early warning and evaluating driving behavior of a bus driver according to the present invention;
fig. 3 is a schematic diagram of scoring index of a method for evaluating early warning of driving behavior of a bus driver in the invention.
Detailed Description
The following is a further detailed description of the embodiments:
example 1
The invention provides a bus driver behavior early warning evaluation method shown in fig. 1, which is beneficial to relieving traffic accidents caused by improper driving operation and reasonably evaluating bus drivers. The method comprises the following steps: a data acquisition step, a data preprocessing step, a deep learning training step and a behavior evaluation step.
The data acquisition step is used to acquire vehicle running state data, and fig. 2 illustrates a vehicle running data acquisition process in the present embodiment. In the process, the MCU main control board is connected to the CAN bus of the vehicle, and the integrated CAN processing chip acquires CAN bus data such as speed, acceleration, steering angle and the like in the driving process. Meanwhile, a GPS module of the NEO7N module is used as positioning equipment, real-time position information of the vehicle is continuously acquired, the parameters comprise latitude, longitude, altitude and the like, and the vehicle-mounted camera equipment is responsible for capturing real-time images in the running process of the vehicle, including facial expressions of a driver, and the images are used for image analysis and processing subsequently. After data acquisition, the MCU main control board integrates various data, and then the integrated 5G communication module is used for transmitting the data to the server terminal in real time. The high-speed and low-delay characteristics of the 5G network can ensure the real-time transmission requirement of a large amount of data. After the data is transmitted to the server, the server stores, processes and analyzes the received data, and the processes are used for tasks such as monitoring the state of the vehicle and evaluating the driving safety.
The data preprocessing step comprises data cleaning, data complement, map matching and journey division, wherein the data cleaning comprises deleting of abnormal data values, the data complement is used for processing missing data segments, and the map matching is used for identifying a normal operation route of a bus. The task of the data preprocessing module is to filter the data collected during the operation of the bus. The module can extract effective information in the actual running process by eliminating data deviation caused by environmental noise and instability factors, and divide the data into travel segments. In actual operation, data cleaning, data completion, map matching and travel division are not strictly performed in sequence, but flexibly performed alternately according to data characteristics and requirements.
The data cleaning part is used for deleting abnormal data values, wherein the abnormal values refer to abnormal values of the speed exceeding the highest speed per hour of the bus and abnormal values of the longitude and latitude deviating from the normal operation route of the bus. The abnormal value of the speed exceeding the highest speed of the bus is replaced by the highest speed of the bus, so that data deflection caused by the abnormal value can be avoided, and the authenticity of the data is ensured; meanwhile, the speed change condition of the bus can be reflected more accurately in subsequent analysis, and related analysis such as vehicle running efficiency, fuel consumption and the like can be facilitated.
And deleting the longitude and latitude abnormal values deviating from the normal operation route of the bus directly, wherein deleting refers to deleting all data fields where the abnormal values are located. Such deletion is primarily intended to exclude invalid data that may be generated due to equipment failure, signal interference, or unusual operation (e.g., temporary rerouting of a bus off-normal route). Through such washing, data analysis can be ensured to be more concentrated on a normal operation route.
The data complement strategy is to make up for blank or missing parts in the data, so as to maintain the data integrity and keep the information of the original data to the maximum extent. The data complement in this embodiment includes a first complement policy, a second complement policy, and a third complement policy.
In this embodiment, when the data complement is performed, it is necessary to determine the type of deletion and the time of deletion, where the type of deletion includes continuous deletion of time-series data and partial deletion of data content (such as speed or status information).
For the continuous deletion of time-series data, a first complementation strategy is used in the present embodiment. For missing data, if the data missing time exceeds a set threshold value, deleting the data segment, so as to avoid misleading analysis caused by insufficient information; if the data is lower than the threshold value, the data is filled by adopting a linear interpolation or cubic spline interpolation method, so that the trend of the missing data segment can be better simulated and presumed, and the continuity and consistency of the data are maintained.
For partial deletions of data content, a second complement strategy is used in this embodiment. In the second completion strategy, when the data segment accords with the sampling frequency but part of information is missing, the average speed calculation based on the front and rear adjacent data points is used for completing the speed data, so that the continuity and smoothness of the speed data can be maintained, and the influence of data jump on subsequent analysis is avoided. If the front data point and the rear data point show the state change of the vehicle, the data in the flameout state of the vehicle is adopted to complement, so that important information of the state change of the vehicle can be kept, even if the data is incomplete, the real state of the vehicle in the time period can be reflected as far as possible, for example, in the running process of a bus, the speed data of the bus cannot be acquired due to the fault or signal interference of GPS equipment in a period of time. During this time, the bus transitions from the driving state to the flameout state (possibly due to stop-to-stop or due to traffic congestion, etc.). If the average of the previous and subsequent data is used to pad, the important information of this state change may be lost. If the missing data is complemented with data at the time of vehicle flameout (i.e. speed 0), then the change of this state can be seen in the subsequent analysis.
If the data relates to a lack of accelerator pedal and brake pedal status, a third completion strategy is used in this embodiment. For missing data of accelerator pedal and brake pedal states, leverage is used for data complementation.
During the running of the vehicle, the driver typically does not simultaneously depress the accelerator pedal and the brake pedal. This behavior pattern can help address the problem of data loss. Specifically, if it is seen in the data that the accelerator pedal is depressed, it can be assumed that at that time the brake pedal is not depressed. And vice versa. The lever method fills the blank of data according to the mutual exclusivity between the operation of the accelerator pedal and the operation of the brake pedal, and the strategy can help to know the running state of the vehicle more accurately.
The map matching part in the data preprocessing step is used for identifying the normal operation route of the bus.
The map matching function is used for identifying whether the bus runs along a normal operation route or not, and meanwhile route deviation caused by factors such as charging, maintenance or accidents can be eliminated. The "normal operation route" refers to a one-way travel track of a bus from a start station to a stop station through all predetermined stations. The map matching mainly uses longitude and latitude values as references, and the vehicle speed is used as an auxiliary reference. Because the longitude and latitude displayed by the map software may have a certain deviation from the actual acquired longitude and latitude, a reasonable deviation range is set to match the position of the actual sampling point in the map. The deviation range is determined according to factors such as the length of the bus, the number of lanes and the width of the road, the length of the stop board and the like, so that the normal operation route of the bus can be effectively identified, and possible route deviation is eliminated.
Extracting features possibly related to abnormal behaviors of a driver from the preprocessed data, selecting the features, screening the features with the most influence on abnormal behavior detection, constructing a model by using machine learning algorithms such as deep learning and the like, training by using marked normal behavior and abnormal behavior data, and performing abnormal driving behaviors including: whether to drive smoothly, whether to drive tiredly, drive distraction, drive emotionally, whether to give away pedestrians, etc. And detecting the behavior of the driver in real time by using the trained model, and reminding the driver or taking other measures in time if abnormality is found.
In this embodiment, the deep learning training step includes training the collected and preprocessed data using a deep learning algorithm model composed of a long short term memory network (LSTM), a Convolutional Neural Network (CNN), and YOLOv 5. The LSTM is capable of efficiently processing time series data to capture dynamic changes during driving. The LSTM processed data is input into the CNN component, from which key features of driving behavior are extracted. By applying the YOLOv5 algorithm and adopting the ciou_loss function, key targets in the driving process can be accurately identified and located.
Wherein LSTM may be represented by the formula:
wherein,for the input data at time t, < >>The memory of the previous time information is shown, and the memory is mainly the preprocessed vehicle running time series data, including the speed transverse and longitudinal speed, the transverse and longitudinal acceleration, the steering angle and the like.
Is composed of input data and cell state, < >>Input representing input gate,/->Input representing forget gate, < >>Input representing output gate and +.>Representing the input of the memory cell. The input gate is responsible for processing the input of the current sequence position and combining the input data with the memory of the previous moment; the forgetting door controls whether to forget the state of the upper layer of hidden cells with a certain probability; updating the cell state by combining the forgotten old data with the input new data; the updated cell state is transferred outwards or to the next moment by means of the output gate.
The outputs of the parts after different activation functions are respectively, wherein +.>Representing the output of the input gate,/->Output representing forgetting gate, < >>Representing the output sum of the output gatesRepresenting an output of the memory cell; />Respectively a weight matrix of the network, wherein +.>Weight matrix representing input gate relative to current input, +.>Weight matrix representing forgetting gate relative to current input, +.>Weight matrix and +.>Representing a weight matrix of the memory unit relative to the current input; />Weight matrix representing the state of the input gate relative to the cell at the previous time, < >>Weight matrix representing the state of the forgetting gate relative to the cell at the previous moment, < >>A weight matrix representing the state of the output gate relative to the cell at the previous time; />The final output of the module is expressed.
The LSTM processed data is imported into the CNN component, and features are further extracted from the results. The mathematical model of CNN is expressed as a function of:
wherein the method comprises the steps ofFor convolution result +.>And->The convolution kernel and offset are represented as parameters in the convolution process. />And->Is a simple expression of the pooling process.
In the network structure of YOLOv5 algorithm, the Head output end is used for outputting the target detection result by the final detection part, and usually comprises a classification branch and a regression branch. Wherein the IoU _loss function is used to handle the overlap area between them. But it is not reflective that the distance loss function between two frames is not conductive when the prediction and GT frames do not intersect, or that the function does not distinguish between the intersection of two prediction frames when they are equal in size. In order to solve the two problems, the GIOU_Loss function is adopted, and the problem that the bounding boxes are not coincident is solved by increasing the measurement mode of the intersection scale. The dious_loss function is used to handle minimizing the normalized distance between the prediction and GT frames. The cious_loss function takes into account the dimension information of the aspect ratio of the bounding box on this basis. The specific calculation method is as follows.
CIOU_Loss=1-CIOU=1-(IOU-)
v=
Wherein IOU (Intersection over Union) is used to measure the degree of overlap between two bounding boxes (typically rectangular boxes). It determines the degree of overlap by calculating the area of the intersection area of the two bounding boxes divided by the area of the union of the two bounding boxes. CIOU (Complete IOU) is a modified version of the IOU (Intersection over Union, cross-over). It takes into account not only the degree of overlap between the target frames, but also their differences in center point distance and size. The IOU has a value ranging from 0 to 1, where 0 indicates that the two bounding boxes do not overlap and 1 indicates that the two bounding boxes completely overlap. When the IOU is greater than a set threshold, it is generally considered a proper detection or match. distance_C represents a penaltyThe coefficient of the frame center point distance. Distance_2 represents the Euclidean Distance between two center points of the prediction box and the real box.A term for the aspect ratio of the penalty box is represented. />、/>、/>、/>Representing the width and height of the prediction and real frames, respectively.
The model compares the data acquired in real time with the learned normal behavior and abnormal behavior characteristics, and if the input data does not accord with the normal behavior mode, the model judges that the abnormal behavior occurs. In order to improve the accuracy and the robustness of detection, the system combines a plurality of algorithms and models, and the abnormal behavior is determined through the combination of the algorithms, so that the risks of false alarm and false alarm missing are effectively reduced.
And the behavior evaluation step is based on the trained model, comprehensively evaluates the behavior of the driver according to different scoring indexes and weights, and sends the evaluation result to the management platform.
The scoring indexes in this embodiment are shown in fig. 3, and mainly include vehicle running stability, driver standardability, driving lane germanity and comprehensive energy consumption of driving.
The vehicle running stability is used to evaluate the stability of the vehicle while the driver is driving. The vehicle running stability also comprises various sub-indexes such as vehicle speed, acceleration, steering, braking times and the like. For example, excessive vehicle speed, abrupt acceleration or deceleration, excessive steering, and frequent braking may all cause the vehicle to run unstably.
Driver normalization is used to evaluate whether a driver is driving according to traffic regulations and regulations. Sub-indicators of driver normalization include whether the driver is driving fatigue, distracted, emotional, etc. For example, facial expressions may be used to assess whether a driver is tired or emotional.
The driver's road germanity is used to evaluate whether the driver has good moral style and social responsibility. Sub-indicators of driver's road germany include whether the driver is giving a pedestrian, whether the driver is traveling on a prescribed route, whether the driver is at a prescribed stop, etc. For example, by means of a car front camera and GPS positioning, it can be determined whether the driver complies with these regulations.
The integrated energy consumption of driving is used for evaluating the energy consumption in the driving process. Sub-indicators of the integrated energy consumption of driving include fuel consumption, energy consumption of auxiliary equipment, brake energy consumption and air resistance. For example, fuel consumption, electricity consumption, kinetic energy consumption by braking, and kinetic energy consumption by air resistance of a vehicle are all important factors for evaluating energy consumption. Through these sub-indices, the driving efficiency and environmental awareness of the driver can be evaluated.
Of course, the present embodiment further includes weight distribution of each evaluation index to reflect the relative importance of the index in the overall score. For example, if driver normalization is considered most important for overall evaluation, it may be assigned the greatest weight. Meanwhile, the vehicle running stability, the driver's moral and the driving comprehensive energy consumption are also the indexes of the evaluation, and the weights are allocated to the indexes according to the importance of the indexes in the overall evaluation. Similarly, weights may be further set for sub-indicators of driver standardability (such as fatigue driving, distraction driving, emotional driving, etc.), sub-indicators of vehicle running smoothness (such as vehicle speed, acceleration, steering, number of braking, etc.), sub-indicators of driver's moral (whether to give pedestrians, whether to run on a prescribed route, whether to stop on a prescribed stop, etc.), sub-indicators of driving integrated energy consumption (air resistance, fuel consumption, auxiliary equipment energy consumption, braking energy consumption, etc.), and these weights reflect the importance of each sub-indicator in each upper level indicator score.
The invention also provides a warning method for the driver at the vehicle terminal in an acousto-optic and electric mode. As shown in fig. 1, the embodiment further includes a speaker and a warning light. When abnormal driving behaviors are detected, the driver is warned in time through the loudspeaker and the warning lamp.
In addition, the speaker and warning lights may also be activated when the driver's overall score is below a set threshold. The warning light may provide a visual warning and the speaker may provide a voice warning, specifically indicating where the driver needs improvement. Meanwhile, certain sub-indexes (fatigue driving, emotional driving and the like) can be set with certain thresholds, and when the set thresholds are exceeded, the loudspeaker and the warning lamp can give an alarm.
The evaluation results are also visually presented to bus drivers and management personnel. Wherein the scoring results include overall scores, scores of various scoring indicators, records of abnormal behavior, and the like. Based on the scoring results, the bus company can provide feedback and improvement advice to the driver, helping the driver improve driving behavior and quality of service. Meanwhile, the system can also monitor the improvement condition of the driver and regularly re-score.
According to the invention, the vehicle running state is acquired, sent and received in real time through the vehicle-mounted terminal, and the information acquisition of the driver is more comprehensive through referencing more data sources. The system can identify abnormal driving behaviors through real-time analysis, and gives real-time early warning to a driver, so that the driving safety is improved. In addition, the scoring evaluation system not only considers single driving data, but also synthesizes a plurality of external factors, further reduces errors and improves scoring accuracy. Meanwhile, the invention combines the scoring evaluation system of the bus driver with the big data analysis technology to deeply mine and analyze a large amount of driver behavior data, so that not only can the rule and trend of the bus driver behavior be found, but also personalized improvement suggestions and training schemes can be provided for each driver, thereby improving the overall quality of bus service.
In addition, the invention also discloses a bus driver driving behavior early warning evaluation system and a bus driver driving behavior early warning evaluation device.
The bus driver driving behavior early warning and evaluating system comprises a data acquisition module, a data preprocessing module, a deep learning training module, a behavior recognition and early warning module and a behavior evaluating module; the data acquisition module is used for acquiring vehicle running state data; the data preprocessing module comprises a data cleaning sub-module, a data complement sub-module, a map matching sub-module and a travel dividing sub-module; the data cleaning submodule is used for deleting abnormal data values; the data complement submodule is used for processing missing data segments; the map matching submodule is used for identifying a normal operation route of the bus; the journey division submodule is used for carrying out journey division on collected data; the deep learning training module comprises a deep learning algorithm model combining a long-term memory network (LSTM), a Convolutional Neural Network (CNN) and a YOLOv5 for training the collected and preprocessed data; the behavior recognition and early warning module analyzes vehicle driving data and vehicle-mounted images in real time based on the trained deep learning model so as to recognize abnormal driving behaviors and early warn a driver in time; the behavior evaluation module is used for evaluating the behavior of the driver and sending the evaluation result to the management platform.
The bus driver driving behavior early warning and evaluating device comprises a data acquisition unit, a data preprocessing unit, a deep learning processing unit, a behavior recognition and early warning unit and a behavior evaluating unit; the data acquisition unit is arranged on the bus and is used for acquiring vehicle running state data, including speed, acceleration, steering angle and GPS positioning data; the data preprocessing unit is used for receiving the collected data and preprocessing the collected data, and comprises data cleaning, data complement, map matching and travel division; the deep learning processing unit integrates a long-term memory network (LSTM), a Convolutional Neural Network (CNN) and a YOLOv5 algorithm, and performs deep learning training on the preprocessed data; the behavior recognition and early warning unit analyzes vehicle driving data and vehicle-mounted images in real time based on the trained deep learning model so as to recognize abnormal driving behaviors and early warn a driver in time; the behavior evaluation unit is used for evaluating the behavior of the driver and sending the evaluation result to the management platform.
The foregoing is merely exemplary of the present invention, and the specific structures and features well known in the art are not described in any way herein, so that those skilled in the art will be able to ascertain all prior art in the field, and will not be able to ascertain any prior art to which this invention pertains, without the general knowledge of the skilled person in the field, before the application date or the priority date, to practice the present invention, with the ability of these skilled persons to perfect and practice this invention, with the help of the teachings of this application, with some typical known structures or methods not being the obstacle to the practice of this application by those skilled in the art. It should be noted that modifications and improvements can be made by those skilled in the art without departing from the structure of the present invention, and these should also be considered as the scope of the present invention, which does not affect the effect of the implementation of the present invention and the utility of the patent. The protection scope of the present application shall be subject to the content of the claims, and the description of the specific embodiments and the like in the specification can be used for explaining the content of the claims.
Claims (9)
1. The bus driver driving behavior early warning evaluation method is characterized by comprising the following steps of: the method comprises a data acquisition step, a data preprocessing step, a deep learning training step, a behavior recognition and early warning step and a behavior evaluation step; the data acquisition step is used for acquiring vehicle running state data and driver state data; the data preprocessing step comprises data cleaning, data complement, map matching and travel division, wherein the data cleaning comprises deleting of abnormal data values, the data complement is used for processing missing data segments, and the map matching is used for identifying a normal operation route of a bus; the deep learning training step comprises training acquired and preprocessed data by adopting a deep learning algorithm model consisting of LSTM, CNN and YOLOv 5; the behavior recognition and early warning step is based on a trained deep learning model, and vehicle driving data and vehicle-mounted images are analyzed in real time to recognize abnormal driving behaviors; the behavior recognition and early warning step further comprises timely early warning of a driver when abnormal behaviors are recognized; the behavior evaluation step is used for evaluating the behavior of the driver and sending the evaluation result to the management platform.
2. The bus driver driving behavior early warning evaluation method according to claim 1, characterized in that: the data complement comprises a first complement strategy, a second complement strategy and a third complement strategy; in the first completion strategy, if the data missing time exceeds a set threshold value for missing data, deleting the data segment; if the data is lower than the threshold value, adopting a linear interpolation or cubic spline interpolation method to fill the data; in the second completion strategy, when the data segment accords with the sampling frequency and part of information is missing, the average speed calculation based on the front data point and the rear data point is used for completing speed data; if the front and rear data points show the vehicle state change, the data in the flameout state of the vehicle is adopted for complement; in the third completion strategy, the data is completed by using a lever method for the missing data of the states of the accelerator pedal and the brake pedal.
3. The bus driver driving behavior early warning evaluation method according to claim 2, characterized in that: the LSTM in the deep learning algorithm model is used for processing time series data, and the CNN is used for extracting features from the LSTM processed data.
4. The bus driver driving behavior early warning evaluation method according to claim 3, characterized in that: the LSTM of the deep learning training step may be represented by the following formula:
wherein,for the input data at time t, < >>Representing a memory of information from a previous time, < >>Is composed of input data and unit state, wherein, < ->Input representing input gate,/->Input representing forget gate, < >>Input representing output gate and +.>An input representing a memory cell; />The outputs of the parts after different activation functions are respectively, wherein +.>Representing the output of the input gate,/->Representing the output of the forgetting gate,Output representing output gate and +.>Representing an output of the memory cell; />Respectively a weight matrix of the network, wherein +.>Weight matrix representing input gate relative to current input, +.>Weight matrix representing forgetting gate relative to current input, +.>Indicating the output gate relative to the currentWeight matrix of front inputs and +.>Representing a weight matrix of the memory unit relative to the current input; />Weight matrix representing the state of the input gate relative to the cell at the previous time, < >>Weight matrix representing the state of the forgetting gate relative to the cell at the previous moment, < >>A weight matrix representing the state of the output gate relative to the cell at the previous time;the final output of the module is expressed.
5. The bus driver driving behavior early warning evaluation method according to claim 1, characterized in that: the mathematical model of CNN in the deep learning training step is represented by the following function:
wherein,for convolution result +.>And->Representing convolution kernels and offsets as parameters in the convolution process; />Andis a simple expression of the pooling process.
6. The bus driver driving behavior early warning evaluation method according to claim 5, characterized in that: the Yolov5 algorithm in the deep learning training step adopts a CIOU_Loss function, and the sum calculation method of the CIOU_Loss function is as follows:
CIOU_Loss=1-CIOU=1-(IOU-)
v=
the IOU (Intersection over Union) is used for measuring the overlapping degree between two bounding boxes, the value range of the IOU is between 0 and 1, 0 indicates that the two bounding boxes are not overlapped, and 1 indicates that the two bounding boxes are completely overlapped; distance_C represents a coefficient for punishing the Distance of the center point of the frame; distance_2 represents the Euclidean Distance between two center points of the prediction frame and the real frame;an item representing the aspect ratio for the penalty box,/->Is a constant coefficient for adjusting the penalty weight for the aspect ratio difference; />、/>、/>、/>Representing the width and height of the prediction and real frames, respectively, wherein +.>、/>Representing the width and height of the real frame, respectively, < >>、/>Representing the width and height of the prediction frame, respectively.
7. The bus driver driving behavior early warning evaluation method according to claim 6, characterized in that: the behavior evaluation step comprises evaluating the behavior of the driver according to different scoring indexes and weights.
8. The bus driver driving behavior early warning and evaluating system is characterized by comprising a data acquisition module, a data preprocessing module, a deep learning training module, a behavior recognition and early warning module and a behavior evaluating module; the data acquisition module is used for acquiring vehicle running state data; the data preprocessing module comprises a data cleaning sub-module, a data complement sub-module, a map matching sub-module and a travel dividing sub-module; the data cleaning submodule is used for deleting abnormal data values; the data complement submodule is used for processing missing data segments; the map matching submodule is used for identifying a normal operation route of the bus; the journey division submodule is used for carrying out journey division on collected data; the deep learning training module comprises a deep learning algorithm model combining LSTM, CNN and YOLOv5 to train the collected and preprocessed data; the behavior recognition and early warning module analyzes vehicle driving data and vehicle-mounted images in real time based on the trained deep learning model so as to recognize abnormal driving behaviors and early warn a driver in time; the behavior evaluation module is used for evaluating the behavior of the driver and sending the evaluation result to the management platform.
9. The bus driver driving behavior early warning and evaluating device is characterized by comprising a data acquisition unit, a data preprocessing unit, a deep learning processing unit, a behavior recognition and early warning unit and a behavior evaluating unit; the data acquisition unit is arranged on the bus and is used for acquiring vehicle running state data, including speed, acceleration, steering angle and GPS positioning data; the data preprocessing unit is used for receiving the collected data and preprocessing the collected data, and comprises data cleaning, data complement, map matching and travel division; the deep learning processing unit integrates LSTM, CNN and YOLOv5 algorithm, and performs deep learning training on the preprocessed data; the behavior recognition and early warning unit analyzes vehicle driving data and vehicle-mounted images in real time based on the trained deep learning model so as to recognize abnormal driving behaviors and early warn a driver in time; the behavior evaluation unit is used for evaluating the behavior of the driver and sending the evaluation result to the management platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311646112.XA CN117574088A (en) | 2023-12-04 | 2023-12-04 | Bus driver driving behavior early warning evaluation method, system and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311646112.XA CN117574088A (en) | 2023-12-04 | 2023-12-04 | Bus driver driving behavior early warning evaluation method, system and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117574088A true CN117574088A (en) | 2024-02-20 |
Family
ID=89895450
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311646112.XA Pending CN117574088A (en) | 2023-12-04 | 2023-12-04 | Bus driver driving behavior early warning evaluation method, system and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117574088A (en) |
-
2023
- 2023-12-04 CN CN202311646112.XA patent/CN117574088A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3533681B1 (en) | Method for detecting safety of driving behavior, apparatus and storage medium | |
| CN107782564A (en) | A kind of automatic driving vehicle evaluation system and method | |
| CN110395258A (en) | Pavement state apparatus for predicting and pavement state estimating method | |
| CN107843440A (en) | A kind of automatic driving vehicle Performance Test System and method | |
| CN104730949A (en) | Affective user interface in an autonomous vehicle | |
| CN104859662A (en) | Fault handling in an autonomous vehicle | |
| CN110866677A (en) | Driver relative risk evaluation method based on benchmark analysis | |
| CN110120153A (en) | A kind of public transport drives accident risk assessment system and its method | |
| WO2010011806A1 (en) | System and method for evaluating and improving driving performance based on statistical feedback | |
| CN113609016B (en) | Method, device, equipment and medium for constructing automatic driving test scene of vehicle | |
| CN106340205A (en) | Traffic monitoring method and traffic monitoring apparatus | |
| CN115081508B (en) | Traffic running risk parallel simulation system based on traffic digital twin | |
| CN114821968B (en) | Intervention method, device and equipment for fatigue driving of motor car driver and readable storage medium | |
| CN115291515A (en) | Automatic driving simulation test system and evaluation method based on digital twinning | |
| Wu et al. | Clustering of several typical behavioral characteristics of commercial vehicle drivers based on GPS data mining: Case study of highways in China | |
| CN114771548A (en) | Data logging for advanced driver assistance system testing and verification | |
| CN112214530B (en) | Social driving behavior tracking evaluation method and related device | |
| CN117574088A (en) | Bus driver driving behavior early warning evaluation method, system and device | |
| CN115871682A (en) | Method and system for monitoring, evaluating and early warning safe driving behavior of driver | |
| CN113192340B (en) | Method, device, equipment and storage medium for identifying highway construction vehicles | |
| Bäumler et al. | Report on validation of the stochastic traffic simulation (Part B) | |
| Peng et al. | A Method for Vehicle Collision Risk Assessment through Inferring Driver's Braking Actions in Near-Crash Situations | |
| Wang | Evaluation of traffic speed control devices and its applications | |
| CN110956072B (en) | Driving skill training method based on big data analysis | |
| Sun et al. | A recognition model of driving risk based on belief rule-base methodology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |