CN116401620A - Real-time evaluation method for pressure load of driver based on city street view image - Google Patents

Real-time evaluation method for pressure load of driver based on city street view image Download PDF

Info

Publication number
CN116401620A
CN116401620A CN202310333102.4A CN202310333102A CN116401620A CN 116401620 A CN116401620 A CN 116401620A CN 202310333102 A CN202310333102 A CN 202310333102A CN 116401620 A CN116401620 A CN 116401620A
Authority
CN
China
Prior art keywords
sample
driver
real
pressure load
pressure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310333102.4A
Other languages
Chinese (zh)
Inventor
何杰
叶云涛
吴晓雨
张长健
秦鹏程
柏春广
严欣彤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN202310333102.4A priority Critical patent/CN116401620A/en
Publication of CN116401620A publication Critical patent/CN116401620A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention discloses a real-time evaluation method for pressure load of a driver based on a city street view image, which comprises the following steps: constructing a data set containing U samples; determining the pressure level of the sample by using the heart rate variability index; then extracting a street view image element characteristic variable and a dynamic parameter characteristic variable in sample video data, and forming a sample characteristic vector; adopting a correlation analysis and random forest importance analysis method to reduce the dimension of the sample feature vector, and converting or eliminating feature variables with strong correlation; the pressure grade is used as a label, a feature-label matrix is formed by the pressure grade and the sample feature vector, the feature-label matrix is used for training a fusion model comprising m machine learning classifiers, and the fusion model with the best performance is used for real-time evaluation of the pressure load of a driver. The method has higher evaluation precision, and the real-time evaluation process has almost no influence on the driving experience of the driver, so that the feasibility of popularization and application is ensured.

Description

Real-time evaluation method for pressure load of driver based on city street view image
Technical Field
The invention belongs to the technical field of urban road traffic safety, and particularly relates to a real-time evaluation method for pressure load of a driver based on urban street view images.
Background
Road traffic accidents are pain points which obstruct the development of the traffic transportation industry, and current researches show that driver factors are core reasons for the traffic accidents, particularly bad driving states in the driving process, and are main reasons for the accidents, wherein the influence on safety by driving pressure load is most remarkable. Therefore, the real-time, efficient and accurate assessment of the pressure load of the driver is of great significance in reducing the accident risk. At present, the research on the pressure load of a driver at home and abroad mainly comprises two types:
(1) The first category evaluates the stress state of a driver when encountering different driving scenarios based on the driver's subjective intention. The method is mainly developed based on a questionnaire form, personal information of a driver is collected in the questionnaire, meanwhile, a typical driving scene question is designed, the driver scores the pressure load value of the driving scene based on personal subjective feeling, and finally, the relation between the driving scene and the pressure load is obtained according to the questionnaire analysis result. The disadvantage of such methods is that the scene that can be covered is limited and is completely governed by the subjective intention of the individual, and the reliability and reference significance of the results are insufficient.
(2) The second category is based on the psychological parameter index of the driver, and data is collected by using a driving simulator or a real vehicle driving experiment mode. Such methods determine the stress level of the driver by asking the driver's head to answer a stress questionnaire during driving, and then utilize advanced models or algorithms to evaluate the driving stress load via the driver psychophysiological parameters. The method can evaluate the driving pressure load more objectively and accurately, and fully proves the close relation between the psychological parameter index and the driving pressure load. However, depending on the psychological parameter index of the driver, if the parameter index is applied to the actual driving process, the invasion and interference to the driver cannot be avoided, and the driving safety is affected.
The invention with the publication number of CN114852088A discloses a driver assistance system and a method for identifying and warning dangerous and fatigue driving behaviors, which do not need to collect normal driving characteristics and known dangerous driving characteristics or train a detection model, reduce the implementation cost and have more universality; the three factors of road condition, driver state and vehicle driving characteristics are combined, and the self-adaptive adjustment mechanism is provided, so that the driving behavior judgment is more comprehensive, the reference value is more provided, and the detection is more accurate and effective; the fatigue driving behavior detection which is difficult to discover is considered, and the driver can be effectively assisted to carry out safe driving; the driver is small in interference, traffic accidents caused by dangerous and fatigue driving are effectively avoided, the driver is helped to find out the slowly-changing driving behavior, and the driver is enabled to realize the dangerous in time when the driving level and the decision level of the driver are reduced, so that the driver can be effectively assisted to drive safely. However, the invention mainly collects various factors causing dangerous driving, fuses the factors through a fuzzy reasoning algorithm to obtain whether dangerous driving occurs, does not involve quantification processing of the pressure load of the driver, and is difficult to obtain an objective conclusion of the pressure load of the driver through simple fusion of various factors.
The invention with publication number CN107145835A discloses a driver load detection vehicle-mounted device based on image recognition, which consists of an acquisition unit, an image processing unit, a road linear load grade calculation unit, a visual load grade calculation unit, a display unit and a storage unit. The display unit is used for receiving the data of the road linear load level calculation unit and the visual load level calculation unit, quantitatively fusing the data, displaying the comprehensive driving load of the road environment in real time, and displaying the systematic running state. The invention combines the visual load of the driver and the mental load caused by the linearity of the road, and displays the comprehensive driving load of the driver, so that the comprehensive driving load can be analyzed by a management department and a design department. However, in the invention, roads are divided into two types according to road lines, driving load degrees are evaluated according to heart rate variability, and then visual loads of drivers and mental loads caused by the road lines are fused, so that no effective solution is provided in the article on how to analyze or introduce other environmental factors. In addition, the invention only uses one time domain index, namely the standard deviation of heart rate sampling interval when evaluating the driving load, and single index quantization of the driving load can lead to one-sided performance of the result, so that a quantization method of a plurality of indexes is necessary to be explored. In addition, the level of the visual load is obtained by calculating the information quantity of the element, belongs to a static division method, and ignores the dynamic characteristic of the vision.
The invention with publication number CN115035687A discloses a driver fatigue state monitoring system based on seat pressure analysis, which consists of a seat pressure sensing monitoring unit, a sensing monitoring data analysis unit, a vehicle control unit and an alarm prompting unit. The system mainly judges the fatigue state of the driver through analyzing the dynamic frequency of the driver and the multi-point pressure distribution condition of the seat, controls the running state of the vehicle according to the judging result, and simultaneously sends an alarm voice prompt. The invention integrates fatigue detection, vehicle control and voice alarm, can be distributed on a physical medium through a program code, is used as a device for implementing functions, has higher practicability, and can be applied to large-scale safety guarantee of operating vehicles. However, the invention only carries out vehicle control through the fatigue state of the driver, lacks consideration of the actual condition of the road scene, and does not consider the coupling influence of the pressure load and the fatigue state.
Disclosure of Invention
The technical problems to be solved are as follows: the invention discloses a real-time evaluation method for the pressure load of a driver based on a city street view image, which fully considers the influence of street view image elements and dynamic parameters on the pressure load of the driver, has higher evaluation precision, has almost no influence on the driving experience of the driver in the real-time evaluation process, and ensures the feasibility of popularization and application.
The technical scheme is as follows:
the method for evaluating the pressure load of the driver in real time based on the city street view image comprises the following steps:
s10, carrying out a real-vehicle driving experiment on a target urban road, matching three types of data on a time sequence according to the collection frequency of the data of the heart rate of a driver, the dynamics of the vehicle and the street view image, and each interval t 1 Second extract 1 t 2 Samples of second length, constructing a dataset comprising U samples;
s20, carrying out time domain analysis and frequency domain analysis on heart rate data of each sample, extracting heart rate variability indexes according to the interval between two adjacent heartbeats, and determining the pressure grade of the sample by utilizing the heart rate variability indexes; then extracting a street view image element characteristic variable and a dynamic parameter characteristic variable in sample video data, and forming a sample characteristic vector;
s30, adopting a correlation analysis and random forest importance analysis method to reduce the dimension of the sample feature vector, converting or eliminating feature variables with strong correlation, and finally keeping the importance not lower than an importance threshold I o R characteristic variables of (2);
s40, taking the pressure grade as a label, forming a characteristic-label matrix with the sample characteristic vector, training a fusion model comprising m machine learning classifiers, and using the fusion model with the best performance for real-time evaluation of the pressure load of a driver.
Further, in step S10, the target urban road includes an urban main road or secondary main road scene with large traffic flow, an urban secondary branch road section with serious mixing, and an intersection with frequent vehicle interaction.
Further, in step S20, the heart rate variability index includes: average RR interval mRR; RR interval range eRR; RR interval standard deviation SDRR; root mean square value RMSSD of adjacent RR interval difference values; adjacent RR interval differences are lower than T d Millisecond ratio pNNT d The method comprises the steps of carrying out a first treatment on the surface of the Low-frequency energy LF is more than or equal to 0.04 and less than 0.15Hz; HF with high frequency energy of 0.15-0<0.40Hz; (3) the low-frequency energy proportion LFnu,
Figure BDA0004155541880000031
high frequency energy ratio HFnu, < ->
Figure BDA0004155541880000032
(5) Low high frequency energy ratio LF/HF.
Further, the process of determining the pressure level of the sample using the heart rate variability index includes:
performing factor analysis on heart rate variability indexes, reserving N main factors with characteristic values larger than 1, and obtaining the score F of each main factor N of the sample i in And variance contribution pn, calculating pressure index SI of sample i by using the following formula i ：
Figure BDA0004155541880000033
Where n=1, 2,.. i The larger the pressure is, the higher the pressure is, and the pressure index SI is obtained by using the K-means clustering algorithm i And performing unsupervised clustering to obtain samples with high, medium or low pressure levels.
Further, the process of extracting the street view image element characteristic variable in the sample video data comprises the following steps:
taking a picture from video data at intervals of delta t, obtaining K pictures from each sample,
Figure BDA0004155541880000034
performing element semantic segmentation on the image by using an image analysis model, and calculating the duty ratio of each element:
Figure BDA0004155541880000035
wherein prop is is The s-th picture element duty ratio representing the j-th picture of sample i, pixel is Pixel count representing the s-th picture element of sample i, j-th picture, pixel total Representing the total pixel number of the j-th picture of the sample i;
calculating the change rate of two adjacent picture elements:
Figure BDA0004155541880000041
wherein df_prop ijs The change rate of the s-th image element of the j-th picture and the j+1-th picture of the sample i;
calculating to obtain corresponding street view element characteristic variables:
Figure BDA0004155541880000042
Figure BDA0004155541880000043
Figure BDA0004155541880000044
Figure BDA0004155541880000045
wherein Mi s Mean value, SD, of element s ratio in sample i is Represents the standard deviation, MD, of the ratio of the element s in the sample i is Mean value representing change rate of element s of sample i, SDD is The standard deviation of the rate of change of the element s of the sample i is shown.
Further, the process of extracting the dynamic parameter characteristic variable in the sample video data comprises the following steps:
calculating the average value and standard deviation of each variable of the sample vehicle speed, the triaxial acceleration, the triaxial angular speed, the pitch angle, the roll angle and the yaw angle variation value as micro dynamics characteristic variables;
the calculation formula of the yaw angle change value is as follows:
Y＝Y t+1 -Y t
wherein Y represents a yaw angle variation value, Y t+1 And Y t The yaw angles of the vehicle at times t+1 and t are shown, respectively.
Further, in step S30, spearman rank correlation test is performed on the street view element feature variable and the dynamic parameter feature variable, and feature dimension reduction is performed by adopting a means of calculating the importance of the error outside the random forest bag;
the calculation formula of the rank correlation coefficient is:
Figure BDA0004155541880000046
wherein ρ is the inter-variable rank correlation coefficient, U is the total number of samples, d θ Representing rank differences between variables;
the importance of the feature variable is calculated by adopting the following formula:
Figure BDA0004155541880000047
wherein I is f Representing the importance of the feature f, dt represents the number of decision trees,
Figure BDA0004155541880000048
representing the error in prediction using all feature variables, +.>
Figure BDA0004155541880000051
The error in prediction using all the characteristic variables except the variable f is represented.
Further, in step S40, the process of using the pressure level as a label, forming a feature-label matrix with the sample feature vector, and using the fusion model with the best performance for real-time assessment of the pressure load of the driver includes the following steps:
carrying out rebalancing treatment on the sample data set after dimension reduction by adopting a synthetic minority class oversampling technology, and training a decision tree, a multi-element logic regression, a gradient lifting tree, a limit gradient lifting tree, a class gradient lifting tree and a light gradient lifting tree by utilizing the rebalancing data set;
according to the training result of a single classifier, adopting a stacking method to take each classifier as a meta classifier, taking other classifiers as base classifiers to perform structural design of fusion models one by one, and reserving the fusion models with G-means and F1 scores more than 80%;
and (3) further optimizing the performance of the reserved fusion model by adopting an Adaboost data set resampling technology, performing final performance evaluation on the fusion model, and using the fusion model with the best performance for real-time evaluation of the pressure load of the driver.
The beneficial effects are that:
firstly, the real-time evaluation method for the pressure load of the driver based on the city street view image avoids the excessive influence of subjective factors, calibrates the pressure grade through objective indexes, comprehensively considers the influence of street view image elements and dynamic parameters on the pressure load of the driver, and adopts a fusion model with higher evaluation precision and efficiency.
Secondly, according to the real-time evaluation method for the pressure load of the driver based on the city street view image, the environmental element information in the street view image is accurately extracted through means of image semantic segmentation and numerical analysis, and a new thought for evaluating the driving pressure from the visual perception angle of the driver is provided;
the method for evaluating the pressure load of the driver in real time based on the city street view image fully confirms the remarkable correlation between the street view image and the dynamic parameters and the pressure load of the driver through a fusion model, realizes the real-time evaluation of the pressure load of the driver without invading the driving process, and ensures the feasibility of popularization and application of the method.
Drawings
FIG. 1 is a schematic diagram of a data acquisition area segment in an example;
FIG. 2 is a schematic diagram of a data acquisition device for a real vehicle test and a vehicle triaxial;
FIG. 3 is a flow chart of calibrating a driver pressure level using an HRV indicator in accordance with an embodiment of the present invention;
FIG. 4 is a schematic diagram of the results after feature dimension reduction;
FIG. 5 is a modeling flow diagram of a fusion model;
fig. 6 is a flowchart of a method for evaluating the pressure load of a driver in real time based on a city street image according to an embodiment of the present invention.
Detailed Description
The following examples will provide those skilled in the art with a more complete understanding of the invention, but are not intended to limit the invention in any way.
The embodiment of the invention discloses a real-time evaluation method for pressure load of a driver based on a city street view image, which comprises the following steps:
s10, carrying out actual driving experiments on the target urban road, matching according to the acquisition frequencies of three types of data and time series, and carrying out t at each interval 1 Second extract 1 t 2 Samples of second length, constructing a dataset comprising U samples;
s20, carrying out time domain analysis and frequency domain analysis on heart rate data of each sample, extracting heart rate variability (Heart rate variability, HRV) indexes according to adjacent two heart beat intervals (RR intervals), and determining the pressure grade of the sample by using the HRV indexes; extracting a street view image element characteristic variable and a dynamic parameter characteristic variable in sample video data, and constructing a characteristic vector;
s30, adopting a correlation analysis and random forest importance analysis method to reduce the dimension of the feature vector, combining or eliminating feature variables with strong correlation, and finally keeping the importance not lower than 1 o R characteristic variables of (2);
s40, taking the pressure grade as a label, forming a characteristic-label matrix with the characteristic vector, and training a fusion model comprising m machine learning classifiers to obtain a fusion model frame with the best performance, and carrying out real-time evaluation on the pressure load of a driver.
Specifically, the HRV index extracted per sample includes:
time domain index: (1) average RR interval (mRR, units ms); (2) RR interval is extremely poor (eRR, units ms); (3) RR interval standard deviation (SDRR, units ms); (4) root mean square value (RMSSD, unit ms) of adjacent RR interval differences; (5) adjacent RR interval differences are lower than T d Millisecond ratio (pNNT) d Unit%);
frequency domain index: (1) low frequency energy (LF, 0.04. Ltoreq.LF)<0.15Hz, unit ms 2); (2) high frequency energy (HF, 0.15.ltoreq.HF)<0.40Hz, unit ms 2); (3) the low frequency energy ratio (LFnu,
Figure BDA0004155541880000061
unit%); (4) high frequency energy ratio (HFnu, ">
Figure BDA0004155541880000062
Unit%); (5) low high frequency energy ratio (LF/HF).
For calibrating the pressure level of a sample based on the HRV index, firstly, extracting n main factors with characteristic values larger than 1 from the HRV index by using a verification factor analysis method, and obtaining a factor n score F of the sample i in Sum factor n variance contribution p n Further calculate the pressure index SI of the sample i i ：
Figure BDA0004155541880000063
SI i The larger the pressure is, the higher the pressure is, and the pressure index SI is obtained by using the K-means clustering algorithm i And performing unsupervised clustering, and calibrating the pressure level of the sample to be high, medium or low.
In order to extract driving environment element information in a street view image, the method for extracting the street view image element feature variable, which is provided by the embodiment of the invention, comprises the following steps:
taking a picture from video data at intervals of delta t, and obtaining K pictures from each sample
Figure BDA0004155541880000064
Performing G-type element semantic segmentation on the image by using an image analysis model, and calculating the duty ratio of each element:
Figure BDA0004155541880000071
wherein prop is is The s-th picture element duty ratio representing the j-th picture of sample i, pixel is Pixel count representing the s-th picture element of sample i, j-th picture, pixel total The total number of pixels of the j-th picture of sample i is represented.
And then calculating the change rate of two adjacent picture elements:
Figure BDA0004155541880000072
wherein df_propi js The change rate of the s-th image element of the j-th picture and the j+1-th picture of the sample i is represented.
The street view element feature variables that can be calculated from this are as follows:
Figure BDA0004155541880000073
Figure BDA0004155541880000074
Figure BDA0004155541880000075
Figure BDA0004155541880000076
wherein M is is Is the average value of the ratio of the element s in the sample i, SD is Represents the standard deviation, MD, of the ratio of the element s in the sample i is Mean value representing change rate of element s of sample i, SDD is The standard deviation of the rate of change of the element s of the sample i is shown.
The extraction method of the dynamic parameter characteristic variables is as follows.
In order to ensure that the yaw angle of the vehicle accurately reflects the steering condition of the vehicle, the following difference value processing is carried out:
Y＝Y t+1 -Y t
wherein Y is a yaw angle variation value, Y t+1 And Y t The yaw angles of the vehicle at times t+1 and t are shown, respectively.
And then calculating the mean value and standard deviation of each variable of the sample vehicle speed, the triaxial acceleration, the triaxial angular speed, the pitch angle, the roll angle and the yaw angle variation value as the micro-dynamics characteristic variable.
On the other hand, the embodiment also discloses a method for evaluating the pressure load of the driver by applying the extracted characteristic variable, which comprises the following steps:
adopting a data set rebalancing technology and a data set diversification technology to improve the evaluation performance and generalization capability of the fusion model;
and training a fusion model according to the feature-label matrix, comparing performance indexes (G-mean and F1 scores) of different classifier combinations to obtain an optimal fusion model frame, and carrying out real-time assessment on the pressure load of the driver by using the optimal fusion model frame.
This method will be described in detail below by taking a section of a district of the city Jiang Ningou of Nanjing as an example. As shown in fig. 1, the selected section belongs to the region under Jiang Ning in south kyo city, and three typical urban road driving scenes are covered in the region, which are respectively:
(1) Urban arterial or secondary arterial road scenes with greater traffic (as in points D1 and D2 of fig. 1);
(2) Urban minor branch sections with serious mixing (as points B1 and B2 in fig. 1);
(3) Intersections with frequent vehicle interactions (e.g., points C1 and C2 of fig. 1).
Fig. 1 is only for showing the target road section, and the text contents such as place names in the drawing are not focused on the protection of the present application.
In this example, an actual driving experiment is performed in a research area, as shown in fig. 2, the experimental apparatus and the vehicle triaxial indication mainly collect the heart rate data of the driver, the micro dynamics parameter data of the vehicle and the street view video data in front of the vehicle, and referring to fig. 6, the disclosed real-time evaluation method for the pressure load of the driver comprises the following main steps:
in step S10, the three types of data are matched according to the order of time sequence, and in consideration of the shortest meaningful period of the HRV index being 100 seconds, 1 sample of 100 seconds is extracted every 10 seconds for the example data set, and 5274 samples are acquired in total.
Step S20, flow is as shown in FIG. 3, according to [0012 ]]-[0014]The extracted HRV index, wherein T d 20 and 50, respectively, pNN20 and pNN50, were taken and the index was then subjected to a validation factor analysis, and the results are shown in Table 1.
TABLE 1 validation factor analysis results
Figure BDA0004155541880000081
Calculating pressure indexes of 5274 samples respectively based on the results of the validation factor analysis, and calculating pressure index SI of sample i i The method comprises the following steps:
Figure BDA0004155541880000091
wherein F is in Factor n score, p, representing the i-th sample n The variance contribution of the factor n is represented.
Pressure index SI using K-means clustering algorithm i Unsupervised clustering was performed to obtain 168 high pressure samples, 4162 medium pressure samples and 944 low pressure samples.
Step S30, 1 street view image is taken from the street view video in front of the vehicle every 1 second, the image analysis model selected in this example is a deep labv3 model, the open source street view image dataset cinthscape is used for training the deep labv3 model, the number of pixels of the g=19 type elements (roads, sidewalks, buildings, walls, guardrails, poles, signal lamps, signboards, vegetation, topography, sky, pedestrians, chess, sedans, trucks, buses, trains, motorcycles, bicycles) are extracted from the sample street view image data, and the ratio of each element is calculated:
Figure BDA0004155541880000092
wherein prop is is The s-th picture element duty ratio representing the j-th picture of sample i, pixel is Pixel count representing the s-th picture element of sample i, j-th picture, pixel total The total number of pixels of the j-th picture of sample i is represented.
And then calculating the change rate of two adjacent picture elements:
Figure BDA0004155541880000093
wherein df_prop ijs The change rate of the s-th image element of the j-th picture and the j+1-th picture of the sample i is represented.
The street view element feature variables that can be calculated from this are as follows:
Figure BDA0004155541880000094
Figure BDA0004155541880000095
Figure BDA0004155541880000096
Figure BDA0004155541880000097
wherein M is is Mean value, SD, of element s ratio in sample i is Represents the standard deviation, MD, of the ratio of the element s in the sample i is Mean value representing change rate of element s of sample i, SDD is The standard deviation of the rate of change of the element s of the sample i is represented, K represents the total number of pictures of the sample i, in this example k=100.
Step S40, extracting characteristic variables of dynamic parameters of a sample vehicle, and firstly, performing difference processing on a yaw angle:
Y＝Y t+1 -Y t
wherein Y is a yaw angle variation value, Y t+1 And Y t The yaw angles of the vehicle at times t+1 and t are shown, respectively.
The mean and standard deviation of each variable of the sample vehicle speed, triaxial acceleration, triaxial angular velocity, pitch angle, roll angle and yaw angle variation values were then calculated as micro-dynamic characteristic variables, and the dynamic parameter characteristic variables are shown in table 2.
TABLE 2 vehicle dynamics parameters characteristic variables
Figure BDA0004155541880000101
And S50, carrying out Spearman rank correlation test on the street view element characteristic variable and the dynamic parameter characteristic variable, and carrying out characteristic dimension reduction by adopting a random forest out-of-bag error importance calculation method. The rank correlation coefficient is calculated as follows:
Figure BDA0004155541880000102
where ρ is the inter-variable rank correlation coefficient, n is the total number of samples, i.e., n=5274, d i Representing rank differences between variables
The importance of the feature variables is calculated as follows:
Figure BDA0004155541880000103
wherein I is f Representing the importance of the feature f, dt representing the number of decision trees, errOOB i1 Representing the error in prediction using all feature variables, errOOB i2 The error in prediction using all the characteristic variables except the variable f is represented.
The feature variable is a sudden importance drop at a portion smaller than 0.04, so this example I 0 Taking 0.04, the feature variable remained after feature dimension reduction is shown in fig. 4.
In step S60, the construction flow of the fusion model is shown in fig. 5. And (3) carrying out rebalancing treatment on the sample data set after the dimension reduction by adopting a synthetic minority class oversampling technology (SMOTE). Decision Trees (DTs), multiple Logic Regression (MLR), gradient boost trees (GBDTs), limiting gradient boost trees (XGBoost), class-type gradient boost trees (Catboost), and lightweight gradient boost trees (LightGBM) are trained using the rebalanced data sets.
According to the training result of a single classifier, a Stacking method (Stacking) is adopted to take each classifier as a meta classifier, other classifiers are taken as base classifiers to carry out structural design of fusion models one by one, and the fusion models with G-average and F1 scores more than 80% are reserved.
In step S70, the data set resampling technology of Adaboost is adopted to further optimize the performance of the reserved fusion model, and the final performance evaluation result of the fusion model is shown in table 3.
Table 3 results of performance comparisons of fusion models
Figure BDA0004155541880000111
Figure BDA0004155541880000121
Thus, the fusion model obtained in this embodiment, which uses GBDT, XGBoost, catboost and LightGBM as the base classifier and DT as the meta classifier, can be used to perform real-time driver pressure load assessment.
The foregoing is merely a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all other embodiments obtained by those skilled in the art without making any inventive effort based on the embodiments of the present invention are within the scope of the present invention; any simple modification or equivalent variation of the above embodiments falls within the scope of the present invention.

Claims (8)

1. The real-time evaluation method for the pressure load of the driver based on the city street images is characterized by comprising the following steps of:
s10, carrying out a real-vehicle driving experiment on a target urban road, matching three types of data on a time sequence according to the collection frequency of the data of the heart rate of a driver, the dynamics of the vehicle and the street view image, and each interval t 1 Second extract 1 t 2 Samples of second length, constructing a dataset comprising U samples;
s20, carrying out time domain analysis and frequency domain analysis on heart rate data of each sample, extracting heart rate variability indexes according to the interval between two adjacent heartbeats, and determining the pressure grade of the sample by utilizing the heart rate variability indexes; then extracting a street view image element characteristic variable and a dynamic parameter characteristic variable in sample video data, and forming a sample characteristic vector;
s30, adopting a correlation analysis and random forest importance analysis method to reduce the dimension of the sample feature vector, converting or eliminating feature variables with strong correlation, and finally keeping the importance not lower than an importance threshold I o R characteristic variables of (2);
s40, taking the pressure grade as a label, forming a characteristic-label matrix with the sample characteristic vector, training a fusion model comprising m machine learning classifiers, and using the fusion model with the best performance for real-time evaluation of the pressure load of a driver.
2. The method for real-time evaluation of driver' S pressure load based on city street images according to claim 1, wherein in step S10, the target city road includes a major road or minor road scene with large traffic flow, a major road and minor road section with serious mixing, and an intersection with frequent vehicle interactions.
3. The method for real-time assessment of driver' S pressure load based on city street images according to claim 1, wherein in step S20, the heart rate variability index comprises: average RR interval mRR; RR interval range eRR; RR interval standard deviation SDRR; root mean square value RMSSD of adjacent RR interval difference values; adjacent RR interval differences are lower than T d Millisecond ratio pNNT d The method comprises the steps of carrying out a first treatment on the surface of the Low-frequency energy LF is more than or equal to 0.04 and less than 0.15Hz; high-frequency energy HF is more than or equal to 0.15 and less than 0.40Hz; (3) the low-frequency energy proportion LFnu,
Figure FDA0004155541870000011
high frequency energy ratio HFnu, < ->
Figure FDA0004155541870000012
(5) Low high frequency energy ratio LF/HF.
4. The method for real-time assessment of pressure load of a driver based on city street images according to claim 1, wherein the process of determining the pressure level of the sample using the heart rate variability index comprises:
performing factor analysis on heart rate variability indexes, reserving N main factors with characteristic values larger than 1, and obtaining the score F of each main factor N of the sample i in Sum of variance contribution rate p n The pressure index SI of the sample i is calculated using the following formula i ：
Figure FDA0004155541870000013
Where n=1, 2,.. i The larger the representation pressureThe higher the force, the higher the stress index SI is by using the K-means clustering algorithm i And performing unsupervised clustering to obtain samples with high, medium or low pressure levels.
5. The method for real-time assessment of driver pressure load based on city street images according to claim 1, wherein the process of extracting the street image element feature variables in the sample video data comprises the steps of:
taking a picture from video data at intervals of delta t, obtaining K pictures from each sample,
Figure FDA0004155541870000021
performing element semantic segmentation on the image by using an image analysis model, and calculating the duty ratio of each element:
Figure FDA0004155541870000022
wherein prop is ijs The s-th picture element duty ratio representing the j-th picture of sample i, pixel ijs Pixel count representing the s-th picture element of sample i, j-th picture, pixel total Representing the total pixel number of the j-th picture of the sample i;
calculating the change rate of two adjacent picture elements:
Figure FDA0004155541870000023
wherein df_prop ijs The change rate of the s-th image element of the j-th picture and the j+1-th picture of the sample i;
calculating to obtain corresponding street view element characteristic variables:
Figure FDA0004155541870000024
Figure FDA0004155541870000025
Figure FDA0004155541870000026
Figure FDA0004155541870000027
wherein M is is Mean value, SD, of element s ratio in sample i is Represents the standard deviation, MD, of the ratio of the element s in the sample i is Mean value representing change rate of element s of sample i, SDD is The standard deviation of the rate of change of the element s of the sample i is shown.
6. The method for real-time assessment of driver's pressure load based on city street images according to claim 1, wherein the process of extracting the kinetic parameter feature variables from the sample video data comprises the steps of:
calculating the average value and standard deviation of each variable of the sample vehicle speed, the triaxial acceleration, the triaxial angular speed, the pitch angle, the roll angle and the yaw angle variation value as micro dynamics characteristic variables;
the calculation formula of the yaw angle change value is as follows:
Y＝Y t+1 -Y t
wherein Y represents a yaw angle variation value, Y t+1 And Y t The yaw angles of the vehicle at times t+1 and t are shown, respectively.
7. The method for evaluating the pressure load of a driver in real time based on the urban street view image according to claim 1, wherein in step S30, spearman rank correlation test is performed on the street view element characteristic variable and the dynamic parameter characteristic variable, and feature dimension reduction is performed by adopting a random forest extrabag error importance calculation method;
the calculation formula of the rank correlation coefficient is:
Figure FDA0004155541870000031
wherein ρ is the inter-variable rank correlation coefficient, U is the total number of samples, d θ Representing rank differences between variables;
the importance of the feature variable is calculated by adopting the following formula:
Figure FDA0004155541870000032
wherein I is f Representing the importance of the feature f, dt represents the number of decision trees,
Figure FDA0004155541870000033
representing the error in prediction using all feature variables, +.>
Figure FDA0004155541870000034
The error in prediction using all the characteristic variables except the variable f is represented.
8. The method for real-time assessment of pressure load of driver based on city street images according to claim 1, wherein in step S40, the process of using the best fusion model for real-time assessment of pressure load of driver by using pressure grade as label, forming feature-label matrix with sample feature vector, and training fusion model comprising m machine learning classifiers comprises the following steps:
carrying out rebalancing treatment on the sample data set after dimension reduction by adopting a synthetic minority class oversampling technology, and training a decision tree, a multi-element logic regression, a gradient lifting tree, a limit gradient lifting tree, a class gradient lifting tree and a light gradient lifting tree by utilizing the rebalancing data set;
according to the training result of a single classifier, adopting a stacking method to take each classifier as a meta classifier, taking other classifiers as base classifiers to perform structural design of fusion models one by one, and reserving the fusion models with G-means and F1 scores more than 80%;
and (3) further optimizing the performance of the reserved fusion model by adopting an Adaboost data set resampling technology, performing final performance evaluation on the fusion model, and using the fusion model with the best performance for real-time evaluation of the pressure load of the driver.
CN202310333102.4A 2023-03-30 2023-03-30 Real-time evaluation method for pressure load of driver based on city street view image Pending CN116401620A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310333102.4A CN116401620A (en) 2023-03-30 2023-03-30 Real-time evaluation method for pressure load of driver based on city street view image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310333102.4A CN116401620A (en) 2023-03-30 2023-03-30 Real-time evaluation method for pressure load of driver based on city street view image

Publications (1)

Publication Number Publication Date
CN116401620A true CN116401620A (en) 2023-07-07

Family

ID=87017240

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310333102.4A Pending CN116401620A (en) 2023-03-30 2023-03-30 Real-time evaluation method for pressure load of driver based on city street view image

Country Status (1)

Country Link
CN (1) CN116401620A (en)

Similar Documents

Publication Publication Date Title
EP3533681B1 (en) Method for detecting safety of driving behavior, apparatus and storage medium
Chen et al. A graphical modeling method for individual driving behavior and its application in driving safety analysis using GPS data
Das et al. Automated vehicle collisions in California: Applying Bayesian latent class model
CN109278753B (en) Intelligent vehicle auxiliary driving method based on visual visible information of driver
CN103247091A (en) Driving evaluation system and driving evaluation method
CN112793576B (en) Lane change decision method and system based on rule and machine learning fusion
CN113743471B (en) Driving evaluation method and system
CN113753059B (en) Method for predicting takeover capacity of driver under automatic driving system
Shirazi et al. Detection of intoxicated drivers using online system identification of steering behavior
CN106651210A (en) CAN data-based driver comprehensive quality evaluation method
CN110781873A (en) Driver fatigue grade identification method based on bimodal feature fusion
Bustos et al. Predicting driver self-reported stress by analyzing the road scene
CN116028884A (en) Prototype network-based vehicle lane change risk assessment method under small sample
Li et al. Research on lane change prediction model based on GBDT
CN112308136A (en) SVM-Adaboost-based driving distraction detection method
CN116401620A (en) Real-time evaluation method for pressure load of driver based on city street view image
CN106651191A (en) Mountain area highway horizontal curve dangerous section identification method
CN111016914A (en) Dangerous driving scene identification system based on portable terminal information and identification method thereof
CN116461546A (en) Vehicle early warning method, device, storage medium and processor
Ma et al. Lane change analysis and prediction using mean impact value method and logistic regression model
CN111985386A (en) Method for identifying pedestrian illegal-passing behavior based on planned behavior theory
Wei et al. An AHP‐ME‐Based Vehicle Crash Prediction Model considering Driver Intention and Real‐Time Traffic/Road Condition
CN111932829A (en) Fatigue driving prevention facility utility testing method and system
Tang Application of information entropy algorithm in safety risk prediction of road traffic driving behaviors.
Wang et al. Fatigue driving detection of urban road at night based on multimodal information fusion.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination