LU600178B1

LU600178B1 - Simulation method and system for fatigue detection of steeringwheel grip force based on dil

Info

Publication number: LU600178B1
Application number: LU600178A
Authority: LU
Inventors: Zimin Liu; Dong Guo; Ning Yu
Original assignee: Univ Chongqing Technology
Priority date: 2024-12-31
Filing date: 2024-12-31
Publication date: 2026-06-30
Also published as: WO2025232250A1

Abstract

The invention provides a simulation method and system for fatigue detection of steering wheel grip force based on DIL, which belongs to the field of automobile auxiliary driving and driving safety technology, including the construction of driver-in-the-loop system platform and virtual driving scene library, the integration of hardware and software to realize system construction, the construction of fatigue driving detection module, the design of experimental scheme, the establishment of fatigue driving sample data set, the construction of fatigue state recognition model, and finally the verification of experimental results, so as to comprehensively evaluate and optimize the fatigue driving detection system. The invention adopts the above-mentioned simulation method and system for fatigue detection of steering wheel grip force based on DIL, which improves the accuracy of fatigue detection, avoids the danger of collecting driving data under actual driving conditions, and improves the efficiency of driving experiment data collection. (Figure 1)

Description

DESCRIPTION 7600178

SIMULATION METHOD AND SYSTEM FOR FATIGUE DETECTION OF STEERING

WHEEL GRIP FORCE BASED ON DIL

TECHNICAL FIELD

[01] The invention relates to the field of automobile auxiliary driving and driving safety technology, especially to a simulation method and system for fatigue detection of steering wheel grip force based on Driver-In-the-Loop (DIL).

BACKGROUND

[02] The existing visual-based fatigue detection technology is susceptible to environmental factors such as illumination changes and driver posture changes, most studies based on physiological signal detection technology still need to wear complex devices, these devices are not only cumbersome to wear, but also may cause unnecessary interference to the driver’s driving experience, and may even affect driving safety to a certain extent.

[03] Although the existing fatigue detection method based on steering wheel grip force has achieved certain research results, it faces many challenges in actual operation.

The most prominent problem is that the labeling of the fatigue state is highly subjective, which may lead to great differences between different researchers or annotators when judging the fatigue state of the same driver. At the same time, due to the complexity of the steering wheel grip data, how to accurately extract the features related to the fatigue state from these data and construct a high-precision fatigue recognition algorithm is also a key problem to be solved in the current research.

[04] With the continuous development of DIL technology, through the combination of virtual driving scene modeling software and an actual driving steering wheel, a simulation test system that maximizes the simulation of real driving scenes can be constructed. It can collect steering wheel grip driving data in real-time and provide a ridH/6001 78 data source for constructing the fatigue detection model.

[05] Based on this background, the invention proposes a new simulation method and system for fatigue detection of steering wheel grip force based on DIL.

SUMMARY

[06] The purpose of the invention is to provide a simulation method and system for fatigue detection of steering wheel grip force based on DIL, overcome the limitations of existing technologies, and improve the accuracy of fatigue detection. Through the simulation test of DIL system with infinite test scenarios, the risk of collecting driving data under actual driving conditions can be avoided.

[07] In order to achieve the above purpose, the invention provides a simulation method and system for fatigue detection of steering wheel grip force based on DIL, including the following steps:

[08] S1, a driver-in-the-loop system platform is established: the hardware-in-the-loop part and the software-in-the-loop part are established;

[09] S2, a virtual driving scene library is established: a virtual driving scene library that includes different road scenes and external environments are established;

[10] S3, the driver-in-the-loop system is established: the hardware-in-the-loop part and the software-in-the-loop part are integrated to realize the driver-in-the-loop system;

[11] S4, a fatigue driving detection module is established: a flexible pressure sensor,

EEG equipment, signal acquisition, and detection module are added to the driver-in-the-loop system to construct a fatigue driving simulation system;

[12] S5, a simulated driving experiment is designed: two experimental schemes of normal driving and fatigue driving are designed;

[13] S6, a fatigue driving sample data set is established: the subjective experiment of the Stanford Sleepiness Scale is combined with the objective evaluation of EEG signals to complete the labeling of different fatigue states;

[14] S7, a fatigue state recognition model is established; LU600178

[15] S8, the experimental results are verified.

[16] Preferably, in S1, the specific steps for establishing the driver-in-the-loop system platform are as follows:

[17] S11, hardware-in-the-loop: the construction of the driver operation device, image simulation display device, sound simulation device, and seat hardware is completed, and the vehicle driving state is simulated through a dynamic driver simulator to realize the hardware-in-the-loop;

[18] S12, software-in-the-loop: the software-in-the-loop is realized through the construction of the scene, sensor simulation software, and vehicle dynamics simulation software.

[19] Preferably, in S6, the optimal threshold determination process of the EEG signal is as follows:

[20] S61, pre-processing: band-pass filtering, notch filtering, re-reference, down-sampling rate, and ICA pre-processing are performed for EEG signals;

[21] S62, feature extraction: the power spectral density characteristics of EEG signals are extracted after preprocessing;

[22] S63, determination of the optimal threshold: the optimal threshold of EEG signals is determined by performing wavelet transform denoising, pre-labeling, ROC analysis, and calculation of confusion matrix steps on the extracted power spectral density features.

[23] Preferably, in S7, the steps for establishing a fatigue state recognition model are as follows:

[24] S71, data preprocessing: linear interpolation, Kalman filtering, and normalization are performed for grip force data;

[25] S72, feature extraction: the fatigue driving sample data set is extracted to reflect the time domain characteristics of driver fatigue;

[26] S73, feature screening and optimization: normalization, correlation analysisU600178 significance test, feature effect analysis, significant feature screening, smooth filtering processing, and effect analysis are performed for the extracted time-domain features;

[27] S74, model training, and optimization: based on the selected time-domain features, the model is trained and optimized to obtain the optimal fatigue state recognition model.

[28] Preferably, in S8, the steps for verifying the experimental results are as follows:

[29] S81, model deployment: the fatigue state recognition model is deployed in the driver-in-the-loop system;

[30] S82, experimental verification: simulated driving experiments are performed on different drivers, and fatigue state recognition is performed based on the fatigue state recognition model;

[31] S83, human-computer interaction interface adjustment. according to the identified fatigue state, the human-computer interaction interface of the vehicle is dynamically adjusted, and the early warning module responds accordingly.

[32] Preferably, the grip force data are derived from the pressure value of the flexible pressure sensor arranged on the steering wheel.

[33] Therefore, the invention adopts the above-mentioned simulation method and system for fatigue detection of steering wheel grip force based on DIL. The technical effects are as follows:

[34] 1. High-precision simulation: through the combination of the virtual driving scene and the actual steering wheel, the real driving scene is simulated to the maximum extent, and the accuracy of fatigue detection is improved.

[35] 2. High security: through the simulation test of the system, the danger of the actual driving data collection is avoided and the efficiency of data collection is improved;

[36] 3. Timely warning: it can effectively identify the driver's fatigue state, timely warning, dynamically adjust the human-computer interaction interface, and make driving safer.

BRIEF DESCRIPTION OF THE DRAWINGS

[371 FIG. 1 is an overall frame diagram of the driver-in-the-loop system platform;

[38] FIG. 2 is a construction diagram for the virtual driving scene library;

[39] FIG. 3 is a construction diagram for the driver-in-the-loop system;

[40] FIG. 4 is an establishment diagram of the fatigue driving sample data set;

[41] FIG. 5 is a flow chart for determining the optimal threshold of EEG signals;

[42] FIG. 6 is an establishment diagram of the fatigue state recognition model;

[43] FIG. 7 is a verification diagram of the fatigue state of the steering wheel grip force of the driver-in-the-loop.

DETAILED DESCRIPTION OF THE EMBODIMENTS LU600178

[44] The following is a further explanation of the technical scheme of the invention through drawings and implementation examples.

[45] Unless otherwise defined, the technical or scientific terms used in the invention shall be the general meaning understood by persons with general skills in the field to which the invention belongs.

[46] Example 1

[47] FIG. 1 is the overall frame diagram of the driver-in-the-loop system platform of the invention, a complete driver-in-the-loop system platform is constructed, it is necessary to complete the construction of hardware and software, the specific construction method is as follows: the data acquisition module is used to capture the torque, angle and force sensor data of the driver when operating the steering wheel, brake and accelerator pedal in real-time, which is processed by PC and real-time simulation platform, combined with rolling resistance and wind resistance information to calculate the vehicle motion state. The scene simulation model renders the driving scene according to the vehicle state, feeds back to the driver through the image and sound simulator, and uses the PreScan software to simulate the sensor function. The main controller generates control signals, drives the electric cylinder and suspension, simulates the real vehicle motion, and realizes force feedback and vibration perception.

The whole system is highly integrated, providing an immersive driving simulation experience for drivers, and effectively supporting the research and application of the fatigue detection of steering wheel grip force.

[48] FIG. 2 is a construction diagram for the virtual driving scene library, the construction of this scene library is based on the actual road driving scene, and the main construction process is as follows: By installing a data acquisition device on the real vehicle, different driving scene data are collected, after processing, it is divided into the vehicle driving state and the driving scene image, the image data is filtered and combined with the high-precision map library to construct a virtual driving scene library, which provides realistic driving feedback for the simulated driving platform and improves the realism of driving simulation and the accuracy of fatigue detection simulation. LU600178

[49] FIG. 3 is the construction diagram for the driver-in-the-loop system, which is a closed-loop description of the trinity of the driver, the constructed driving platform, and the simulated driving system, the main construction process is as follows: the driver interacts with the simulated driving software by manipulating the pedal and steering wheel on the driving platform to operate in real-time and receive visual feedback. The driver can customize the virtual scene, such as road type and weather, to realize the real feedback interaction between simulated driving and real driving, and meet the requirements of the driving experiment. [S0] FIG. 4 is an establishment diagram of the fatigue driving sample data set, the specific establishment process is as follows: Based on the driver-in-the-loop simulation system, a fatigue driving simulation system is constructed through a flexible pressure sensor on the steering wheel and an EEG signal acquisition device. The experimental design included two parts: normal driving and fatigue driving, which are carried out in an energetic morning and an afternoon after sleep restriction, respectively. During data collection, the driver's different fatigue states are marked by the subjective and objective comprehensive evaluation method combined with the Stanford Sleepiness Scale and

EEG signal analysis, and the fatigue driving sample data set is constructed to distinguish the awake, mild fatigue, and moderate and severe fatigue states.

[51] FIG. 5 is the flow chart for determining the optimal threshold of EEG signals, the specific construction process is as follows: EEG signal data acquisition covers key brain areas, and noise and artifacts are removed by preprocessing. The power spectral density features strongly related to fatigue are extracted. After denoising by wavelet transform, the changing trend of power spectral density characteristics is combined with the subjective scoring results of the Stanford Sleepiness Scale to complete the pre-labeling of EEG signals under different fatigue states. Through ROC analysis, the corresponding EEG signal fatigue threshold is determined. After the confusion matrix verification, the overall classification accuracy is greater than 95 % as the standard. If it is not satisfied, it is searched again, finally, the optimal threshold is determined to classify awake, mild, moderate, and severe fatigue states to achieve fatigue driving detection.

[52] FIG. 6 is an establishment diagram of the fatigue state recognition model, tHaJ600178 specific establishment process is as follows: In the data processing stage, linear interpolation, Kalman filtering, and normalization techniques are first used to effectively deal with missing values, noise, and individual differences in grip force data.

Subsequently, the time-domain features are extracted, and the features closely related to the degree of fatigue are accurately selected through correlation analysis and significance tests. In order to ensure the integrity and smoothness of feature information, feature optimization is carried out using moving average smoothing, wavelet denoising, and Savitzky-Golay filtering. Finally, based on these carefully processed feature parameters, random forest, one-dimensional convolutional neural network (1D-CNN), long short-term memory network LSTM, and bidirectional LSTM are used to construct and optimize the fatigue state recognition model, we strive to obtain the best fatigue detection effect through the optimization of the verification set.

[53] FIG. 7 is a verification diagram of the fatigue state of the steering wheel grip force of the driver-in-the-loop, the specific verification process is as follows: The fatigue state recognition model is deployed in the driver-in-the-loop simulation system to identify different fatigue states and dynamically adjust the early warning measures and human-computer interaction interface. Mild fatigue is slightly reminded, and multiple coercive measures are taken when moderate and severe. The display is simplified or the control is adjusted according to the degree of fatigue to improve driving safety.

[54] Therefore, the invention adopts the above-mentioned simulation method and system for fatigue detection of steering wheel grip force based on DIL, and combines the virtual driving scene with the actual steering wheel to construct a simulation test system to realize the high simulation and infinite test of the real driving scene, which is convenient for the fatigue identification of the steering wheel grip force. At the same time, the driver is warned by the fatigue state recognition model, and the human-computer interaction interface is dynamically adjusted to significantly improve driving safety and experience.

[55] Finally, it should be explained that the above example is only used to explain the technical scheme of the invention rather than restrict it, although the invention is described in detail concerning the better example, the ordinary technical personnel #600178 this field should understand that they can still modify or replace the technical scheme of the invention, and these modifications or equivalent substitutions cannot make the modified technical scheme out of the spirit and scope of the technical scheme of the invention.

Claims

CLAIMS LU600178

1. À simulation method and system for fatigue detection of steering wheel grip force based on DIL, including the following steps: S1, a driver-in-the-loop system platform is established: the hardware-in-the-loop part and the software-in-the-loop part are established; S2, a virtual driving scene library is established: a virtual driving scene library that includes different road scenes and external environments are established: S3, the driver-in-the-loop system is established: the hardware-in-the-loop part and the software-in-the-loop part are integrated to realize the driver-in-the-loop system; S4, a fatigue driving detection module is established: a flexible pressure sensor, EEG equipment, signal acquisition, and detection module are added to the driver-in-the-loop system to construct a fatigue driving simulation system; S5, a simulated driving experiment is designed: two experimental schemes of normal driving and fatigue driving are designed; S6, a fatigue driving sample data set is established: the subjective experiment of the Stanford Sleepiness Scale is combined with the objective evaluation of EEG signals to complete the labeling of different fatigue states; S7, a fatigue state recognition model is established: S8, the experimental results are verified.

2. The simulation method and system for fatigue detection of steering wheel grip force based on DIL according to claim 1, wherein in S1, the specific steps for establishing the driver-in-the-loop system platform are as follows: S11, hardware-in-the-loop: the construction of the driver operation device, image simulation display device, sound simulation device, and seat hardware is completed, and the vehicle driving state is simulated through a dynamic driver simulator to realize the hardware-in-the-loop; S12, software-in-the-loop: the software-in-the-loop is realized through the construction of the scene, sensor simulation software, and vehicle dynamics simulatid#/6001 78 software.

3. The simulation method and system for fatigue detection of steering wheel grip force based on DIL according to claim 1, wherein in S6, the optimal threshold determination process of the EEG signal is as follows: S61, pre-processing: band-pass filtering, notch filtering, re-reference, down-sampling rate, and ICA pre-processing are performed for EEG signals; S62, feature extraction: the power spectral density characteristics of EEG signals are extracted after preprocessing; S63, determination of the optimal threshold: the optimal threshold of EEG signals is determined by performing wavelet transform denoising, pre-labeling, ROC analysis, and calculation of confusion matrix steps on the extracted power spectral density features.

4. The simulation method and system for fatigue detection of steering wheel grip force based on DIL according to claim 1, wherein in S7, the steps for establishing a fatigue state recognition model are as follows: S71, data preprocessing: linear interpolation, Kalman filtering, and normalization are performed for grip force data; S72, feature extraction: the fatigue driving sample data set is extracted to reflect the time domain characteristics of driver fatigue; S73, feature screening and optimization: normalization, correlation analysis, significance test, feature effect analysis, significant feature screening, smooth filtering processing, and effect analysis are performed for the extracted time-domain features; S74, model training, and optimization: based on the selected time-domain features, the model is trained and optimized to obtain the optimal fatigue state recognition model.

5. The simulation method and system for fatigue detection of steering wheel grip force based on DIL according to claim 1, wherein in S8, the steps for verifying the experimental results are as follows: S81, model deployment: the fatigue state recognition model is deployed in the driver-in-the-loop system; LU600178 S82, experimental verification: simulated driving experiments are performed on different drivers, and fatigue state recognition is performed based on the fatigue state recognition model; S83, human-computer interaction interface adjustment: according to the identified fatigue state, the human-computer interaction interface of the vehicle is dynamically adjusted, and the early warning module responds accordingly.

6. The simulation method and system for fatigue detection of steering wheel grip force based on DIL according to claim 4, wherein the grip force data are derived from the pressure value of the flexible pressure sensor arranged on the steering wheel.