CN114506335A - Driver fatigue and health monitoring system for automatic driving - Google Patents

Driver fatigue and health monitoring system for automatic driving Download PDF

Info

Publication number
CN114506335A
CN114506335A CN202210038454.2A CN202210038454A CN114506335A CN 114506335 A CN114506335 A CN 114506335A CN 202210038454 A CN202210038454 A CN 202210038454A CN 114506335 A CN114506335 A CN 114506335A
Authority
CN
China
Prior art keywords
driver
heartbeat
vehicle
frequency
fatigue
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210038454.2A
Other languages
Chinese (zh)
Inventor
白蓉蓉
罗振刚
魏娜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan Haozhirong Technology Co ltd
Original Assignee
Sichuan Haozhirong Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan Haozhirong Technology Co ltd filed Critical Sichuan Haozhirong Technology Co ltd
Priority to CN202210038454.2A priority Critical patent/CN114506335A/en
Publication of CN114506335A publication Critical patent/CN114506335A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0872—Driver physiology
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/221—Physiology, e.g. weight, heartbeat, health or special needs
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/229—Attention level, e.g. attentive to driving, reading or sleeping

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention relates to a driver fatigue and health monitoring system for automatic driving, which belongs to the field of auxiliary driving and comprises a vehicle communication module, a vehicle state sensing module, a radar sensing module and a power supply module, wherein the vehicle communication module, the vehicle state sensing module, the fatigue and health monitoring module and the radar sensing module are respectively and electrically connected with the power supply module, and the vehicle communication module, the vehicle state sensing module and the radar sensing module are respectively and electrically connected with the fatigue and health monitoring module through signals. The invention has the beneficial effects that: the millimeter wave radar has good penetrating power for the shelters such as clothes, masks and the like, and the influence of the shelters is eliminated; based on the specific frequency range of heartbeat respiration, the influence of vehicle vibration on a detection result can be eliminated by combining a radar algorithm and vehicle information, and the accuracy and reliability of detection are improved.

Description

Driver fatigue and health monitoring system for automatic driving
Technical Field
The invention belongs to the field of auxiliary driving, and particularly relates to a driver fatigue and health monitoring system for automatic driving.
Background
The automatic driving function is increasingly powerful, and under the condition of good road conditions, a driver does not need to control a steering wheel and an accelerator pedal of a vehicle. According to the requirements of low-level automatic driving grades, drivers still need to monitor road conditions in real time and control vehicles in case of emergency. In practical applications, long-term automatic driving can cause drivers to get tired, so the driver monitoring system takes place at the right moment and tries to judge whether the driver is tired or has fallen asleep.
The current driver monitoring system judges whether the driver is tired or not by capturing the changes of facial expressions, head swings and upper limb movements of the driver based on the in-vehicle camera. However, when the driver drives the vehicle automatically for a long time, the driver does not need to move the upper limbs and the head, and objects such as sunglasses and a mask can also shield facial expressions, and the privacy of the driver and passengers in the vehicle is greatly invaded by the camera in the vehicle.
Disclosure of Invention
The invention provides a driver fatigue and health monitoring system for automatic driving, which monitors the heartbeat and the breath of a driver in real time when a vehicle enters an automatic driving state, judges that the driver is tired or abnormal breathing and heartbeat judges that a health problem occurs when the breath and the heartbeat are in low amplitude and low frequency for a long time, and gives an alarm.
In order to achieve the purpose, the technical scheme is as follows: a driver fatigue and health monitoring system for automatic driving comprises a vehicle communication module, a vehicle state sensing module, a radar sensing module and a power supply module, wherein the vehicle communication module, the vehicle state sensing module, the fatigue and health monitoring module and the radar sensing module are respectively and electrically connected with the power supply module, and the vehicle communication module, the vehicle state sensing module and the radar sensing module are respectively in signal connection with the fatigue and health monitoring module;
the vehicle communication module is used for reading in vehicle state signals from a vehicle bus and transmitting the vehicle state signals to the vehicle state sensing module;
the vehicle state sensing module is used for judging whether the vehicle is driven stably according to a vehicle state signal transmitted by the vehicle communication module, judging whether the driving state of the vehicle meets the accurate measurement condition of the radar on the breathing and heartbeat frequency of the human body, and outputting the state of whether the vehicle is driven stably to the fatigue and health monitoring module;
the radar sensing module is used for measuring the heartbeat, the breathing frequency and the intensity of a driver by transmitting and receiving millimeter wave signals and transmitting the heartbeat, the breathing frequency and the intensity of the driver to the fatigue and health monitoring module;
the fatigue and health monitoring module is used for receiving information transmitted by the vehicle communication module, the vehicle state sensing module and the radar sensing module, judging the fatigue state and the health state of a driver and sending an alarm signal when the state of the driver is abnormal;
the vehicle state comprises a steering angle, a vehicle running speed and an automatic driving starting condition.
Further, through radar perception module measures driver's breathing characteristic and heartbeat characteristic, specifically includes: echo signals are obtained through a millimeter wave radar, signal frequency spectrums are calculated, then corresponding frequency spectrums in frequency range where heartbeat and respiration are located are independently extracted, and influence of vehicle vibration is reduced and eliminated.
Further, acquiring a database of heartbeat and breath of a driver and passengers of a company, and calculating the characteristics of the heartbeat and breath of the driver along with the change of time in the modes of statistics, machine learning and deep learning so as to separate the heartbeat and breath frequency and amplitude of the driver independently and distinguish the heartbeat and breath frequency from the characteristics of vehicle vibration;
in the driving process of the vehicle, a driver cannot change the radar in the driving process of the vehicle, and the radar further corrects the characteristics of heartbeat and respiration changing along with time aiming at the unique database of heartbeat and respiration continuously accumulated by a detected object, so that the characteristic range is more accurate, the screening is more efficient, and the misjudgment is further reduced.
Further, the method for acquiring the characteristics of the heartbeat respiratory frequency and amplitude of the driver changing along with time comprises the following steps:
counting the heartbeat frequency, the breathing amplitude and the heartbeat amplitude of a driver within the time T, and calculating the respective average value and variance of the heartbeat frequency, the breathing amplitude and the heartbeat amplitude to an N-order moment and the covariance between every two of the heartbeat frequency, the breathing amplitude and the heartbeat amplitude, wherein the average value calculation formula is as follows:
Figure BDA0003469087950000021
wherein X represents the heart rate and amplitudeAny one of degree, respiratory rate, respiratory amplitude, XiA sample representing X;
within a certain time, X is sampled N times, N samples are generated, X is respectively1To XN，
The variance is calculated by
Var{X}＝E{[X-E{X}]2}＝E{X2}-(E{X})2
The covariance is calculated by
Cij＝Cov{Xi,Xj}＝E{(Xi-E{Xi})(Xj-E{Xj)}
Wherein Xi，XjThe statistical characteristics of a certain number of people under different conditions are summarized, and the breathing and heartbeat characteristics of the driver are summarized. .
Further, the method for calculating the characteristics of the heartbeat and the breathing of the driver along with the time change through machine learning comprises the following steps: inputting a sample and a training set of the heartbeat frequency, the heartbeat amplitude, the breathing frequency and the breathing amplitude of which the states of the driver are marked, then training based on a machine learning algorithm to finally form a trained machine learning model, and inputting millimeter wave signals into the trained machine learning model to judge the fatigue state and the health state of the driver.
The method for calculating the characteristics of the heartbeat and the respiration of the driver along with the time change through deep learning comprises the following steps: inputting the heartbeat frequency, heartbeat amplitude, respiratory frequency and respiratory amplitude samples marked with the state of the driver into a RadarTransformer for deep learning to obtain a deep training model, and inputting millimeter wave signals into the deep training model of the driver, so that the fatigue state and the health state of the driver can be judged.
Further, the method for judging whether the vehicle runs smoothly by the vehicle state sensing module is as follows:
s1, the vehicle does not turn or the turning radius is lower than the set value within the set time;
s2, the speed of the vehicle is higher than a first threshold value within a set time, and the speed change of the vehicle is lower than a second threshold value;
s3, detecting that the acceleration sensor in the vertical direction does not have large acceleration change in the vertical direction of the vehicle;
if at least one of the determination conditions S1, S2, and S3 is satisfied, it is determined that the vehicle is running smoothly, and this is suitable for driver fatigue and health monitoring.
Further, the method for judging the fatigue state of the driver comprises the steps of judging that the vehicle state is suitable for measuring the heartbeat, the breathing frequency, the heartbeat intensity and the breathing intensity of the driver in the time period of the fatigue and health condition of the driver, carrying out long-time statistics according to a time axis, detecting the change of the heartbeat, the breathing frequency, the heartbeat intensity and the breathing intensity of the driver along with time, and judging whether the heartbeat and the breathing frequency gradually slow down and the intensity is weakened, if the heartbeat or the breathing frequency of the driver is in a low value for a long time and the intensity is also in a low value, judging the fatigue of the driver, and if the road condition changes, such as road bump, vehicle turning becomes more and front vehicles become more, and the frequency and the intensity of the breathing heartbeat of the driver do not change along with the change, judging the fatigue of the driver.
Further, the method for judging the fatigue state of the driver further comprises the step of detecting the operation frequency of the vehicle by the driver, and the method for detecting the operation frequency of the vehicle comprises the following steps: when the vehicle is automatically driven, a driver mainly focuses on vehicle operation on window lifting, windscreen wipers, an air conditioning ventilation system and a vehicle-mounted entertainment system, the operation comprises hand touch operation, voice operation and gesture recognition operation, and the average interval and the interval time of last operation of the driver in the past 30-60 min for operating the system are continuously counted;
determining that the driver is fatigued if the driver has not operated the vehicle above-mentioned device on average or recently for more than a preset time;
when weather changes, such as from raining to raining, or from raining to raining, the system does not control the wiper and the driver does not operate the wiper, or the temperature in the vehicle changes obviously to a temperature higher than a high-temperature preset value or lower than a low-temperature preset value and the driver does not adjust the air conditioner, the driver is judged to be tired.
Further, the vehicle state is judged to be suitable for measuring the frequency and the intensity of the heartbeat and the breath of the driver in the time period of the fatigue and the health condition of the driver, long-time statistics is carried out according to the time axis, the change of the heartbeat and the breath frequency of the driver and the intensity of the heartbeat and the breath frequency of the driver along with the time is detected, and if the heartbeat and the breath frequency are extremely high or extremely low and the heartbeat and the breath intensity are extremely high or extremely low, the health problem of the driver is judged.
The invention has the following beneficial effects:
1. the millimeter wave radar has good penetrating power for the shelters such as clothes, masks and the like, and the influence of the shelters is eliminated;
2. based on the specific frequency range of heartbeat respiration, the influence of vehicle vibration on a detection result can be eliminated by combining a radar algorithm and vehicle information, and the accuracy and reliability of detection are improved;
3. the characteristics of heartbeat and respiration changing along with time can be continuously optimized through a machine learning algorithm and a deep learning algorithm, misjudgment and missed judgment can be further reduced, and the accuracy and reliability of the identification result are further improved.
Drawings
FIG. 1 is a schematic diagram of a driver fatigue and health monitoring system for automatic driving according to the present invention.
Detailed Description
The invention will now be described more fully with reference to the accompanying examples.
Referring to fig. 1, the embodiment discloses a driver fatigue and health monitoring system for automatic driving, which includes a vehicle communication module, a vehicle state sensing module, a radar sensing module and a power supply module, wherein the vehicle communication module, the vehicle state sensing module, the fatigue and health monitoring module and the radar sensing module are respectively electrically connected with the power supply module, and the vehicle communication module, the vehicle state sensing module and the radar sensing module are respectively in signal connection with the fatigue and health monitoring module;
the vehicle communication module is used for reading in vehicle state signals from a vehicle bus and transmitting the vehicle state signals to the vehicle state sensing module;
the vehicle state sensing module is used for judging whether the vehicle is driven stably according to a vehicle state signal transmitted by the vehicle communication module, judging whether the driving state of the vehicle meets the accurate measurement condition of the radar on the breathing and heartbeat frequency of the human body, and outputting the state of whether the vehicle is driven stably to the fatigue and health monitoring module;
the radar sensing module is used for measuring the heartbeat, the breathing frequency and the intensity of a driver by transmitting and receiving millimeter wave signals and transmitting the heartbeat, the breathing frequency and the intensity of the driver to the fatigue and health monitoring module;
the fatigue and health monitoring module is used for receiving information transmitted by the vehicle communication module, the vehicle state sensing module and the radar sensing module, judging the fatigue state and the health state of a driver and sending an alarm signal when the state of the driver is abnormal;
the vehicle state comprises a steering angle, a vehicle running speed and an automatic driving starting condition.
In the above embodiment, through radar perception module measures driver's breathing characteristic and heartbeat characteristic, specifically include: echo signals are obtained through a millimeter wave radar, signal frequency spectrums are calculated, then corresponding frequency spectrums in frequency range where heartbeat and respiration are located are independently extracted, and influence of vehicle vibration is reduced and eliminated.
In the embodiment, a database of heartbeat and breath of a driver and passengers of a company is obtained, and the characteristics of the heartbeat and breath of the driver changing along with time are calculated in a statistical, machine learning and deep learning mode, so that the heartbeat and breath frequency and amplitude of the driver are separated independently and are distinguished from the characteristics of vehicle vibration;
in the driving process of the vehicle, a driver cannot change the radar in the driving process of the vehicle, and the radar further corrects the characteristics of heartbeat and respiration changing along with time aiming at the unique database of heartbeat and respiration continuously accumulated by a detected object, so that the characteristic range is more accurate, the screening is more efficient, and the misjudgment is further reduced.
In the above embodiment, the method for acquiring the characteristics of the heartbeat respiratory frequency and amplitude of the driver changing with time includes: counting heartbeat frequency, respiratory amplitude and heartbeat amplitude of a driver within time T, and calculating respective average values and variances of the heartbeat frequency, the respiratory amplitude and the heartbeat amplitude to an N-th moment and covariance between every two of the average values and the variances, wherein the average value calculation formula is as follows:
Figure BDA0003469087950000051
wherein X represents any one of heartbeat frequency, heartbeat amplitude, respiratory frequency and respiratory amplitude, and XiA sample representing X;
within a certain time, X is sampled N times, N samples are generated, X is respectively1To XN，
The variance is calculated by
Var{X}＝E{[X-E{X}]2}＝E{X2}-(E{X})2
The covariance is calculated by
Cij＝Cov{Xi,Xj}＝E{(Xi-E{Xi})(Xj-E{Xj)}
Wherein Xi，XjThe statistical characteristics of a certain number of people under different conditions are summarized, and the breathing and heartbeat characteristics of the driver are summarized.
In the above embodiment, the method for calculating the characteristics of the heartbeat and the breathing of the driver over time through machine learning includes: inputting a sample and a training set of the heartbeat frequency, the heartbeat amplitude, the breathing frequency and the breathing amplitude of which the states of the driver are marked, then training based on a machine learning algorithm to finally form a trained machine learning model, and inputting millimeter wave signals into the trained machine learning model to judge the fatigue state and the health state of the driver. In the actual implementation process, machine learning is performed in a decision tree manner, and in the actual implementation process, other machine learning algorithms such as naive Bayes classification and support vector machine can also be adopted.
In the above embodiment, the method for calculating the characteristics of the heartbeat and the breathing of the driver over time through deep learning includes: inputting the heartbeat frequency, heartbeat amplitude, respiratory frequency and respiratory amplitude samples marked with the driver state into a Radar transform for deep learning to obtain a deep training model, and inputting millimeter wave signals into the deep training model of the driver, so that the fatigue state and the health state of the driver can be judged.
In the above embodiment, the method for judging whether the vehicle runs smoothly by the vehicle state sensing module is as follows:
s1, the vehicle does not turn or the turning radius is lower than the set value within the set time;
s2, the speed of the vehicle is higher than a first threshold value within a set time, and the speed change of the vehicle is lower than a second threshold value;
s3, detecting that the acceleration sensor in the vertical direction does not have large acceleration change in the vertical direction of the vehicle;
if at least one of the determination conditions S1, S2, and S3 is satisfied, it is determined that the vehicle is running smoothly, and this is suitable for driver fatigue and health monitoring.
In the above embodiment, the method for determining the fatigue state of the driver includes determining that the vehicle state is suitable for measuring the heartbeat, breathing frequency, heartbeat intensity and breathing intensity of the driver in the time period of the fatigue and health condition of the driver, performing long-time statistics on a time axis, detecting the change of the heartbeat, breathing frequency, heartbeat intensity and breathing intensity of the driver along with time, and determining whether the heartbeat and breathing frequency gradually becomes slow and the intensity gradually decreases, if the heartbeat or the breathing frequency of the driver is in a low value for a long time and the intensity is also in a low value, determining that the driver is tired, and if the road condition changes, such as road bump, vehicle turning becomes more, front vehicles become more, and the frequency and the intensity of the breathing heartbeat of the driver do not change, determining that the driver is tired.
In the above embodiment, the method for determining the fatigue state of the driver further includes detecting the frequency of the operation of the vehicle by the driver, and the method for detecting the frequency of the operation of the vehicle includes: when the vehicle is automatically driven, a driver mainly focuses on vehicle operation on window lifting, windscreen wipers, an air conditioning ventilation system and a vehicle-mounted entertainment system, the operation comprises hand touch operation, voice operation and gesture recognition operation, and the average interval and the interval time of last operation of the driver in the past 30-60 min for operating the system are continuously counted;
determining that the driver is fatigued if the driver has not operated the vehicle above-mentioned device on average or recently for more than a preset time;
when weather changes, such as from raining to raining, or from raining to raining, the system does not control the wiper and the driver does not operate the wiper, or the temperature in the vehicle changes obviously to a temperature higher than a high-temperature preset value or lower than a low-temperature preset value and the driver does not adjust the air conditioner, the driver is judged to be tired.
In the embodiment, the vehicle state is judged to be suitable for measuring the frequency and the intensity of the heartbeat and the breath of the driver in the fatigue and health condition time period of the driver, long-time statistics is carried out according to the time axis, the change of the heartbeat and the breath frequency and the intensity of the driver along with time is detected, and if the heartbeat and the breath frequency are extremely high or extremely low and the heartbeat and the breath intensity are extremely high or extremely low, the health problem of the driver is judged.
It should be noted that the specific embodiments are only some embodiments, and those skilled in the art can obtain other embodiments without creative efforts, and all embodiments are within the protection scope of the present invention.

Claims (10)

1. A driver fatigue and health monitoring system for automatic driving is characterized by comprising a vehicle communication module, a vehicle state sensing module, a radar sensing module and a power supply module, wherein the vehicle communication module, the vehicle state sensing module, the fatigue and health monitoring module and the radar sensing module are respectively and electrically connected with the power supply module, and the vehicle communication module, the vehicle state sensing module and the radar sensing module are respectively in signal connection with the fatigue and health monitoring module;
the vehicle communication module is used for reading in vehicle state signals from a vehicle bus and transmitting the vehicle state signals to the vehicle state sensing module;
the vehicle state sensing module is used for judging whether the vehicle is driven stably according to a vehicle state signal transmitted by the vehicle communication module, judging whether the driving state of the vehicle meets the accurate measurement condition of the radar on the breathing and heartbeat frequency of the human body, and outputting the state of whether the vehicle is driven stably to the fatigue and health monitoring module;
the radar sensing module is used for measuring the heartbeat, the breathing frequency and the intensity of a driver by transmitting and receiving millimeter wave signals and transmitting the heartbeat, the breathing frequency and the intensity of the driver to the fatigue and health monitoring module;
the fatigue and health monitoring module is used for receiving information transmitted by the vehicle communication module, the vehicle state sensing module and the radar sensing module, judging the fatigue state and the health state of a driver and sending an alarm signal when the state of the driver is abnormal;
the vehicle state comprises a steering angle, a vehicle running speed and an automatic driving starting condition.
2. The system for monitoring fatigue and health of a driver for automatic driving according to claim 1, wherein the radar sensing module measures respiratory characteristics and heartbeat characteristics of the driver, and specifically comprises:
echo signals are obtained through a millimeter wave radar, signal frequency spectrums are calculated, then corresponding frequency spectrums in frequency range where heartbeat and respiration are located are independently extracted, and influence of vehicle vibration is reduced and eliminated.
3. The system for monitoring fatigue and health of a driver for automatic driving according to claim 2, wherein a database of heartbeat and breath of a company driver and passengers is obtained, and the characteristics of the change of the heartbeat and breath of the driver with time are calculated in a statistical, machine learning and deep learning manner, so that the heartbeat and breath frequency and amplitude of the driver are separated independently and are distinguished from the characteristics of vehicle vibration;
in the driving process of the vehicle, a driver cannot change the heartbeat and the respiration data base continuously accumulated by the radar aiming at the unique tested object, and the characteristics of the heartbeat and the respiration changing along with time are further corrected, so that the heartbeat and the respiration characteristic range is more accurate, the screening is more efficient, and the misjudgment is further reduced.
4. The system for monitoring fatigue and health of a driver for automatic driving according to claim 1, wherein the method for obtaining the characteristics of the heartbeat and the breathing frequency and the amplitude of the heartbeat of the driver changing with time comprises:
counting the heartbeat frequency, the breathing amplitude and the heartbeat amplitude of a driver within the time T, and calculating the respective average value and variance of the heartbeat frequency, the breathing amplitude and the heartbeat amplitude to an N-order moment and the covariance between every two of the heartbeat frequency, the breathing amplitude and the heartbeat amplitude, wherein the average value calculation formula is as follows:
Figure FDA0003469087940000021
wherein X represents any one of heartbeat frequency, heartbeat amplitude, respiratory frequency and respiratory amplitude, and XiA sample representing X;
within a certain time, X is sampled N times, N samples are generated, X is respectively1To XN，
The variance is calculated by
Var{X}＝E{[X-E{X}]2}＝E{X2}-(E{X})2
The covariance is calculated by
Cij＝Cov{Xi,Xj}＝E{(Xi-E{Xi})(Xj-E{Xj)}
Wherein Xi，XjIs any two of heart rate, heart amplitude, respiratory rate and respiratory amplitudeAfter the statistical characteristics of a certain number of people under different conditions are counted, the breathing and heartbeat characteristics of the driver are summarized.
5. The system for monitoring fatigue and health of a driver for automatic driving according to claim 1, wherein the method for calculating the characteristics of the heartbeat and the respiration of the driver over time by machine learning comprises: inputting a sample and a training set of the heartbeat frequency, the heartbeat amplitude, the breathing frequency and the breathing amplitude of which the states of the driver are marked, then training based on a machine learning algorithm to finally form a trained machine learning model, and inputting millimeter wave signals into the trained machine learning model to judge the fatigue state and the health state of the driver.
6. The system for monitoring fatigue and health of a driver for automatic driving according to claim 1, wherein the method for calculating the characteristics of the heartbeat and the respiration of the driver over time through deep learning comprises: inputting the heartbeat frequency, heartbeat amplitude, respiratory frequency and respiratory amplitude samples marked with the state of the driver into a Radar transducer for deep learning to obtain a deep training model, and inputting millimeter wave signals into the deep training model of the driver, so that the fatigue state and the health state of the driver can be judged.
7. The system for monitoring fatigue and health of driver for automatic driving according to claim 1, wherein the method for the vehicle state sensing module to determine whether the vehicle is running smoothly is as follows:
s1, the vehicle does not turn or the turning radius is lower than the set value within the set time;
s2, the speed of the vehicle is higher than a first threshold value within a set time, and the speed change of the vehicle is lower than a second threshold value;
s3, detecting that the acceleration sensor in the vertical direction does not have large acceleration change in the vertical direction of the vehicle;
if at least one of the determination conditions S1, S2, and S3 is satisfied, it is determined that the vehicle is running smoothly, and this is suitable for driver fatigue and health monitoring.
8. The automated driver fatigue and health monitoring system of claim 1, the method for judging the fatigue state of the driver comprises the steps of judging that the vehicle state is suitable for measuring the heartbeat, the breathing frequency, the heartbeat intensity and the breathing intensity of the driver in the time period of the fatigue and health condition of the driver, and counting for a long time according to a time axis, detecting whether the variation of the heartbeat and the breathing frequency, the heartbeat intensity and the breathing intensity of a driver along with the time has the tendency of gradually slowing down the heartbeat and the breathing frequency and weakening the intensity, if the heart rate or breathing rate of the driver is at a low value for a long time, and the intensity is also at a low value, it is determined that the driver is fatigued, and if the road conditions change, such as the road is bumpy, the vehicle turns more, the vehicles in front are more, and the frequency and the intensity of breathing heartbeat of the driver do not change along with the change, judging that the driver is tired.
9. The system for monitoring fatigue and health of a driver for automatic driving according to claim 1, wherein the method for determining the fatigue state of the driver further comprises detecting by detecting the frequency of operation of the vehicle by the driver:
the method for detecting the operation frequency of the vehicle comprises the following steps: when the vehicle is automatically driven, a driver mainly focuses on vehicle operation on window lifting, windscreen wipers, an air conditioning ventilation system and a vehicle-mounted entertainment system, the operation comprises hand touch operation, voice operation and gesture recognition operation, and the average interval and the interval time of last operation of the driver in the past 30-60 min for operating the system are continuously counted;
if the driver does not have the operation on average or recently over the preset time, judging that the driver is tired;
when weather changes, such as from raining to raining, or from raining to raining, the system does not control the wiper and the driver does not operate the wiper, or the temperature in the vehicle changes obviously to a temperature higher than a high-temperature preset value or lower than a low-temperature preset value and the driver does not adjust the air conditioner, the driver is judged to be tired.
10. The system for monitoring fatigue and health of a driver for automatic driving according to claim 4, wherein the state of the vehicle is judged to be suitable for measuring the frequency and intensity of heartbeat and breath of the driver in the time period of the fatigue and health condition of the driver for long-time statistics on a time axis, the frequency and intensity of heartbeat and breath of the driver are detected along with the change of the time, and if the frequency of heartbeat and breath are extremely high or extremely low and the intensity of heartbeat and breath is extremely high or extremely low, the health problem of the driver is judged to occur.
CN202210038454.2A 2022-01-13 2022-01-13 Driver fatigue and health monitoring system for automatic driving Pending CN114506335A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210038454.2A CN114506335A (en) 2022-01-13 2022-01-13 Driver fatigue and health monitoring system for automatic driving

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210038454.2A CN114506335A (en) 2022-01-13 2022-01-13 Driver fatigue and health monitoring system for automatic driving

Publications (1)

Publication Number Publication Date
CN114506335A true CN114506335A (en) 2022-05-17

Family

ID=81549164

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210038454.2A Pending CN114506335A (en) 2022-01-13 2022-01-13 Driver fatigue and health monitoring system for automatic driving

Country Status (1)

Country Link
CN (1) CN114506335A (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150051547A (en) * 2013-11-04 2015-05-13 현대오트론 주식회사 Driver assistance systems and controlling method for the same
CN107336711A (en) * 2017-06-19 2017-11-10 北京汽车股份有限公司 Vehicle and its automated driving system and method
CN108784669A (en) * 2018-06-08 2018-11-13 张洪平 A kind of contactless heartbeat and disordered breathing monitor system and method
CN209795467U (en) * 2018-12-26 2019-12-17 长沙智能驾驶研究院有限公司 Fatigue driving detection system and vehicle
CN111657889A (en) * 2020-06-03 2020-09-15 大连理工大学 Non-contact driver fatigue detection method based on millimeter wave radar
CN111968341A (en) * 2020-08-21 2020-11-20 无锡威孚高科技集团股份有限公司 Fatigue driving detection system and method
CN112220480A (en) * 2020-10-21 2021-01-15 合肥工业大学 Driver state detection system and vehicle based on millimeter wave radar and camera fusion
CN112829767A (en) * 2021-02-22 2021-05-25 清华大学苏州汽车研究院（相城） Automatic driving control system and method based on monitoring misoperation of driver

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150051547A (en) * 2013-11-04 2015-05-13 현대오트론 주식회사 Driver assistance systems and controlling method for the same
CN107336711A (en) * 2017-06-19 2017-11-10 北京汽车股份有限公司 Vehicle and its automated driving system and method
CN108784669A (en) * 2018-06-08 2018-11-13 张洪平 A kind of contactless heartbeat and disordered breathing monitor system and method
CN209795467U (en) * 2018-12-26 2019-12-17 长沙智能驾驶研究院有限公司 Fatigue driving detection system and vehicle
CN111657889A (en) * 2020-06-03 2020-09-15 大连理工大学 Non-contact driver fatigue detection method based on millimeter wave radar
CN111968341A (en) * 2020-08-21 2020-11-20 无锡威孚高科技集团股份有限公司 Fatigue driving detection system and method
CN112220480A (en) * 2020-10-21 2021-01-15 合肥工业大学 Driver state detection system and vehicle based on millimeter wave radar and camera fusion
CN112829767A (en) * 2021-02-22 2021-05-25 清华大学苏州汽车研究院（相城） Automatic driving control system and method based on monitoring misoperation of driver

Similar Documents

Publication Publication Date Title
CN105677039B (en) Method and device for detecting driving state based on gesture and wearable device
CN103824420B (en) Fatigue driving identification system based on heart rate variability non-contact measurement
US7912796B2 (en) System and method for real-time recognition of driving patterns
CN105261151B (en) High-grade highway driver fatigue condition detection method based on operation behavior feature
CN109147279B (en) Driver fatigue driving monitoring and early warning method and system based on Internet of vehicles
CN110766982B (en) Vehicle collision detection system based on vehicle-mounted sensor
CN106571015A (en) Driving behavior data collection method based on Internet
CN105564436A (en) Advanced driver assistance system
CN111753674A (en) Fatigue driving detection and identification method based on deep learning
CN107662613B (en) A kind of extreme driving behavior recognition methods and system based on mobile intelligent perception
CN106882141A (en) Driving behavior detection method and device
CN112644506A (en) Method for detecting driver driving distraction based on model long-time memory neural network LSTM-NN
CN108021875A (en) A kind of vehicle driver's personalization fatigue monitoring and method for early warning
CN111645694A (en) Driver driving state monitoring system and method based on attitude estimation
CN112698660B (en) Driving behavior visual perception device and method based on 9-axis sensor
CN114330449A (en) Vehicle collision detection method and system based on feature time domain matching
CN108898795B (en) Pedestrian state real-time analysis method and system based on mobile device
CN210000130U (en) automobile self-recognition deceleration system
CN114506335A (en) Driver fatigue and health monitoring system for automatic driving
CN109398366B (en) Early warning method for time pressure state of driver
CN116597611A (en) Method, system and device for monitoring and early warning driver state
CN113362565A (en) Fatigue driving early warning method and system based on decision algorithm
CN115635982A (en) Automatic driving method and system based on driving efficiency and comfort
CN205220773U (en) Steering wheel is prevented sleepy electron device and is prevented sleepy system
CN110459034A (en) A kind of fatigue driving method for early warning and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination