CN116588115A - Vehicle safety system based on driver state analysis - Google Patents
Vehicle safety system based on driver state analysis Download PDFInfo
- Publication number
- CN116588115A CN116588115A CN202211639119.4A CN202211639119A CN116588115A CN 116588115 A CN116588115 A CN 116588115A CN 202211639119 A CN202211639119 A CN 202211639119A CN 116588115 A CN116588115 A CN 116588115A
- Authority
- CN
- China
- Prior art keywords
- module
- vehicle
- detection system
- driver
- analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 40
- 238000001514 detection method Methods 0.000 claims abstract description 92
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 claims abstract description 40
- 238000004891 communication Methods 0.000 claims abstract description 19
- 238000007726 management method Methods 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000008451 emotion Effects 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 6
- 230000035622 drinking Effects 0.000 claims description 6
- 230000001815 facial effect Effects 0.000 claims description 6
- 230000008921 facial expression Effects 0.000 claims description 6
- 210000003128 head Anatomy 0.000 claims description 5
- 230000006855 networking Effects 0.000 claims description 5
- 230000006399 behavior Effects 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000002618 waking effect Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 230000002996 emotional effect Effects 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 9
- 238000000034 method Methods 0.000 description 8
- 230000014509 gene expression Effects 0.000 description 5
- 230000008909 emotion recognition Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W50/16—Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B7/00—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00
- G08B7/06—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00 using electric transmission, e.g. involving audible and visible signalling through the use of sound and light sources
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0818—Inactivity or incapacity of driver
- B60W2040/0836—Inactivity or incapacity of driver due to alcohol
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/223—Posture, e.g. hand, foot, or seat position, turned or inclined
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/229—Attention level, e.g. attentive to driving, reading or sleeping
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/24—Drug level, e.g. alcohol
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
The invention discloses a vehicle safety system based on driver state analysis, which comprises: the system comprises a face detection system, an alcohol sensing detection system, a safety early warning system, a vehicle control system, a remote management system and a vehicle-mounted network communication system; the face detection system and the alcohol sensing detection system feed detection results back to the safety early warning system and the vehicle control system through the vehicle-mounted network communication system and further feed detection results back to the remote management system. The invention can comprehensively judge and analyze the bad state of the driver, thereby realizing vehicle early warning, vehicle related function control and remote alarm of the internet of vehicles so as to ensure the driving safety.
Description
Technical Field
The invention relates to the technical field of intelligent transportation, in particular to a vehicle safety system based on driver state analysis.
Background
With the continuous development of intelligent vehicles and intelligent transportation industry, people often neglect the important role of the people in the driving process. At present, the requirements of people on traffic safety and safety in the running process of vehicles are still very high, and drivers need to correctly understand driving responsibility to ensure safety. It is necessary to monitor the driver during driving of a conventional car or using an intelligent vehicle having a driving assistance function.
From the driver's perspective, security-threatening artifacts include, but are not limited to, fatigue, drunk driving, angry driving, and the like. At present, the traffic and transportation industry in China starts to popularize driver monitoring equipment and capture the state of a driver when the vehicle runs. However, firstly, the device is insufficient as a detection device, and at present, a single system is often used for alcohol detection, fatigue detection and emotion detection in the market, so that the factors are not well integrated; secondly, the structure of the equipment and the installation position of the equipment threaten drivers when danger occurs, and the equipment has certain limitation; finally, most of the devices are aimed at traditional vehicles, and some research on emerging intelligent vehicles is lacking.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a vehicle safety system based on driver state analysis, which can comprehensively judge and analyze bad states of drivers, thereby realizing vehicle early warning, vehicle related function control and remote alarm of the Internet of vehicles so as to ensure driving safety.
In order to solve the above technical problems, the present invention provides a vehicle safety system based on driver state analysis, comprising: the system comprises a face detection system, an alcohol sensing detection system, a safety early warning system, a vehicle control system, a remote management system and a vehicle-mounted network communication system; the face detection system and the alcohol sensing detection system feed detection results back to the safety early warning system and the vehicle control system through the vehicle-mounted network communication system and further feed detection results back to the remote management system.
Preferably, the face detection system and the alcohol sensing detection system are integrated together, and are arranged on an upright post in the cockpit or on an instrument panel right below the driver.
Preferably, the face detection system comprises a real-time head and face high-speed camera module and an analysis storage module; the real-time head and face high-speed camera module acquires facial expressions and fatigue states based on a camera and dynamically transmits the results to the analysis storage module; the analysis storage module is used for carrying out intelligent calculation analysis on the acquired data, and independently carrying out calculation analysis on the emotion state or the fatigue state or carrying out information fusion joint analysis.
Preferably, the emotional state is calculated and analyzed, and a space transformation network STN and a Bayesian convolutional neural network are utilized; the space conversion network STN carries out translation, scaling and rotation space transformation on the facial expression image input by the camera, carries out gesture calibration on the facial image, focuses on the divided interesting facial areas and plays a role in image preprocessing;
the Bayesian convolutional neural network is adopted for classification, so that accurate and efficient identification is realized, and ResNet18 is used as a basic network.
Preferably, the fatigue state is calculated and analyzed, the detection of the human eye state is enhanced by utilizing the constraint of the two eyes, and the fatigue characteristics are obtained by using a layered hidden Markov model based on human eye state sequences with different sampling frequencies;
for human eye detection, selecting a target detection network Faster-RCNN as a human eye detector to acquire the position and the open-close state of human eyes, and for a section of driver video sequence, acquiring a section of corresponding human eye observation state sequence O= { O1, O2, …, ot } by carrying out human eye detection on each frame of image;
the layered hidden Markov model nodrrowsy-HHMM in the non-fatigue state and the layered hidden Markov model drawsy-HHMM in the fatigue state are used for representing human eye changes in the two states, and O= { O1, O2, …, ot } is input to obtain the probability of waking up or fatigue.
Preferably, the alcohol sensing detection system comprises an alcohol detection module and an analysis storage module; the alcohol detection module is used for detecting the alcohol concentration in the gas exhaled by the driver and judging whether the driver drives after drinking; the analysis storage module uses the data storage device to store the identified information and communicates the information with the vehicle-mounted network communication system.
Preferably, the safety early warning system comprises an audio early warning module, a touch seat early warning module and a flashing early warning module; the audio early warning module uses voice prompt; the touch seat early warning module is arranged at the bottom of the seat and the backrest by using an arranged vibration device; the flash early warning module uses the light emitting diode device to perform flash early warning.
Preferably, the vehicle control system comprises a speed limiting module, an AEB automatic emergency braking module, an LKA lane keeping auxiliary module, an FCW collision avoiding module, a blind spot detecting module and a lane changing auxiliary module; the speed limiting module is used for controlling the highest running speed of the vehicle when receiving the abnormal monitoring results sent by the face detection system and the alcohol sensing detection system; the AEB automatic emergency braking module is used for performing corresponding braking when receiving an abnormal monitoring result of the alcohol sensing detection system sent by the face detection system; the LKA lane keeping auxiliary module, the FCW collision avoidance module, the blind spot detection module and the lane change auxiliary module are used for being automatically started when abnormal monitoring results sent by the face detection system and the alcohol sensing detection system are received, so that safety is guaranteed.
Preferably, the remote management system comprises an alarm module, a related industry management client and a networking data set; the alarm module is linked with the alcohol sensing detection system, and alarms to related industry management client software through the Internet of vehicles when a driver has drunk driving behaviors, and the networking data set is used as a basic data source of the face detection system.
The beneficial effects of the invention are as follows: the invention can comprehensively detect and analyze the face dynamics of the driver in the process of driving the vehicle, strictly judge the emotion state and fatigue degree of the driver, simultaneously detect the drinking state by utilizing the respiratory alcohol concentration sensor, can react to single emotion, fatigue and drinking driving, and can also react when the negative states exist simultaneously; the method comprises the steps of sending a message to a relevant ECU of a control system of the vehicle by using a vehicle-mounted network communication system when the existence of negative states is detected, and controlling a relevant active and passive safety function; the potential dangerous state of the driver is interfered from the angles of early warning, control and active warning, and the risks caused by abnormal emotion states, fatigue states and drinking states are prevented, so that the running safety of the vehicle is improved.
Drawings
Fig. 1 is a schematic diagram of a system structure according to the present invention.
Fig. 2 is a schematic diagram of a face detection system according to the present invention.
Fig. 3 is a schematic diagram of a emotion recognition implementation process of the present invention.
FIG. 4 is a schematic diagram of the fatigue recognition process according to the present invention.
FIG. 5 is a schematic diagram of the alcohol sensing system according to the present invention.
Fig. 6 is a schematic structural diagram of a safety warning system according to the present invention.
Fig. 7 is a schematic diagram of a vehicle control system according to the present invention.
Fig. 8 is a schematic diagram of a vehicular network communication system according to the present invention.
Fig. 9 is a schematic diagram of a remote management system according to the present invention.
Detailed Description
As shown in fig. 1, a vehicle safety system based on driver state analysis includes: the system comprises a face detection system, an alcohol sensing detection system, a safety early warning system, a vehicle control system, a remote management system and a vehicle-mounted network communication system; the face detection system and the alcohol sensing detection system feed detection results back to the safety early warning system and the vehicle control system through the vehicle-mounted network communication system and further feed detection results back to the remote management system.
The human face detection system and the alcohol sensing detection system are integrated together and are arranged on an upright post in the cockpit or on an instrument panel right below the driver.
As shown in fig. 2, the face detection system comprises a real-time head and face high-speed camera module and an analysis storage module; the real-time head and face high-speed camera module acquires facial expressions and fatigue states based on a camera and dynamically transmits the results to the analysis storage module; the analysis storage module is used for carrying out intelligent calculation analysis on the acquired data, and independently carrying out calculation analysis on the emotion state or the fatigue state or carrying out information fusion joint analysis.
As shown in fig. 3, the emotion recognition algorithm of the present invention is divided into two parts: a spatial transformation network (Spatial Transformer Networks, STN) and a bayesian convolutional neural network, respectively. The spatial transformation network STN mainly performs spatial transformation such as translation, scaling, rotation and the like on the facial expression image input by the camera, so that the gesture calibration is performed on the facial image, the influence of the image on tasks such as classification, positioning and the like due to spatial diversity is reduced, the classified facial region of interest is focused, and the network can better extract valuable expression related features.
Because the expression category is very easy to be confused, a certain uncertainty exists in expression recognition. Therefore, the algorithm in the invention adopts Bayesian convolutional neural network to classify so as to realize accurate and efficient identification. Because of the small number of basic expression categories, the expression recognition algorithm of the invention uses ResNet18 with a simpler structure as a basic network.
As for the dataset, note that the dataset in the present identification procedure uses a networked RAF-DB dataset from a remote management center.
As shown in fig. 4, the general scheme of the invention for fatigue detection is to enhance human eye state detection by utilizing binocular position constraint first; secondly, a layered hidden Markov model based on human eye state sequences of different sampling frequencies is used to obtain fatigue characteristics.
For human eye detection, the device selects an object detection network Faster-RCNN as a human eye detector to acquire the position and the open/close state of human eyes. Therefore, for a section of driver video sequence, by performing human eye detection on each frame of image, a section of corresponding human eye observation state sequence o= { O1, O2, …, ot } can be obtained.
For the fatigue discrimination strategy of the present invention, the device comprises a layered hidden Markov model nodrowsy-HHMM in a non-fatigue state and a layered hidden Markov model drawsy-HHMM in a fatigue state, which are used for representing human eye changes in the two states respectively. By inputting o= { O1, O2, …, ot } as described above, the probability of waking up or tiring can be obtained. And saving the obtained result in a data storage module.
For data sets, note that for data sets in the present identification process, networked NTHU-DDD data sets from a remote management center are used.
As shown in fig. 5, the alcohol sensing detection system includes an alcohol detection module and an analysis storage module; the alcohol detection module is used for detecting the alcohol concentration in the gas exhaled by the driver and judging whether the driver drives after drinking; the analysis storage module uses the data storage device to store the identified information and communicates the information with the vehicle-mounted network communication system.
As shown in fig. 6, the safety precaution system includes an audio precaution module, a haptic seat precaution module, and a flashing precaution module; the audio early warning module uses voice prompt; the touch seat early warning module is arranged at the bottom of the seat and the backrest by using an arranged vibration device; the flash early warning module uses the light emitting diode device to perform flash early warning.
As shown in fig. 7, the vehicle control system includes a speed limit module, an AEB automatic emergency brake module, an LKA lane keeping assist module, an FCW collision avoidance module, a blind spot detection module, and a lane change assist module; the speed limiting module is used for controlling the highest running speed of the vehicle when receiving the abnormal monitoring results sent by the face detection system and the alcohol sensing detection system; the AEB automatic emergency braking module is used for performing corresponding braking when receiving an abnormal monitoring result of the alcohol sensing detection system sent by the face detection system; the LKA lane keeping auxiliary module, the FCW collision avoidance module, the blind spot detection module and the lane change auxiliary module are used for being automatically started when abnormal monitoring results sent by the face detection system and the alcohol sensing detection system are received, so that safety is guaranteed.
As shown in fig. 8, the in-vehicle network communication system includes a network communication module and a data storage calculation module; the network communication module includes available high-low speed buses, which interact with the rest of the subsystems. The vehicle-mounted network communication system is used as a core of information communication, at least has a network communication function and a data storage and calculation function, so as to analyze and transmit data information generated by the face detection system and the alcohol sensing detection system, and can alarm the drunk behavior detected by the alcohol sensing detection system to a related software platform of a remote management system.
As shown in fig. 9, the remote management system includes an alarm module, a related industry management client, and a networking dataset; the alarm module is linked with the alcohol sensing detection system, and alarms to related industry management client software through the Internet of vehicles when a driver has drunk driving behaviors, and the networking data set is used as a basic data source of the face detection system.
Claims (9)
1. A vehicle safety system based on driver state analysis, comprising: the system comprises a face detection system, an alcohol sensing detection system, a safety early warning system, a vehicle control system, a remote management system and a vehicle-mounted network communication system; the face detection system and the alcohol sensing detection system feed detection results back to the safety early warning system and the vehicle control system through the vehicle-mounted network communication system and further feed detection results back to the remote management system.
2. A vehicle safety system based on driver status analysis as claimed in claim 1, wherein the face detection system and the alcohol sensing detection system are integrated together, provided on a pillar in the cockpit, or provided on a dashboard directly under the driver.
3. The vehicle safety system based on driver state analysis according to claim 1, wherein the face detection system comprises a real-time head-face high-speed camera module and an analysis storage module; the real-time head and face high-speed camera module acquires facial expressions and fatigue states based on a camera and dynamically transmits the results to the analysis storage module; the analysis storage module is used for carrying out intelligent calculation analysis on the acquired data, and independently carrying out calculation analysis on the emotion state or the fatigue state or carrying out information fusion joint analysis.
4. A driver state analysis based vehicle safety system according to claim 3, wherein the emotional state is computationally analyzed using a spatial transformation network STN and a bayesian convolutional neural network; the space conversion network STN carries out translation, scaling and rotation space transformation on the facial expression image input by the camera, carries out gesture calibration on the facial image, focuses on the divided interesting facial areas and plays a role in image preprocessing;
the Bayesian convolutional neural network is adopted for classification, so that accurate and efficient identification is realized, and ResNet18 is used as a basic network.
5. A driver state analysis based vehicle safety system according to claim 3, wherein the fatigue state is computationally analyzed, the human eye state detection is enhanced by using the binocular position constraint, and the fatigue characteristics are obtained by using a hierarchical hidden markov model based on human eye state sequences of different sampling frequencies;
for human eye detection, selecting a target detection network Faster-RCNN as a human eye detector to acquire the position and the open-close state of human eyes, and for a section of driver video sequence, acquiring a section of corresponding human eye observation state sequence O= { O1, O2, …, ot } by carrying out human eye detection on each frame of image;
the layered hidden Markov model nodrrowsy-HHMM in the non-fatigue state and the layered hidden Markov model drawsy-HHMM in the fatigue state are used for representing human eye changes in the two states, and O= { O1, O2, …, ot } is input to obtain the probability of waking up or fatigue.
6. The driver state analysis-based vehicle safety system according to claim 1, wherein the alcohol sensing detection system includes an alcohol detection module and an analysis storage module; the alcohol detection module is used for detecting the alcohol concentration in the gas exhaled by the driver and judging whether the driver drives after drinking; the analysis storage module uses the data storage device to store the identified information and communicates the information with the vehicle-mounted network communication system.
7. The driver state analysis-based vehicle safety system of claim 1, wherein the safety pre-warning system comprises an audio pre-warning module, a haptic seat pre-warning module, and a flashing pre-warning module; the audio early warning module uses voice prompt; the touch seat early warning module is arranged at the bottom of the seat and the backrest by using an arranged vibration device; the flash early warning module uses the light emitting diode device to perform flash early warning.
8. The driver state analysis-based vehicle safety system of claim 1, wherein the vehicle control system includes a speed limit module, an AEB automatic emergency braking module, an LKA lane keeping assist module, an FCW collision avoidance module, a blind spot detection module, and a lane change assist module; the speed limiting module is used for controlling the highest running speed of the vehicle when receiving the abnormal monitoring results sent by the face detection system and the alcohol sensing detection system; the AEB automatic emergency braking module is used for performing corresponding braking when receiving an abnormal monitoring result of the alcohol sensing detection system sent by the face detection system; the LKA lane keeping auxiliary module, the FCW collision avoidance module, the blind spot detection module and the lane change auxiliary module are used for being automatically started when abnormal monitoring results sent by the face detection system and the alcohol sensing detection system are received, so that safety is guaranteed.
9. The driver state analysis based vehicle security system of claim 1, wherein the remote management system includes an alarm module, a related industry management client, and a networked data set; the alarm module is linked with the alcohol sensing detection system, and alarms to related industry management client software through the Internet of vehicles when a driver has drunk driving behaviors, and the networking data set is used as a basic data source of the face detection system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211639119.4A CN116588115A (en) | 2022-12-20 | 2022-12-20 | Vehicle safety system based on driver state analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211639119.4A CN116588115A (en) | 2022-12-20 | 2022-12-20 | Vehicle safety system based on driver state analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116588115A true CN116588115A (en) | 2023-08-15 |
Family
ID=87597784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211639119.4A Pending CN116588115A (en) | 2022-12-20 | 2022-12-20 | Vehicle safety system based on driver state analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116588115A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115894A (en) * | 2023-10-24 | 2023-11-24 | 吉林省田车科技有限公司 | Non-contact driver fatigue state analysis method, device and equipment |
-
2022
- 2022-12-20 CN CN202211639119.4A patent/CN116588115A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115894A (en) * | 2023-10-24 | 2023-11-24 | 吉林省田车科技有限公司 | Non-contact driver fatigue state analysis method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112277955B (en) | Driving assistance method, device, equipment and storage medium | |
| CN107832748B (en) | Shared automobile driver replacing system and method | |
| CN105938657B (en) | The Auditory Perception and intelligent decision system of a kind of automatic driving vehicle | |
| Braunagel et al. | Ready for take-over? A new driver assistance system for an automated classification of driver take-over readiness | |
| CN109774471B (en) | Vehicle-mounted equipment suitable for safe driving | |
| CN108995654B (en) | Driver state identification method and system | |
| Doshi et al. | On the roles of eye gaze and head dynamics in predicting driver's intent to change lanes | |
| CN102874259B (en) | A kind of automobile driver mood monitors and vehicle control system | |
| CN110077414A (en) | A kind of vehicle driving safety support method and system based on driver status monitoring | |
| US20080169914A1 (en) | Warning a vehicle operator of unsafe operation behavior based on a 3d captured image stream | |
| CN109145719B (en) | Driver fatigue state identification method and system | |
| US10131362B1 (en) | Automobile detection system | |
| CN107207013A (en) | Automatic Pilot control device and automatic Pilot control method and program | |
| CN106585624A (en) | Driver state monitoring method and device | |
| CN1802273A (en) | Method and arrangement for controlling vehicular subsystems based on interpreted driver activity | |
| CN115027484A (en) | Human-computer fusion perception method for high-degree automatic driving | |
| CN116588115A (en) | Vehicle safety system based on driver state analysis | |
| JP6677126B2 (en) | Interactive control device for vehicles | |
| CN109987090A (en) | Driving assistance system and method | |
| CN107403541A (en) | The system of real-time eye recognition monitoring fatigue driving | |
| CN114092922A (en) | Driver emotion recognition and behavior intervention method based on specificity | |
| Kutila et al. | Driver cognitive distraction detection: Feature estimation and implementation | |
| CN114735010A (en) | Intelligent vehicle driving control method and system based on emotion recognition and storage medium | |
| CN210760742U (en) | Intelligent vehicle auxiliary driving system | |
| CN107958573A (en) | A kind of traffic control method being applied in intelligent vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |