CN111645694A - Driver driving state monitoring system and method based on attitude estimation - Google Patents
Driver driving state monitoring system and method based on attitude estimation Download PDFInfo
- Publication number
- CN111645694A CN111645694A CN202010293941.4A CN202010293941A CN111645694A CN 111645694 A CN111645694 A CN 111645694A CN 202010293941 A CN202010293941 A CN 202010293941A CN 111645694 A CN111645694 A CN 111645694A
- Authority
- CN
- China
- Prior art keywords
- driving
- driver
- server
- vehicle speed
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 title claims abstract description 32
- 241000282414 Homo sapiens Species 0.000 claims abstract description 12
- 238000004891 communication Methods 0.000 claims description 33
- 230000009471 action Effects 0.000 claims description 27
- 238000012360 testing method Methods 0.000 claims description 13
- 238000011156 evaluation Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000001788 irregular Effects 0.000 claims description 3
- 206010039203 Road traffic accident Diseases 0.000 description 6
- 230000006399 behavior Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000033001 locomotion Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010041349 Somnolence Diseases 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006806 disease prevention Effects 0.000 description 1
- 230000035622 drinking Effects 0.000 description 1
- 239000003651 drinking water Substances 0.000 description 1
- 235000020188 drinking water Nutrition 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W40/09—Driving style or behaviour
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo or light sensitive means, e.g. infrared sensors
- B60W2420/403—Image sensing, e.g. optical camera
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2520/00—Input parameters relating to overall vehicle dynamics
- B60W2520/10—Longitudinal speed
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/16—Ratio selector position
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/223—Posture, e.g. hand, foot, or seat position, turned or inclined
Abstract
The invention discloses a driver driving state monitoring system and method based on attitude estimation, the system comprises a local vehicle-mounted end and a remote server end, an image frame is intercepted in real time through a video shot by a camera, the head and body attitude of a driver is monitored in real time by using a human body attitude estimation algorithm, the driving state information of the driver and the real-time speed information of a vehicle are fused, and the driving danger grade discrimination algorithm is used for realizing the early warning of dangerous driving. The system has an off-line mode and an on-line mode, and in the on-line mode, the server side can monitor safe driving in real time and store data; while in the off-line mode, the monitoring is performed only by the local processor. The method for monitoring safe driving of the driver realizes non-invasive real-time monitoring, and can obtain higher reliability and accuracy by combining an off-line mode and an on-line mode.
Description
Technical Field
The invention relates to a computer vision image processing technology used in the field of driving assistance systems and automatic driving, in particular to a driver driving state monitoring system and method based on attitude estimation.
Background
According to the investigation of World Health Organization (World Health Organization), traffic accidents are the eighth leading cause of death of human beings, and 130 million people die of traffic accidents and 205,000 million people are injured/disabled in traffic accidents each year in the World. In recent years, the number of traffic accidents caused by driver distraction has gradually increased, and a disease prevention and control center provides wide distraction driving definitions and divides distraction driving into three types according to vision, manual operation and cognition. The driver uses a mobile phone or sends a short message to cause the distraction that the sight line is far away from the driving route in the driving process, which belongs to the visual distraction; manual distraction mainly involves various activities of the driver leaving the steering wheel with both hands in the driving process, such as eating, drinking, operating a navigator, and the like; the driver is in a safe driving position and may be mentally distracted, such distraction due to inattention or drowsiness being a cognitive distraction. It is reported that the distracted driving is a main cause of the traffic accident, and an in-vehicle monitoring system (in-vehicle monitoring system) can play a key role in preventing and alleviating the traffic accident by warning the distracted driver and adaptively adjusting a safety mechanism. Even autodrive vehicles are being developed that require continuous monitoring of the driver to ensure that the driver can return to the driving task in an emergency.
At present, the recognition methods of driving behaviors are mainly classified into contact driving behavior recognition methods and computer vision-based methods. The contact type driving behavior identification method is mainly used for identifying the current state of a driver based on analysis of physiological and biomedical signals such as Electrocardiogram (ECG), Electroencephalogram (EOG), electroencephalogram (EEG) and optical mark sensors. The driving posture recognition method is an intrusive detection method, reduces user experience and increases hardware cost, and has certain limitation in practical application; the second method is a method for recognizing the posture of the driver based on computer vision, which has lower equipment cost and more natural and comfortable data acquisition for the driver, so that more and more research institutions and scholars intensively research the recognition of the driving posture based on computer vision. Most of the existing monitoring systems only use the head posture, the sight direction or the facial expression of a driver to judge the fatigue and the attention concentration degree, and lack abundant research on the macroscopic motions of limbs of the driver, such as the head position, the body angle, the hand motion and the like.
In the automotive industry, human body pose estimation algorithms are gaining increasing attention for their ability to capture human motion poses without the need for any sensor instruments. The classification method of the human body posture estimation method can be roughly classified into top-down and bottom-up methods. Top-down methods the top-down method first detects human bounding boxes and then decomposes each bounding box into principal key points and skeleton. Although the task of general attitude estimation has made major breakthroughs, few attitude estimation models in the literature are specifically targeted for on-board use. In the driving environment of 10 months in 2019, Chun et al simulate the driving posture for the first time to construct a data set for driving posture estimation, and provide an NADS-Net network for estimating the posture of a front-row driver and whether to fasten a safety belt.
Disclosure of Invention
The invention aims to solve the technical problem of providing a system and a method for monitoring the driving state of a driver based on attitude estimation aiming at the defects involved in the background technology.
The invention adopts the following technical scheme for solving the technical problems:
a driver driving state monitoring system based on attitude estimation comprises a high-definition camera, a vehicle speed sensor, a gear sensor, a local processor, a loudspeaker, a server and a wireless communication module, wherein the high-definition camera is installed in a vehicle and used for collecting driving attitudes;
the local processor is respectively and electrically connected with the high-definition camera, the vehicle speed sensor, the gear sensor, the wireless communication module and the loudspeaker; the wireless communication module is wirelessly connected with the server end;
the high-definition camera is arranged in the vehicle and used for shooting a front driving image of a driver and transmitting the front driving image to the local processor;
the vehicle speed sensor and the gear sensor are respectively used for acquiring the real-time vehicle speed and the gear state of the vehicle and transmitting the real-time vehicle speed and the gear state to the local processor;
the local processor comprises a server connection testing unit, an uploading unit, a local danger level evaluation module and an execution unit; the server comprises a database and a remote danger rating module;
the server connection testing unit is used for testing whether the wireless communication module can be wirelessly connected with the server or not and transmitting the result to the uploading unit;
the uploading unit is used for transmitting a driving image, a real-time vehicle speed and a gear state of a driver to the server when the wireless communication module can be in wireless connection with the server, judging the driving state of the driver through the server and transmitting the result to the execution unit, transmitting the driving image, the real-time vehicle speed and the gear state of the driver to the local danger level evaluation module when the wireless communication module cannot be in wireless connection with the server, judging the driving state of the driver through the local danger level evaluation module and transmitting the result to the execution unit;
the local danger rating module and the remote danger rating module both comprise a human body posture detection unit, a driving state monitoring unit, a first timer, a second timer, a counter and a judgment unit, wherein:
the human body posture estimation unit is used for calculating the deflection angle of the head of the driver in the driving process according to the received driving image, judging whether the driver has actions of non-double-hand driving or not and transmitting the actions to the driving state monitoring unit;
the driving state monitoring unit is used for sending a message of 'normal head deflection angle' to the judging unit when the head deflection angle of the driver is smaller than or equal to a preset angle threshold value, controlling a first timer to time the duration time T2 of the first timer when the head deflection angle of the driver is larger than or equal to the preset angle threshold value, and transmitting the time T2 to the judging unit; sending a message of 'bimanual' to the judging unit when no bimanual action exists by the driver, timing the duration time T1 of the bimanual action exists by the driver and transmitting T1 to the judging unit;
the judging unit is used for judging whether the driving state of the driver belongs to safe driving, dangerous driving or non-standard driving according to the received information of the driving state monitoring unit, controlling the counter to be cleared and then starting counting when the driving is not standard, and transmitting the judging result and the counting times n of the counter to the executing unit;
the execution unit is used for controlling the loudspeaker to warn the driver when the counting number N is greater than or equal to a preset number threshold N or the driving state belongs to dangerous driving;
the database is used for storing the data received by the server and the judgment result of the remote danger rating module.
As a further optimization scheme of the attitude estimation-based driver driving state monitoring system, the local processor adopts an STM32 single chip microcomputer based on an ARM inner core.
As a further optimization scheme of the driver driving state monitoring system based on attitude estimation, the wireless communication module adopts a 4G communication technology to realize data transmission.
The invention also discloses a monitoring method of the driver driving state monitoring system based on attitude estimation, which is characterized by comprising the following steps:
step 1), after a vehicle is started, a high-definition camera shoots a front driving image of a driver and transmits the front driving image to a local processor, and a vehicle speed sensor and a gear sensor respectively obtain the real-time vehicle speed and the gear state of the vehicle and transmit the real-time vehicle speed and the gear state to the local processor;
step 2), the local processor tests whether the wireless communication module can be wirelessly connected with the server, if the wireless communication module can be wirelessly connected with the server, the driving image, the real-time vehicle speed and the gear state of the driver are transmitted to the server, and the driving state of the driver is judged through the server; if the wireless communication module can not be wirelessly connected with the server, the local processor directly judges the driving state of the driver; the specific steps for judging the driving state of the driver are as follows:
step 2.1), searching the real-time vehicle speed and the non-double-hand driving dangerous time Tt corresponding to the gear state in a preset vehicle speed, gear and non-double-hand driving dangerous time comparison table according to the real-time vehicle speed and the gear state of the vehicle, and searching the real-time vehicle speed and the head deflection dangerous time Td corresponding to the gear state in the preset vehicle speed, gear and head deflection dangerous time comparison table;
step 2.2), calculating the deflection angle A of the head of the driver in the driving process according to the received driving image and judging whether the driver has actions of non-bimanual driving;
step 2.3), judging whether the head deflection angle A of the driver is smaller than or equal to a preset angle threshold value B, and judging whether the driver has non-double-hand driving action;
step 2.3.1), if A is larger than or equal to B, controlling a first timer to time the duration T2 of the first timer, and judging whether T2 is larger than or equal to Td;
step 2.3.2), if the driver has the non-double-hand driving action, controlling a second timer to count the duration time T1 and judging whether the T1 is more than or equal to Tt;
step 2.3.3), if A is less than or equal to B and the driver does not have the action of non-bimanual driving, judging that the driving state of the driver is safe driving;
if T2 is more than or equal to Td or T1 is more than or equal to Tt, the driving state of the driver is determined as dangerous driving;
determining that the driving state of the driver is the irregular driving if T1 < Tt and A ≦ B, or T1 < Tt and T2 < Td, or T2 < Td and the driver has no action of the bimanual driving;
step 3), when the driving state of the driver is the abnormal driving, counting the times n of the abnormal driving of the driver after the control counter is cleared;
and 4) when the counting number N is larger than or equal to a preset number threshold N or the driving state belongs to dangerous driving, the local processor controls the loudspeaker to warn the driver.
Compared with the prior art, the invention adopting the technical scheme has the following technical effects:
according to the method for monitoring the driving state of the driver based on the attitude estimation, provided by the invention, the driving attitude information of the driver is obtained in a non-invasive manner, the interference to the driver is small, the hardware requirement is low, and more reliable evaluation can be realized by comprehensively considering the body attitude and the head attitude. The off-line mode of the method can assist the vehicle-mounted equipment to accurately monitor the driving state of the driver aiming at individual driving, and the on-line mode can realize the multi-vehicle cooperative management function on the basis of the off-line mode.
Drawings
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a schematic diagram of a network test flow at startup according to the present invention;
FIG. 3 is a schematic flow chart of a monitoring method according to the present invention;
fig. 4 is a schematic diagram of an image information sample for monitoring a non-bimanual driving state of a driver.
Detailed Description
The technical scheme of the invention is further explained in detail by combining the attached drawings:
the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In the drawings, components are exaggerated for clarity.
As shown in fig. 1, the invention discloses a driver driving state monitoring system based on attitude estimation, which comprises a high-definition camera, a vehicle speed sensor, a gear sensor, a local processor, a loudspeaker, a server and a wireless communication module, wherein the high-definition camera is arranged in a vehicle and is used for acquiring driving attitude;
the local processor is respectively and electrically connected with the high-definition camera, the vehicle speed sensor, the gear sensor, the wireless communication module and the loudspeaker; the wireless communication module is wirelessly connected with the server end;
the high-definition camera is arranged in the vehicle and used for shooting a front driving image of a driver and transmitting the front driving image to the local processor;
the vehicle speed sensor and the gear sensor are respectively used for acquiring the real-time vehicle speed and the gear state of the vehicle and transmitting the real-time vehicle speed and the gear state to the local processor;
the local processor comprises a server connection testing unit, an uploading unit, a local danger level evaluation module and an execution unit; the server comprises a database and a remote danger rating module;
the server connection testing unit is used for testing whether the wireless communication module can be wirelessly connected with the server or not and transmitting the result to the uploading unit;
the uploading unit is used for transmitting the driving image, the real-time vehicle speed and the gear state of the driver to the server when the wireless communication module can be wirelessly connected with the server, judging the driving state of the driver through the server and transmitting the result to the execution unit, transmitting the driving image, the real-time vehicle speed and the gear state of the driver to the local danger rating module when the wireless communication module cannot be wirelessly connected with the server, judging the driving state of the driver through the local danger rating module and transmitting the result to the execution unit, as shown in fig. 2;
the local danger rating module and the remote danger rating module both comprise a human body posture detection unit, a driving state monitoring unit, a first timer, a second timer, a counter and a judgment unit, wherein:
the human body posture estimation unit is used for calculating the deflection angle of the head of the driver in the driving process according to the received driving image, judging whether the driver has actions of non-double-hand driving or not and transmitting the actions to the driving state monitoring unit;
the driving state monitoring unit is used for sending a message of 'normal head deflection angle' to the judging unit when the head deflection angle of the driver is smaller than or equal to a preset angle threshold value, controlling a first timer to time the duration time T2 of the first timer when the head deflection angle of the driver is larger than or equal to the preset angle threshold value, and transmitting the time T2 to the judging unit; sending a message of 'bimanual' to the judging unit when no bimanual action exists by the driver, timing the duration time T1 of the bimanual action exists by the driver and transmitting T1 to the judging unit;
the judging unit is used for judging whether the driving state of the driver belongs to safe driving, dangerous driving or non-standard driving according to the received information of the driving state monitoring unit, controlling the counter to be cleared and then starting counting when the driving is not standard, and transmitting the judging result and the counting times n of the counter to the executing unit;
the execution unit is used for controlling the loudspeaker to warn the driver when the counting number N is greater than or equal to a preset number threshold N or the driving state belongs to dangerous driving;
the database is used for storing the data received by the server and the judgment result of the remote danger rating module.
The local processor preferably adopts an STM32 singlechip based on an ARM core.
The wireless communication module preferentially adopts a 4G communication technology to realize data transmission.
As shown in fig. 3, the invention also discloses a monitoring method of the driver driving state monitoring system based on attitude estimation, which is characterized by comprising the following steps:
step 1), after a vehicle is started, a high-definition camera shoots a front driving image of a driver and transmits the front driving image to a local processor, and a vehicle speed sensor and a gear sensor respectively obtain the real-time vehicle speed and the gear state of the vehicle and transmit the real-time vehicle speed and the gear state to the local processor;
step 2), the local processor tests whether the wireless communication module can be wirelessly connected with the server, if the wireless communication module can be wirelessly connected with the server, the driving image, the real-time vehicle speed and the gear state of the driver are transmitted to the server, and the driving state of the driver is judged through the server; if the wireless communication module can not be wirelessly connected with the server, the local processor directly judges the driving state of the driver; the specific steps for judging the driving state of the driver are as follows:
step 2.1), searching the real-time vehicle speed and the non-double-hand driving dangerous time Tt corresponding to the gear state in a preset vehicle speed, gear and non-double-hand driving dangerous time comparison table according to the real-time vehicle speed and the gear state of the vehicle, and searching the real-time vehicle speed and the head deflection dangerous time Td corresponding to the gear state in the preset vehicle speed, gear and head deflection dangerous time comparison table;
step 2.2), calculating the deflection angle A of the head of the driver in the driving process according to the received driving image and judging whether the driver has actions of non-bimanual driving;
step 2.3), judging whether the head deflection angle A of the driver is smaller than or equal to a preset angle threshold value B, and judging whether the driver has non-double-hand driving action;
step 2.3.1), if A is larger than or equal to B, controlling a first timer to time the duration T2 of the first timer, and judging whether T2 is larger than or equal to Td;
step 2.3.2), if the driver has the non-double-hand driving action, controlling a second timer to count the duration time T1 and judging whether the T1 is more than or equal to Tt;
step 2.3.3), if A is less than or equal to B and the driver does not have the action of non-bimanual driving, judging that the driving state of the driver is safe driving;
if T2 is more than or equal to Td or T1 is more than or equal to Tt, the driving state of the driver is determined as dangerous driving;
determining that the driving state of the driver is the irregular driving if T1 < Tt and A ≦ B, or T1 < Tt and T2 < Td, or T2 < Td and the driver has no action of the bimanual driving;
the following table specifically shows:
step 3), when the driving state of the driver is the abnormal driving, counting the times n of the abnormal driving of the driver after the control counter is cleared;
and 4) when the counting number N is larger than or equal to a preset number threshold N or the driving state belongs to dangerous driving, the local processor controls the loudspeaker to warn the driver.
As shown in fig. 4, various activities of the driver leaving the steering wheel with both hands during driving may cause distraction, such as visual distraction of the driver's eyes away from the driving route by using a mobile phone or sending a short message, or manual distraction of eating things, drinking water, operating a navigator, and the like.
The invention can also add a display at the local end, thereby being convenient for regulation and control.
Without departing from the principle of the invention, the invention may also be modified and modified in several ways, such as: and adding new functions such as weather broadcasting, GPS positioning and the like in an online mode.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The above embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, it should be understood that the above embodiments are only illustrative of the present invention and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (4)
1. A driver driving state monitoring system based on attitude estimation is characterized by comprising a high-definition camera, a vehicle speed sensor, a gear sensor, a local processor, a loudspeaker, a server and a wireless communication module, wherein the high-definition camera is installed in a vehicle and used for collecting driving attitudes;
the local processor is respectively and electrically connected with the high-definition camera, the vehicle speed sensor, the gear sensor, the wireless communication module and the loudspeaker; the wireless communication module is wirelessly connected with the server end;
the high-definition camera is arranged in the vehicle and used for shooting a front driving image of a driver and transmitting the front driving image to the local processor;
the vehicle speed sensor and the gear sensor are respectively used for acquiring the real-time vehicle speed and the gear state of the vehicle and transmitting the real-time vehicle speed and the gear state to the local processor;
the local processor comprises a server connection testing unit, an uploading unit, a local danger level evaluation module and an execution unit; the server comprises a database and a remote danger rating module;
the server connection testing unit is used for testing whether the wireless communication module can be wirelessly connected with the server or not and transmitting the result to the uploading unit;
the uploading unit is used for transmitting a driving image, a real-time vehicle speed and a gear state of a driver to the server when the wireless communication module can be in wireless connection with the server, judging the driving state of the driver through the server and transmitting the result to the execution unit, transmitting the driving image, the real-time vehicle speed and the gear state of the driver to the local danger level evaluation module when the wireless communication module cannot be in wireless connection with the server, judging the driving state of the driver through the local danger level evaluation module and transmitting the result to the execution unit;
the local danger rating module and the remote danger rating module both comprise a human body posture detection unit, a driving state monitoring unit, a first timer, a second timer, a counter and a judgment unit, wherein:
the human body posture estimation unit is used for calculating the deflection angle of the head of the driver in the driving process according to the received driving image, judging whether the driver has actions of non-double-hand driving or not and transmitting the actions to the driving state monitoring unit;
the driving state monitoring unit is used for sending a message of 'normal head deflection angle' to the judging unit when the head deflection angle of the driver is smaller than or equal to a preset angle threshold value, controlling a first timer to time the duration time T2 of the first timer when the head deflection angle of the driver is larger than or equal to the preset angle threshold value, and transmitting the time T2 to the judging unit; sending a message of 'bimanual' to the judging unit when no bimanual action exists by the driver, timing the duration time T1 of the bimanual action exists by the driver and transmitting T1 to the judging unit;
the judging unit is used for judging whether the driving state of the driver belongs to safe driving, dangerous driving or non-standard driving according to the received information of the driving state monitoring unit, controlling the counter to be cleared and then starting counting when the driving is not standard, and transmitting the judging result and the counting times n of the counter to the executing unit;
the execution unit is used for controlling the loudspeaker to warn the driver when the counting number N is greater than or equal to a preset number threshold N or the driving state belongs to dangerous driving;
the database is used for storing the data received by the server and the judgment result of the remote danger rating module.
2. The attitude estimation-based driver driving state monitoring system according to claim 1, wherein the local processor employs an STM32 single chip microcomputer based on an ARM core.
3. The attitude estimation-based driver driving state monitoring system according to claim 1, wherein the wireless communication module adopts 4G communication technology to realize data transmission.
4. The monitoring method of the attitude estimation-based driver driving state monitoring system based on claim 1 is characterized by comprising the following steps:
step 1), after a vehicle is started, a high-definition camera shoots a front driving image of a driver and transmits the front driving image to a local processor, and a vehicle speed sensor and a gear sensor respectively obtain the real-time vehicle speed and the gear state of the vehicle and transmit the real-time vehicle speed and the gear state to the local processor;
step 2), the local processor tests whether the wireless communication module can be wirelessly connected with the server, if the wireless communication module can be wirelessly connected with the server, the driving image, the real-time vehicle speed and the gear state of the driver are transmitted to the server, and the driving state of the driver is judged through the server; if the wireless communication module can not be wirelessly connected with the server, the local processor directly judges the driving state of the driver; the specific steps for judging the driving state of the driver are as follows:
step 2.1), searching the real-time vehicle speed and the non-double-hand driving dangerous time Tt corresponding to the gear state in a preset vehicle speed, gear and non-double-hand driving dangerous time comparison table according to the real-time vehicle speed and the gear state of the vehicle, and searching the real-time vehicle speed and the head deflection dangerous time Td corresponding to the gear state in the preset vehicle speed, gear and head deflection dangerous time comparison table;
step 2.2), calculating the deflection angle A of the head of the driver in the driving process according to the received driving image and judging whether the driver has actions of non-bimanual driving;
step 2.3), judging whether the head deflection angle A of the driver is smaller than or equal to a preset angle threshold value B, and judging whether the driver has non-double-hand driving action;
step 2.3.1), if A is larger than or equal to B, controlling a first timer to time the duration T2 of the first timer, and judging whether T2 is larger than or equal to Td;
step 2.3.2), if the driver has the non-double-hand driving action, controlling a second timer to count the duration time T1 and judging whether the T1 is more than or equal to Tt;
step 2.3.3), if A is less than or equal to B and the driver does not have the action of non-bimanual driving, judging that the driving state of the driver is safe driving;
if T2 is more than or equal to Td or T1 is more than or equal to Tt, the driving state of the driver is determined as dangerous driving;
determining that the driving state of the driver is the irregular driving if T1 < Tt and A ≦ B, or T1 < Tt and T2 < Td, or T2 < Td and the driver has no action of the bimanual driving;
step 3), when the driving state of the driver is the abnormal driving, counting the times n of the abnormal driving of the driver after the control counter is cleared;
and 4) when the counting number N is larger than or equal to a preset number threshold N or the driving state belongs to dangerous driving, the local processor controls the loudspeaker to warn the driver.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010293941.4A CN111645694B (en) | 2020-04-15 | 2020-04-15 | Driver driving state monitoring system and method based on attitude estimation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010293941.4A CN111645694B (en) | 2020-04-15 | 2020-04-15 | Driver driving state monitoring system and method based on attitude estimation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111645694A true CN111645694A (en) | 2020-09-11 |
| CN111645694B CN111645694B (en) | 2021-08-06 |
Family
ID=72341657
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010293941.4A Expired - Fee Related CN111645694B (en) | 2020-04-15 | 2020-04-15 | Driver driving state monitoring system and method based on attitude estimation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111645694B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541413A (en) * | 2020-11-30 | 2021-03-23 | 阿拉善盟特种设备检验所 | Dangerous behavior detection method and system for forklift driver practical operation examination and coaching |
| CN112849149A (en) * | 2021-01-28 | 2021-05-28 | 上海商汤临港智能科技有限公司 | Driver state detection method, driver state detection device, driver state detection apparatus, driver state detection medium, and driver state detection program product |
| CN113173168A (en) * | 2021-04-30 | 2021-07-27 | 重庆工程职业技术学院 | Intelligent driving control system and method for vehicle |
| TWI772078B (en) * | 2021-06-28 | 2022-07-21 | 神達數位股份有限公司 | Method and system for evaluating vehicle driver competency |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101444610B1 (en) * | 2013-07-04 | 2014-09-24 | 현대다이모스(주) | Driving pose inattention warning device |
| US9616809B1 (en) * | 2016-08-01 | 2017-04-11 | Denso International America, Inc. | Lane change prediction and turn signal activation upon observation of head and eye movement |
| US9679487B1 (en) * | 2015-01-20 | 2017-06-13 | State Farm Mutual Automobile Insurance Company | Alert notifications utilizing broadcasted telematics data |
| CN108696725A (en) * | 2018-06-01 | 2018-10-23 | 宜春学院 | A kind of camera long-range control method Internet-based and system |
| CN109747656A (en) * | 2017-11-02 | 2019-05-14 | 罗曦明 | Artificial intelligence vehicle assistant drive method, apparatus, equipment and storage medium |
| CN109866774A (en) * | 2019-01-09 | 2019-06-11 | 南京航空航天大学 | System and method is monitored based on Face datection and the driver fatigue state of authentication |
| CN110582437A (en) * | 2019-05-31 | 2019-12-17 | 驭势（上海）汽车科技有限公司 | driving reminding method, driving state detection method and computing device |
-
2020
- 2020-04-15 CN CN202010293941.4A patent/CN111645694B/en not_active Expired - Fee Related
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101444610B1 (en) * | 2013-07-04 | 2014-09-24 | 현대다이모스(주) | Driving pose inattention warning device |
| US9679487B1 (en) * | 2015-01-20 | 2017-06-13 | State Farm Mutual Automobile Insurance Company | Alert notifications utilizing broadcasted telematics data |
| US9616809B1 (en) * | 2016-08-01 | 2017-04-11 | Denso International America, Inc. | Lane change prediction and turn signal activation upon observation of head and eye movement |
| CN109747656A (en) * | 2017-11-02 | 2019-05-14 | 罗曦明 | Artificial intelligence vehicle assistant drive method, apparatus, equipment and storage medium |
| CN108696725A (en) * | 2018-06-01 | 2018-10-23 | 宜春学院 | A kind of camera long-range control method Internet-based and system |
| CN109866774A (en) * | 2019-01-09 | 2019-06-11 | 南京航空航天大学 | System and method is monitored based on Face datection and the driver fatigue state of authentication |
| CN110582437A (en) * | 2019-05-31 | 2019-12-17 | 驭势（上海）汽车科技有限公司 | driving reminding method, driving state detection method and computing device |
Non-Patent Citations (1)
| Title |
|---|
| 汪旭，陈仁文，黄斌: "基于Android系统的司机驾驶安全监测系统的实现", 《电子测量技术》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541413A (en) * | 2020-11-30 | 2021-03-23 | 阿拉善盟特种设备检验所 | Dangerous behavior detection method and system for forklift driver practical operation examination and coaching |
| CN112541413B (en) * | 2020-11-30 | 2024-02-23 | 阿拉善盟特种设备检验所 | Dangerous behavior detection method and system for forklift driver real operation assessment and coaching |
| CN112849149A (en) * | 2021-01-28 | 2021-05-28 | 上海商汤临港智能科技有限公司 | Driver state detection method, driver state detection device, driver state detection apparatus, driver state detection medium, and driver state detection program product |
| CN113173168A (en) * | 2021-04-30 | 2021-07-27 | 重庆工程职业技术学院 | Intelligent driving control system and method for vehicle |
| TWI772078B (en) * | 2021-06-28 | 2022-07-21 | 神達數位股份有限公司 | Method and system for evaluating vehicle driver competency |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111645694B (en) | 2021-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111645694B (en) | Driver driving state monitoring system and method based on attitude estimation | |
| Biswal et al. | IoT-based smart alert system for drowsy driver detection | |
| Wang et al. | Driver fatigue detection: a survey | |
| JP7204739B2 (en) | Information processing device, mobile device, method, and program | |
| JP6911841B2 (en) | Image processing device, image processing method, and moving object | |
| US20190391581A1 (en) | Passenger Health Monitoring and Intervention for Autonomous Vehicles | |
| CN107832748B (en) | Shared automobile driver replacing system and method | |
| Kaplan et al. | Driver behavior analysis for safe driving: A survey | |
| Dong et al. | Driver inattention monitoring system for intelligent vehicles: A review | |
| Hossain et al. | IOT based real-time drowsy driving detection system for the prevention of road accidents | |
| KR102053794B1 (en) | Apparatus and method for delivering driver's movement intention based on bio-signals | |
| CN106571015A (en) | Driving behavior data collection method based on Internet | |
| CN110448316A (en) | Data processing equipment and method, wake up system and recording medium at monitoring system | |
| WO2020145161A1 (en) | Information processing device, mobile device, method, and program | |
| US20200104617A1 (en) | System and method for remote monitoring of a human | |
| US11751784B2 (en) | Systems and methods for detecting drowsiness in a driver of a vehicle | |
| WO2020161610A2 (en) | Adaptive monitoring of a vehicle using a camera | |
| CN108021875A (en) | A kind of vehicle driver's personalization fatigue monitoring and method for early warning | |
| CN111105594A (en) | Vehicle and recognition method and device for fatigue driving of driver | |
| CN109886148A (en) | A kind of driver's active warning system and device based on recognition of face | |
| CN205722310U (en) | A kind of train driver Activity recognition system based on video sequence | |
| Guria et al. | Iot-enabled driver drowsiness detection using machine learning | |
| CN106874831A (en) | Driving behavior method for detecting and its system | |
| CN109383516A (en) | A kind of anomaly analysis behavioral value system based on user behavior analysis | |
| Byrnes et al. | On Using Drivers' Eyes to Predict Accident-Causing Drowsiness Levels |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210806 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |