US20170210357A1 - Driver monitoring device - Google Patents
Driver monitoring device Download PDFInfo
- Publication number
- US20170210357A1 US20170210357A1 US15/324,015 US201515324015A US2017210357A1 US 20170210357 A1 US20170210357 A1 US 20170210357A1 US 201515324015 A US201515324015 A US 201515324015A US 2017210357 A1 US2017210357 A1 US 2017210357A1
- Authority
- US
- United States
- Prior art keywords
- driver
- posture
- unbalanced
- habitual
- monitoring device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012806 monitoring device Methods 0.000 title claims abstract description 24
- 230000002650 habitual effect Effects 0.000 claims abstract description 59
- 238000001514 detection method Methods 0.000 claims abstract description 20
- 230000000694 effects Effects 0.000 claims description 14
- 230000036544 posture Effects 0.000 description 223
- 238000010586 diagram Methods 0.000 description 9
- 210000003128 head Anatomy 0.000 description 6
- 238000000034 method Methods 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 5
- 208000003443 Unconsciousness Diseases 0.000 description 2
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
- B60T7/00—Brake-action initiating means
- B60T7/12—Brake-action initiating means for automatic initiation; for initiation not subject to will of driver or passenger
- B60T7/14—Brake-action initiating means for automatic initiation; for initiation not subject to will of driver or passenger operated upon collapse of driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60C—VEHICLE TYRES; TYRE INFLATION; TYRE CHANGING; CONNECTING VALVES TO INFLATABLE ELASTIC BODIES IN GENERAL; DEVICES OR ARRANGEMENTS RELATED TO TYRES
- B60C9/00—Reinforcements or ply arrangement of pneumatic tyres
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K28/00—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions
- B60K28/02—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver
- B60K28/06—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver responsive to incapacity of driver
- B60K28/066—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver responsive to incapacity of driver actuating a signalling device
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/21—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using visual output, e.g. blinking lights or matrix displays
- B60K35/22—Display screens
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- G06K9/00617—
-
- G06K9/00845—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/197—Matching; Classification
-
- B60K2350/106—
-
- B60K2350/1076—
-
- B60K2350/2013—
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/16—Type of output information
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/20—Optical features of instruments
- B60K2360/21—Optical features of instruments using cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/103—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using camera systems provided with artificial illumination device, e.g. IR light source
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8006—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
- B60T2220/00—Monitoring, detecting driver behaviour; Signalling thereof; Counteracting thereof
- B60T2220/02—Driver type; Driving style; Driver adaptive features
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0062—Adapting control system settings
- B60W2050/0075—Automatic parameter input, automatic initialising or calibrating means
- B60W2050/0083—Setting, resetting, calibration
- B60W2050/0088—Adaptive recalibration
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/223—Posture, e.g. hand, foot, or seat position, turned or inclined
Definitions
- the present disclosure relates to a driver monitoring device for monitoring a vehicle driver.
- a monitoring device which determines whether or not a driver is in a dangerous state, for example, looking aside or drowsing, and which alerts the driver when the driver is determined to be in a dangerous state.
- the driver monitoring device disclosed in Patent Literature 1 determines, based on an image taken by photographing a pattern of bright spots projected on the face of a driver, angles of inclination of the driver's face when the driver turns his/her face up or down or left or right or when the driver otherwise inclines his/her face.
- the device disclosed in Patent Literature 1 determines whether or not the driver is in a dangerous state, for example, looking aside or drowsing and, when the driver is determined to be in a dangerous state, alerts the driver.
- Patent Literature 1 JP H10-960A
- a driver may possibly be determined to be in a dangerous state and be alerted not only when the driver's posture is found unbalanced, for example, with the driver looking aside or drowsing but also when the driver is in his/her habitual unbalanced posture.
- An object of the present disclosure is to provide a driver monitoring device that can provide a driver with appropriate notification corresponding to the unbalanced posture of the driver.
- a driver monitoring device comprises: a detection unit that detects an unbalanced posture of a driver based on an image of a driver's seat of a vehicle captured by a camera; a determination unit that determines whether or not the unbalanced posture of the driver detected by the detection unit is a habitual posture of the driver; and a notification unit that notifies the driver of the unbalanced posture in different manners in accordance with whether the unbalanced posture determined is the habitual posture of the driver or the unbalanced posture determined is other than the habitual posture of the driver.
- an unbalanced posture of a driver is detected based on a camera-captured image and whether or not the detected unbalanced posture of the driver is a habitual posture of the driver is determined.
- the action to be taken may be of lower urgency than when the unbalanced posture is other than a habitual posture of the driver, but the driver may be preferably urged to improve his/her unbalanced posture.
- notifications about the unbalanced postures of the driver are given to the driver in different manners in accordance with whether the unbalanced posture is a habitual posture of the driver or the unbalanced posture is other than a habitual posture of the driver. In this way, it is possible to provide a driver with appropriate notification corresponding to the driver's unbalanced posture.
- FIG. 1 is a diagram illustrating a configuration of a driver monitoring device
- FIG. 2 is a diagram illustrating a vehicle interior where the driver monitoring device is installed
- FIG. 3 is a diagram illustrating an example of a posture unbalance magnitude
- FIG. 4A is a diagram illustrating variation with time of the magnitude of posture unbalance of a driver in a proper posture
- FIG. 4B is a diagram illustrating variation with time of the magnitude of posture unbalance of a driver in a habitual unbalanced posture
- FIG. 4C is a diagram illustrating variation with time of the magnitude of posture unbalance of a driver in an unbalanced posture due to fatigue or reduced attentiveness;
- FIG. 5 is a flowchart of a procedure for posture unbalance notification
- FIG. 6 is a diagram illustrating an example of a proper driving posture to be provided to a driver
- FIG. 7A is a diagram illustrating example effects of an unbalanced posture to be provided to a driver.
- FIG. 7B is a diagram illustrating other example effects of an unbalanced posture to be provided to a driver.
- the driver monitoring device 10 includes a driver monitoring ECU 20 , a camera 31 , lighting 32 , a setting switch 36 , a display 33 and a speaker 34 .
- the camera 31 is, for example, a CCD camera having a high sensitivity characteristic in a near-infrared region.
- the lighting 32 including, for example, near-infrared LEDs is for emitting near-infrared light and is installed near the camera 31 .
- the camera 31 is, as shown in FIG. 2 , installed in a meter hood 53 and photographs the driver's seat of the vehicle illuminated by the lighting 32 .
- the lighting 32 emits near-infrared light toward around a head rest 51 a of a driver's seat 51 and the camera 31 photographs an area around the headrest 51 a illuminated by the lighting 32 .
- the camera 31 photographs the driver's face and its periphery. Every time the camera 31 captures one frame of image data, the camera 31 sends the image data to the driver monitoring ECU 20 .
- the camera 31 may be installed over the dashboard or on the steering column or on an A-pillar. Also, the camera 31 may include plural cameras and such plural cameras may be installed in different locations.
- the driver monitoring ECU 20 is a microcomputer having, for example, a CPU, an ROM, an RAM and I/O parts and realizes such functions as a detection unit 21 , a determination unit 22 , an opening degree calculation unit 23 and a vehicle stopping unit 24 .
- the driver monitoring ECU 20 adjusts the amount of light of the lighting 32 depending on the time of day so as to facilitate photographing of the driver's face.
- the detection unit 21 corresponds to an example of detection means.
- the determination unit 22 corresponds to an example of determination means.
- the opening degree calculation unit 23 corresponds to an example of opening degree calculation means.
- the vehicle stopping unit 24 corresponds to an example of vehicle stopping means.
- the detection unit 21 detects an unbalanced posture of a driver based on an image captured by the camera 31 .
- the detection unit 21 recognizes an image captured by the camera 31 and calculates magnitude L of the driver's posture unbalance.
- the posture unbalance magnitude L may be defined appropriately. For example, as shown in FIG. 3 , the distance between the center of the head rest 51 a set as a reference point and the center of the driver's face may be defined as the posture unbalance magnitude L. Instead of the center of the head rest 51 a, the center of the steering wheel 52 may be set as a reference point.
- the center of the face of the driver who is seated in a proper posture on the driver's seat so as to face forward may be set as a reference point, and the distance between this reference point and the center of the driver's face in the captured image may be regarded as the posture unbalance magnitude L.
- the detection unit 21 detects the driver's posture as an unbalanced posture.
- the threshold Th is set, for example, to be a threshold for allowing proper and effective operation of an airbag at the time of a collision.
- the threshold Th is set to such a value that when the posture unbalance magnitude L is larger than the threshold Th, the proper and effective operation of an airbag is difficult.
- the airbag may be, for example, a driver seat airbag to be deployed over the steering wheel 52 or a side airbag to be deployed between the seat 51 and a door panel.
- the threshold Th may be set to be appropriate for the location where the airbag is installed and the size of the airbag. Different thresholds Th may be set for different directions of posture unbalance, i.e. for different directions of posture shifting from a proper driving posture.
- the determination unit 22 determines whether or not a driver's unbalanced posture detected by the detection unit 21 is a habitual posture of the driver. As shown in FIG. 4A , when a driver is in a proper posture, the posture unbalance magnitude L is smaller than the threshold Th. Also, as shown in FIG. 4B , when a driver's posture is habitually unbalanced, the posture unbalance magnitude L is kept larger than the threshold Th continuously from immediately after start of driving. Also, as shown in FIG. 4C , when a driver's unbalanced posture is due to fatigue or reduced attentiveness of the driver, the posture unbalance magnitude L increases with time although the posture unbalance magnitude L is small immediately after the start of driving.
- the posture unbalance magnitude L stays being larger than the threshold Th.
- a driver's unbalanced posture is a habitual posture of the driver, it is not necessary to immediately alert the driver.
- his/her fatigue easily accumulates, so that it is desirable to urge the driver to correct his/her posture.
- the unbalanced posture may lead to an accident, so that it is necessary to immediately alert the driver and urge the driver to take a rest.
- the determination unit 22 determines that the unbalanced posture is a habitual posture of the driver. Also, when the posture unbalance magnitude L exceeds the threshold Th by increasing with time after being initially not larger than the threshold Th, the determination unit 22 determines that the unbalanced posture is due to fatigue or reduced attentiveness of the driver, namely, that the unbalanced posture is other than a habitual posture of the driver.
- the determination unit 22 determines that the unbalanced posture is other than a habitual posture of the driver.
- the determination unit 22 determines that an unbalanced posture of the driver is a habitual posture of the driver, the determination unit 22 instructs the display 33 and the speaker 34 to output posture information at predetermined timing.
- the determination unit 22 determines that an unbalanced posture of the driver is other than a habitual posture of the driver, the determination unit 22 instructs the display 33 and the speaker 34 to immediately alert the driver.
- the posture unbalance magnitude L after sharply increasing to over the threshold Th decreases to or below the threshold L before the time T 1 elapses, the driver is supposed to have regained a proper posture, for example, after once looking aside, so that it is not necessary to notify the driver of the unbalanced posture.
- the opening degree calculation unit 23 recognizes an image captured by the camera 31 and calculates the degree of opening of the driver's eyes. The degree of opening of an eye is calculated based on the distance between the opposing edges of the top and bottom lids of the eye.
- the vehicle stopping unit 24 sends a signal for safely stopping the vehicle to a vehicle control device 40 .
- the vehicle stopping unit 24 determines that the driver's eyes are closed, when the driver's eye opening degree calculated by the opening degree calculation unit 23 remains smaller than threshold Tho (opening degree threshold) longer than time T 2 (opening degree determination time).
- the display 33 (corresponding to a notification unit and an example of notification means) is, for example, a display of an in-vehicle information terminal or an in-vehicle display installed in a meter panel.
- the display 33 may be a touch display having an LCD panel or an organic EL panel.
- the speaker 34 (corresponding to a notification unit and an example of notification means) is an in-vehicle speaker used, for example, along with an in-vehicle information terminal and an in-vehicle audio device.
- the display 33 and the speaker 34 notify the driver of an unbalanced posture in different manners in accordance with whether the determination unit 22 determines that the unbalanced posture is a habitual posture of the driver or the determination unit 22 determines that the unbalanced posture is other than a habitual posture of the driver.
- the display 33 and the speaker 34 when an unbalanced posture of the driver is a habitual posture of the driver, the display 33 and the speaker 34 output posture information at predetermined timing.
- the posture information is for urging the driver to correct his/her posture and preferably includes information either about a proper driving posture or about effects of the unbalanced posture.
- the predetermined timing is set to be when the vehicle is stopped, more posture information can be safely outputted than when the vehicle is running.
- the display 33 and the speaker 34 immediately alert the driver.
- the setting switch 36 (corresponding to a setting unit and an example of setting means) is for use by a driver to set the frequency of output of posture information.
- the setting switch 36 may be a mechanical switch or a touch switch formed integrally on the display 33 .
- frequent outputs of posture information after detection of a habitual unbalanced posture of the driver may be annoying.
- the setting switch 36 allows such a driver to set a desired frequency of posture information output, for example, to have posture information outputted only once in a trip of driving or once in several hours.
- the procedure is executed by the driver monitor ECU 20 .
- the unbalanced posture is a habitual posture of the driver (S 12 ).
- the unbalanced posture has been continuously detected longer than the time T 1 from immediately after the vehicle was started, it is determined that the unbalanced posture is a habitual posture of the driver (S 12 : YES).
- “immediately after the vehicle was started” means “within a predetermined amount of time after the shift position was changed from a parking position” or “within a predetermined amount of time after the vehicle was started.”
- the shift position and starting of the vehicle are determined based on values detected by a vehicle sensor 41 .
- the unbalanced posture is determined to be a habitual posture of the driver, it is determined whether or not the vehicle is stopped (S 13 ).
- a determination of whether or not the vehicle speed is V km/h or lower is made based on a value detected by a vehicle sensor 41 .
- the “V km/h” may represent either 0 km/h (vehicle completely stopped) or a speed low enough (e.g. 1 km/h) for the vehicle to be regarded as stopped.
- processing of S 13 is repeated.
- FIG. 6 shows an example of a proper driving posture to be displayed.
- illustrations of a proper driving posture are displayed on the display 33 along with steps A to D to be followed to achieve the proper driving posture.
- the steps A to D are also voice-outputted from the speaker 34 .
- Step A reads, “Be seated facing front and adjust the position of the back so as to keep the back supporting you during driving.”
- Step B reads, “Adjust the seat position such that you can firmly step on the pedals and such that, when you grasp the steering wheel, your elbows are slightly bent.”
- Step C reads, “Make sure that the center in height of the headrest is around the upper ends of your ears.”
- Step D reads, “Wear the seat belt correctly.”
- Information about effects of an unbalanced posture is displayed on the display 33 and is also voice-outputted from the speaker 34 .
- Examples of information about effects of an unbalanced posture include “Continuing driving in an unbalanced posture will accumulate your fatigue and will increase the physical burden on your body” and “Driving in an unbalanced posture will make your driving operation awkward and will increase chances of causing an accident.”
- the information to be outputted when a habitual unbalanced posture of a driver is detected may be changed depending on the type of the unbalanced posture.
- the habitual unbalanced posture of a driver may be classified into types, for example, lean-forward, overly relaxed (bent backwards) and leaning against a door and the posture information corresponding to the detected type of unbalanced posture may be outputted.
- a table in which unbalanced posture types are classified may be prepared and a driver's posture detected by image recognition may be compared with the classified types.
- An alternative method may also be used to classify unbalanced posture types. Examples of posture information to be outputted for different types of unbalanced postures will be described in the following.
- an illustration of a lean-forward posture is displayed on the display 33 .
- the following messages ( 1 ) and ( 2 ) are displayed on the display 33 as information about effects of the lean-forward posture and are also voice-outputted from the speaker 34 .
- the message ( 1 ) reads, “In the event the airbag is deployed, you will be hit hard.”
- the message ( 2 ) reads, “The ranges of a rearward view and rearward side views that can be seen on the rearview mirror and side mirrors are shifted to expand your blind spots, making it difficult to sufficiently check the situation around the vehicle.”
- a possible measure which may be taken in this case is to output only the description of step A shown in FIG. 6 as information about a proper driving posture.
- an illustration of an overly relaxed posture as shown in FIG. 7B is displayed on the display 33 .
- the following messages ( 1 ) and ( 2 ) are displayed on the display 33 as information about effects of the overly relaxed posture and are also voice-outputted from the speaker 34 .
- the message ( 1 ) reads, “In the event of a collision, your body may get under the seat belt.”
- the message ( 2 ) reads, “You can't have an adequate forward view.”
- a possible measure which may be taken in this case is is to output only the description of step B shown in FIG. 6 as information about a proper driving posture.
- the driver is immediately alerted (S 15 ).
- an alarm sound is outputted from the speaker 34 .
- an alarm message for example, “Have a rest” is displayed on the display 33 and, in addition, the message is voice-outputted from the speaker 34 .
- the driver's posture is determined not to have improved (S 16 : NO) and whether or not the driver's eyes have been closed longer than the time T 2 is determined (S 17 ). Namely, whether or not the driver's eye opening degree has been smaller than the threshold Tho longer than the time T 2 is determined.
- a habitual unbalanced posture of a driver continues from immediately after driving is started. Therefore, when an unbalanced posture of a driver is continuously detected longer than the time T 1 from immediately after driving is started, the unbalanced posture can be determined to be a habitual posture of the driver.
- posture information is outputted at predetermined timing.
- the driver is immediately alerted.
- the driver can be given appropriate notifications depending on whether the detected unbalanced posture requires no urgent action to be taken or requires urgent action to be taken.
- posture information about a proper driving posture or/and information about effects of the unbalanced posture can be outputted to urge the driver to improve his/her driving posture.
- the predetermined timing is set to be when the vehicle is stopped, more information can be safely outputted than when the vehicle is running.
- the posture information to be outputted may be determined based on the type of the habitual unbalanced posture of the driver. In this way, the driver can efficiently obtain posture information corresponding to the type of his/her unbalanced posture.
- posture information related with a habitual unbalanced posture of a driver When posture information related with a habitual unbalanced posture of a driver is frequently outputted, the driver may feel annoyed. Therefore, the driver is allowed to set a desired frequency of posture information output. In this way, the driver can obtain posture information at a desired frequency.
- posture information may be outputted dividedly at different vehicle stop times. For example, an illustration of a proper driving posture and information about effects of the unbalanced posture may be outputted at different vehicle stop times. Also, when outputting illustrations of a proper driving posture as shown in FIG. 6 , the description of steps A to D may be outputted in sequence, for example, such that steps A and B are outputted at a first time when the vehicle is stopped and such that steps C and D are outputted when the vehicle is stopped next time. Outputting the divided posture information at different vehicle stop times enables posture information to be provided to a driver safely. In cases where, in the process of outputting divided posture information at different vehicle stop times, the driver's posture is improved before completion of the process, the output of the remaining posture information may be canceled.
- the driver monitoring device 10 may be provided with an external storage device 35 for storing information about the unbalanced posture in association with the driver when the unbalanced posture of the driver is determined to be a habitual posture of the driver.
- the driver may have an image of his/her habitual driving posture captured by the camera 31 beforehand and may have the captured image stored in the external storage device 35 . Then, when a posture of the driver detected by image recognition immediately after driving is started agrees with the information about the driver's habitual driving posture stored in the external storage device 35 , the determination unit 22 may determine that the detected posture of the driver is the driver's habitual posture.
- Posture information and an alert may be outputted to a driver using either one or both of the display 33 and the speaker 34 .
- Embodiments and example configurations according to the present disclosure have been illustrated, but the embodiments and configurations according to the present disclosure are not limited to those illustrated above. Embodiments and configurations that can be realized by appropriately combining technical elements disclosed in different embodiments and configurations are also included in the embodiments and configurations according to the present disclosure.
Landscapes
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Transportation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Emergency Management (AREA)
- Business, Economics & Management (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Theoretical Computer Science (AREA)
- Ophthalmology & Optometry (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Traffic Control Systems (AREA)
- Auxiliary Drives, Propulsion Controls, And Safety Devices (AREA)
- Emergency Alarm Devices (AREA)
Abstract
A driver monitoring device is provided. The driver monitoring device includes a detection unit that detects an unbalanced posture of a driver based on an image of a driver's seat of a vehicle captured by a camera; a determination unit that determines whether or not the unbalanced posture of the driver detected by the detection unit is a habitual posture of the driver; and a notification unit that notifies the driver of the unbalanced posture in different manners in accordance with whether the unbalanced posture determined is the habitual posture of the driver or the unbalanced posture determined is other than the habitual posture of the driver.
Description
- This application is based on Japanese Patent Application No. 2014-162875 filed on Aug. 8, 2014, the disclosure of which is incorporated herein by reference.
- The present disclosure relates to a driver monitoring device for monitoring a vehicle driver.
- A monitoring device have been proposed which determines whether or not a driver is in a dangerous state, for example, looking aside or drowsing, and which alerts the driver when the driver is determined to be in a dangerous state. For example, the driver monitoring device disclosed in
Patent Literature 1 determines, based on an image taken by photographing a pattern of bright spots projected on the face of a driver, angles of inclination of the driver's face when the driver turns his/her face up or down or left or right or when the driver otherwise inclines his/her face. Based on the angles of inclination of the driver's face determined, the device disclosed inPatent Literature 1 determines whether or not the driver is in a dangerous state, for example, looking aside or drowsing and, when the driver is determined to be in a dangerous state, alerts the driver. - Patent Literature 1: JP H10-960A
- According to the studies made by the inventor of the present application, in the case of the device disclosed in
Patent Literature 1, a driver may possibly be determined to be in a dangerous state and be alerted not only when the driver's posture is found unbalanced, for example, with the driver looking aside or drowsing but also when the driver is in his/her habitual unbalanced posture. - An object of the present disclosure is to provide a driver monitoring device that can provide a driver with appropriate notification corresponding to the unbalanced posture of the driver.
- In an example of the present disclosure, a driver monitoring device comprises: a detection unit that detects an unbalanced posture of a driver based on an image of a driver's seat of a vehicle captured by a camera; a determination unit that determines whether or not the unbalanced posture of the driver detected by the detection unit is a habitual posture of the driver; and a notification unit that notifies the driver of the unbalanced posture in different manners in accordance with whether the unbalanced posture determined is the habitual posture of the driver or the unbalanced posture determined is other than the habitual posture of the driver.
- According to the above driver monitoring device, an unbalanced posture of a driver is detected based on a camera-captured image and whether or not the detected unbalanced posture of the driver is a habitual posture of the driver is determined. When the unbalanced posture is a habitual posture of the driver, the action to be taken may be of lower urgency than when the unbalanced posture is other than a habitual posture of the driver, but the driver may be preferably urged to improve his/her unbalanced posture. Thus, notifications about the unbalanced postures of the driver are given to the driver in different manners in accordance with whether the unbalanced posture is a habitual posture of the driver or the unbalanced posture is other than a habitual posture of the driver. In this way, it is possible to provide a driver with appropriate notification corresponding to the driver's unbalanced posture.
- The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description made with reference to the accompanying drawings. In the drawings:
-
FIG. 1 is a diagram illustrating a configuration of a driver monitoring device; -
FIG. 2 is a diagram illustrating a vehicle interior where the driver monitoring device is installed; -
FIG. 3 is a diagram illustrating an example of a posture unbalance magnitude; -
FIG. 4A is a diagram illustrating variation with time of the magnitude of posture unbalance of a driver in a proper posture; -
FIG. 4B is a diagram illustrating variation with time of the magnitude of posture unbalance of a driver in a habitual unbalanced posture; -
FIG. 4C is a diagram illustrating variation with time of the magnitude of posture unbalance of a driver in an unbalanced posture due to fatigue or reduced attentiveness; -
FIG. 5 is a flowchart of a procedure for posture unbalance notification; -
FIG. 6 is a diagram illustrating an example of a proper driving posture to be provided to a driver; -
FIG. 7A is a diagram illustrating example effects of an unbalanced posture to be provided to a driver; and -
FIG. 7B is a diagram illustrating other example effects of an unbalanced posture to be provided to a driver. - In the following, an embodiment of a driver monitoring device will be described with reference to drawings.
- First, the configuration of a
driver monitoring device 10 of the present embodiment will be described with reference toFIGS. 1 and 2 . Thedriver monitoring device 10 includes adriver monitoring ECU 20, acamera 31,lighting 32, asetting switch 36, adisplay 33 and aspeaker 34. - The
camera 31 is, for example, a CCD camera having a high sensitivity characteristic in a near-infrared region. Thelighting 32 including, for example, near-infrared LEDs is for emitting near-infrared light and is installed near thecamera 31. Thecamera 31 is, as shown inFIG. 2 , installed in ameter hood 53 and photographs the driver's seat of the vehicle illuminated by thelighting 32. To be specific, thelighting 32 emits near-infrared light toward around ahead rest 51 a of a driver'sseat 51 and thecamera 31 photographs an area around theheadrest 51 a illuminated by thelighting 32. Namely, when a driver is seated on the driver's seat, thecamera 31 photographs the driver's face and its periphery. Every time thecamera 31 captures one frame of image data, thecamera 31 sends the image data to thedriver monitoring ECU 20. As long as the driver's face and its periphery can be photographed, thecamera 31 may be installed over the dashboard or on the steering column or on an A-pillar. Also, thecamera 31 may include plural cameras and such plural cameras may be installed in different locations. - The
driver monitoring ECU 20 is a microcomputer having, for example, a CPU, an ROM, an RAM and I/O parts and realizes such functions as adetection unit 21, a determination unit 22, an openingdegree calculation unit 23 and avehicle stopping unit 24. The driver monitoringECU 20 adjusts the amount of light of thelighting 32 depending on the time of day so as to facilitate photographing of the driver's face. Each of the above functions will be described in the following. Thedetection unit 21 corresponds to an example of detection means. The determination unit 22 corresponds to an example of determination means. The openingdegree calculation unit 23 corresponds to an example of opening degree calculation means. Thevehicle stopping unit 24 corresponds to an example of vehicle stopping means. - The
detection unit 21 detects an unbalanced posture of a driver based on an image captured by thecamera 31. To be specific, thedetection unit 21 recognizes an image captured by thecamera 31 and calculates magnitude L of the driver's posture unbalance. The posture unbalance magnitude L may be defined appropriately. For example, as shown inFIG. 3 , the distance between the center of the head rest 51 a set as a reference point and the center of the driver's face may be defined as the posture unbalance magnitude L. Instead of the center of the head rest 51 a, the center of thesteering wheel 52 may be set as a reference point. Alternatively, the center of the face of the driver who is seated in a proper posture on the driver's seat so as to face forward may be set as a reference point, and the distance between this reference point and the center of the driver's face in the captured image may be regarded as the posture unbalance magnitude L. - When the posture unbalance magnitude L is larger than threshold Th, the
detection unit 21 detects the driver's posture as an unbalanced posture. The threshold Th is set, for example, to be a threshold for allowing proper and effective operation of an airbag at the time of a collision. In this case, the threshold Th is set to such a value that when the posture unbalance magnitude L is larger than the threshold Th, the proper and effective operation of an airbag is difficult. The airbag may be, for example, a driver seat airbag to be deployed over thesteering wheel 52 or a side airbag to be deployed between theseat 51 and a door panel. In the case of a driver seat airbag, for example, when the driver's head projected on the upper surface of thesteering wheel 52 is not inside an airbag deployment area, the airbag cannot be properly effective at the time of a collision. Also, when the distance between the airbag and the driver's head is too short, deployment of the airbag subjects the driver's head to a strong impact. This prevents the airbag from being properly effective at the time of a collision. The threshold Th may be set to be appropriate for the location where the airbag is installed and the size of the airbag. Different thresholds Th may be set for different directions of posture unbalance, i.e. for different directions of posture shifting from a proper driving posture. - The determination unit 22 determines whether or not a driver's unbalanced posture detected by the
detection unit 21 is a habitual posture of the driver. As shown inFIG. 4A , when a driver is in a proper posture, the posture unbalance magnitude L is smaller than the threshold Th. Also, as shown inFIG. 4B , when a driver's posture is habitually unbalanced, the posture unbalance magnitude L is kept larger than the threshold Th continuously from immediately after start of driving. Also, as shown inFIG. 4C , when a driver's unbalanced posture is due to fatigue or reduced attentiveness of the driver, the posture unbalance magnitude L increases with time although the posture unbalance magnitude L is small immediately after the start of driving. In this case, after exceeding the threshold Th, the posture unbalance magnitude L stays being larger than the threshold Th. When a driver's unbalanced posture is a habitual posture of the driver, it is not necessary to immediately alert the driver. However, when the driver stays in an unbalanced posture, his/her fatigue easily accumulates, so that it is desirable to urge the driver to correct his/her posture. When a driver's unbalanced posture is due to fatigue or reduced attentiveness of the driver, the unbalanced posture may lead to an accident, so that it is necessary to immediately alert the driver and urge the driver to take a rest. - When a driver's unbalanced posture is continuously detected longer than time T1 (posture unbalance determination time) from immediately after driving is started (from within a predetermined time after driving is started), the determination unit 22 determines that the unbalanced posture is a habitual posture of the driver. Also, when the posture unbalance magnitude L exceeds the threshold Th by increasing with time after being initially not larger than the threshold Th, the determination unit 22 determines that the unbalanced posture is due to fatigue or reduced attentiveness of the driver, namely, that the unbalanced posture is other than a habitual posture of the driver. Or, when, after driving is started and a predetermined amount of time elapses, an unbalanced posture of the driver is continuously detected by the
detection unit 21 longer than the time T1, the determination unit 22 determines that the unbalanced posture is other than a habitual posture of the driver. - When the determination unit 22 determines that an unbalanced posture of the driver is a habitual posture of the driver, the determination unit 22 instructs the
display 33 and thespeaker 34 to output posture information at predetermined timing. When the determination unit 22 determines that an unbalanced posture of the driver is other than a habitual posture of the driver, the determination unit 22 instructs thedisplay 33 and thespeaker 34 to immediately alert the driver. When the posture unbalance magnitude L after sharply increasing to over the threshold Th decreases to or below the threshold L before the time T1 elapses, the driver is supposed to have regained a proper posture, for example, after once looking aside, so that it is not necessary to notify the driver of the unbalanced posture. - In cases where, after the unbalanced posture of a driver is determined to be other than a habitual posture of the driver and the driver is alerted, the driver's posture remains unimproved and the driver's eyes are closed, the driver is feared to be unconscious and be unable to drive. Hence, the opening
degree calculation unit 23 recognizes an image captured by thecamera 31 and calculates the degree of opening of the driver's eyes. The degree of opening of an eye is calculated based on the distance between the opposing edges of the top and bottom lids of the eye. When, after an unbalanced posture of the driver is determined to be other than a habitual posture of the driver and the driver is alerted, the driver's posture is not improved and the driver's eyes are determined to be closed, thevehicle stopping unit 24 sends a signal for safely stopping the vehicle to avehicle control device 40. Thevehicle stopping unit 24 determines that the driver's eyes are closed, when the driver's eye opening degree calculated by the openingdegree calculation unit 23 remains smaller than threshold Tho (opening degree threshold) longer than time T2 (opening degree determination time). - The display 33 (corresponding to a notification unit and an example of notification means) is, for example, a display of an in-vehicle information terminal or an in-vehicle display installed in a meter panel. The
display 33 may be a touch display having an LCD panel or an organic EL panel. The speaker 34 (corresponding to a notification unit and an example of notification means) is an in-vehicle speaker used, for example, along with an in-vehicle information terminal and an in-vehicle audio device. Thedisplay 33 and thespeaker 34 notify the driver of an unbalanced posture in different manners in accordance with whether the determination unit 22 determines that the unbalanced posture is a habitual posture of the driver or the determination unit 22 determines that the unbalanced posture is other than a habitual posture of the driver. - To be specific, when an unbalanced posture of the driver is a habitual posture of the driver, the
display 33 and thespeaker 34 output posture information at predetermined timing. The posture information is for urging the driver to correct his/her posture and preferably includes information either about a proper driving posture or about effects of the unbalanced posture. When the predetermined timing is set to be when the vehicle is stopped, more posture information can be safely outputted than when the vehicle is running. When an unbalanced posture of the driver is not a habitual posture of the driver, thedisplay 33 and thespeaker 34 immediately alert the driver. - The setting switch 36 (corresponding to a setting unit and an example of setting means) is for use by a driver to set the frequency of output of posture information. The setting
switch 36 may be a mechanical switch or a touch switch formed integrally on thedisplay 33. Depending on the driver, frequent outputs of posture information after detection of a habitual unbalanced posture of the driver may be annoying. The settingswitch 36 allows such a driver to set a desired frequency of posture information output, for example, to have posture information outputted only once in a trip of driving or once in several hours. - Next, a procedure for posture unbalance notification will be described with reference to the flowchart shown in
FIG. 5 . The procedure is executed by thedriver monitor ECU 20. - First, whether or not an unbalanced posture has been detected is determined (S10). Namely, whether or not the posture unbalance magnitude L has exceeded the threshold Th is determined. When no unbalanced posture has been detected (S10: NO), processing of S10 is repeated. When an unbalanced posture has been detected (S10: YES), whether or not the detection of the unbalanced posture has been continuous longer than the time T1 is determined (S11). When the detection of the unbalanced posture has not continued longer than the time T1 (S11: NO), processing returns to S10.
- When the detection of the unbalanced posture has continued longer than the time T1 (S11: YES), it is determined whether or not the unbalanced posture is a habitual posture of the driver (S12). When the unbalanced posture has been continuously detected longer than the time T1 from immediately after the vehicle was started, it is determined that the unbalanced posture is a habitual posture of the driver (S12: YES). In the present case, “immediately after the vehicle was started” means “within a predetermined amount of time after the shift position was changed from a parking position” or “within a predetermined amount of time after the vehicle was started.” The shift position and starting of the vehicle are determined based on values detected by a
vehicle sensor 41. When the detection of the unbalanced posture has been continuous longer than the time T1 not from immediately after the vehicle was started but from some time after the vehicle was started, it is determined that the unbalanced posture is other than a habitual posture of the driver (S12: NO). - When the unbalanced posture is determined to be a habitual posture of the driver, it is determined whether or not the vehicle is stopped (S13). To be specific, a determination of whether or not the vehicle speed is V km/h or lower is made based on a value detected by a
vehicle sensor 41. The “V km/h” may represent either 0 km/h (vehicle completely stopped) or a speed low enough (e.g. 1 km/h) for the vehicle to be regarded as stopped. When the vehicle is not determined to be stopped (S13: NO), processing of S13 is repeated. - When it is determined that the vehicle is in stop (S13: YES), information about a proper driving posture and effects of the unbalanced posture is outputted (S14).
FIG. 6 shows an example of a proper driving posture to be displayed. In the example shown inFIG. 6 , illustrations of a proper driving posture are displayed on thedisplay 33 along with steps A to D to be followed to achieve the proper driving posture. The steps A to D are also voice-outputted from thespeaker 34. Step A reads, “Be seated facing front and adjust the position of the back so as to keep the back supporting you during driving.” Step B reads, “Adjust the seat position such that you can firmly step on the pedals and such that, when you grasp the steering wheel, your elbows are slightly bent.” Step C reads, “Make sure that the center in height of the headrest is around the upper ends of your ears.” Step D reads, “Wear the seat belt correctly.” - Information about effects of an unbalanced posture is displayed on the
display 33 and is also voice-outputted from thespeaker 34. Examples of information about effects of an unbalanced posture include “Continuing driving in an unbalanced posture will accumulate your fatigue and will increase the physical burden on your body” and “Driving in an unbalanced posture will make your driving operation awkward and will increase chances of causing an accident.” - Also, the information to be outputted when a habitual unbalanced posture of a driver is detected may be changed depending on the type of the unbalanced posture. To be specific, the habitual unbalanced posture of a driver may be classified into types, for example, lean-forward, overly relaxed (bent backwards) and leaning against a door and the posture information corresponding to the detected type of unbalanced posture may be outputted.
- For use in determining the type of an unbalanced posture, a table in which unbalanced posture types are classified may be prepared and a driver's posture detected by image recognition may be compared with the classified types. An alternative method may also be used to classify unbalanced posture types. Examples of posture information to be outputted for different types of unbalanced postures will be described in the following.
- For example, when the driver is leaning forward with his/her back off the back of the driver's
seat 51 as shown inFIG. 7A , an illustration of a lean-forward posture is displayed on thedisplay 33. This will enable the driver to objectively grasp his/her habitual posture. In addition, the following messages (1) and (2) are displayed on thedisplay 33 as information about effects of the lean-forward posture and are also voice-outputted from thespeaker 34. The message (1) reads, “In the event the airbag is deployed, you will be hit hard.” The message (2) reads, “The ranges of a rearward view and rearward side views that can be seen on the rearview mirror and side mirrors are shifted to expand your blind spots, making it difficult to sufficiently check the situation around the vehicle.” A possible measure which may be taken in this case is to output only the description of step A shown inFIG. 6 as information about a proper driving posture. - When the driver is in an overly relaxed posture with his/her lower back not in contact with the
seat 51, an illustration of an overly relaxed posture as shown inFIG. 7B is displayed on thedisplay 33. In addition, the following messages (1) and (2) are displayed on thedisplay 33 as information about effects of the overly relaxed posture and are also voice-outputted from thespeaker 34. The message (1) reads, “In the event of a collision, your body may get under the seat belt.” The message (2) reads, “You can't have an adequate forward view.” A possible measure which may be taken in this case is is to output only the description of step B shown inFIG. 6 as information about a proper driving posture. - When the unbalanced posture is determined to be other than a habitual posture of the driver (S12: NO), the driver is immediately alerted (S15). To be specific, an alarm sound is outputted from the
speaker 34. Or, an alarm message, for example, “Have a rest” is displayed on thedisplay 33 and, in addition, the message is voice-outputted from thespeaker 34. - Subsequently, whether or not the driver's posture has been improved is determined (S16). When the posture unbalance magnitude L is found to have decreased to be not larger than the threshold Th, the driver's posture is determined to have improved (S16: YES) and processing is terminated.
- On the other hand, when the posture unbalance magnitude L is kept being larger than the threshold Th, the driver's posture is determined not to have improved (S16: NO) and whether or not the driver's eyes have been closed longer than the time T2 is determined (S17). Namely, whether or not the driver's eye opening degree has been smaller than the threshold Tho longer than the time T2 is determined.
- When the driver's eyes are determined to have been continuously closed longer than the time T2 (S17: YES), an instruction is issued to the
vehicle control device 40 to safely stop the vehicle through appropriate braking and steering. Also, to warn vehicles in the vicinity of the dangerous condition, instructions are issued to thevehicle control device 40 to light the headlights and sound the horn. Furthermore, to inform the other passengers of the vehicle about the dangerous condition, an instruction is issued to thespeaker 34 to output an alarm sound. - When the driver's eyes are determined not to have been continuously closed longer than the time T2 (S17: NO), whether or not the vehicle is stopped is determined (S19) as in S13. When the vehicle is determined to be stopped (S19: YES), information about a proper driving posture and effects of driving in the unbalanced posture is outputted (S20) as in S14. The processing is then terminated.
- The present embodiment described above has the following technical effects.
- When an unbalanced posture of a driver is detected based on a camera-captured image, whether or not the unbalanced posture is a habitual posture of the driver is determined. The driver is notified of the unbalanced posture in different manners in accordance with a result of determining whether or not the unbalanced posture is a habitual posture of the driver. In this way, notifications appropriate for the detected unbalanced postures can be given to the driver.
- Generally, a habitual unbalanced posture of a driver continues from immediately after driving is started. Therefore, when an unbalanced posture of a driver is continuously detected longer than the time T1 from immediately after driving is started, the unbalanced posture can be determined to be a habitual posture of the driver.
- When an unbalanced posture of a driver is determined to be a habitual posture of the driver, posture information is outputted at predetermined timing. When the unbalanced posture of the driver is determined to be other than a habitual posture of the driver, the driver is immediately alerted. Thus, the driver can be given appropriate notifications depending on whether the detected unbalanced posture requires no urgent action to be taken or requires urgent action to be taken.
- When an unbalanced posture of a driver is determined to be a habitual posture of the driver, posture information about a proper driving posture or/and information about effects of the unbalanced posture can be outputted to urge the driver to improve his/her driving posture.
- When the predetermined timing is set to be when the vehicle is stopped, more information can be safely outputted than when the vehicle is running.
- Depending types of habitual unbalanced posture, a difference between a proper driving posture and the habitual unbalanced posture of the driver and the effect of the unbalanced posture vary. Thus, the posture information to be outputted may be determined based on the type of the habitual unbalanced posture of the driver. In this way, the driver can efficiently obtain posture information corresponding to the type of his/her unbalanced posture.
- When posture information related with a habitual unbalanced posture of a driver is frequently outputted, the driver may feel annoyed. Therefore, the driver is allowed to set a desired frequency of posture information output. In this way, the driver can obtain posture information at a desired frequency.
- In cases where, after an unbalanced posture of a driver is determined to be other than a habitual posture of the driver and the driver is alerted, the driver's posture remains unimproved and his/her eyes are kept closed longer than the time T2, the driver is highly likely unconscious and unable to drive. In such cases, the vehicle is safely stopped to prevent occurrence of an accident.
- When an unbalanced posture of a driver is determined to be a habitual posture of the driver, posture information may be outputted dividedly at different vehicle stop times. For example, an illustration of a proper driving posture and information about effects of the unbalanced posture may be outputted at different vehicle stop times. Also, when outputting illustrations of a proper driving posture as shown in
FIG. 6 , the description of steps A to D may be outputted in sequence, for example, such that steps A and B are outputted at a first time when the vehicle is stopped and such that steps C and D are outputted when the vehicle is stopped next time. Outputting the divided posture information at different vehicle stop times enables posture information to be provided to a driver safely. In cases where, in the process of outputting divided posture information at different vehicle stop times, the driver's posture is improved before completion of the process, the output of the remaining posture information may be canceled. - The
driver monitoring device 10 may be provided with anexternal storage device 35 for storing information about the unbalanced posture in association with the driver when the unbalanced posture of the driver is determined to be a habitual posture of the driver. The driver may have an image of his/her habitual driving posture captured by thecamera 31 beforehand and may have the captured image stored in theexternal storage device 35. Then, when a posture of the driver detected by image recognition immediately after driving is started agrees with the information about the driver's habitual driving posture stored in theexternal storage device 35, the determination unit 22 may determine that the detected posture of the driver is the driver's habitual posture. - Posture information and an alert may be outputted to a driver using either one or both of the
display 33 and thespeaker 34. - Embodiments and example configurations according to the present disclosure have been illustrated, but the embodiments and configurations according to the present disclosure are not limited to those illustrated above. Embodiments and configurations that can be realized by appropriately combining technical elements disclosed in different embodiments and configurations are also included in the embodiments and configurations according to the present disclosure.
Claims (9)
1. A driver monitoring device comprising:
a detection unit that detects an unbalanced posture of a driver based on an image of a driver's seat of a vehicle captured by a camera;
a determination unit that determines whether or not the unbalanced posture of the driver detected by the detection unit is a habitual posture of the driver; and
a notification unit that notifies the driver of the unbalanced posture in different manners in accordance with whether the unbalanced posture determined is the habitual posture of the driver or the unbalanced posture determined is other than the habitual posture of the driver.
2. The driver monitoring device according to claim 1 , wherein:
when the unbalanced posture of the driver is detected by the detection unit continuously from immediately after a start of driving longer than a posture unbalance determination threshold time, the determination unit determines that the unbalanced posture is a habitual posture of the driver.
3. The driver monitoring device according to claim 1 , wherein:
when the determination unit determines that the unbalanced posture is the habitual posture of the driver, the notification unit outputs posture information at a predetermined timing; and
when the determination unit determines that the unbalanced posture is other than the habitual posture of the driver, the notification unit immediately alerts the driver.
4. The driver monitoring device according to claim 3 , wherein:
the posture information includes at least information about a proper driving posture or information about an effect of the unbalanced posture.
5. The driver monitoring device according to claim 3 , wherein:
the predetermined timing is when the vehicle is stopped.
6. The driver monitoring device according to claim 5 , wherein:
the notification unit outputs the posture information dividedly at a plurality of times at each which the vehicle is stopped.
7. The driver monitoring device according to claim 3 , wherein:
the notification unit outputs the posture information contents of which are changed in accordance with how the driver's habitual posture is unbalanced.
8. The driver monitoring device according to claim 3 , further comprising:
a setting unit that allows the driver to set a frequency at which the posture information is outputted.
9. The driver monitoring device according to claim 3 , further comprising:
an opening degree calculation unit that calculates a degree of opening of an eye of the driver; and
a vehicle stopping unit that, when the driver's posture is not improved after the driver is alerted by the notification unit with the driver's eye opening degree calculated by the opening degree calculation unit having been continuously smaller than an opening degree threshold exceeding an opening degree determination threshold time, outputs a signal for safely stopping the vehicle to a vehicle control device.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014162875A JP6303907B2 (en) | 2014-08-08 | 2014-08-08 | Driver monitoring device |
JP2014-162875 | 2014-08-08 | ||
PCT/JP2015/003742 WO2016021142A1 (en) | 2014-08-08 | 2015-07-27 | Driver monitoring device |
Publications (2)
Publication Number | Publication Date |
---|---|
US20170210357A1 true US20170210357A1 (en) | 2017-07-27 |
US10160426B2 US10160426B2 (en) | 2018-12-25 |
Family
ID=55263439
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/324,015 Active 2035-08-08 US10160426B2 (en) | 2014-08-08 | 2015-07-27 | Driver monitoring device |
Country Status (5)
Country | Link |
---|---|
US (1) | US10160426B2 (en) |
JP (1) | JP6303907B2 (en) |
CN (1) | CN106575478B (en) |
DE (1) | DE112015003672T5 (en) |
WO (1) | WO2016021142A1 (en) |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180334097A1 (en) * | 2017-05-19 | 2018-11-22 | Yazaki Corporation | Monitoring system |
US20190071095A1 (en) * | 2017-09-01 | 2019-03-07 | Alpine Electronics, Inc. | Driver monitoring apparatus, driver monitoring method, and program |
US20190092236A1 (en) * | 2016-05-25 | 2019-03-28 | Denso Corporation | Safe driving assistance device and safe driving assistance program product |
US20190095734A1 (en) * | 2017-09-25 | 2019-03-28 | Yazaki Corporation | Monitoring system |
CN110290349A (en) * | 2019-06-17 | 2019-09-27 | 苏州佳世达电通有限公司 | The method of the sitting posture state of lamps and lanterns and detecting user |
US20190362167A1 (en) * | 2018-05-25 | 2019-11-28 | Subaru Corporation | Occupant monitoring device for vehicle |
US10699143B2 (en) * | 2017-03-10 | 2020-06-30 | Gentex Corporation | System and method for vehicle occupant identification and monitoring |
US11062575B2 (en) * | 2016-01-13 | 2021-07-13 | Travis Copen | Wireless tilt sensor system and method |
US11120283B2 (en) | 2019-03-08 | 2021-09-14 | Subaru Corporation | Occupant monitoring device for vehicle and traffic system |
US11245874B2 (en) * | 2017-03-07 | 2022-02-08 | Mando Corporation | Display system of vehicle and method of driving the same |
US20220126772A1 (en) * | 2020-10-27 | 2022-04-28 | GM Global Technology Operations LLC | Vision-based airbag enablement |
US11345298B2 (en) * | 2019-12-26 | 2022-05-31 | Panasonic Intellectual Property Management Co., Ltd. | Driver monitoring device and driver monitoring method |
US20220242438A1 (en) * | 2021-02-03 | 2022-08-04 | Magna Mirrors Of America, Inc. | Vehicular driver monitoring system with posture detection and alert |
US11919462B2 (en) | 2019-10-08 | 2024-03-05 | Subaru Corporation | Vehicle occupant monitoring apparatus and vehicle occupant protecting system |
Families Citing this family (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2543259B1 (en) | 2011-07-06 | 2017-09-27 | Kraft Foods R & D, Inc. | Method of manufacturing confectionery shells |
EP2543260A1 (en) | 2011-07-06 | 2013-01-09 | Kraft Foods R & D, Inc. | Method for manufacturing an aerated confectionery shell |
CN106428019A (en) * | 2016-10-12 | 2017-02-22 | 防城港市港口区升聚科技有限公司 | Automobile driving posture control device |
KR20180047522A (en) * | 2016-10-31 | 2018-05-10 | 주식회사 티노스 | Electrical system of a vehicle for recognizing a driver |
CN108304745A (en) * | 2017-01-10 | 2018-07-20 | 普天信息技术有限公司 | A kind of driver's driving behavior detection method, device |
JP6948559B2 (en) * | 2017-05-09 | 2021-10-13 | オムロン株式会社 | Driver monitoring device and driver monitoring method |
JP6786451B2 (en) * | 2017-05-19 | 2020-11-18 | 矢崎総業株式会社 | Monitoring system |
EP3657465A4 (en) | 2017-07-21 | 2020-10-21 | Sony Semiconductor Solutions Corporation | Vehicle control device and vehicle control method |
JP7080598B2 (en) | 2017-07-21 | 2022-06-06 | ソニーセミコンダクタソリューションズ株式会社 | Vehicle control device and vehicle control method |
JP6944308B2 (en) | 2017-08-18 | 2021-10-06 | ソニーセミコンダクタソリューションズ株式会社 | Control devices, control systems, and control methods |
JP2019095892A (en) * | 2017-11-20 | 2019-06-20 | シャープ株式会社 | Vehicle drive supporting device and vehicle drive supporting program |
JP6684295B2 (en) * | 2018-02-09 | 2020-04-22 | 株式会社日立物流 | Management support system |
KR102575147B1 (en) * | 2018-04-25 | 2023-09-06 | 현대자동차주식회사 | Apparatus and Method for notifying correction of posture |
CN109334573B (en) * | 2018-08-09 | 2020-06-30 | 智车优行科技(北京)有限公司 | Automobile exterior rearview mirror adjusting method and device and automobile |
US20220358835A1 (en) * | 2019-09-16 | 2022-11-10 | Joshua Grady Graver | Methods and systems for automatically detecting violation of a driving-related law |
JP7452402B2 (en) * | 2020-12-18 | 2024-03-19 | トヨタ自動車株式会社 | Vehicle door leaning detection system |
JP2023086452A (en) * | 2021-12-10 | 2023-06-22 | パナソニックIpマネジメント株式会社 | Driver monitoring device |
EP4239598A1 (en) | 2022-03-02 | 2023-09-06 | Bayerische Motoren Werke Aktiengesellschaft | Method for determining an attentiveness of a driver of an automated vehicle |
WO2023243066A1 (en) * | 2022-06-17 | 2023-12-21 | 三菱電機株式会社 | Abnormal posture determination device, abnormal posture determination method, and vehicle control system |
WO2024069785A1 (en) * | 2022-09-28 | 2024-04-04 | 三菱電機株式会社 | Occupant state determination device, occupant state determination system, occupant state determination method, program, and vehicle control system |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5689241A (en) * | 1995-04-24 | 1997-11-18 | Clarke, Sr.; James Russell | Sleep detection and driver alert apparatus |
US6049747A (en) * | 1996-06-12 | 2000-04-11 | Yazaki Corporation | Driver monitoring device |
US20080080741A1 (en) * | 2006-09-28 | 2008-04-03 | Takata Corporation | Occupant detection system |
US20080130953A1 (en) * | 2006-12-04 | 2008-06-05 | Denso Corporation | Operation estimating apparatus and program |
US20100090839A1 (en) * | 2007-02-01 | 2010-04-15 | Takuhiro Omi | Driver management apparatus and travel management system |
US20110178680A1 (en) * | 2009-09-09 | 2011-07-21 | Yumiko Kato | Vehicle control device and vehicle control method |
US20110295466A1 (en) * | 2009-02-06 | 2011-12-01 | Masprodenkoh Kabushikikaisha | Seating Status Detection Apparatus and Occupant Monitoring System for a Moving Body |
US20130338881A1 (en) * | 2011-03-04 | 2013-12-19 | Toyota Jidosha Kabushiki Kaisha | Approaching vehicle detection device |
US20140121927A1 (en) * | 2011-07-11 | 2014-05-01 | Kiyoto Hanita | Vehicle emergency evacuation device |
US20150166059A1 (en) * | 2013-12-18 | 2015-06-18 | Automotive Research & Testing Center | Autonomous vehicle driving support system and autonomous driving method performed by the same |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4916755B2 (en) * | 2006-04-06 | 2012-04-18 | 富士重工業株式会社 | Vehicle driving support device |
JP4877650B2 (en) * | 2006-11-09 | 2012-02-15 | 株式会社デンソー | Car navigation system |
JP2011238133A (en) * | 2010-05-12 | 2011-11-24 | Toyota Motor Corp | Driving support apparatus |
CN102097003B (en) * | 2010-12-31 | 2014-03-19 | 北京星河易达科技有限公司 | Intelligent traffic safety system and terminal |
US9844344B2 (en) | 2011-07-05 | 2017-12-19 | Saudi Arabian Oil Company | Systems and method to monitor health of employee when positioned in association with a workstation |
JP2014525086A (en) | 2011-07-05 | 2014-09-25 | サウジ アラビアン オイル カンパニー | Floor mat system and associated computer media for monitoring and improving employee health and productivity, and computer-implemented methods |
US9256711B2 (en) | 2011-07-05 | 2016-02-09 | Saudi Arabian Oil Company | Systems, computer medium and computer-implemented methods for providing health information to employees via augmented reality display |
US20130012802A1 (en) | 2011-07-05 | 2013-01-10 | Saudi Arabian Oil Company | Systems, Computer Medium and Computer-Implemented Methods For Monitoring and Improving Cognitive and Emotive Health of Employees |
US9492120B2 (en) | 2011-07-05 | 2016-11-15 | Saudi Arabian Oil Company | Workstation for monitoring and improving health and productivity of employees |
US8872640B2 (en) | 2011-07-05 | 2014-10-28 | Saudi Arabian Oil Company | Systems, computer medium and computer-implemented methods for monitoring health and ergonomic status of drivers of vehicles |
US9526455B2 (en) | 2011-07-05 | 2016-12-27 | Saudi Arabian Oil Company | Systems, computer medium and computer-implemented methods for monitoring and improving health and productivity of employees |
US9710788B2 (en) | 2011-07-05 | 2017-07-18 | Saudi Arabian Oil Company | Computer mouse system and associated, computer medium and computer-implemented methods for monitoring and improving health and productivity of employees |
US9833142B2 (en) | 2011-07-05 | 2017-12-05 | Saudi Arabian Oil Company | Systems, computer medium and computer-implemented methods for coaching employees based upon monitored health conditions using an avatar |
US10108783B2 (en) | 2011-07-05 | 2018-10-23 | Saudi Arabian Oil Company | Systems, computer medium and computer-implemented methods for monitoring health of employees using mobile devices |
CN102393989B (en) * | 2011-07-28 | 2013-03-27 | 山西智济电子科技有限公司 | Real-time monitoring system of driver working state |
JP6071296B2 (en) | 2012-07-20 | 2017-02-01 | キヤノン株式会社 | Communication control device and communication control method |
JP6212992B2 (en) * | 2012-08-01 | 2017-10-18 | 日産自動車株式会社 | Driver abnormality support device and driver abnormality support method |
-
2014
- 2014-08-08 JP JP2014162875A patent/JP6303907B2/en active Active
-
2015
- 2015-07-27 US US15/324,015 patent/US10160426B2/en active Active
- 2015-07-27 CN CN201580040333.XA patent/CN106575478B/en active Active
- 2015-07-27 DE DE112015003672.5T patent/DE112015003672T5/en active Pending
- 2015-07-27 WO PCT/JP2015/003742 patent/WO2016021142A1/en active Application Filing
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5689241A (en) * | 1995-04-24 | 1997-11-18 | Clarke, Sr.; James Russell | Sleep detection and driver alert apparatus |
US6049747A (en) * | 1996-06-12 | 2000-04-11 | Yazaki Corporation | Driver monitoring device |
US20080080741A1 (en) * | 2006-09-28 | 2008-04-03 | Takata Corporation | Occupant detection system |
US20080130953A1 (en) * | 2006-12-04 | 2008-06-05 | Denso Corporation | Operation estimating apparatus and program |
US20100090839A1 (en) * | 2007-02-01 | 2010-04-15 | Takuhiro Omi | Driver management apparatus and travel management system |
US20110295466A1 (en) * | 2009-02-06 | 2011-12-01 | Masprodenkoh Kabushikikaisha | Seating Status Detection Apparatus and Occupant Monitoring System for a Moving Body |
US20110178680A1 (en) * | 2009-09-09 | 2011-07-21 | Yumiko Kato | Vehicle control device and vehicle control method |
US20130338881A1 (en) * | 2011-03-04 | 2013-12-19 | Toyota Jidosha Kabushiki Kaisha | Approaching vehicle detection device |
US20140121927A1 (en) * | 2011-07-11 | 2014-05-01 | Kiyoto Hanita | Vehicle emergency evacuation device |
US20150166059A1 (en) * | 2013-12-18 | 2015-06-18 | Automotive Research & Testing Center | Autonomous vehicle driving support system and autonomous driving method performed by the same |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11062575B2 (en) * | 2016-01-13 | 2021-07-13 | Travis Copen | Wireless tilt sensor system and method |
US20190092236A1 (en) * | 2016-05-25 | 2019-03-28 | Denso Corporation | Safe driving assistance device and safe driving assistance program product |
US10676025B2 (en) * | 2016-05-25 | 2020-06-09 | Denso Corporation | Safe driving assistance device and safe driving assistance program product |
US11245874B2 (en) * | 2017-03-07 | 2022-02-08 | Mando Corporation | Display system of vehicle and method of driving the same |
US10699143B2 (en) * | 2017-03-10 | 2020-06-30 | Gentex Corporation | System and method for vehicle occupant identification and monitoring |
US11117515B2 (en) * | 2017-05-19 | 2021-09-14 | Yazaki Corporation | Monitoring system |
US20180334097A1 (en) * | 2017-05-19 | 2018-11-22 | Yazaki Corporation | Monitoring system |
US20190071095A1 (en) * | 2017-09-01 | 2019-03-07 | Alpine Electronics, Inc. | Driver monitoring apparatus, driver monitoring method, and program |
US10457293B2 (en) * | 2017-09-01 | 2019-10-29 | Alpine Electronics, Inc. | Driver monitoring apparatus, driver monitoring method, and program |
US20190095734A1 (en) * | 2017-09-25 | 2019-03-28 | Yazaki Corporation | Monitoring system |
CN109561277A (en) * | 2017-09-25 | 2019-04-02 | 矢崎总业株式会社 | Monitoring system |
US10949688B2 (en) * | 2017-09-25 | 2021-03-16 | Yazaki Corporation | Monitoring system for person in a vehicle |
US10824886B2 (en) * | 2018-05-25 | 2020-11-03 | Subaru Corporation | Occupant monitoring device for vehicle |
US20190362167A1 (en) * | 2018-05-25 | 2019-11-28 | Subaru Corporation | Occupant monitoring device for vehicle |
US11120283B2 (en) | 2019-03-08 | 2021-09-14 | Subaru Corporation | Occupant monitoring device for vehicle and traffic system |
CN110290349A (en) * | 2019-06-17 | 2019-09-27 | 苏州佳世达电通有限公司 | The method of the sitting posture state of lamps and lanterns and detecting user |
US11919462B2 (en) | 2019-10-08 | 2024-03-05 | Subaru Corporation | Vehicle occupant monitoring apparatus and vehicle occupant protecting system |
US11345298B2 (en) * | 2019-12-26 | 2022-05-31 | Panasonic Intellectual Property Management Co., Ltd. | Driver monitoring device and driver monitoring method |
US20220126772A1 (en) * | 2020-10-27 | 2022-04-28 | GM Global Technology Operations LLC | Vision-based airbag enablement |
US11807181B2 (en) * | 2020-10-27 | 2023-11-07 | GM Global Technology Operations LLC | Vision-based airbag enablement |
US20220242438A1 (en) * | 2021-02-03 | 2022-08-04 | Magna Mirrors Of America, Inc. | Vehicular driver monitoring system with posture detection and alert |
US11851080B2 (en) * | 2021-02-03 | 2023-12-26 | Magna Mirrors Of America, Inc. | Vehicular driver monitoring system with posture detection and alert |
Also Published As
Publication number | Publication date |
---|---|
CN106575478A (en) | 2017-04-19 |
CN106575478B (en) | 2019-05-14 |
JP2016038793A (en) | 2016-03-22 |
WO2016021142A1 (en) | 2016-02-11 |
US10160426B2 (en) | 2018-12-25 |
DE112015003672T5 (en) | 2017-04-27 |
JP6303907B2 (en) | 2018-04-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10160426B2 (en) | Driver monitoring device | |
US9767374B2 (en) | Driver check apparatus | |
US10793149B2 (en) | Control apparatus, control method, and program | |
JP6341055B2 (en) | In-vehicle control device | |
US11345298B2 (en) | Driver monitoring device and driver monitoring method | |
JP6331875B2 (en) | In-vehicle control device | |
JP4658899B2 (en) | Vehicle occupant detection device | |
JP5082834B2 (en) | Aside look detection device and method, and program | |
US20060287779A1 (en) | Method of mitigating driver distraction | |
JP2008199515A (en) | Fellow passenger sitting posture detecting/determining apparatus and method | |
JP6730096B2 (en) | Occupant condition monitoring device | |
JP2008305096A (en) | Driving support device | |
JP4840146B2 (en) | Sleepiness detection device | |
JP2008204107A (en) | Carelessness warning device, vehicle equipment control method for the device and program for vehicle control device | |
JP6593486B2 (en) | In-vehicle control device and in-vehicle system | |
JP4534789B2 (en) | Vehicle alarm device | |
JP2018133007A (en) | Alarm apparatus | |
JP2010282350A (en) | Collision safety system | |
JP2021105981A (en) | Driver monitoring system and driver monitoring method | |
JP2008305190A (en) | Doze warning device and car | |
JP2004133749A (en) | Face direction detector | |
JP2004041485A (en) | Closed/open eye monitoring device | |
JP2017205368A (en) | Driver state detector | |
JP2007204032A (en) | On-vehicle warning device | |
JP2005280483A (en) | Seat belt device of vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: DENSO CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NAGAI, FUMIYA;REEL/FRAME:040858/0186 Effective date: 20161216 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |