JP7046748B2 - Driver status determination device and driver status determination method - Google Patents

Description

The present invention relates to a driver state determination device and a driver state determination method.

Conventionally, various devices for determining whether or not a driver is drowsy have been proposed so that the driver can drive safely. For example, in Patent Document 1, the driver's eye opening amount is detected, the distribution of the eye opening amount and the detection frequency is approximated by a curve, and whether the driver feels drowsy from the degree of change in the maximum value in the curve. A state estimation device for determining the above is disclosed.

Japanese Unexamined Patent Publication No. 2008-99884

However, in the above-mentioned state estimation device, when the face portion cannot be detected by the camera due to the driver standing down, even if the driver feels drowsy, there is a risk of overlooking it. However, when a plurality of cameras are attached and processing is performed using the image data captured and generated by each of them, the processing load may increase.

An object of the present invention is to provide a driver state determination device and a driver state determination method for comprehensively observing a driver's condition and determining the driver's condition while reducing the processing load.

A first aspect of the present invention is a driver including a first driver state detecting means for detecting a first state of a driver and a second driver state detecting means for detecting a second state of the driver. In the state determination device, when the value indicating the first state is equal to or higher than the first threshold value, the second driver state detecting means is operated.

A second aspect of the present invention includes an operation including a first driver state detection step for detecting the first state of the driver and a second driver state detection step for detecting the second state of the driver. It is a driver state determination method executed by the person state determination device, and when the value indicating the first state is equal to or greater than the first threshold value, the second driver state detection step is executed.

According to the present invention, it is possible to comprehensively observe the state of the driver and determine the state of the driver while reducing the processing load.

It is a figure which shows the functional block of the driver state determination apparatus which concerns on this embodiment. It is a flowchart which shows the process by the driver state determination apparatus which concerns on this embodiment. It is a figure which shows the functional block of the driver state determination apparatus which concerns on an example of this embodiment. It is a flowchart which shows the process by the driver state determination apparatus which concerns on an example of this embodiment.

A suitable embodiment of the driver state determination device and the driver state determination method according to the present invention will be described in detail below with reference to the accompanying drawings.

[Embodiment]
FIG. 1 is a diagram showing a functional block of the driver state determination device 10 according to the present embodiment. The driver state determination device 10 includes a first driver state detection unit 12, a second driver state detection unit 14, and the like. In the following description, it is assumed that the target on which the driver state determination device 10 is mounted is a vehicle, but the present invention is not limited to this.

The first driver state detection unit 12 detects the first state of the driver. The first state corresponds to the state of consciousness of the driver, for example, the facial expression of the driver, the number of times or the operation time of the automatic brake, the driving operation by the driver regardless of the surrounding situation of the vehicle, and the like. Can be mentioned.

The second driver state detection unit 14 detects the second state of the driver. The second state corresponds to the state of consciousness of the driver, and examples thereof include the posture of the upper body of the driver and the gesture of the driver.

The driver state determination device 10 operates the second driver state detection unit 14 when the value indicating the first state becomes equal to or higher than a predetermined threshold value (also referred to as the first threshold value).

FIG. 2 is a flowchart showing processing by the driver state determination device 10 according to the present embodiment. In step S1, the first driver state detection unit 12 detects the first state. When the value indicating the first state is equal to or greater than the first threshold value (step S2: YES), the second driver state detection unit 14 executes the operation (step S3). When the value indicating the first state is less than the first threshold value (step S2: NO), the process of the driver state determination device 10 returns to step S1.

Hereinafter, the driver state determination device 10 will be described in detail. FIG. 3 is a diagram showing a functional block of the driver state determination device 10 according to an example of the present embodiment. The driver state determination device 10 includes an image pickup unit 16, a vehicle peripheral sensor 18, a brake sensor 20, a handle sensor 22, an automatic brake sensor 24, an ECU (Electronic Control Unit) 26, and the like.

The image pickup unit 16 is, for example, a CCD (Charge Coupled Device) camera, which is provided in the upper part of the instrument panel, the dashboard, the vicinity of the rear-view mirror, the ceiling portion near the windshield, etc. in front of the driver's seat, and is provided in the driver's seat. The upper body of the seated driver is imaged. The image pickup unit 16 appropriately magnifies and captures the driver's face portion according to the control from the ECU 26. Further, the image pickup unit 16 continuously executes the image pickup process in time, and outputs the image data obtained by the image pickup to the ECU 26 at any time. The image pickup unit 16 may output the data of the time to the ECU 26 together with the image data obtained by taking a picture at each time.

The vehicle peripheral sensor 18 detects the peripheral condition of the vehicle. The brake sensor 20 detects a manual vehicle stop process such as operation of the brake pedal by the driver. The steering wheel sensor 22 detects the steering wheel operation by the driver. The automatic brake sensor 24 detects the operation of the automatic brake.

The ECU 26 uses the image data input from the image pickup unit 16, the detection results of the brake sensor 20 and the handle sensor 22 according to the peripheral conditions detected by the vehicle peripheral sensor 18, the detection results of the automatic brake sensor 24, and the like. , The state of the driver is determined, and the process for guiding the state of the driver to a state suitable for driving is executed. In order to execute such processing, the ECU 26 includes an image processing unit 28, an image pickup control unit 30, a facial expression data derivation unit 32, a driving operation data derivation unit 34, an automatic brake operation data derivation unit 36, a storage unit 38, and a posture data derivation unit. A unit 40, a state determination unit 42, an awakening support unit 44, and the like are provided.

The ECU 26 can be configured by, for example, a processor such as a CPU (Central Processing Unit), a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory), various interfaces, and the like. The memory realizes the function of the storage unit 38. The processor executes processing using the program and various information stored in the memory, so that the image processing unit 28, the facial expression data derivation unit 32, the driving operation data derivation unit 34, the automatic brake operation data derivation unit 36, and the attitude data Each function of the derivation unit 40 and the state determination unit 42 is realized. In addition, the processor executes processing using programs and various information stored in the memory, and realizes each function of the image pickup control unit 30 and the awakening support unit 44 through various interfaces.

The image processing unit 28 performs image processing of the image data input from the image pickup unit 16. The image processing unit 28 detects the driver's face portion from the image data input from the image pickup unit 16. The image processing unit 28 detects, for example, an eye portion from the detected face portion of the driver. In addition, it is assumed that the face portion in the example of the present embodiment includes the eye portion.

The image pickup control unit 30 controls the image pickup unit 16 so as to magnify and take an image of the driver's face portion detected by the image processing unit 28.

The image processing unit 28 acquires the image data of the face portion from the image pickup unit 16 and outputs the image data after the image processing to the facial expression data derivation unit 32. Further, the image processing unit 28 performs image processing on the image data of the upper body imaged by the image pickup unit 16 without using the enlargement function, and outputs the image data to the posture data derivation unit 40. The image processing unit 28 associates the image data input from the image pickup unit 16 with the data related to the time when the image data is input, and outputs the data to the facial expression data derivation unit 32 and the posture data derivation unit 40.

The facial expression data deriving unit 32 derives the opening degree of the driver's eyelids by using the image data of the driver's face portion. In one example of the present embodiment, the opening degree is the number of pixels or the length of the maximum length portion (maximum length portion) between the upper eyelid and the lower eyelid of the driver. However, it is assumed that weighting is appropriately performed according to the value of the magnification of the enlargement of the image pickup unit 16. Further, the opening degree is the maximum length between the upper and lower eyelids in the image data of the driver in a state where he / she does not feel drowsiness, which is the number of pixels or the length of the maximum length portion between the upper and lower eyelids in the image data acquired from the image processing unit 28. It may be a ratio to the number of pixels or the length of the portion. In this case, the image data of the driver who does not feel drowsy is stored in advance in the storage unit 38, which will be described later. Further, the opening degree is derived by combining the magnification related to the enlargement of the image data acquired from the image processing unit 28 and the magnification related to the enlargement of the image data of the driver who does not feel drowsy.

The facial expression data deriving unit 32 derives at least one of the frequency with which the eyelid opening is equal to or less than the predetermined opening in a predetermined time and the maintenance time with the eyelid opening being equal to or less than the predetermined opening. These frequencies and maintenance times are examples of first indicators of the degree of drowsiness of the driver. The first index is obtained from image data of the face portion, information on the driving operation of the driver, the operation status of the automatic brake, and the like. The first index may also be, for example, the frequency at which the pupil portion cannot be detected in the eye portion detected in the image data, the maintenance time in which the pupil portion cannot be detected, or the like. Further, the facial expression data deriving unit 32 may derive an ambiguity indicating the degree to which the derived value of the first index is unclear for determining the presence or absence of drowsiness and the strength. This ambiguity is an example of a value indicating the first state.

When the brake sensor 20 or the steering wheel sensor 22 does not detect the operation of the brake or the steering wheel according to the peripheral situation (for example, a situation where there is an obstacle in front) detected by the vehicle peripheral sensor 18 in the driving operation data derivation unit 34. When the detection is delayed, the number of non-detections (hereinafter, also referred to as the number of non-detections) and the delay time of detection (hereinafter, also referred to as the delay time) are derived. These non-detection times and delay times are examples of the first index. Further, the driving operation data deriving unit 34 may derive an ambiguity indicating the degree to which the derived value of the first index is unclear for determining the presence or absence of drowsiness and the strength of the drowsiness. This ambiguity is an example of a value indicating the first state.

The automatic brake operation data derivation unit 36 derives the operation number, operation time, or operation frequency (operation number per predetermined time) of the automatic brake detected by the automatic brake sensor 24. The number of times of operation, the operation time, and the frequency of operation are examples of the first index. Further, the automatic brake operation data derivation unit 36 may derive an unclearness indicating the degree to which the derived value of the first index is unclear for determining the presence or absence of drowsiness and the strength. This ambiguity is an example of a value indicating the first state.

The storage unit 38 has a lower limit value of the first index (also referred to as a second threshold value) indicating the case where the driver feels strong drowsiness, and a lower limit value of the first index indicating the case where the driver feels a little drowsy. The value (also referred to as the third threshold value) is stored. The storage unit 38 has at least one of a first index obtained from the image data of the face portion, a first index obtained from the information related to the driving operation, and a first index obtained from the information related to the operation status of the automatic brake. Memorize the second and third thresholds. As these second and third threshold values, values obtained in advance by experiments, machine learning, etc. are used.

The storage unit 38 has a lower limit of the degree of ambiguity when the presence or absence of drowsiness and the strength cannot be determined from the image data of the face portion, and a lower limit of the degree of ambiguity when the presence or absence of drowsiness and the strength cannot be determined from the information related to the driving operation. , At least one of the lower limit of the unclearness when it is not possible to determine the presence or absence of drowsiness or the strength of the automatic brake from the operating status of the automatic brake is stored. The lower limit of each ambiguity in these cases is an example of the first threshold. In one example of the present embodiment, it is not possible when the image processing unit 28 cannot detect the face portion and when the value of the first index derived by the facial expression data derivation unit 32 is equal to or more than the third threshold value and less than the second threshold value. The clarity is greater than or equal to the first threshold. Similarly, when the value of the first index derived by the driving operation data derivation unit 34 is equal to or greater than the third threshold value and less than the second threshold value, the degree of ambiguity is equal to or greater than the first threshold value. Similarly, when the value of the first index derived by the automatic braking operation data derivation unit 36 is equal to or greater than the third threshold value and less than the second threshold value, the degree of ambiguity becomes equal to or greater than the first threshold value. However, it is not limited to these.

The posture data derivation unit 40 detects the posture of the driver from the image data of the upper body of the driver processed by the image processing unit 28. It is assumed that the posture includes the direction of the face. The posture data deriving unit 40 derives the degree to which the detected posture deviates from the posture in the state of not feeling drowsiness. For example, the posture data deriving unit 40 detects the degree of deviation of the driver's face from the traveling direction of the vehicle. When this degree is equal to or higher than a certain limit allowed for safety, the attitude data deriving unit 40 derives the duration in which the deviation occurs. This duration is an example of a second indicator of the degree of drowsiness of the driver.

The posture data deriving unit 40 detects the gesture to be performed when the driver feels drowsy from the image data for a certain period of time or longer. Examples of this gesture are yawning, the movement of the face suddenly turning downward or diagonally downward from the previous position due to the weakness of the neck, and the sudden tilting of the body due to the weakness of the body. There are operations and so on.

The storage unit 38 stores information related to the posture in the awake state in association with, for example, the behavior of the vehicle (forward movement, curve, etc.). The posture in this awake state may be 1 or more with respect to the behavior of one type of vehicle. The posture in this awake state includes, for example, a posture in which the face is turned in the traveling direction of the vehicle. The storage unit 38 stores the minimum value (also referred to as a deviation lower limit value) among the deviation degrees of various postures deviating from the posture in the awake state. Information on the posture in the awake state and information on the lower limit of deviation are obtained in advance by, for example, machine learning or an experiment. The storage unit 38 may store information related to the posture in the awake state and information related to the posture deviating from the posture in the awake state instead of the lower limit value of deviation from the posture in the awake state.

The storage unit 38 stores information (which is set as a fourth threshold value) for distinguishing the presence or absence of drowsiness of the driver based on the image data related to the posture. The fourth threshold value is the duration of the posture deviating from the posture that can be taken in the awake state, and includes the duration in the state of starting to feel drowsiness. In addition, the storage unit 38 stores information related to gestures performed when the driver feels drowsy or in a sleeping state. It should be noted that the fourth threshold value, information related to gestures, and the like are obtained in advance by, for example, machine learning or experiments.

The state determination unit 42 determines whether or not the value of the first index derived by the facial expression data derivation unit 32 is equal to or higher than the second and third threshold values. Alternatively, the state determination unit 42 determines whether or not the value indicating the first state is equal to or higher than the first threshold value.

The state determination unit 42 refers to the posture data derivation unit 40 when the value of the first index is less than the second threshold value and equal to or higher than the third threshold value, or when the image processing unit 28 cannot detect the driver's face portion. Instruct the derivation of the value of the second index. The state determination unit 42 determines whether or not the value of the second index derived by the posture data derivation unit 40 is equal to or greater than the fourth threshold value.

The state determination unit 42 determines that the driver feels drowsy when the value of the first index is equal to or higher than the second threshold value or when the value of the second index is equal to or higher than the fourth threshold value. The state determination unit 42 determines that the driver feels drowsy when the posture data derivation unit 40 detects a gesture to be performed when the driver feels drowsiness.

The awakening support unit 44 controls the control target device included in the control target device group 46 to awaken the driver when the state determination unit 42 determines that the driver is drowsy, and performs a predetermined operation. To execute. For example, the awakening support unit 44 may control an audio device or the like included in the controlled target device group 46 to output sound. Further, the awakening support unit 44 controls an audio device, a navigation device, and the like included in the controlled target device group 46 by using technologies such as artificial intelligence (AI) and intelligent agent (IA). And may have a conversation with the driver.

Further, as the control target device included in the control target device group 46, a massage device may be built in the driver's seat, and the awakening support unit 44 may operate this massage device. Further, as the control target device included in the control target device group 46, an aroma diffuser that diffuses the awakening scent may be provided in the vehicle, and the awakening support unit 44 may operate the diffuser. Further, as the control target device included in the control target device group 46, a lighting device may be provided facing the driver's seat, and the awakening support unit 44 controls the lighting device to irradiate the driver with light. You may let me.

The awakening support unit 44 may control the navigation device after the above-mentioned processing such as sound output and execution of dialogue, and may execute guidance processing for a road shoulder or a resting place for stopping.

FIG. 4 is a flowchart showing processing by the driver state determination device 10 according to an example of the present embodiment. The driver state determination device 10 starts its operation when the engine of the vehicle is operated. In step S11, the imaging unit 16 images the upper body. The image processing unit 28 appropriately processes the image data imaged and generated by the image pickup unit 16 to detect the driver's face portion (step S12).

When the image processing unit 28 detects the face portion (step S13: YES), the processing by the driver state determination device 10 shifts to step S14. When the image processing unit 28 cannot detect the face portion (step S13: NO), the processing by the driver state determination device 10 proceeds to step S17.

In step S14, the facial expression data derivation unit 32, the driving operation data derivation unit 34, and the automatic brake operation data derivation unit 36 each derive the value of each first index. When the state determination unit 42 determines in step S15 that the value of the first index is equal to or greater than the second threshold value (step S15: YES), the above-mentioned constant control operation by the awakening support unit 44 is executed in step S21. .. After that, the process by the driver state determination device 10 shifts to the process after step S11 after the elapse of a predetermined time.

When the value of the first index is less than the second threshold value (step S15: NO) and is determined by the state determination unit 42 to be less than the third threshold value (step S16: NO), the driver state determination device The process of 10 returns to step S11. When the value of the first index is less than the second threshold value (step S15: NO) and is determined by the state determination unit 42 to be equal to or higher than the third threshold value (step S16: YES), the driver state determination device The process of 10 proceeds to step S17.

In step S17, the posture data derivation unit 40 derives the value of the second index by using the image data of the upper body of the driver, which is imaged and generated by the image pickup unit 16 in step S11. When the value of the second index is determined by the state determination unit 42 in step S18 to be equal to or higher than the fourth threshold value (step S18: YES), the above-mentioned constant control operation by the awakening support unit 44 is executed in step S21. After that, the process by the driver state determination device 10 shifts to the process after step S11 after the elapse of a predetermined time.

When the value of the second index is determined to be less than the fourth threshold value by the state determination unit 42 in step S18 (step S18: NO), the posture data derivation unit 40 executes a gesture detection process indicating drowsiness of the driver. (Step S19). In this detection process, a plurality of image data at a fixed time are used. The plurality of image data includes the latest image data used for deriving the value of the second index in step S17, as well as image data captured before the image data.

When the posture data deriving unit 40 detects a gesture indicating drowsiness of the driver in the next step S20 (step S20: YES), the above-mentioned constant control operation by the awakening support unit 44 is executed in step S21. After that, the process by the driver state determination device 10 shifts to the process after step S11 after the elapse of a predetermined time. If the gesture indicating drowsiness of the driver is not detected in step S20 (step S20: NO), the process of the driver state determination device 10 returns to step S11.

The driver state determination device 10 according to the present embodiment executes the detection process of the first state of the driver, and when the value indicating the first state is equal to or higher than the first threshold value, the second driver state detection unit 14 is used. It is activated to execute the detection process of the second state. The value indicating the first state is, for example, the degree of drowsiness derived from the image data of the driver's face and the degree of drowsiness derived from the driver's driving operation data. The second state is, for example, the posture of the driver's upper body and the gesture of the driver.

When the driver state determination device 10 can determine from the first state that the driver's conscious state is suitable for driving, or when it can determine from the first state that the driver's conscious state is not suitable for driving. , The second state is not detected, and the processing corresponding to each case is executed. As a result, it is possible to omit procedures such as the detection process of the second state and the derivation process of the value indicating the second state, and it is possible to reduce the processing load.

Further, when the driver state determination device 10 cannot determine from the first state whether or not the driver's consciousness state is suitable for driving, the driver state determination device 10 executes the detection process of the second state, and other than the first state. Also, since the driver's consciousness state is determined using the information related to the second state, an accurate determination can be made. Further, for example, when the first state is the facial expression of the driver and the second state is the posture of the upper body of the driver, these image data can be acquired by one image pickup unit 16, which is simple. The configuration enables accurate judgment.

[Modification example]
The following modifications can be made to the above embodiment.
(Modification 1)
In the driver state determination device 10 according to the above embodiment, the frequency or maintenance time in a predetermined time when the opening degree of the driver's eyelids is equal to or less than the predetermined opening degree is used as the first index. The driver state determination device 10 according to this modification determines whether or not the driver is drowsy by using not only the eye portion but also the cheek portion, the mouth portion, and the like. For this determination, the storage unit 38 stores the first classification information obtained by machine learning such as SVM (Support Vector Machine). This first classification information is information that classifies the feature quantities derived from the state of each part on the entire face of the driver into each group according to the presence or absence of drowsiness of the driver. Here, for more accurate classification of the feature amount near the boundary between the group in the case of drowsiness and the group in the awake state, the driver state determination device 10 detects the second state. At this time, the value indicating the first state of a certain feature amount is, for example, the reciprocal of the distance from the boundary line, the boundary surface, or the like when the boundary is approximated by a line or a surface. At this time, the first threshold value is, for example, the reciprocal of the distance from the boundary line or the boundary surface at which an error begins to occur in the determination of the state of consciousness.

According to the driver state determination device 10 according to the present modification, it is possible to detect a comprehensive and detailed state change based on the facial expression of the entire face.

(Modification 2)
The driver state determination device 10 according to this modification includes, for example, an optical sensor such as an illuminance meter on the bonnet or near the windshield. Then, when the illuminance of the detected light is above a certain illuminance or when the illuminance suddenly changes from a low state to a high state, the driver's eyelid opening remains low due to glare. In this case, the state determination unit 42 may wait for the driver's subsequent actions such as setting a awning to determine the driver's state, or set the above-mentioned second and third threshold values to be higher than the usual values. It may be changed to a large value and used at the time of judgment. The data indicating the behavior of the driver, such as setting the awning, is obtained in advance by machine learning or the like as data indicating that the driver does not feel drowsy, and is stored in the storage unit 38. Further, it is assumed that the second and third threshold values after the change include the time until the eyes get used to the light, and these are also obtained by machine learning or the like and are stored in the storage unit 38 in advance.

According to the driver state determination device 10 according to the present embodiment, it is possible to make an appropriate determination in consideration of light from the outside, and it is possible to prevent an erroneous determination of the driver state.

[Technical Thought Obtained from the Embodiment]
The technical ideas that can be grasped from the above embodiments are described below.

<First technical idea>
A driver state determination device (12) including a first driver state detecting means (12) for detecting the first state of the driver and a second driver state detecting means (14) for detecting the second state of the driver. 10) activates the second driver state detecting means (14) when the value indicating the first state is equal to or higher than the first threshold value.

This makes it possible to comprehensively observe the driver's condition and determine the driver's condition while reducing the processing load.

The first state may be detected from the image of the driver's face portion imaged by the image pickup means (16). This makes it possible to detect a lot of information about the driver's state of consciousness.

The second state may be detected from the driver's image captured by the imaging means (16). This makes it possible to more accurately determine the state of consciousness of the driver.

The second state may include at least one of the posture of the driver's upper body and the gesture of the driver. This makes it possible to determine the driver's state of consciousness even more accurately.

The first state may include at least one of a state detected from the driving operation of the driver and a state detected from the operating state of the automatic brake. This makes it possible to determine the driver's state of consciousness even more accurately.

The first driver state detecting means (12) derives the value of the first index indicating the degree of drowsiness of the driver based on the first state, and the second driver state detecting means (14) changes to the second state. Based on this, the value of the second index indicating the degree of drowsiness of the driver is derived, the behavior of the driver is detected based on the second state, and the driver state determination device (10) uses the derived value of the first index as the value. Awakening support that causes the controlled device to perform an action to awaken the driver when the driver's drowsiness is detected from at least one of the derived second index value and the gesture when the threshold value is equal to or higher than the second threshold value. Means (44) may be further provided. As a result, the driver can be returned to a state of consciousness suitable for driving, and the safety of the driver can be ensured.

The first index may be the frequency with which the eyelid opening of the driver is equal to or less than the predetermined opening in a predetermined time, or the maintenance time in which the eyelid opening is equal to or less than the predetermined opening. .. This makes it possible to prevent erroneous determination that the driver is drowsy due to blinking or the like.

The driver state determination device (10) may increase the second threshold value when the illuminance of the outside light is high. As a result, even if the driver tends to close his eyelids due to glare, it is possible to prevent erroneous determination of the state of consciousness.

The second index may be the duration during which the posture of the driver's upper body deviates from the posture in the awake state. As a result, it is possible to prevent an erroneous determination from being made every time there is a movement of the body.

The posture in the awake state may be a posture in which the driver faces his / her face in the direction of travel. This makes it possible to accurately determine the state of consciousness of the driver.

<Second technical idea>
The driver state determination method executed by the driver state determination device (10) includes a first driver state detection step for detecting the first state of the driver and a second driver for detecting the second state of the driver. When the value indicating the first state is equal to or greater than the first threshold value including the state detection step, the second driver state detection step is executed.

This makes it possible to comprehensively observe the driver's condition and determine the driver's condition while reducing the processing load.

10 ... Driver state determination device 12 ... First driver state detection unit (first driver state detection means)
14 ... Second driver state detection unit (second driver state detection means)
16 ... Imaging unit (imaging means) 18 ... Vehicle peripheral sensor 20 ... Brake sensor 22 ... Handle sensor 24 ... Automatic brake sensor 26 ... Electronic Control Unit (ECU)
28 ... Image processing unit 30 ... Image control unit 32 ... Facial expression data derivation unit 34 ... Driving operation data derivation unit 36 ... Automatic brake operation data derivation unit 38 ... Storage unit 40 ... Posture data derivation unit 42 ... State determination unit 44 ... Awakening support Department (awakening support means) 46 ... Control target device group

Claims (2)

The first driver state detecting means for detecting the first state of the driver, and
A second driver state detecting means for detecting the second state of the driver is provided.
A driver state determination device that activates the second driver state detecting means when the value indicating the first state is equal to or higher than the first threshold value.
The first state is detected from the image of the driver's face portion captured by the imaging means, and is detected from the driving operation of the driver or the operating state of the automatic brake. Including at least one
The second state is detected from the image of the driver captured by the imaging means, and includes at least one of the posture state of the driver's upper body and the gesture state of the driver.
The first driver state detecting means derives a value of a first index indicating the degree of drowsiness of the driver based on the first state, and derives a value.
The second driver state detecting means derives a value of a second index indicating the degree of drowsiness of the driver based on the second state, detects the behavior of the driver based on the second state, and detects the behavior of the driver.
When the value of the derived first index is equal to or greater than the second threshold value, or when the driver's drowsiness is detected from at least one of the derived value of the second index and the gesture, the driver is referred to. A driver state determination device further provided with awakening support means for causing a controlled device to execute an operation for awakening . The first driver state detection step for detecting the first state of the driver, and
A second driver state detection step for detecting the second state of the driver is provided.
A driver state determination method for executing the second driver state detection step when the value indicating the first state is equal to or higher than the first threshold value.
The first state is detected from the image of the driver's face portion captured by the imaging means, and is detected from the driving operation of the driver or the operating state of the automatic brake. Including at least one
The second state is detected from the image of the driver captured by the imaging means, and includes at least one of the posture state of the driver's upper body and the gesture state of the driver.
The first driver state detection step derives a value of a first index indicating the degree of drowsiness of the driver based on the first state.
The second driver state detection step derives a value of a second index indicating the degree of drowsiness of the driver based on the second state, detects the behavior of the driver based on the second state, and detects the behavior of the driver.
When the value of the derived first index is equal to or greater than the second threshold value, when the value of the derived second index is equal to or greater than the fourth threshold value, or when the value of the derived second index is equal to or greater than that of the gesture. A driver state determination method for executing an awakening support step of causing a controlled target device to perform an operation for awakening the driver when the driver's drowsiness is detected .

Images