JP2010128649A - Awakening state determining device and awakening state determining method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an awakening state determining system 100 which determines the awakening state of a driver with a high degree of accuracy. <P>SOLUTION: The awakening state determining system includes: a behavior detecting unit 10 comprising a first indicator detecting unit 11 which detects a behavior of driver's eyes and a second indicator detecting unit 12 which detects a behavior of driver's head based on an image of the driver; a criterion setting unit 20 which sets a criterion for the determination of a driver's awakening level based on the driver's head behavior when driving on a monotonic section where a driving operation of a vehicle is determined to be monotonic according acquired road information, vehicle information, and/or inter-vehicular distance information; a determination unit 30 which determines the driver's awakening state based on the detected head behavior of the driver and the set criterion. The awakening state determining system 100 transmits and receives information to/from at least one of car navigation equipment 200, a vehicle controller 300, and an inter-vehicular distance measuring device 400. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a wakefulness determination device and a wakefulness determination method for determining a driver's wakefulness.

  Detects long blinks longer than a predetermined time, and shows the ratio of long blinks obtained from the total number of blinks and the number of long blinks in a given evaluation period, and behavior indicating the driver's behavioral characteristics (for example, horizontal movement of the eyeball) There is known a technique for evaluating a driver's arousal level according to a prediction formula obtained by a regression analysis using a sleepiness prediction model based on a detection result of (standard deviation of) (see Patent Document 1).

Japanese Patent Laid-Open No. 10-272959

  However, with this technology, when the driver wears spectacles or the like, scenery or the like is reflected in the spectacles, so the blink of the driver cannot be detected and whether or not the driver is awake. There has been a problem that the accuracy of judgment may decrease.

  The problem to be solved by the present invention is to determine the driver's arousal state with high accuracy even when the driver wears glasses or the like.

  Driving detected using criteria set based on the behavior of the driver's head detected in a monotonous section where the driving of the vehicle obtained from road information, vehicle information and / or inter-vehicle distance information is monotonous The above problem is solved by determining the driver's arousal state based on the behavior of the person's head.

  According to the present invention, since the driver's arousal level can be determined even when the driver wears glasses or the like, the driver's arousal state can be determined with high accuracy.

  The best mode of the awake state determination system 100 according to the present invention will be described based on the drawings.

  FIG. 1 is a functional block diagram of a driving support system 1000 including an arousal state determination system 100.

  As shown in FIG. 1, the driving support system 1000 includes an awake state determination system 100, a navigation device 200, a vehicle controller 300, an inter-vehicle distance measuring device 400, a notification device 500, a driving support device 600, and a camera 700. And an image processing apparatus 800. The awake state determination system 100 and each of the devices 200 to 800 are connected by a CAN (Controller Area Network) or other in-vehicle LAN, and exchange information with each other.

  FIG. 2 is a diagram for explaining the interrelation between the components included in the driving support system 1000.

  As shown in FIG. 2, the camera 700 and the near-infrared illumination 702 are installed at a position where the head including the driver's eyes can be imaged, such as in the meter panel of the vehicle or near the steering column. The camera 700 includes a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), and other image sensors 701 having a sensitivity distribution up to near infrared rays. The camera 700 preferably has an angle of view that allows the entire head of the driver to be imaged. The camera 700 sends the acquired video signal to the image processing apparatus 800. The image processing apparatus 800 sends a digital image of the driver's head, face, and / or eyes included in the face to the awake state determination system 100 based on the acquired video signal.

  Further, the vehicle controller 300 sends the detected vehicle signal to the awake state determination system 100, and the inter-vehicle distance measuring device 400 sends the positional relationship with other vehicles to the awake state determination system.

  Further, the navigation device 200 sends the stored road information to the awake state determination system 100.

  Then, the arousal state determination system 100 sends an awakening state determination signal related to the determination of the driver's arousal state to the notification device 500 and the driving support device 600.

  The arousal state determination system 100 according to the present embodiment includes a behavior detection unit 10 that realizes a function of detecting a driver's behavior, a determination criterion setting unit 20 that realizes a function of setting a determination criterion, and a determination unit 30. .

  The behavior detection unit 10, the determination criterion setting unit 20, and the determination unit 30 of the arousal state determination system 100 include a ROM (Read Only Memory) that stores a program for executing the arousal state determination process according to the present embodiment, By executing a program stored in the ROM, a CPU (Central Processing Unit) as an operation circuit functioning as the awake state determination system 100 and a RAM (Random Access Memory) functioning as an accessible storage device are provided. Prepare. As an operation circuit, instead of or in addition to a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. Can be used. These hardware cooperates with software that realizes the driver's head detection function, determination criterion setting function, and determination function, and executes the arousal state determination process.

  Each configuration will be described below.

  First, the behavior detection unit 10 will be described. The behavior detection unit 10 detects the behavior of the driver's head based on the captured image of the driver. The behavior detection unit 10 includes a first index detection unit 11 and a second index detection unit 12, and detects two indexes related to the driver's behavior.

  The first index detection unit 11 of the behavior detection unit 10 uses the first index having a high correlation with the decrease in the arousal level as a detection target. As the first index, it is preferable to adopt an electroencephalogram (α wave, β wave), skin potential, closed eye rate, occurrence frequency of long closed eyes, and the like.

The 1st index detection part 11 of this embodiment detects the closed eye rate regarding the behavior of a driver | operator's eyes based on a driver | operator's captured image as a 1st parameter | index. For this reason, the first index detection unit 11 of the present embodiment includes an eye region specifying function 111, an eye closing determination function 112, and an eye closing rate calculation function 113.

  The eye area specifying function 111 specifies an eye area related to the eye opening operation of the driver's face included in the video signal acquired from the image processing apparatus 800. The eye area specifying function 111 specifies an eye area based on the video signal acquired from the image processing apparatus 800.

  The image processing apparatus 800 extracts a region corresponding to the driver's head portion and a region corresponding to the eye portion included in the face based on the captured image, and sends the extracted information to the first index detection unit 21. . Specifically, the image processing apparatus 800 extracts eyes (pupils), upper eyelids, and lower eyelids included in the imaging information, and sets the positions as coordinate values (coordinate value ranges) on the image to the first index detection unit 21. Send it out.

  The method of extracting an area corresponding to the driver's head from the captured image and the method of extracting an area corresponding to the eye included in the face from the captured image, which are performed by the image processing apparatus 800, are particularly limited. Instead, a known method can be used. For example, the technique disclosed in Japanese Patent Laid-Open No. 10-40361 can be used.

  Further, when the driver wears glasses or sunglasses, the image processing device 800 is, for example, an image processing device 800 disclosed in Japanese Patent Application Laid-Open No. 2008-146356, eyes included in the face, The head region and the face region can be extracted by using a method of combining one or a plurality of features such as an eyebrow, a nose, a mouth, and a contour.

  The closed eye determination function 112 determines the eye open state of the driver's eyes from the video signal based on the captured image of the eye area. FIG. 3 is a diagram illustrating an example of a video signal in the eye area. The closed eye determination function 112 can determine the opening / closing of the eye by calculating a change in the vertical width of the eye included in the captured image, using the method disclosed in Japanese Patent Laid-Open No. 10-40361. Further, the closed eye determination function 112 uses the method disclosed in Japanese Patent Application Laid-Open No. 2007-151798, and uses the upper end (upper eyelid), the upper and lower end of the lower end of the eye (lower eyelid), and the left end (upper eye) of the eye. The aspect ratio of the eye may be obtained from the right and left ends of the right edge (eye corner) of the eye, and the opening / closing of the eye may be determined based on the aspect ratio of the eye.

  Furthermore, the closed eye rate calculation function 113 calculates the closed eye rate of the driver based on the determination result of the closed eye determination function 112. The eye closure rate in the present embodiment is a ratio of time during which the eyes are closed per certain time. In general, as the degree of arousal decreases, the speed at which a person's eyes are closed becomes slow, and the closing time becomes longer. The closed eye rate calculation function 113 observes the video signal acquired from the image processing apparatus 800 over time, and obtains the ratio of the time during which the eyes are closed for a predetermined time or more in a certain time. The method for obtaining the closed eye rate is not particularly limited, and for example, the method disclosed in Japanese Patent Laid-Open No. 7-156682 can be used.

As an index similar to the eye closure rate, the number of occurrences (occurrence frequency) of the phenomenon of closing the eyes for a predetermined time or more in a certain time may be obtained. The method for obtaining the number of times the eyes are closed is not particularly limited, and for example, the method described in Japanese Patent Application No. 10-40361 can be used.

  Next, the second index detection unit 12 included in the behavior detection unit 10 will be described. The second index detection unit 12 of the behavior detection unit 10 sets the second index having a high correlation with the conflict state with respect to the decrease in the arousal level as a detection target. Of course, the 2nd index detection part 12 of this embodiment is good also considering the index with a high correlation with the fall of arousal level as a detection target. As the second index, it is preferable to adopt heartbeat, respiration, secondary behavior, head behavior and other body movements.

  The 2nd parameter | index detection part 12 of this embodiment detects the behavior of the driver | operator's head based on a driver | operator's captured image as a 1st parameter | index. Thus, by detecting an index different from the first index, the arousal level can be accurately determined. In addition, by using the behavior of the head different from the closed eye rate of the eyes as the first index, the driver's arousal level is determined even when the driver is wearing glasses or the like. be able to.

The second index detection unit 12 of the present embodiment displays an index highly correlated with the process of lowering the arousal level in a situation where the driver tries to withstand this in the process of lowering the arousal level (a state of conflict with sleepiness). It is preferable to make it a detection target. In general, an index (for example, behavior such as a closed eye rate) having a high correlation with a decrease in the driver's arousal level is suppressed from changing in value in a state of conflict with drowsiness in the process of decreasing the arousal level. That is, when the arousal level starts to decrease, the driver resists this, and the behavior (first index) highly correlated with the decrease in the arousal level is suppressed. Similarly, when the degree of arousal starts to decrease, the driver resists this, and a change appears in the driver's other behavior (second index). Although there are conscious movement and unconscious movement in the movement of the human head, the behavior of the driver's head detected by the present embodiment as the second index is the latter unconscious movement. . Humans exercise reflexively (involuntarily) to maintain posture. When the degree of arousal is high, the behavior of the head is stabilized because of the reflexive movement that keeps the posture. However, when the arousal level starts to decrease, the posture maintenance function gradually decreases, the reflexive movement for maintaining the posture is not maintained, and the behavior of the head is disturbed. According to experiments, this disturbance in head behavior tends to increase in a state of conflict with sleepiness.

  The 2nd parameter | index detection part 12 of this embodiment detects the behavior of the driver | operator's head based on a driver | operator's captured image as a 2nd parameter | index.

  The second index detection unit 12 has a head reference position specifying function 121 and a head behavior calculating function 122.

  As a first function of the second index detection unit 12, the head reference position specifying function 121 specifies a reference position included in the driver's head area based on the captured image. This reference position is a reference when detecting the behavior of the driver's head. The head reference position specifying function 121 of the present embodiment detects the barycentric coordinates of the driver's head area included in the captured image as the reference position. The head reference position specifying function 121 obtains the coordinates of the center of gravity based on a plurality of facial feature coordinate sequences included in the extracted video data of the head region. The obtained head center-of-gravity coordinates are specified as a reference position.

  As a second function of the second index detection unit 12, the head behavior amount calculation function 122 calculates the behavior amount of the driver's head with respect to the specified reference position. The head behavior amount calculation function 122 calculates a standard deviation with respect to a predetermined number of head reference positions specified most recently, and uses this as the head behavior amount. Specifically, as shown in FIG. 4, the reference position (head center of gravity coordinates) at time t is set as coordinates (Hx, Hy), the latest n pieces of data are stored in the memory, and the horizontal standard deviation and the vertical A horizontal head behavior amount Mx and a vertical head behavior amount My, which are standard deviations of directions, are calculated. In the present embodiment, the head deviation amount (Mx, My) is obtained by obtaining the standard deviation of the latest n pieces of data for each of Hx and Hy.

  Next, the determination criterion setting unit 20 will be described. Based on the behavior of the head of the driver in a monotonous section where the driving of the vehicle determined from the acquired road information and / or vehicle information is monotonous, the determination criterion setting unit 20 determines a criterion for determining the driver's arousal level. Set.

  The determination criterion setting unit 20 of the present embodiment includes a first determination criterion setting function 21 that sets a first determination criterion that is a determination criterion based on a first index that includes a closed eye rate, and a first criterion that includes the behavior of the driver's head. A second judgment criterion setting function 22 that is a judgment criterion based on two indicators, a monotonic section detection unit 23 that detects that the vehicle travels in a monotonous section of the traveling section, and a vehicle that is a traveling section other than the monotonous section And a normal section detection unit 24 that detects that the section travels.

  The first determination criterion setting function 21 sets a determination criterion relating to the closed eye rate for determining that the driver's arousal level is low. The first determination criterion setting function 21 sets a determination criterion based on the driver's closed eye rate in a monotonous section where the driving of the vehicle becomes monotonous and the driver's arousal level tends to decrease.

  The second determination criterion setting function 22 sets a determination criterion related to the behavior of the head for determining that the driver's arousal level is low. The second determination criterion setting function 22 sets a determination criterion according to the behavior of the driver's head in a monotonous section where the driving of the vehicle becomes monotonous and the driver's arousal level tends to decrease.

  Here, the monotone section detection unit 23 that detects the monotone section used when setting the first determination criterion and the second determination criterion will be described.

  The monotone section detection unit 23 determines whether or not the travel section in which the vehicle travels is a monotone section based on the road information acquired from the navigation device 200. The road information of this embodiment includes one or more of a road shape, a road type, and a road attribute. The road shape includes road curvature, road intersection, etc., the road type includes information on whether the road is an automobile-only road such as an expressway, and the road attribute is a single road with one direction. Includes information about roads.

  Specifically, the monotone section detection unit 23 is a single road in which the road on which the vehicle travels is a highway, the road on which the vehicle travels is a straight road, or the road on which the vehicle travels is unidirectional. If it is, it is determined that the travel section in which the vehicle travels is a monotone section.

  Further, the monotone section detection unit 23 determines that the travel section in which the vehicle travels is a monotone section based on the vehicle information acquired from the vehicle controller 300. The vehicle information includes one or more of steering angle information, vehicle speed information, accelerator opening information, and brake information. Specifically, the determination criterion setting unit 20 changes the accelerator opening value within a predetermined time when the steering amount within a predetermined time is less than a predetermined threshold, when the number of brake operations within the predetermined time is less than a predetermined value. When the amount is less than the predetermined value, or when the change amount of the vehicle speed within the predetermined time is less than the predetermined value, it is determined that the travel section in which the vehicle travels is a monotone section.

  Further, the monotone section detection unit 23 determines that the travel section in which the vehicle travels is a monotone section based on the inter-vehicle distance information with the other vehicle acquired from the inter-vehicle distance measurement device 400. The inter-vehicle distance information of the present embodiment includes any one or more of presence / absence information on the presence of other vehicles around the host vehicle or distance information between the host vehicle and the other vehicle. Specifically, the determination criterion setting unit 20 determines that the travel section in which the vehicle travels is a monotonous section when the amount of change in the inter-vehicle distance with other vehicles within a predetermined time is less than a predetermined value or when there is no preceding vehicle. Judge.

  Whether or not the travel section is a monotonous section may be determined based on any of road information, vehicle information, or inter-vehicle distance, or any one of road information, vehicle information, and inter-vehicle distance. You may judge combining the above information.

  Further, the monotone section detection unit 23 has a function of determining whether the travel section in which the vehicle travels is a straight section or a curve section based on the road shape included in the road information or the steering angle included in the vehicle information. Have. Specifically, the determination criterion setting unit 20 defines in advance a steering angle pattern for traveling in a straight section and a steering angle pattern for traveling in a curve section (for each of a gentle curve section and a curved section having a large curvature). Then, the steering angle pattern is compared with the change in the steering angle obtained from the vehicle controller, and according to the degree of coincidence, the section in which the vehicle travels is a straight section or a curved section (a gentle curve section or a large curvature) It is determined whether it is a curve section). Of course, the determination criterion setting unit 20 may determine whether the traveling section of the vehicle is a straight section or a curved section based on the road curvature included in the road information obtained from the navigation device 200.

  Although not particularly limited, when the travel section is a straight travel section and when the travel section is a gentle curve section, the travel section is determined to be a monotone section where the driving is monotonous.

  The monotone section detection unit 23 determines that the travel section is a monotone section when the travel section in which the host vehicle travels meets the predefined monotone section requirements. The monotonous section detection unit 23 of the present embodiment is a section where the vehicle speed is 60 km / h or more and the speed change is within ± 10 km / h, or a section where the absolute value of the steering angle can pass through 10 degrees or more and 30 degrees or less, Judged as a monotone section with a gentle curve.

  Further, the normal section detection unit 24 detects a travel section other than the monotone section detected by the monotone section detection unit 23 as a normal section.

  Here, the description returns to the first determination criterion setting function 21 and the second determination criterion setting function 22.

  The first determination criterion setting function 21 sets a first determination criterion based on the driver's eye closure rate when traveling in a monotonous section. The method for deriving the closed eye rate as the first determination criterion is not particularly limited, but the average value of the closed eye rate or the maximum detected closed eye rate within a predetermined time after the vehicle transitions from the normal zone to the monotonous zone. Can be set based on.

  The second determination criterion setting function 22 of the present embodiment sets the second determination criterion based on the behavior of the driver's head when traveling in a monotone section. The method of deriving the head behavior amount as the second determination criterion is not particularly limited, but in the present embodiment, the vertical head of the driver detected within a predetermined time after transition from the normal section to the monotone section. Based on the average value of the behavior of the driver, a criterion for judging the driver's arousal level is set.

  Further, as shown in FIG. 5, the second determination criterion setting function 22 uses the data at the start of driving for the time series data of the standard deviation of the vertical center of gravity coordinates (reference position) obtained as the head behavior. Then, an average value is obtained, and a threshold value that serves as a determination criterion is set based on the average value. The section for determining the threshold (learning section) is set to a certain time after the vehicle transitions from the normal section (section other than the monotone section) to the monotone section. In addition, the threshold value serving as a determination criterion to be set is obtained by multiplying the average value of the head behavior during the learning time by a certain coefficient. This coefficient can be set arbitrarily by experiment or the like, and may be set for each vehicle type or driver.

  Further, the second determination criterion setting function 22 sets a determination criterion based on the behavior of the driver's head in a predetermined curve section as one aspect of the monotone section. The predetermined curve section is set based on road information, vehicle information, and / or inter-vehicle distance. For example, the second criterion setting function 22 is a section where the vehicle speed is 60 km / h or more and the speed change is within ± 10 km / h, or a section where the absolute value of the steering angle is 10 degrees or more and 30 degrees or less can be passed. A predetermined curve section having a gentle curve is used.

  In addition, the second determination criterion setting function 22 is detected within a predetermined time (learning section) after the vehicle starts traveling when the vehicle travels in a predetermined curve section after transitioning from the normal section to the monotone section. Based on the behavior of the driver's horizontal head, a criterion for determining the driver's arousal level is set. In the present embodiment, the driver's arousal is determined based on the driver's horizontal head behavior that is determined to have the highest arousal level among the driver's horizontal head behavior detected in the learning section. Set the criteria for judging the degree.

  As shown in FIG. 6, the second determination criterion setting function 22 is configured such that the vehicle enters the curve section of the time series data of the standard deviation of the horizontal head center-of-gravity coordinates (reference position) obtained as the head behavior. Then, a threshold value serving as a determination criterion is set based on the maximum standard deviation of the horizontal head of the driver detected during a predetermined time (learning time) immediately after that. Note that the learning time for determining the threshold is a fixed time after the vehicle transitions from the normal zone to the monotone zone, and is a travel zone determined as a predetermined curve zone. Further, it is assumed that the criterion (threshold value) is obtained by multiplying the maximum value in the learning time by a certain coefficient. This coefficient can be set arbitrarily by experiment or the like, and may be set for each vehicle type or driver.

  Next, the determination unit 30 will be described.

  The determination unit 30 determines the driver's arousal state based on the driver's behavior detected by the behavior detection unit 10 in accordance with the determination criterion set by the determination criterion setting unit 20.

  If the closed eye rate (first index) detected by the first index detection unit 11 is higher than the determination criterion based on the determination criterion set by the first determination criterion setting function 21, the determination unit 30 determines that the driver When the eye closure rate (first index) detected by the first index detection unit 11 is lower than the criterion, it is determined that the driver's arousal level is high.

In addition, the determination unit 30 determines whether the behavior amount (second index) of the driver's head detected by the second index detection unit 12 is based on the determination criterion set by the second determination criterion setting function 22. Is higher, the driver's arousal level is determined to be low. If the behavior amount (second index) of the driver's head detected by the second index detection unit 12 is lower than the criterion, driving is performed. Judgment is high.

  Furthermore, the determination unit 30 determines the driver's arousal level using both the first index and the second index. That is, the determination unit 30 determines the driver's eye-closing rate and the driver based on the first determination criterion set by the first determination criterion setting function 21 and the second determination criterion set by the second determination criterion setting function 22. The amount of head behavior is evaluated.

  FIG. 7 is a diagram illustrating a change over time (broken line) of the index when there is no conflict between the first index and the second index, and a change over time (solid line) of the index when there is a conflict.

  As shown in FIG. 7, when there is no conflict, the first index is a characteristic that a change over time (broken line) increases with a decrease in the degree of arousal and a subject (driver) who tries to endure drowsiness. If there is a conflict, the change is suppressed (see solid line). On the other hand, when there is no conflict, the second index has a characteristic that a change with time (broken line) accompanying a decrease in the arousal level is smaller than that of the first index. In addition, the second index is a change when there is a conflict in the subject (driver) such as trying to endure drowsiness despite a small change over time (broken line) accompanying a decrease in arousal level. There is a characteristic that it is large.

  That is, when the driver is in a normal psychological situation such as enduring drowsiness, the first index including the closed eye rate in the present embodiment is observed to change markedly during awakening, while the behavior of the head In the second index including the amount, a remarkable change is observed in the process from wakefulness to non-wakefulness. Thus, when considering the driver's conflict, the first index is mainly suitable for the determination of awakening / non-wakening, and the second index is an arousal or non-awakening that causes resistance and conflicts against a decrease in arousal level. Suitable for judging the process leading to

  Therefore, in the present embodiment, by combining these two indexes (first index and second index), the driver's arousal state is more specifically analyzed, and the actual driver's state is determined with high accuracy. .

  Specifically, the determination unit 30 of the present embodiment determines that the driver's arousal level is suitable for driving operation based on the combination of the first index and the second index, and the driver's arousal level is determined based on the driving operation. Although it is a state suitable for driving, the degree of arousal is decreasing, and a conflict state in which the driver resists a decrease in the degree of arousal and an abnormal state in which the driver's arousal level is not suitable for driving operation are determined.

  Furthermore, the determination unit 30 of the present embodiment is a conflict state in which the driver resists a decrease in the arousal level, although the arousal level is not suitable for driving operation based on the combination of the first index and the second index. And an abnormal state that is not in a conflict state in which the driver resists a decrease in arousal level.

  The determination unit 30 of the present embodiment defines the correspondence between the combination of the first index and the second index and the driver's state in advance, and stores this. FIG. 8 is a diagram illustrating an example of a determination map in the case where the driver's arousal level is determined based on the first index (closed eye rate) and the second index (head behavior).

  In FIG. 8, the horizontal axis A indicates the closed eye rate that is the first index, and the vertical axis B indicates the amount of behavior of the head that is the second index.

In FIG. 8, “normal” of the first index is a state in which the detected eye closing rate is equal to or lower than the first determination criterion set by the first determination criterion setting function 21 (the probability of closing the eyes is low). Further, “abnormality” of the first index is a state in which the detected eye-closing rate is higher than the first determination criterion set by the first determination criterion setting function 21 (the probability of closing the eyes is high). In addition, along the horizontal axis, the first column shows the case where the closed eye rate is “normal”, the second column shows the case where the closed eye rate has transitioned from “normal” to “abnormal”, and the third column shows The case where the closed eye rate transitions from “abnormal” to “normal” is shown.

Further, in FIG. 8, “normal” of the second index indicates that the detected behavior amount of the head of the driver is equal to or less than the second determination criterion set by the second determination criterion setting function 22 (head shaking). Small). Further, “abnormality” of the second index indicates that the detected behavior amount of the head of the driver is larger than the second determination criterion set by the second determination criterion setting function 22 (the head shake is large). State. In addition, along the vertical axis, the first column shows the case where the head behavior amount is “normal”, and the second column shows the case where the head behavior amount transitions from “normal” to “abnormal”. The third column shows the case where the behavior amount of the head transitions from “abnormal” to “normal”.

A determination method of the determination unit 30 of this embodiment will be described. For the following determination, any one determination method may be adopted, two or more determination methods may be combined, or all determination methods may be used.

First, the determination unit 30 has the driver's closed eye rate detected based on the driver's captured image equal to or lower than the first determination criterion, and the driver's head detected based on the driver's captured image. When the behavior amount is equal to or less than the second determination criterion, it is determined that the driver's arousal level is normal. That is, when the probability of closing the eyes is lower than a predetermined threshold and the head shake is lower than the predetermined threshold, it is determined that the driver's arousal level is suitable for driving.

That is, the driver's closed eye rate is normal, and since there is no conflict seen in the process of lowering the arousal level, it can be determined that the driver is in a high arousal state. Thus, by combining the first index and the second index, the driver's arousal level can be determined with high accuracy.

In addition, the determination unit 30 has a driver's closed eye rate detected based on the driver's captured image equal to or lower than the first determination criterion, and the driver's head detected based on the driver's captured image. If the behavior amount is higher than the second criterion, it is determined that the driver's arousal level is normal, but the driver is in conflict and the driver's arousal level is in the process of decreasing. That is, if the probability of closing eyes is lower than a predetermined threshold and the head shake is higher than a predetermined threshold, the driver's current arousal level is suitable for driving, but the degree of concentration on driving decreases. Therefore, it is determined that the amount of momentum of the driver's head is increasing due to the subconsciousness that resists the decrease in arousal level. That is, the driver is in a normal state with respect to the closed eye rate, but can see that there is a conflict in the process of lowering the arousal level, and therefore the driver is in a state where the arousal level is decreasing.

As described above, according to this determination method, it is possible to foresee the sign before the arousal level actually decreases (falls into a state unsuitable for driving). Thus, if a sign of a decrease in the arousal level can be predicted, the fact can be notified before the awakening level actually decreases, and a decrease in the driver's arousal level can be prevented. Furthermore, if a sign of a decrease in arousal level can be predicted, it can also be used as a trigger for starting the operation of the driving support device 600.

As described above, by combining the first index and the second index, the actual driver's arousal level can be specifically determined with high accuracy.

Further, the determination unit 30 determines that the driver's arousal level is abnormal when the driver's closed eye rate detected based on the driver's captured image is higher than the first determination criterion. Since the closed eye rate is an index that changes in response to a decrease in the driver's arousal level, when the closed eye rate is higher than the first criterion, it is determined that the driver's arousal level is low. The arousal level can be determined with high accuracy.

Further, when determining that the driver's arousal level is in an abnormal state, the determination unit 30 further specifically identifies the abnormal state.

First, the determination unit 30 has a driver's eye closure rate detected based on the driver's captured image higher than the first determination criterion, and the driver's head detected based on the driver's captured image. When the behavior amount of the vehicle is equal to or less than the second determination criterion, it is determined that the driver's arousal level is abnormal and the driver's conflict is weak in the first abnormal state. This first abnormal state is, for example, a state in which the driver is drowsy or a state in which the driver feels sleepy. In this first abnormal state, the driver maintains an exercise that maintains his posture.

Secondly, the determination unit 30 has a driver's eye closure rate detected based on the driver's captured image higher than the first determination criterion, and the driver's head detected based on the driver's captured image. When the behavior amount of the vehicle part transitions from a state that is equal to or lower than the second determination criterion to a state that is higher than the second determination criterion, it is determined that the driver's arousal level is in the second abnormal state. This second abnormal state is, for example, a state where the driver has entered a light sleep. In the second abnormal state, the movement for maintaining the driver's posture is reduced, and is in a state of resisting a reduction in arousal level.

Thirdly, the determination unit 30 has a driver's eye closure rate detected based on the driver's captured image higher than the first determination criterion, and the driver's head detected based on the driver's captured image. When the behavior amount of the vehicle part transitions from a state higher than the second determination criterion to a state equal to or lower than the second determination criterion, the driver's arousal level is lower than that in the second abnormal state, and the driver's conflict is weak. Judged to be in an abnormal state. This third abnormal state is, for example, a state in which the driver enters a deep sleep. In the second abnormal state, the driver is in a state in which the exercise of maintaining the posture is reduced and the consciousness of the conflict against the reduction in the arousal level is also weakened.

  FIG. 9 is a diagram illustrating an example of a change over time of the first index (closed eye rate) and the second index (behavior of the head in the vertical direction) and a change in the driver's arousal level over time. In this example, the threshold value of the closed eye rate is 10%, and the first 3 minutes is a learning interval, and a value 7.6 obtained by multiplying the average value of the standard deviation of head behavior in this interval by a multiplier of 2.0 is set as the threshold value.

The driver's arousal state can be determined on the basis of the temporal change of the first index (eye closing rate) and the second index (vertical behavior of the head) and the map shown in FIG. That is, a section where the driver is in a high arousal state (or time or timing), a section where he / she feels a conflict (or time or timing) due to a decrease in the arousal state, a section where he / she feels sleepy (or time or timing) Can be estimated.

In the example illustrated in FIG. 9, the first index is normal and the second index is normal, and the driver's arousal level is normal, and the second index changes prior to the first index, indicating a conflict state. . Thereafter, when the first index changes from normal to abnormal, the driver's arousal state becomes sleepy (small conflict). In addition, when the second index changes from normal to abnormal, the driver transitions to a state of shallow dozing (conflict large). After that, when the second index changes from abnormal to normal, the driver deeply doesze (small conflict). Transition to the state.

  The determination unit 30 of the present embodiment can determine such changes in the driver's arousal state sequentially, specifically and with high accuracy, using the map shown in FIG.

The determination unit 30 outputs the determination result to an external device. The external device to which the determination result is output is not particularly limited. The determination unit 30 includes a speaker 501 that conveys information to the driver by voice, a display 502 that conveys information to the driver by image or text, or a vibration device 503 or a pressing device that conveys information via the driver's sense of touch. The information can be output to the driver notification device 500 including the number 504. In addition, the determination unit 30 uses the determination result as a key plane system 601 that supports driving in which the vehicle travels in a certain positional relationship with the road lane, and the vehicle travels in a certain positional relationship with other vehicles. This can be output to the driving support device 600 including the inter-vehicle maintenance system 602 that supports driving.

  The notification device 500 outputs notification information suitable for the arousal state based on the acquired driver's arousal level. In addition, the driving support device 600 performs driving support suitable for the awake state based on the acquired driver's arousal level.

  In the present embodiment, since a specific arousal state can be output, the content of the notification information and the notification timing can be set according to the driver's arousal state, and the content and the operation timing of the driving assistance can be driven. Depending on the awakening state of the person.

  Furthermore, in this embodiment, since the driver's actual arousal state can be output with high accuracy, the notification information suitable for the driver's arousal state is output with high accuracy, which is suitable for the driver's arousal state. Driving assistance can be executed with high accuracy.

  Next, the control procedure of the arousal state determination system 100 of this embodiment will be described. 10 to 13 are flowcharts for explaining the control procedure of the arousal state determination system 100. FIG. 10 is a flowchart for explaining the determination processing of the arousal level, FIG. 11 is a flowchart for explaining the calculation processing for the eye closing rate, and FIG. 12 is a flowchart for explaining the calculation processing for the behavior amount of the head. FIG. 13 is a flowchart for explaining the determination criterion setting process and the arousal state determination process.

  First, based on FIG. 10, the overall flow of the processing of the awake state determination 100 of the embodiment will be described.

  In step 1, the behavior detection unit 10 acquires a face image via the camera 700 and the image processing apparatus 800 (Step 1). The face image is extracted from the video signal based on the positional relationship between the eyes and the mouth constituting the face.

  In step 2, the second index detection unit 12 detects the driver's head region (Step 2).

In Step 3 in parallel with Step 2, the first index detection unit 11 detects the first index (eye closure rate) based on the driver's face image (Step 3).

Further, in Step 4 in parallel with Step 3, the second index detection unit 12 calculates a second index (head behavior amount) with respect to the head reference position based on the image of the head region detected in Step 2. Detect (Step 4).

Furthermore, in step 5, the determination criterion setting unit 20 determines a first determination criterion regarding the driver's eye closure rate and a second determination criterion regarding the behavior of the driver's head in order to determine the driver's arousal level. Is set (Step 5). Specifically, in step 5, the determination criterion setting unit 20 determines the driver's closed eye rate in a monotonous section in which the driving of the vehicle determined from the acquired road information, vehicle information, and / or inter-vehicle distance information is monotonous and / or. Based on the behavior of the driver's head, a criterion for determining the driver's arousal level is set.

Subsequently, in step 6, the determination unit 30 follows the first determination criterion and the second determination criterion set in step 5, and the first index data (driver's eye closing rate) detected in step 3 and in step 4. Based on the detected second index data (the amount of behavior of the driver's head), the driver's arousal state is determined (Step 6).

The determination unit 30 sends a wakefulness signal related to the determination result toward an external device (the notification device 500 or the driving support device 600).

FIG. 11 is a flowchart showing a subroutine of first index detection processing in step 3 of FIG. By this processing, the driver's eye closure rate, which is the first index, is detected.

First, in step 301, the eye region specifying function 111 of the first index detection unit 11 detects the position of each face part based on the captured image, and the eye opening / closing operation of the driver's face based on the positional relationship. The eye area (coordinates) related to the is specified (Step 301). At this time, the eye region may be specified using the head reference position specified by the second index detection unit 12.

In step 302, the closed eye determination function 112 of the first index detection unit 11 determines the closed eye state of the driver based on the captured image of the eye area specified in step 301 (Step 302).

In step 302, the eye-close rate calculation function 113 of the first index detection unit 11 calculates the driver's eye-close rate based on the determination result of the eye-closed state (and the eye-open state) determined in step 302 (Step 303).

FIG. 12 is a flowchart showing a subroutine of the second index detection process in step 4 of FIG. By this process, the behavior of the driver's head, which is the second index, is detected.

First, in step 401, the head reference position specifying function 121 of the second index detection unit 12 specifies a reference position included in the driver's head region based on the captured image. In this process, the head reference position specifying function 121 specifies the barycentric coordinates of the head region (Step 401). At this time, the reference position of the head region may be specified using the face image and the position of the face specified by the first index detection unit 11.

  In subsequent step 402, the head behavior calculation function 122 of the second index detection unit 12 calculates the behavior amount of the driver's head relative to the specified reference position (Step 402).

  Further, FIG. 13 is a flowchart showing a subroutine of the determination criterion setting process in step 5 and the arousal level determination process in step 6 of FIG. By this process, a reference for determining the driver's arousal level is set, and the driver's arousal level is determined based on the detected first index and second index.

  First, in step S <b> 501, the first determination criterion setting function 21 uses the road information acquired from the navigation device 200, the vehicle information acquired from the vehicle controller 300 and / or the inter-vehicle distance measurement device 400. Based on the closed eye rate of the driver in a monotonous section where the driving of the vehicle determined from the acquired inter-vehicle distance information is monotonous, a first criterion for determining the driver's arousal level is set.

  In step 502, the second determination criterion setting function 22 determines from the road information acquired from the navigation device 200, the vehicle information acquired from the vehicle controller 300 and / or the inter-vehicle distance information acquired from the inter-vehicle distance measuring device 400. Based on the behavior of the driver's head in a monotonous section where the driving of the vehicle is monotonous, a second criterion for determining the driver's arousal level is set.

  In subsequent step 601, the driver's arousal state is determined using the driver's eye-closing rate and the behavior amount of the driver's head, and the set first and second determination criteria.

Specifically, referring to the map illustrated in FIG. 8 that associates the combination of the first index with respect to the first criterion and the second index with respect to the second criterion and the driver's arousal level, The driver's arousal state is determined based on the index (the closed eye rate of the driver) and the second index (the amount of behavior of the driver's head).

Since the arousal state determination system 100 of the present embodiment is configured and functions as described above, the following effects can be obtained.

The arousal state determination system 100 of the present embodiment detects the behavior of the driver's head, and the behavior of the driver's head in a monotonous section where the driving of the vehicle obtained from the road information and / or vehicle information is monotonous. In order to determine the driver's arousal state using this criterion, the determination criteria for determining the driver's arousal level are set based on the driver's head behavior change. Since the degree of arousal can be determined, the driver's arousal state can be determined with high accuracy even when the driver is wearing glasses or the like.

In addition, in order to determine a criterion for determining the driver's arousal level based on the behavior of the driver's head in a monotonous section, the determination based on the driver's behavior when the driver's arousal level decreases A standard can be set. That is, when traveling in a monotonous driving environment, it is possible to set a determination criterion based on the behavior of the head in the process in which the driver's arousal level decreases. By using such a criterion, the driver's arousal level can be determined with high accuracy.

Further, since the reference position (for example, the center of gravity) of the head region is specified and the behavior of the driver's head is calculated using a deviation from the reference position, the behavior amount of the head can be accurately calculated. . By using the behavior amount thus calculated, the driver's arousal level can be determined with high accuracy.

In addition, the movement of the head in the sitting posture of the driver is generally accompanied by a movement for maintaining the balance of the head, and there is no disturbance in the movement of the head when the arousal level is high. In addition, as the driver's arousal level decreases, the movement for maintaining the balance decreases, and the movement of the head is disturbed. In addition, when the user is aware of a decrease in the arousal level, body movement is generated to maintain the awakening. However, these two actions are lost when the computer becomes doze. As described above, based on the behavior of the driver's head, it is possible to determine the state of the process in which the arousal level is reduced or the arousal level is reduced.

The head behavior includes horizontal movement and vertical movement. By extracting the behavioral component in the horizontal direction from the behavior of the head, the behavioral amount of the head can be accurately determined by calculating the behavioral amount of the driver's head by eliminating the influence of vertical movement. Can be calculated. By using the behavior amount thus calculated, the driver's arousal level can be determined with high accuracy.

Further, when the horizontal component of the head behavior is used as the second index data, when the vehicle travels in a predetermined curve section after the travel section of the vehicle transitions from the normal section other than the monotonous section to the monotonous section, the curve Based on the behavior of the driver's horizontal head that is determined to have the highest arousal level among the behavior of the driver's horizontal head detected within a predetermined time after the start of driving, Set the criteria for arousal level. Since the determination criterion is set in the curve section, it is possible to set the determination criterion that excludes the influence of the horizontal head movement that occurs in the curve section for the safety confirmation operation. In addition, although individual differences occur in the head behavior, in this embodiment, since the criterion is set based on the head behavior during actual running, the criterion that excludes the influence caused by the individual difference in the head behavior is set. Can be set. By using the criterion set in this way, the driver's arousal level can be determined with high accuracy.

Furthermore, by extracting the behavioral component in the vertical direction from the behavior of the head, the behavioral amount of the head is calculated by calculating the behavioral amount of the driver's head by eliminating the influence of the horizontal movement. It can be calculated accurately. By using the behavior amount thus calculated, the driver's arousal level can be determined with high accuracy.

Further, when the vertical component of the head behavior is used as the second index data, the vertical direction of the driver detected within a predetermined time after the vehicle travel section transitions from the normal section other than the monotone section to the monotonous section. Based on the average value of the behavior of the head, a criterion for determining the driver's arousal level is set. In the present embodiment, since the determination criterion is set based on the head behavior during actual running, it is possible to set the determination criterion that excludes the effects caused by individual differences in head behavior. By using the criterion set in this way, the driver's arousal level can be determined with high accuracy.

In this way, with respect to the behavior of the driver's head detected from the captured image, the horizontal and vertical behavior components with respect to the apparent head reference coordinates (head center-of-gravity coordinates) are separated, and the standard for each time-series data Since the deviation is obtained, it is possible to separate and handle a horizontal direction component that includes a large amount of behavior due to driving operations such as a safety check in the left-right direction and a vertical direction component that has a small behavior due to driving operations. Further, by acquiring the head behavior of the horizontal component and the head behavior of the vertical component, and setting the judgment standard for each, it is possible to set the judgment standard excluding the influence of the behavior due to the driving operation. By using the criterion set in this way, the driver's arousal level can be determined with high accuracy.

  Further, by specifying an eye region based on the acquired captured image, determining the eye-closed state based on the captured image of the eye region, and calculating the eye-closing rate, the eye-closing rate can be accurately calculated. In general, eye opening / closing information is considered to have a high correlation with arousal level, and if the eyes are closed for a long time, it is judged that the driver cannot pay attention to the surroundings, that is, it is unsafe. Therefore, it is possible to accurately determine a state in which the arousal level is reduced. By using the closed eye rate calculated in this manner, the driver's arousal level can be determined with high accuracy.

  Furthermore, the arousal state determination system 100 of the present embodiment determines the driver's arousal level based on a combination of the evaluation related to the first index and the evaluation related to the second index, thereby increasing the driver's arousal level with high accuracy. Can be judged. In particular, in the present embodiment, a first index that changes as the driver's arousal level decreases, and a second index that changes when the driver consciously or unconsciously conflicts with the decrease in the arousal level, Therefore, it is possible to determine the specific arousal level of the driver with high accuracy. Specifically, according to the determination unit 30 of the present embodiment, based on the combination of the first index and the second index, the normal state where the driver's arousal level is suitable for driving operation and the driver's arousal level are: Although it is in a state suitable for driving operation, it is judged that there is a conflict state where the arousal level is decreasing and the driver resists the decrease in the arousal level, and an abnormal state where the driver's arousal level is not suitable for driving operation be able to. Furthermore, according to the determination unit 30 of the present embodiment, the driver resists a decrease in the arousal level based on the combination of the first index and the second index, although the arousal level is not suitable for driving operation. It is possible to determine an abnormal state that is in a conflict state and an abnormal state that is not in a conflict state in which the driver resists a reduction in arousal level.

  By the way, conventionally, it has been reported that when the degree of arousal is low or when fatigue is increased, the driver's behavior decreases and the body motion decreases (Japanese Patent Laid-Open No. 10-272959, particularly No. 2005-095408). In contrast, the present invention focuses on the fact that in the process where the driver's arousal level decreases, the driver resists the arousal level and conflicts, resulting in an increase in the driver's behavior and an increase in body movement. This behavior is used to determine the driver's arousal level. In other words, according to the present invention, in a monotonous driving environment, the arousal level is determined by combining the first index that represents a decrease in the arousal level and the second index that represents a conflict with respect to the decrease in the arousal level. The arousal level can be determined with higher accuracy than when the arousal level is determined only by the first index representing the degree of decrease. Specifically, in this embodiment, in a monotonous driving environment, the closed eye rate (first index) indicating a decrease in arousal level and the behavior of the head (second index) indicating a conflict with the decrease in the arousal level. By combining the above, it is possible to determine the arousal level with higher accuracy than in the case of determining the arousal level based only on the closed eye rate (first index) because it does not depend only on the information about the eyes (the closed eye rate). In addition, since it does not depend only on the information about the eyes (eye closing rate), it is possible to determine the arousal level even in a scene where the eyes cannot be detected due to the influence of glasses.

Specifically, the determination unit 30 determines that the driver's arousal level is normal when the driver's eye closure rate is equal to or lower than the first determination criterion and the behavior amount of the driver's head is equal to or less than the second determination criterion. to decide. In this case, the driver's eye-closing rate is normal, and there is no conflict seen in the process of lowering the arousal level, so the driver can determine with high accuracy that the driver is in a high arousal state. Thus, by combining the first index and the second index, the driver's arousal level can be determined with high accuracy.

Further, when the driver's closed eye rate is equal to or lower than the first determination criterion and the behavior amount of the driver's head is higher than the second determination criterion, the driver's arousal level is normal, but the driver It is determined that the driver is in conflict and is in the process of reducing the driver's arousal level. In this case, although the driver is in the normal state in terms of the closed eye rate, the driver can determine that the degree of wakefulness is decreasing because of the conflict seen in the process of decreasing the degree of wakefulness. In other words, the signs can be foreseen before the arousal level actually decreases (falls into a state unsuitable for driving). Thus, if a sign of a decrease in arousal level can be foreseen, the fact can be notified before the awakening level actually decreases, and a decrease in the driver's arousal level can be prevented. Further, if a sign of a decrease in arousal level can be predicted, it can also be used as a trigger for starting the operation of the driving support device 600. Thus, by combining the first index and the second index, the actual driver's arousal level can be specifically determined with high accuracy.

In addition, the determination unit 30 determines that the driver's arousal level is abnormal when the driver's closed eye rate is higher than the first determination criterion. In this determination, the closed eye rate is an index that changes in accordance with a decrease in the driver's arousal level. Therefore, when the closed eye rate is higher than the first criterion, the driver's arousal level is determined to be low. It is possible to determine a person's actual arousal level with high accuracy.

Further, when the driver's eye closure rate is higher than the first determination criterion and the behavior amount of the driver's head is equal to or less than the second determination criterion, the determination unit 30 indicates that the driver's arousal level is abnormal. It is determined that the driver is in a first abnormal state with a weak conflict. As a result, it is possible to determine a state in which the driver is drowsy or a state in which the driver feels sleepy, among the states in which the driver's arousal level is reduced.

Furthermore, the determination unit 30 changes the state in which the driver's eye closure rate is higher than the first determination criterion and the behavior amount of the driver's head is equal to or lower than the second determination criterion to a state higher than the second determination criterion. When making a transition, it is determined that the driver's arousal level is in the second abnormal state. As a result, among the states in which the driver's arousal level has decreased, the state in which the driver has entered a light sleep, the state of maintaining the driver's posture has decreased, and the state in which the driver's awakening level has been resisted. Judgment can be made.

In addition, the determination unit 30 transitions from a state in which the driver's eye closure rate is higher than the first determination criterion and the behavior amount of the driver's head is higher than the second determination criterion to a state equal to or lower than the second determination criterion. When it does, it judges that it is in the 3rd abnormal state where a driver's arousal level is still lower than a 2nd abnormal state, and a driver's conflict is weak. As a result, among the states where the driver's arousal level has decreased, the driver's state of deep sleep, and the driver's movement to maintain posture will decrease, and the consciousness of the conflict against the decrease in the arousal level will also fade Can be judged.

As described above, the determination of the driver's arousal level can be performed with high accuracy, so that the accuracy of the notification information for driving support using this information can be improved and the operation accuracy of driving support can be increased. Can do.

Furthermore, by specifically determining the driver's arousal level, it is possible to diversify the manner of notification information for driving assistance using this information, and to finely set the driving assistance operation.

Further, when setting the determination criteria, it is determined whether the traveling section in which the vehicle travels is a straight traveling section or a curved section based on the road shape included in the road information or the steering angle included in the vehicle information. Since the behavior of the head in the section and the behavior of the head outside the curve section (straight section) can be measured separately, it is possible to set a judgment criterion according to the traveling section.

Further, road information including any one or more of road shape, road type or road attribute, vehicle information including any one or more of steering angle information, vehicle speed information, accelerator opening information, or brake information, and surroundings of own vehicle Since the monotone section is determined from the inter-vehicle distance information including at least one of the presence / absence information of the presence of the other vehicle and the distance information between the host vehicle and the other vehicle, the monotone section can be accurately determined. By accurately determining the monotonous section, it is possible to identify a traveling section where a decrease in the driver's arousal level occurs, and to set a criterion based on the behavior of the driver's head in the monotonous section. That is, it is possible to prevent the determination standard from being set based on the behavior of the driver's head in the normal section in which the degree of arousal is unlikely to occur.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  That is, in this specification, the wakefulness determination system 100 constituting a part of the driving support system 1000 will be described as an example of the wakefulness determination device according to the present invention, but the present invention is not limited to this. It is not something.

  Further, in this specification, as one aspect of the arousal state determination device, the behavior detection unit 10 as an example of the head behavior detection unit, the determination criterion setting unit 20 as an example of the determination criterion setting unit, and an example of the determination unit Although the arousal state determination system 100 including the determination unit 30 as will be described, the present invention is not limited to this.

  Furthermore, in the present specification, as one aspect of the arousal state determination device including a head behavior detection unit having a head reference position specification unit and a head behavior amount calculation unit, a head reference position specification function 121, The awake state determination system 100 including the second index detection unit 12 having the behavior calculation function 122 will be described, but is not limited thereto.

  Moreover, in this specification, although the wakefulness determination system 100 further provided with the 1st parameter | index detection part 11 is demonstrated as one aspect | mode of the wakefulness determination apparatus further provided with an eye part behavior detection means, it is not limited to this. Absent.

  The eye behavior detecting unit according to the present invention includes an eye region specifying unit, an eye closing determining unit, and an eye closing rate calculating unit. In this specification, an eye region specifying function 111 as an example of an eye region specifying unit, Although the 1st index detection part 11 which has the closed eye determination function 112 as an example of a determination part and the closed eye rate calculation function 113 as an example of a closed eye rate calculation part is demonstrated, it is not limited to this.

  Further, in the present specification, the second determination as an example of the determination criterion setting means for setting the determination criterion (second determination criterion) of the driver's arousal level based on the behavior of the driver's head according to the present invention. Although the arousal state determination system 100 including the determination reference setting unit 20 having the reference setting function 22 will be described, the present invention is not limited to this.

  Similarly, in the present specification, the first determination criterion setting as an example of the determination criterion setting means for setting the determination criterion (first determination criterion) of the driver's arousal degree based on the driver's eye closure rate according to the present invention. Although the arousal state determination system 100 including the determination criterion setting unit 20 having the function 21 will be described, the present invention is not limited to this.

2 is a functional block diagram of a driving support system 1000 including an arousal state determination system 100. FIG. FIG. 6 is a diagram for explaining a mutual relationship between components included in a driving support system 1000. It is a figure which shows an example of the video signal of an eye area | region. A method of obtaining a horizontal head behavior amount Mx and a vertical head behavior amount My that are a standard deviation in the horizontal direction and a standard deviation in the vertical direction with respect to the coordinates (Hx, Hy) with respect to the reference position (head center of gravity coordinates) at time t. It is a figure for demonstrating. It is a figure for demonstrating the method of setting a judgment standard based on the time series data of the standard deviation of the head center-of-gravity coordinate (reference position) of the vertical direction calculated | required as head behavior. Of the time series data of the standard deviation of the horizontal head center-of-gravity coordinates (reference position) obtained as head behavior, the driver detected in a predetermined time (learning time) immediately after the vehicle entered the curve section It is a figure for demonstrating the method of setting a judgment reference | standard based on the standard deviation of the horizontal head. It is a figure which shows the time-dependent change (dashed line) when there is no conflict between the first index and the second index, and the time-dependent change (solid line) when there is a conflict. It is a figure which shows an example of the judgment map in the case of judging a driver | operator's arousal level based on a 1st parameter | index (eye-closing rate) and a 2nd parameter | index (behavior of a head). It is a figure which shows an example of a time-dependent change of a 1st parameter | index (eye-closing rate) and a 2nd parameter | index (behavior of a head), and a time-dependent change of a driver | operator's alertness. It is a flowchart for demonstrating the determination process of the arousal level by the arousal state determination system 100 of this embodiment. It is a flowchart for demonstrating the calculation process of an eye-closing rate. It is a flowchart figure for demonstrating the calculation process of the behavior amount of a head. It is a flowchart for demonstrating the determination reference setting process and the arousal state determination process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1000 ... Driving assistance system 100 ... Arousal state determination system 10 ... Behavior detection part 11 ... 1st parameter | index detection part 111 ... Eye area | region specific function 112 ... Eye-closed determination function 113 ... Eye-closed rate calculation function 12 ... 2nd parameter | index detection part 121 ... Head Part reference position specifying function 122 ... Head behavior calculation function 20 ... Judgment reference setting part 21 ... First judgment reference setting function 22 ... Second judgment reference setting function 23 ... Monotone section detection part 231 ... Curve section detection part 24 ... Normal section Detection unit
DESCRIPTION OF SYMBOLS 30 ... Judgment part 200 ... Navigation apparatus 300 ... Vehicle controller 400 ... Inter-vehicle distance measurement apparatus 500 ... Notification apparatus 600 ... Driving assistance apparatus 600
700 ... Camera 701 ... Image sensor 702 ... Near-infrared illumination 800 ... Image processing apparatus

Claims (17)

Head behavior detecting means for detecting the behavior of the driver's head based on the driver's captured image;
Criteria for determining the driver's arousal level based on the behavior of the driver's head detected in the monotonous section where the driving of the vehicle is determined monotonically based on the acquired road information, vehicle information and / or inter-vehicle distance information A criterion setting means for setting
A wakefulness state determination device comprising: a determination unit that determines the wakefulness state of the driver based on the detected behavior of the head of the driver according to the set determination criterion. In the awakening state determination apparatus according to claim 1,
The head behavior detecting means includes a head reference position specifying unit that specifies a reference position included in the driver's head region based on the captured image;
A wakefulness state determination device comprising: a head behavior amount calculation unit that calculates a behavior amount of the driver's head with respect to the specified reference position. In the wakefulness state determination device according to claim 2,
The head behavior amount calculation unit is a wakefulness state determination device that calculates a behavior amount of the driver's head based on a behavior component of the driver's head in a horizontal direction with respect to the reference position. In the wakefulness state determination device according to claim 3,
Based on the acquired road information, vehicle information, and / or inter-vehicle distance information, the determination criterion setting unit determines whether the vehicle has a predetermined curve section after the vehicle travel section transitions from a normal section other than the monotonous section to a monotonous section. Judgment of the case of traveling, based on the behavior of the driver's horizontal head detected within a predetermined time after the start of traveling of the curve section, to set a criterion for determining the driver's arousal level,
The determination means determines that the driver's arousal level is low when the horizontal behavior of the driver's head detected by the head behavior detection means is equal to or greater than the determination criterion, and the head behavior An arousal state determination device that determines that the degree of arousal of the driver is high when the horizontal behavior of the driver's head detected by the detection means is lower than the determination criterion. In the wakefulness determination apparatus as described in any one of Claims 2-4,
The head behavior observation unit is a wakefulness state determination device that calculates a behavior amount of the driver's head based on a behavior component of the driver's head perpendicular to the reference position. In the wakefulness state determination device according to claim 5,
The determination criterion setting unit determines, based on the acquired road information, vehicle information, and / or inter-vehicle distance information, a case where the traveling section of the vehicle transitions from a normal section other than the monotonous section to a monotonic section, and Based on the behavior of the driver's vertical head detected within a predetermined time after the transition, set a criterion for determining the driver's arousal level,
The determination means determines that the driver's arousal level is low when the vertical behavior of the driver's head detected by the head behavior detection means is equal to or greater than the determination criterion, and the head behavior An arousal state determination device that determines that the degree of arousal of the driver is high when the vertical behavior of the driver's head detected by the detection means is lower than the determination criterion. In the wakefulness determination apparatus as described in any one of Claims 1-6,
Further comprising eye behavior detection means for detecting the behavior of the eye region included in the driver's face based on the driver's captured image,
The eye behavior detecting means, based on the captured image, an eye region specifying unit that specifies an eye region related to an eye opening / closing operation of the driver's face;
Based on the captured image of the specified eye region, the closed eye determination unit that determines the closed eye state of the driver;
A wakefulness state determination device comprising: an eye closure rate calculation unit that calculates an eye closure rate of the driver based on a determination result of the eye closure determination unit. In the awakening state determination apparatus according to claim 7,
The determination criterion setting unit is configured to determine a first determination criterion related to a driver's arousal level based on the driver's closed eye rate in a monotonous section in which driving of the vehicle is monotonous obtained from the acquired road information and / or vehicle information. And the degree of arousal of the driver based on the behavior amount of the driver's head in a monotonous section where the driving of the vehicle is monotonous obtained from the acquired road information, vehicle information and / or inter-vehicle distance information. Set the second criteria,
The determination means determines the driver's arousal state using the driver's eye-closing rate and the driver's head behavior amount, and the set first determination criterion and second determination criterion. State judgment device. In the wakefulness state determination device according to claim 8,
The determination means includes a driver's head whose eye closure rate detected based on the driver's captured image is equal to or lower than the first determination criterion, and detected based on the driver's captured image. A wakefulness determination device that determines that the driver's wakefulness is normal when the behavior amount of the vehicle is below the second determination criterion. In the wakefulness state determination device according to claim 8 or 9,
The driver's closed eye rate detected based on the driver's captured image is less than or equal to the first criterion, and the behavior amount of the driver's head detected based on the driver's captured image is When higher than the second determination criterion, the driver's arousal level is normal, but the driver is in a conflict state and determines that the driver's arousal level is in the process of decreasing. . In the wakefulness determination apparatus as described in any one of Claims 8-10,
The determination means is an awakening state determination device that determines that the driver's arousal level is abnormal when the driver's closed eye rate detected based on the captured image of the driver is higher than the first determination criterion. In the wakefulness state determination device according to claim 11,
The determination means includes a driver's head detected by the driver's closed eye rate based on the driver's captured image that is higher than the first determination criterion and based on the driver's captured image. A waking state determination device that determines that the driver is in a weak first abnormal state that the driver feels drowsy when the behavior amount is less than or equal to the second determination criterion. In the wakefulness state determination device according to claim 11,
The determination means includes a driver's head detected by the driver's closed eye rate based on the driver's captured image that is higher than the first determination criterion and based on the driver's captured image. An awakening state determination device that determines that the driver is in a second abnormal state in which the driver enters a light sleep when transitioning from a state where the behavior amount of the vehicle is less than or equal to the second determination criterion to a state higher than the second determination criterion . In the wakefulness state determination device according to claim 11,
The determination means includes a driver's head detected by the driver's closed eye rate based on the driver's captured image that is higher than the first determination criterion and based on the driver's captured image. An awakening state determination device that determines that the driver is in a third abnormal state in which the driver enters deep sleep when the behavior amount of the vehicle transitions from a state higher than the second determination criterion to a state equal to or lower than the second determination criterion. In the awake state determination apparatus as described in any one of Claims 1-14,
The determination criterion setting means determines whether the travel section in which the vehicle travels is a straight section or a curve section based on a road shape included in the road information or a steering angle included in the vehicle information. A wakefulness state determination device. In the wakefulness determination apparatus as described in any one of Claims 1-15,
The road information includes any one or more of road shape, road type, or road attribute, and the vehicle information includes any one or more of steering angle information, vehicle speed information, accelerator opening information, or brake information. The distance information is an awakening state determination device including any one or more of presence / absence information on the presence of another vehicle around the host vehicle or distance information between the host vehicle and the other vehicle. Detect the behavior of the driver's head from the captured image of the driver,
Based on the acquired road information, vehicle information and / or inter-vehicle distance information, a monotonous section where the driving of the vehicle is monotonous is obtained, and the driver's arousal level based on the behavior of the driver's head detected in the monotonic section Set the criteria for judging
An arousal state determination method for determining the arousal state of the driver based on the detected behavior of the head of the driver according to the set determination criterion.

Images