JP5644414B2 - Awakening level determination device, awakening level determination method, and program - Google Patents

Description

  The present invention relates to an arousal level determination device, an arousal level determination method, and a program, and more particularly, to an arousal level determination device that determines a driver's arousal level, and an arousal level determination method and a program for determining a driver's arousal level.

  Although the number of fatalities in traffic accidents has been decreasing in recent years, the number of accidents has remained at a high level. There are various causes of accidents, but driving a vehicle while the driver's arousal level is reduced is one of the causes of accidents.

  Thus, various devices have been proposed for avoiding traffic accidents by issuing a warning when the driver's arousal level is reduced (see, for example, Patent Document 1). The apparatus disclosed in Patent Literature 1 calculates the opening degree of the driver's eyes based on the driver's face image. And when the state where the calculated opening degree of the eye is equal to or less than the threshold value continues, it is determined that the driver is dozing, and an alarm is issued.

JP 2000-142164 A

  It is a statistical fact that the driver's eye opening often falls below a threshold when the driver is asleep. However, even if the opening degree of the eye is less than or equal to the threshold value, the driver may be awake to an extent that does not hinder driving. In such a case, if an alarm is issued to the driver, the driver may feel uncomfortable and distract from the driver's attention.

  Even when the driver's eyes are wide open, the driver's arousal level may be reduced. In such a case, it is necessary to determine the state of the driver based on information other than the eye opening of the driver.

  The present invention has been made under the above-described circumstances, and an object thereof is to accurately detect a decrease in a driver's arousal level.

In order to achieve the above object, a wakefulness level determination device according to a first aspect of the present invention includes:
An angle calculating means for calculating an angle defined by the right eye line of sight and the left eye line of sight based on an image obtained by photographing the face of the driver;
A standard deviation calculating means for calculating a standard deviation of the calculated angle;
When the standard deviation is greater than or equal to a threshold value, a determination unit that determines that the driver's arousal level is reduced;
Is provided.

  The angle may be a convergence angle defined by the line of sight of the right eye and the line of sight of the left eye.

  The angle may be a difference between an angle formed by a horizontal plane and the line of sight of the right eye and an angle formed by a horizontal plane and the line of sight of the left eye.

The determination means includes
When the standard deviation is less than the threshold value, it may be determined that the driver's arousal level is lowered when the calculated angle is not within a predetermined range.

The arousal level determination method according to the second aspect of the present invention includes:
A step of calculating the angle by an angle calculating means for calculating an angle defined by a gaze of the right eye and a gaze of the left eye of the driver based on an image obtained by photographing the driver's face;
A step of calculating the standard deviation by a standard deviation calculating means for calculating a standard deviation of the calculated angle;
When the standard deviation is greater than or equal to a threshold value , a step of determining by the determination unit that determines that the driver's arousal level is reduced;
including.

The program according to the third aspect of the present invention is:
On the computer,
A procedure for calculating the angle by an angle calculation means for calculating an angle defined by the right eye line of sight and the left eye line of sight based on an image obtained by photographing the driver's face;
A step of calculating the standard deviation by a standard deviation calculating means for calculating a standard deviation of the calculated angle;
When the standard deviation is greater than or equal to a threshold value , a procedure for determining by the determining means that determines that the driver's arousal level has decreased; and
Is executed.

  According to the present invention, the angle formed by the lines of sight of the eyes of the driver is sequentially calculated based on the image of the face of the driver driving the vehicle. Then, based on the calculated standard deviation of the angle formed by the line of sight, it is determined whether or not the driver's arousal level is reduced. For this reason, even if there is no correlation between the opening degree of the eye and the driver's arousal level, it is possible to accurately detect a decrease in the driver's arousal level.

It is a block diagram of the arousal level determination apparatus which concerns on 1st Embodiment. It is a figure which shows arrangement | positioning of an imaging device. It is a figure which shows the image image | photographed with the imaging device. It is a flowchart for demonstrating operation | movement of a wakefulness determination apparatus. It is a figure which shows the image of both eyes of a driver. It is a figure which shows the image of both eyes of a driver. It is a figure which shows a driver | operator's eyes | visual_axis. It is a figure which shows the time change of a convergence angle. It is a figure which shows the time change of the standard deviation regarding a convergence angle. It is a flowchart for demonstrating operation | movement of the arousal level determination apparatus which concerns on 2nd Embodiment. It is a figure which shows the image of both eyes of a driver. It is a figure which shows a driver | operator's eyes | visual_axis. It is a figure which shows a driver | operator's eyes | visual_axis. It is a figure which shows the time change of a difference angle. It is a figure which shows the time change of the standard deviation regarding a difference angle. It is a block diagram of the arousal level determination apparatus which concerns on 3rd Embodiment.

<< First Embodiment >>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a wakefulness determination device 10 according to the present embodiment. The awakening level determination device 10 is a device that determines whether or not the awakening level of a driver who drives a vehicle has decreased. As shown in FIG. 1, the arousal level determination device 10 includes a calculation device 20 and a photographing device 30.

  FIG. 2 is a diagram illustrating the arrangement of the imaging device 30. As shown in FIG. 2, the photographing device 30 is attached to a steering column or a dashboard that supports the steering 62 of the vehicle. The photographing apparatus 30 is a video camera having a frame rate of about 30. Then, an image acquired by photographing the face of the driver 50 seated on the seat 61 is converted into an electric signal and sequentially output to the arithmetic device 20.

  In FIG. 3, an image IM photographed by the photographing device 30 is shown. As can be seen by referring to the image IM, the face of the driver 50 seated on the seat 61 appears in the center of the image photographed by the photographing device 30. Here, for convenience of explanation, an XY coordinate system having the origin at the lower left corner of the image IM is defined, and the following description will be given using the XY coordinate system as appropriate.

  Returning to FIG. 1, the arithmetic device 20 is a computer having a CPU (Central Processing Unit) 21, a main storage unit 22, an auxiliary storage unit 23, a display unit 24, an input unit 25, and an interface unit 26.

  The CPU 21 reads and executes the program stored in the auxiliary storage unit 23. Specific operations of the CPU 21 will be described later.

  The main storage unit 22 has a volatile memory such as a RAM (Random Access Memory). The main storage unit 22 is used as a work area for the CPU 21.

  The auxiliary storage unit 23 includes a nonvolatile memory such as a ROM (Read Only Memory), a magnetic disk, and a semiconductor memory. The auxiliary storage unit 23 stores programs executed by the CPU 21, various parameters, and the like. In addition, information about an image output from the image capturing device 30 and information including a processing result by the CPU 21 are sequentially stored.

  The display unit 24 includes a display unit such as an LCD (Liquid Crystal Display). The display unit 24 displays the processing result of the CPU 21 and the like.

  The input unit 25 includes input keys and a pointing device such as a touch panel. The operator's instruction is input via the input unit 25 and is notified to the CPU 21 via the system bus 27.

  The interface unit 26 includes a serial interface, a parallel interface, a LAN (Local Area Network) interface, and the like. The imaging device 30 is connected to the system bus 27 via the interface unit 26.

  The flowchart of FIG. 4 corresponds to a series of processing algorithms of a program executed by the CPU 21. Hereinafter, the operation of the arousal level determination device 10 will be described with reference to FIG. A series of processing shown in the flowchart of FIG. 4 is executed when the ignition switch of the vehicle is turned on.

  First, in the first step S201, the CPU 21 instructs the driver 50 to look far away. This instruction is performed, for example, by displaying a text for prompting a remote view on the display unit 24 or outputting a sound.

  In the next step S202, the CPU 21 measures a first reference value for calculating a convergence angle defined by the line of sight of the driver 50 on the assumption that the driver 50 is viewing a distance.

  FIG. 5 is a diagram showing images of both eyes of the driver 50 shown in the image IM. The CPU 21 executes, for example, pattern matching or an estimation method based on the distribution of feature points on the image to identify the image of the right eye 51 and the image of the left eye 52 of the driver 50 included in the image IM. Then, the distance X1 in the X-axis direction from the eye head to the pupil 51a in the image of the right eye 51 is measured. This distance can be represented, for example, by the number of pixels that exist between the pixels constituting the head of the right eye 51 and the pixels constituting the pupil 51a. Similarly, the CPU 21 measures the distance X2 from the eye head to the pupil 52a in the image of the left eye 52.

  After measuring the distances X1 and X2, the CPU 21 calculates a convergence angle defined by the line of sight of both eyes of the driver 50 using the distances X1 and X2 as reference values.

  In the next step S203, the CPU 21 measures the second reference value for calculating the depression angles of the eyes 51 and 52 of the driver 50 on the assumption that the driver 50 is viewing a distance.

  For example, the CPU 21 measures a Y-axis direction distance Y1 from the eye head to the pupil 51a in the image of the right eye 51 and a Y-axis direction distance Y2 from the eye head to the pupil 52a in the image of the left eye 52. Then, after measuring the distances Y1 and Y2, the CPU 21 calculates the depression angles of the right eye 51 and the left eye 52 of the driver 50 using the distances Y1 and Y2 as reference values.

  In the next step S204, the CPU 21 measures the direction of the line of sight of both eyes of the driver 50. Specifically, the CPU 21 extracts the eye image of the driver 50 from the latest image photographed by the photographing device 30. Then, by executing processing similar to the processing in step S203, the distance YY1 in the Y-axis direction from the eye to the pupil 51a in the image of the right eye 51, and Y from the eye to the pupil 52a in the image of the left eye 52 The axial distance YY2 is measured.

  In the next step S205, the CPU 21 determines whether or not the driver 50 is looking forward outside the vehicle. As described above, the processing in step S203 is performed on the assumption that the driver 50 is viewing the front outside the vehicle. Therefore, the CPU 21 compares the distances Y1 and Y2 as the second reference values calculated in step S202 with the distances YY1 and YY2 measured in step S204.

  Then, when the difference between the distance Y1 and the distance YY1 or the difference between the distance Y2 and the distance YY2 is equal to or greater than a predetermined threshold, the CPU 21 determines that the driver 50 is not viewing the front outside the vehicle (step S205). No). On the other hand, when the difference between the distance Y1 and the distance YY1 and the difference between the distance Y2 and the distance YY2 are each less than a predetermined threshold, the CPU 21 determines that the driver 50 is viewing the front outside the vehicle (step) S205: Yes).

  If the determination in step S205 is negative, the CPU 21 returns to step S204, and repeatedly executes the processes in step S204 and step S205 until the determination in step S205 is affirmed. If the determination in step S205 is affirmed, the CPU 21 proceeds to step S206.

  In step S <b> 206, the CPU 21 calculates a convergence angle that is defined by the line of sight of the driver 50. Specifically, the CPU 21 extracts the images of the eyes 51 and 52 of the driver 50 from the latest image captured by the imaging device 30. Next, by executing processing similar to the processing in step S202, the distance XX1 in the X-axis direction from the eye to the pupil 51a in the image of the right eye 51 and the distance from the eye to the pupil 52a in the image of the left eye 52 are performed. A distance XX2 in the X-axis direction is measured. Then, the convergence angle defined by the line of sight of both eyes of the driver 50 is calculated from the distances X1, X2 and the distances XX1, XX2.

  For example, in FIG. 6, the pupils 51a and 52a of both eyes 51 and 52 when the driver 50 is looking at infinity are indicated by broken lines, and the gazing point located in front of the infinity is viewed. The pupils 51a and 52a of both eyes 51 and 52 are indicated by solid lines. When the driver 50 who looks at infinity looks at a gazing point located closer to infinity, the pupil 51a of the right eye 51 of the driver 50 is indicated by a broken line in the drawing. It moves in the + X direction from the indicated position to the position indicated by the solid line. At the same time, the pupil 52a of the left eye 52 moves in the −X direction from the position indicated by the broken line in the drawing to the position indicated by the solid line. For this reason, the distance XX1 is smaller than the distance X1, and the distance XX2 is smaller than the distance X2.

  Therefore, the CPU 21 is based on the difference between the distances X1 and X2 and the distances XX1 and XX2 on the assumption that the line of sight of the right eye 51 and the line of sight of the left eye 52 when viewing infinity are almost parallel. Thus, the convergence angle defined by the line of sight of the right eye 51 and the line of sight of the left eye 52 is calculated.

A straight line LH1 ₁ shown in FIG. 7 indicates the line of sight of the right eye 51 when the driver 50 is looking far away, and the straight line LH2 ₁ indicates the line of sight of the left eye 52. Then, a straight line LH1 _2, the driver 50 indicates the line of sight of the right eye 51 when you are looking at the fixation point, a straight line LH2 ₂ shows the line of sight of the left eye 52. Further, the straight line LH1 ₁ and the straight line LH2 ₁ are parallel.

As seen with reference to FIG. 7, the convergence angle [theta] h which is defined by a straight line LH1 ₂ and the straight line LH2 ₂ is an angle [theta] h ₁ defined by a straight line LH1 ₁ and the straight line LH1 _2, straight lines LH2 ₁ and the straight line LH2 ₂ And an angle θh ₂ defined by the above. Therefore, the CPU 21 first calculates the angle θh ₁ based on the difference between the distance X1 and the distance XX1. For example, the angle θh ₁ can be approximately calculated using the following equation, where r is the radius of the eyeball.

Next, when calculating the angle θh _{2 in} the same procedure, the CPU 21 adds the angles θh ₁ and θh ₂ to calculate the convergence angle θh.

  In the next step S207, the CPU 21 calculates a standard deviation of the convergence angle θh every predetermined time. For example, the CPU 21 calculates the standard deviation of the convergence angle calculated based on the images taken in the past one second. The frame rate of the photographing apparatus 30 according to this embodiment is about 30. Therefore, the CPU 21 calculates a standard deviation for the convergence angle θh calculated based on 30 images including the latest image. If one second has not elapsed since the processing shown in FIG. 4 was started, the CPU 21 calculates the standard deviation for the already calculated convergence angle θh.

  In the next step S208, it is determined whether or not the calculated standard deviation is greater than or equal to a threshold value. FIG. 8 is a diagram illustrating a temporal change in the convergence angle θh after the reduction in the arousal level is recognized. FIG. 9 is a diagram showing a temporal change in the standard deviation regarding the convergence angle θh. When the driver's arousal level is high, the convergence angle θh varies little and settles near 0 degrees. However, as shown in FIG. 8, the convergence angle θh deviates greatly from 0 degrees as the driver's arousal level decreases and the forward gaze becomes difficult. In this case, the standard deviation of the convergence angle θh also increases in synchronization with the fluctuation of the convergence angle θh. Therefore, when the calculated standard deviation exceeds the threshold value N1 (step S208: Yes), the CPU 21 determines that the arousal level of the driver 50 has decreased, and proceeds to step S210.

  In step S210, the CPU 21 issues an alarm. The warning can be issued by, for example, displaying a text indicating a decrease in arousal level on the display unit 24 or issuing a warning to the driver 50 using an alarm sound or voice. Moreover, CPU21 is good also as outputting the signal which shows a wakefulness fall to the exterior.

  On the other hand, when the calculated standard deviation is equal to or smaller than the threshold value in step S208 (step S208: No), the CPU 21 proceeds to step S209.

  In step S209, the CPU 21 determines whether or not the convergence angle θh calculated in step S206 is within an appropriate range. As described above, the convergence angle θh deviates greatly from 0 degrees as the driver's arousal level decreases and the forward gaze becomes difficult. Therefore, as can be seen with reference to FIG. 8, the CPU 21 determines whether or not the convergence angle θh is included in a range from a value a1 smaller than zero to a value a2 larger than zero. Then, when the convergence angle is greater than or equal to the value a1 and less than or equal to the value a2 (step S209: No), the CPU 21 returns to step S204, and thereafter, step S204 to step until the determination in step S209 is affirmed. The processing up to S209 is repeatedly executed.

  On the other hand, if the convergence angle θh is less than the value a1 or greater than the value a2 in step S209 (step S209: Yes), the CPU 21 proceeds to step S210 and issues an alarm.

  When the process of step S210 is completed, the CPU 21 returns to step S204, and thereafter repeats the processes of steps S204 to S210 until the ignition switch is turned off.

  As described above, in the present embodiment, the convergence angle θh defined by the line of sight of the eyes 51 and 52 of the driver 50 is sequentially calculated based on the image showing the face of the driver 50 driving the vehicle. Then, based on the calculated standard deviation of the convergence angle θh, it is determined whether or not the awakening level of the driver 50 is reduced (step S208). For this reason, even if there is no correlation between the opening degree of the eyes 51 and 52 of the driver 50 and the arousal level, a decrease in the arousal level of the driver 50 can be accurately detected. Thereby, it becomes possible to issue an alarm to the driver 50 accurately.

  When the standard deviation of the convergence angle θh is equal to or smaller than the threshold value, it is further determined whether or not the convergence angle θh itself is in a certain range. Then, based on this determination result, it is determined whether or not the awakening level of the driver 50 is reduced. For this reason, even if there is no correlation between the opening degree of the eyes 51 and 52 of the driver 50 and the arousal level, a decrease in the arousal level of the driver 50 can be accurately detected similarly.

When the driver's arousal level decreases, the direction of the right eye 51 and the direction of the left eye 52 change without correlation. For this reason, the driver 50 cannot watch an object such as a vehicle traveling ahead. In this case, the value of the angle θh _{1 and} the value of the angle θh ₂ in FIG. In the present embodiment, based on the standard deviation of the convergence angle [theta] h to be calculated as the sum of the angle [theta] h ₁ and the angle [theta] h _2, decrease in awareness of the driver 50 is detected. Therefore, it is possible to accurately detect a decrease in the arousal level of the driver 50 based on the line-of-sight variation caused by the decrease in the arousal level.

  In the present embodiment, the convergence angle is calculated only when the driver 50 is looking outside the vehicle (steps S205 and S206). This convergence angle changes greatly when the line of sight is moved from the outside of the vehicle to an in-vehicle instrument or when the attention is directed to a certain object from a state of looking out of the vehicle. In this case, the convergence angle changes relatively steeply. Therefore, the standard deviation of the convergence angle may be calculated only when the convergence angle changes relatively slowly, and the driver's arousal level may be evaluated based on the standard deviation.

<< Second Embodiment >>
Next, a second embodiment of the present invention will be described. In addition, about the structure which is the same as that of 1st Embodiment, or equivalent, an equivalent code | symbol is used.

  The arousal level determination apparatus 10 according to the present embodiment is the first in that it determines a decrease in the arousal level based on the depression angle of the eyes 51 and 52 of the driver 50 instead of the convergence angle defined by the line of sight of the driver 50. This is different from the arousal level determination device 10 according to the first embodiment.

  Hereinafter, the operation of the arousal level determination apparatus 10 according to the second embodiment will be described with reference to FIG. The flowchart of FIG. 10 corresponds to a series of processing algorithms of a program executed by the CPU 21.

  In step S201 to step S205, the CPU 21 executes a process similar to the process in the first embodiment described above.

  In step S306, the CPU 21 calculates the depression angles of the eyes 51 and 52 of the driver 50. Specifically, the CPU 21 extracts the images of the eyes 51 and 52 of the driver 50 from the latest image captured by the imaging device 30. Then, by executing processing similar to the processing in step S203, the distance YY1 in the Y-axis direction from the eye to the pupil 51a in the image of the right eye 51, and Y from the eye to the pupil 52a in the image of the left eye 52 The axial distance YY2 is measured. Then, the depression angles of the eyes 51 and 52 of the driver 50 are calculated from the distances Y1 and Y2 and the distances YY1 and YY2.

  For example, FIG. 11 shows the pupils 51a and 52a of both eyes 51 and 52 as broken lines when the driver 50 gazes at infinity and the line of sight is almost horizontal, so that the driver 50 can visually recognize the instrument, for example. The pupils 51a and 52a of both eyes 51 and 52 when the line of sight is moved downward are indicated by solid lines. When the driver 50 moves the line of sight downward, as shown in FIG. 11, the pupils 51a and 52a of both eyes 51 and 52 of the driver 50 are indicated by solid lines from the positions indicated by broken lines in the drawing. Move in the -Y direction to the position. For this reason, the distance YY1 is smaller than the distance Y1, and the distance YY2 is smaller than the distance Y2.

  Therefore, the CPU 21 assumes that the line of sight is almost horizontal when viewing at infinity, and sets the depression angles of the right eye 51 and the right eye 52 based on the difference between the distances Y1, Y2 and the distances YY1, YY2, respectively. calculate.

Straight LV1 ₁ shown in FIG. 12 shows the line of sight of the right eye 51 when the driver 50 is visually far. Then, a straight line LV1 _2, the driver 50 indicates the line of sight of the right eye 51 of while visually downward. The CPU 21 calculates the depression angle θv ₁ based on the difference between the distance Y1 and the distance YY1. For example, the depression angle θv ₁ can be approximately calculated using the following equation, where r is the radius of the eyeball.

A straight line LV2 ₁ shown in FIG. 13 shows the line of sight of the left eye 52 when the driver 50 is visually far. Then, a straight line LV2 _2, the driver 50 indicates the line of sight of the left eye 52 when you are looking at the lower. The CPU 21 similarly calculates the depression angle θv ₂ based on the difference between the distance Y2 and the distance YY2.

In the next step S307, the CPU 21 calculates the standard deviation of the difference angle θv (= θv ₁ −θv ₂ ) between the depression angle θv ₁ of the right eye 51 and the depression angle θv _{2 of the} left eye 52 at predetermined time intervals. . For example, the CPU 21 calculates the standard deviation of the difference angle θv calculated based on the images taken in the past one second. The frame rate of the photographing apparatus 30 according to this embodiment is about 30. Therefore, the CPU 21 calculates a standard deviation for the difference angle θv calculated based on 30 images including the latest image. If one second has not elapsed since the processing shown in FIG. 4 was started, the CPU 21 calculates a standard deviation for the already calculated difference angle θv.

  In the next step S308, it is determined whether or not the calculated standard deviation is greater than or equal to a threshold value. FIG. 14 is a diagram illustrating a temporal change in the difference angle θv after the decrease in the arousal level is recognized. FIG. 15 is a diagram showing a temporal change in the standard deviation regarding the difference angle θv. When the driver's 50 awakening level is high, the depression angle of the right eye 51 and the depression angle of the left eye 52 generally coincide substantially. For this reason, the difference angle θv, which is the difference between the depression angles of both eyes, varies little and settles in the vicinity of 0 degrees when the driver's arousal level is high. However, as shown in FIG. 14, the difference angle θv deviates greatly from 0 degrees as the driver's arousal level decreases and the forward gaze becomes difficult. In this case, the standard deviation of the difference angle θv also increases in synchronization with the fluctuation of the difference angle θv. Therefore, when the calculated standard deviation exceeds the threshold value N2 (step S308: Yes), the CPU 21 determines that the arousal level of the driver 50 has decreased, and proceeds to step S210.

  On the other hand, when the calculated standard deviation is equal to or smaller than the threshold value in step S308 (step S308: No), the CPU 21 proceeds to step S309.

  In step S309, the CPU 21 determines whether or not the difference angle θv calculated in step S307 is within an appropriate range. As described above, as the driver's arousal level decreases and the forward gaze becomes difficult, the difference angle θv deviates greatly from 0 degrees. Therefore, as can be seen with reference to FIG. 14, the CPU 21 determines whether or not the difference angle θv is included in a range from a value b1 smaller than zero to a value b2 larger than zero. Then, when the difference angle θv is greater than or equal to the value b1 and less than or equal to the value b2 (step S309: No), the CPU 21 returns to step S204, and thereafter proceeds to steps S204 to S204 until the determination in step S309 is denied. The processing up to step S309 is repeatedly executed.

  On the other hand, when the difference angle θv is less than the value a1 or greater than the value a2 in step S309 (step S309: Yes), the CPU 21 proceeds to step S210 and issues an alarm.

  When the process of step S210 is completed, the CPU 21 returns to step S204 and repeatedly executes the processes after step S204 until the ignition switch is turned off.

As described above, in the present embodiment, the difference angle between the depression angle θv ₁ of the right eye 51 of the driver 50 and the depression angle θv _{2 of the} left eye 52 based on the image of the face of the driver 50 driving the vehicle. θv is sequentially calculated. Then, based on the calculated standard deviation of the difference angle θv, it is determined whether or not the awakening level of the driver 50 is reduced (step S308). For this reason, even if there is no correlation between the opening degree of the eyes 51 and 52 of the driver 50 and the arousal level, a decrease in the arousal level of the driver 50 can be accurately detected. Thereby, it becomes possible to issue an alarm to the driver 50 accurately.

  If the standard deviation of the difference angle θv is less than or equal to the threshold value, it is further determined whether or not the difference angle θv itself is within a certain range. Then, based on this determination result, it is determined whether or not the awakening level of the driver 50 is reduced. For this reason, even if there is no correlation between the opening degree of the eyes 51 and 52 of the driver 50 and the arousal level, a decrease in the arousal level of the driver 50 can be accurately detected similarly.

When the awakening level of the driver 50 decreases, the direction of the right eye 51 and the direction of the left eye 52 change without correlation. For this reason, the driver 50 cannot watch an object such as a vehicle traveling ahead. For example, in this case, the depression angle θv ₁ in FIG. 12 and the depression angle θv ₂ in FIG. In the present embodiment, a decrease in the arousal level of the driver 50 is detected based on the standard deviation of the difference angle θv calculated as the difference between the depression angle θv ₁ and the depression angle θv ₂ . Therefore, it is possible to accurately detect the decrease in the arousal level of the driver 50 based on the line-of-sight variation caused by the decrease in the arousal level.

  In the present embodiment, the arousal level of the driver 50 is evaluated based on the difference (difference angle θv) between the depression angle of the right eye 51 and the depression angle of the left eye 52. For this reason, it is not always necessary to calculate the depression angle of the right eye 51 and the depression angle of the left eye 52 based on the line of sight when viewing at infinity. For example, the difference between the depression angle of the right eye 51 and the depression angle of the left eye 52 may be calculated based on the line of sight when the instrument is viewed.

<< Third Embodiment >>
Next, a third embodiment of the present invention will be described. In addition, about the structure which is the same as that of the said embodiment, or equivalent, an equivalent code | symbol is used.

  The arousal level determination device 10A according to the present embodiment is different from the arousal level determination device 10 according to the above-described embodiment in that the arithmetic device 20 is configured by hardware.

  FIG. 16 is a block diagram of the arousal level determination device 10A configured by hardware. As shown in FIG. 10, the arithmetic device 20 constituting the arousal level determination device 10A includes a storage unit 20a, a reference value measurement unit 20b, a visual direction identification unit 20c, a convergence angle calculation unit 20d, a standard deviation calculation unit 20e, and a determination. 20f and alarm issuing unit 20g.

  The storage unit 20a sequentially stores information about the image output from the imaging device 30, and information including the processing results of the units 20b to 20f.

  The reference value measurement unit 20b is based on the assumption that the driver 50 is looking far away, and a reference value for calculating the convergence angle defined by the line of sight of the driver 50 and the depression angles of both eyes 51 and 52 of the driver 50. X1, X2, Y1, and Y2 are measured.

  The visual direction specifying unit 20 c measures the direction of both eyes 51 and 52 of the driver 50. Specifically, the CPU 21 extracts the images of the eyes 51 and 52 of the driver 50 from the latest image captured by the imaging device 30. Then, the distance YY1 in the Y-axis direction from the eye head to the pupil 51a in the image of the right eye 51 and the distance YY2 in the Y-axis direction from the eye head to the pupil 52a in the image of the left eye 52 are measured.

  Then, the visual direction identification unit 20c compares the distances Y1 and Y2 with the measured distances YY1 and YY2, and the difference between the distance Y1 and the distance YY1 or the difference between the distance Y2 and the distance YY2 is less than a predetermined threshold value. In this case, it is determined that the driver 50 is viewing the front outside the vehicle.

  The convergence angle calculation unit 20d calculates a convergence angle defined by the line of sight of the driver 50 when it is determined that the driver 50 is viewing the front outside the vehicle. Specifically, the CPU 21 extracts the eye image of the driver 50 from the latest image photographed by the photographing device 30. Next, a distance XX1 in the X-axis direction from the eye head to the pupil 51a in the image of the right eye 51 and a distance XX2 in the X-axis direction from the eye head to the pupil 52a in the image of the left eye 52 are measured. Then, a convergence angle θh defined by the line of sight of both eyes of the driver 50 is calculated from the distances X1, X2 and the distances XX1, XX2.

  The standard deviation calculator 20e calculates a standard deviation of the convergence angle θh. For example, the standard deviation calculation unit 20e calculates a standard deviation of the convergence angle θh calculated based on images taken in the past one second. The frame rate of the photographing apparatus 30 according to this embodiment is about 30. Therefore, the CPU 21 calculates a standard deviation for the convergence angle θh calculated based on 30 images including the latest image.

  When the calculated standard deviation exceeds the threshold value N1, the determination unit 20f determines that the arousal level of the driver 50 has decreased, and notifies the warning issuing unit 20g of the determination result. In addition, even when the calculated standard deviation does not exceed the threshold value N1, the determination unit 20f determines that the arousal level of the driver 50 is reduced when the convergence angle θh is not in the proper range. Then, the determination result is notified to the alarm issuing unit 20g.

  The warning issuing unit 20g issues a warning to the driver 50 when the determination result is notified from the determining unit 20f. The warning can be issued by, for example, displaying a text indicating a decrease in arousal level on a liquid crystal monitor (not shown) or issuing a warning to the driver 50 using an alarm sound or voice. Moreover, CPU21 is good also as outputting the signal which shows a wakefulness fall to the exterior.

  As described above, in the present embodiment, it is determined based on the calculated standard deviation of the convergence angle θh whether or not the awakening level of the driver 50 driving the vehicle is in a reduced state. For this reason, even if there is no correlation between the opening degree of the eyes 51 and 52 of the driver 50 and the arousal level, a decrease in the arousal level of the driver 50 can be accurately detected. Thereby, it becomes possible to issue an alarm to the driver 50 accurately.

  As mentioned above, although embodiment of this invention was described, this invention is not limited by the said embodiment. For example, in the above embodiment, the convergence angle and the depression angle are calculated based on the distance between the eyes and the pupil. Not limited to this, the convergence angle and the depression angle may be calculated based on the distance between the lower eyelid and the pupil. In addition, as a method for calculating the convergence angle and the depression angle, various known techniques may be used.

  Moreover, in the said embodiment, when CPU21 runs the program memorize | stored in the auxiliary memory part, a series of processes shown by FIG. 4 or FIG. 10 are performed. However, the present invention is not limited to this, and part or all of the arithmetic device 20 may be configured by hardware.

  The programs stored in the auxiliary storage unit 23 of the arithmetic unit 20 are computers such as a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), and an MO (Magneto-Optical disk). A device that executes the above-described processing may be configured by storing and distributing the program in a readable recording medium and installing the program in a computer.

  Further, the program may be stored in a disk device or the like included in a predetermined server device on a communication network such as the Internet, and may be downloaded onto a computer by being superimposed on a carrier wave, for example.

  It should be noted that the present invention can be variously modified and modified without departing from the broad spirit and scope of the present invention. Further, the above-described embodiment is for explaining the present invention, and does not limit the scope of the present invention.

  The arousal level determination device, the arousal level determination method, and the program according to the present invention are suitable for detecting a decrease in a driver's arousal level.

10, 10A Arousal level determination device 20 Arithmetic device 20a Storage unit 20b Reference value measurement unit 20c Viewing direction identification unit 20d Convergence angle calculation unit 20e Standard deviation calculation unit 20f Judgment unit 20g Alarm issuing unit 21 CPU
DESCRIPTION OF SYMBOLS 22 Main memory part 23 Auxiliary memory part 24 Display part 25 Input part 26 Interface part 27 System bus 30 Image pick-up device 50 Driver 51 Right eye 52 Left eye 51a, 52a Eye 61 Sheet 62 Steering IM image

Claims (6)

An angle calculating means for calculating an angle defined by the right eye line of sight and the left eye line of sight based on an image obtained by photographing the face of the driver;
A standard deviation calculating means for calculating a standard deviation of the calculated angle;
When the standard deviation is greater than or equal to a threshold value, a determination unit that determines that the driver's arousal level is reduced;
A wakefulness determination device comprising:   The arousal level determination apparatus according to claim 1, wherein the angle is a convergence angle defined by the line of sight of the right eye and the line of sight of the left eye.   The wakefulness determination device according to claim 1, wherein the angle is a difference between an angle formed by a horizontal plane and the line of sight of the right eye and an angle formed by a horizontal plane and the line of sight of the left eye. The determination means includes
The arousal level according to any one of claims 1 to 3, wherein when the standard deviation is less than a threshold value, it is determined that the awakening level of the driver is lowered when the calculated angle is not within a predetermined range. Judgment device. A step of calculating the angle by an angle calculating means for calculating an angle defined by a gaze of the right eye and a gaze of the left eye of the driver based on an image obtained by photographing the driver's face;
A step of calculating the standard deviation by a standard deviation calculating means for calculating a standard deviation of the calculated angle;
When the standard deviation is greater than or equal to a threshold value , a step of determining by the determination unit that determines that the driver's arousal level is reduced;
Awakening degree determination method including: On the computer,
A procedure for calculating the angle by an angle calculation means for calculating an angle defined by the right eye line of sight and the left eye line of sight based on an image obtained by photographing the driver's face;
A step of calculating the standard deviation by a standard deviation calculating means for calculating a standard deviation of the calculated angle;
When the standard deviation is greater than or equal to a threshold value , a procedure for determining by the determining means that determines that the driver's arousal level has decreased; and
A program for running

Images