JP2010142410A - Awakening degree determining apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an awakening degree determining apparatus appropriately determining the degree of awakening of a driver based on the result of detection of the eyeball movement of the driver driving a vehicle. <P>SOLUTION: The awakening degree determining apparatus for determining the degree of awakening of the driver driving the vehicle includes an eyeball movement detecting means 20, a difference storing means 12, and an awakening degree determining means 16. The eyeball movement detecting means 20 detects the eyeball movement of the driver. The difference storing means 12 measures the quantity of movement of the eyes of the driver every prescribed time Ta based on the result of detection of the eyeball movement, and stores the difference ΔX<SB>i</SB>(ΔX<SB>i</SB>=X<SB>i+1</SB>-X<SB>i</SB>(i is 1 or a natural number larger than 1)) from the quantity X<SB>i</SB>measured last time sequentially. The awakening degree determining means 16 determines that the degree of awakening of the driver declines if a standard deviation SD of the difference ΔX<SB>i</SB>decreases in the elapse of time. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an arousal level determination device that determines the level of arousal of a driver who drives a vehicle.

Conventionally, as a wakefulness determination device for determining the wakefulness of a user of a head mounted display (HMD), the eye movement of the user is detected, the amount of movement of the user's eyes is measured, and the standard deviation of the amount of movement is One that determines the degree of arousal from time dependence is known (see, for example, Patent Document 1). When the standard deviation of the movement amount increases with the passage of time, this device determines that the eyes are not focused on the target and the arousal level is reduced, and the HMD of the user is reduced. Display a message to suspend usage.
JP-A-8-19520

  However, the wakefulness level determination device described in Patent Document 1 determines that the wakefulness level has decreased when the standard deviation of the movement amount increases with the passage of time. The arousal level cannot be properly determined.

  Therefore, in order to appropriately determine the arousal level of the driver who drives the vehicle, when the standard deviation of the movement amount decreases with the passage of time, it is determined that the eye movement is slow and the arousal level is decreased. It is possible. However, even if the eye movement is active, if the eye movement is monotonous, it is considered that the degree of arousal is reduced, so the determination accuracy is not sufficient.

  The present invention has been made in view of the above, and provides an arousal level determination device that can appropriately determine a driver's arousal level based on a detection result of an eye movement of a driver driving a vehicle. Objective.

In order to achieve the above object, the present invention provides a wakefulness determination device that determines the wakefulness of a driver who drives a vehicle.
Eye movement detection means for detecting the driver's eye movement;
Difference storage means for measuring the amount of movement of the driver's eyes every predetermined time based on the detection result of the eye movement detection means and sequentially storing the difference from the previous movement amount;
Awakening level determination means for determining that the driver's awakening level is decreasing when the standard deviation of the difference decreases with time;

  ADVANTAGE OF THE INVENTION According to this invention, the arousal level determination apparatus which can determine a driver | operator's arousal level appropriately based on the detection result of the driver | operator's eye movement of a vehicle is obtained.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a functional block diagram showing an embodiment of the arousal level determination apparatus of the present invention. The arousal level determination device is configured around an electronic control unit (hereinafter referred to as “wake level determination ECU”) 10 that determines the level of arousal of the driver who drives the vehicle.

  The wakefulness determination ECU 10 is configured by a microcomputer, and measures, for example, a CPU, a ROM that stores a control program, a readable / writable RAM that stores calculation results, a timer that measures time, and the number of processes such as calculations. It has a counter, an input interface, and an output interface. The eye movement detection means 20, the notification means 30, and the information input device 40 are connected to the arousal level determination ECU 10.

  The eye movement detection means 20 is a means for detecting the driver's eye movement. The eye movement detection means 20 may have a known configuration, for example, an infrared light source that illuminates the driver's eyeball, an image sensor such as a CCD camera or a CMOS camera that images the driver's eyeball, and an image is captured by the image sensor. And a microcomputer that performs image processing on the processed image (hereinafter referred to as “face image”).

  The eye movement detection means 20 performs image processing on the face image to extract areas of the cornea (black eyes) and sclera (white eyes), and detects the eye movement of the driver based on the shape change of each area. For example, first, the face image is coordinate-transformed from a coordinate system based on the image sensor to a coordinate system based on the driver's eye. Next, the center point of the image is moved or the image is enlarged or reduced to create a corrected image without distortion. Finally, the cornea (black eye) and sclera (white eye) regions are extracted from the corrected image, and the driver's eye movement is detected based on the shape change of each region.

  The eye movement detection means 20 of the present embodiment detects the eye movement by performing image processing on the face image, but the method or type of the eye movement detection means 20 is not limited. For example, the detection is performed using the scleral reflection method in which the difference in reflectance between the cornea (black eye) and the sclera (white eye) is detected, or the movement of a virtual image of infrared light on the cornea (black eye). A corneal reflection method may be used, or an EOG method in which detection is performed using a potential difference between the cornea and the retina.

  The eye movement detection means 20 outputs the detection result of the driver's eye movement to the arousal level determination ECU 10.

  The notification means 30 is a means for notifying the driver of a decrease in the arousal level. The notification unit 30 includes, for example, an image display device 32 such as a dedicated indicator 32a and a liquid crystal display 32b, and a speaker 34. The notification means 30 is controlled by the arousal level determination ECU 10 and notifies the driver by lighting or blinking display by the dedicated indicator 32a, character or image display by the liquid crystal display 32b, voice guidance by the speaker 34, and the like.

  The information input device 40 is a device that inputs information indicating a state around the vehicle such as a traffic jam, information about a driver, and vehicle speed information. As shown in FIG. 1, for example, the information input device 40 includes a navigation device 42, a camera outside the vehicle 44, a wireless communication device 46, and a vehicle speed sensor 48.

  The navigation device 42 recognizes the position of the host vehicle on the map based on the information received from the GPS satellite by the GPS receiver and the map information in the map database, and displays a map image indicating the position of the host vehicle on the map. It is a device for displaying images.

  The map database is a recording medium such as an HDD, CD, or DVD-ROM stored in association with latitude and longitude. The map information in the map database includes information on general roads and highways on which vehicles run and information on buildings such as stores and buildings, especially information on the location of traffic lights and road signs. Yes.

  The vehicle exterior camera 44 is a camera that captures an image around the vehicle. The vehicle outside camera 44 performs image processing on a peripheral image obtained by imaging the periphery of the vehicle, and detects the presence or absence of traffic lights, road signs, and pedestrians. For example, the presence of traffic lights, road signs, and pedestrians is detected by matching processing with standard image data registered in a memory such as a ROM. Further, the vehicle exterior camera 44 creates a saliency map by performing image processing on a front image obtained by capturing the front of the vehicle. The saliency map is a probability distribution of positions that a driver sees naturally due to the visual characteristics of an object.

  The wireless communication device 46 is a device that performs wireless communication. For example, the wireless communication device 46 receives traffic information such as traffic jam information transmitted from the VICS center. Further, the wireless communication device 46 receives an ID code transmitted from a portable device (for example, smart key) carried by the driver.

  The vehicle speed sensor 48 is a sensor that detects the vehicle speed V. For the vehicle speed sensor 48, for example, a Hall element that outputs a pulse signal corresponding to the rotational speed of each wheel is used.

  The information input device 40 outputs various types of information to the arousal level determination ECU 10 at predetermined time intervals. Various information includes the presence or absence of traffic on the road, whether the road is in the city or the suburbs, the presence of traffic lights and road signs around the vehicle, the presence or absence of pedestrians around the vehicle, the saliency map, the ID code of the mobile device, The vehicle speed V is included.

  FIG. 2 is a block diagram showing the main functions of the arousal level determination ECU 10. As shown in FIG. 2, the arousal level determination ECU 10 includes a difference storage unit (difference storage unit) 12, a standard deviation calculation unit (standard deviation calculation unit) 14, a wakefulness level determination unit (wake level determination unit) 16, and notification control. Unit (notification control means) 18 is provided.

  The arousal level determination ECU 10 stores the program corresponding to each unit 12-18 in a memory such as a ROM, and causes the CPU to process these programs to realize the function of each unit 12-18. Details of each part 12-18 will be described later.

The difference storage unit 12 measures the movement amount of the driver's eyes every predetermined time Ta based on the detection result of the eye movement detection means 20, and the difference ΔX _i (ΔX _i = X _{i + 1)} from the previous movement amount X _i. -X _i (i is a natural number of 1 or more)) is sequentially stored. The difference storage unit 12 may delete the stored data of the difference ΔX _i after the set time has elapsed.

  Here, the movement amount of the driver's eyes is measured by sequentially processing a plurality of face images taken at predetermined time intervals Ta. For example, the center position of the cornea (black eye) is sequentially detected by image processing, and the two-dimensional movement amount in the horizontal direction and the vertical direction is measured based on the change in the center position. Note that the contour shape of the cornea (black eye) may be sequentially detected by image processing, and the three-dimensional movement amount may be measured based on the change in the contour shape.

The standard deviation calculation unit 14 repeatedly calculates the standard deviation SD of the difference ΔX _i (i = 1, 2,... N) stored in the difference storage unit 12 during a predetermined time Tb. The standard deviation SD is calculated according to the well-known formula 1.

図３は、横軸を時間軸として標準偏差ＳＤの推移を示す図である。より詳しくは、図３は、横軸に時間軸に相当する繰り返し回数ｊ（ｊは１以上の自然数）を配し、縦軸に各回の標準偏差ＳＤ_ｊと初回の標準偏差ＳＤ_１との差分（ＳＤ_ｊ−ＳＤ_１）を配し、標準偏差ＳＤの推移を示す図である。

FIG. 3 is a diagram showing the transition of the standard deviation SD with the horizontal axis as the time axis. More specifically, in FIG. 3, the horizontal axis indicates the number of repetitions j corresponding to the time axis (j is a natural number of 1 or more), and the vertical axis indicates the difference between the standard deviation SD _{j of} each time and the initial standard deviation SD _1. arranged _(SD j -SD _1), is a graph showing transition of standard deviation SD.

FIG. 3 shows a measurement of the right eye movement of a right-handed driver every 0.01 seconds and sequentially stores the difference ΔX _i from the previous movement, and the standard deviation SD of the difference ΔX _i for 20 minutes is stored. This is a result of repeated calculation four times. FIG. 3 shows data of nine drivers who have driven vehicles of the same travel route and the same vehicle type.

  As is clear from FIG. 3, the standard deviation SD tended to decrease with time for all the drivers. This is presumably because the driver's eyes became dull with the result of time, and the driver's eye movement became dull, or the driver's eye movement became monotonous.

  The arousal level determination unit 16 determines that the driver's arousal level is decreasing when the standard deviation SD calculated by the standard deviation calculation unit 14 decreases with time.

  For example, the arousal level determination unit 16 determines that the driver's arousal level is decreased when the standard deviation SD is decreased by a predetermined width or more with time. In other words, it is determined that the driver's arousal level has decreased when the standard deviation SD has decreased over a predetermined time with the passage of time and the absolute value of the difference between the first and the last has exceeded a threshold value. To do.

For example, when the standard deviation SD decreases by a predetermined width SDa or more after a predetermined time Tc has elapsed from the start of determination (for example, (SD ₁ −SD _j ) ≧ SDa), it is determined that the driver's arousal level has decreased. Further, when the standard deviation SD decreases by a predetermined width SDb or more during the predetermined time Td (for example, (SD _j −SD _{j + 1} ) ≧ SDb), it may be determined that the driver's arousal level is decreasing. The predetermined widths SDa and SDb are stored in advance in a memory such as a ROM.

  Here, the predetermined widths SDa and SDb may be made variable in accordance with information indicating a state around the vehicle from the information input device 40. Information indicating the state of the surroundings of the vehicle includes the presence or absence of traffic congestion on the road, whether the road is urban or suburban, presence of traffic lights or road signs around the vehicle, presence of pedestrians around the vehicle, and saliency maps It is done. Thereby, the arousal level can be determined in consideration of the state around the vehicle, and the arousal level can be accurately determined.

  Further, the predetermined widths SDa and SDb may be variable in accordance with information related to the driver. As information about the driver, identification information (ID code) of the driver and the age of the driver are used. Information regarding these drivers is stored in advance in a memory such as a ROM in association with each other, and is read out based on a collation result with the ID code of the portable device output from the information input device 40. Thereby, the arousal level can be determined in consideration of individual differences among the drivers, and the arousal level can be accurately determined.

  In addition, the arousal level determination unit 16 may determine that the driver's arousal level has decreased when the standard deviation SD has decreased by a predetermined percentage or more with the passage of time. When the predetermined percentage is used, the arousal level can be determined in consideration of individual differences among the drivers as compared with the case where the predetermined widths SDa and SDb are used, and the arousal level can be accurately determined. . Note that the predetermined percentage may be made variable in accordance with information indicating the state around the vehicle and information regarding the driver, as with the predetermined widths SDa and SDb.

  When it is determined by the arousal level determination unit 16 that the driver's arousal level is decreasing, the notification control unit 18 controls the notification unit 30 to perform notification control for the driver. Thereby, the driver's attention can be alerted or the driver can be rested.

  FIG. 4 is a diagram illustrating an example of processing executed by the arousal level determination ECU 10. The awakening degree determination ECU 10 performs the processing after S100 in FIG. 4 every predetermined time Tb after the travel drive source such as the engine or the electric motor is started until the travel drive source is stopped.

  Note that the arousal level determination ECU 10 may prevent erroneous determination by performing the processing after S100 in FIG. 4 under a specific situation. For example, each process (especially S100) may be temporarily stopped when the vehicle speed V is less than a threshold value Va (for example, 60 km / h), and each process may be resumed when the vehicle speed V becomes equal to or greater than the threshold value Va. Thereby, it is possible to prevent erroneous determination due to an intersection or a curve run. The vehicle speed V is calculated based on a pulse signal from the vehicle speed sensor 48, and the threshold value Va of the vehicle speed V is stored in advance in a memory such as a ROM.

In S100 of FIG. 4, first, the difference storage unit 12, based on the detection result of the eye movement measuring the amount of movement of the driver's eye every predetermined time Ta, the difference [Delta] X _i of the moving amount X _i of the previous ( ΔX _i = X _{i + 1} −X _i (i is a natural number of 1 or more)) are sequentially stored.

Next, the standard deviation calculation unit 14 calculates the standard deviation SD during the predetermined time Tb of the difference ΔX _i (i = 1, 2,... N) (S102).

  Next, the arousal level determination unit 16 determines whether or not the standard deviation SD has decreased by a predetermined width SDa or more after a predetermined time Tc has elapsed since the determination process was started (for example, when the engine was started) (S104). If it has not decreased by the predetermined width SDa or more (S104, NO), it is determined that the driver's arousal level is sufficient (S106), and the current process is terminated.

  On the other hand, when it falls below the predetermined width SDa (S104, YES), it is determined that the driver's arousal level is lowered (S108), and then the notification control unit 18 controls the notification means 30 to drive the vehicle. Information control is performed on the person (S110), and the current process is terminated.

As described above, the arousal level determination device according to the present embodiment measures the movement amount X _{i + 1} of the driver's eyes every predetermined time Ta, sequentially stores the difference ΔX _i from the previous movement amount X _i, and the difference ΔX _i Since the arousal level is determined from the time dependence of the standard deviation SD during the predetermined time Tb, the driver's arousal level can be appropriately determined.

If the degree of arousal is determined from the time dependence of the standard deviation of the driver's eye movement amount X _i , the case where the eye movement is monotonous cannot be identified. Therefore, the determination accuracy of the arousal level is not sufficient.

On the other hand, in this embodiment, since the degree of arousal is determined from the time dependence of the standard deviation SD of the difference ΔX _i of the movement amount X _i , it is possible to identify a case where the eye movement is monotonous. Therefore, the arousal level can be determined with high accuracy.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, in the above-described embodiment, the arousal level determination unit 16 determines that the driver's arousal level is decreased when the standard deviation SD calculated by the standard deviation calculation unit 14 decreases with time. However, when the standard deviation SD is equal to or less than the threshold value regardless of the passage of time, it may be determined that the driver's arousal level is reduced. Thereby, even if it is a case where a driver | operator's arousal level is not enough from the time of boarding, a driver | operator's arousal level can be determined appropriately.

In the above-described embodiment, the standard deviation calculation unit 14 repeatedly calculates the standard deviation SD during the predetermined time Tb of the difference ΔX _i , but the present invention is not limited to this. For example, the standard deviation calculation unit 14 may calculate the standard deviation SD for a predetermined time Tb of the difference ΔX _i every predetermined time Te (Te> Tb or Te <Tb). Here, the predetermined time Tb and the predetermined time Te may be variable. For example, after the standard deviation calculation unit 14 calculates the standard deviation SD of the difference ΔX _i for 10 minutes, the standard of the difference ΔX _i for 20 minutes. The deviation SD may be calculated.

  Further, in FIG. 3 described above, the movement amount of the driver's right eye is measured every predetermined time Ta, but the movement amount of the driver's left eye may be measured every predetermined time Ta.

  Further, in FIG. 4 described above, the arousal level determination ECU 10 starts the processing after S100 after the travel drive source is started, but the present invention is not limited to this. For example, the arousal level determination ECU 10 may start the processing after S100 after the travel drive source is started and the vehicle speed V becomes equal to or higher than the threshold value Vb.

It is a functional block diagram which shows one Example of the arousal level determination apparatus of this invention. It is a block diagram which shows the main functions of wakefulness determination ECU10. It is a figure which shows transition of the standard deviation SD by making a horizontal axis into a time axis. It is a flowchart which shows an example of the process which awakening degree determination ECU10 performs.

Explanation of symbols

10 Arousal level judgment ECU
12 Difference storage unit (difference storage means)
14 Standard deviation calculation part (standard deviation calculation means)
16 Arousal level determination unit (wake level determination means)
18 Notification Control Unit (Notification Control Unit)
20 eye movement detection means 30 notification means 40 information input device 42 navigation device 44 vehicle camera 46 wireless communication device 48 vehicle speed sensor

Claims (1)

In the wakefulness determination device for determining the wakefulness of the driver driving the vehicle,
Eye movement detection means for detecting the driver's eye movement;
Difference storage means for measuring the amount of movement of the driver's eyes every predetermined time based on the detection result of the eye movement detection means and sequentially storing the difference from the previous movement amount;
An arousal level determination device comprising: an awakening level determination unit that determines that the driver's arousal level is decreasing when the standard deviation of the difference decreases with time.

Images