JP2003025866A - Driver's awakening degree detecting apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a driver's awakening-degree detecting apparatus capable of continuing awakening degree detection without losing sight of a region to be detected of the driver. SOLUTION: As a driving attitude of the driver sitting on a front seat 3 changes, the driver's awakening-degree detecting apparatus of the present invention expands a search area A2 in the direction of the change of the driver's attitude according to the change of the driver's posture, and facilitates tracking for detecting a pupil. Even if the face direction of the driver changes when steering, operating winkers, operating an in-vehicle apparatus, operating a seating posture, etc., the awakening degree detection may be continued without losing sight of an eye of the driver.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a driver alertness detection device for detecting the alertness of a driver.

[0002]

2. Description of the Related Art In automatic (vehicle) traveling, the occurrence of drowsy driving is a problem.

As a countermeasure against this, there has been proposed a system which detects the awakening degree of a driver who is driving and issues a warning to the driver when the awakening degree is lowered, that is, when the driver determines to be sleepy.

An apparatus for estimating a driver's arousal level used in such a system, a so-called awakening level detecting apparatus, uses an imaging camera for capturing an image of the driver's face. A detection target part that is likely to appear, such as the driver's eye, is searched and stored in the search area, and the blinking condition from the image of the eye in this search area, specifically, the blinking frequency of the eye (the number of blinks per unit time) or the closed eye A method of detecting the time (blinking time) or the like and estimating the awakening degree of the driver who is currently driving is adopted.

In order to continue such blink detection, it is required to track the detected target part of the searched face image.

Therefore, conventionally, the position of the eye is tracked by capturing the feature of the eye while keeping the detection target site, here the eye in the search area.

[0007]

By the way, a driver operates a steering wheel, a winker, an air conditioner or a navigation device in order to drive the vehicle in a desired direction or to spend comfortably in the vehicle interior. Operate in-vehicle devices such as music and music devices.

At this time, for example, when the steering wheel or blinker is operated suddenly, or when the operation unit of the in-vehicle device is far from the driver, the driver's face is suddenly changed. The direction changes greatly, and the position of the eyes changes accordingly.

[0009] However, since the search area for detecting blinks is usually formed to have a size that can accommodate the periphery of the pupil, if the face direction suddenly changes in this way, the captured eye will fall out of the eye search area. , I can't capture the features of my eyes and my eyes cannot track them.

When this happens, the eyes to be detected are lost, and the driver's arousal level cannot be estimated. This also occurs when the seating posture of the driver is changed midway.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a driver's arousal level detection device capable of continuing the arousal level detection without losing sight of the driver's detection target portion. .

[0012]

In order to achieve the above object, the invention of claim 1 is provided with a detecting means for detecting a change in the driving posture of the driver, and when the change in the driving posture of the driver is detected, a tracking means is provided. By expanding the search area in the direction in which the driver's posture changes, it is easier to track the detection target part, even if the driver's face direction is the handle operation,
Even if the blinker operation, the operation of the in-vehicle device, the operation of the sitting posture, etc. change, the driver's detection target part is not lost, or even if it is lost, it is immediately found and the arousal level detection can be continued.

The invention of claim 2 is applied to a method of detecting the position of the pupil of the driver in the search area so that the effect is exhibited particularly in the blink detection which is easy to lose sight.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described below based on the first embodiment shown in FIGS.

FIG. 1 shows a schematic structure of a driver's arousal level detection device mounted on an automobile (vehicle). In FIG. 1, reference numeral 1 denotes an instrument panel 2a, which is provided along the width direction of the vehicle body. 2b is an electronic operation unit of an in-vehicle device assembled in the center of the instrument panel 1 in the vehicle width direction,
For example, it is an operation panel of an air conditioner or an operation panel of a navigation device. Also front seat 3
In the instrument panel portion in front of, an imaging camera for imaging the face of the driver seated on the front seat 3,
For example, a CCD camera 4 is attached. It should be noted that the front seat 3 is of an electrically adjustable type that can electrically adjust the posture of each part.

The CCD camera 4 is connected to a control section 5 composed of a microcomputer together with the operation panel sections 2a and 2b of the on-vehicle equipment.

The control section 5 has a function of detecting the position of the nostril in the center of the face by using the nostril search area (search area) from the image information picked up by the CCD camera 4.

Using the pupil search area (search area) based on the position of the nostril, a detection target portion in the driver's face where changes in awakening are most likely to appear, for example, the driver's left eye (the influence of outside light of the vehicle is small. A function to search the eyes in the center of the passenger compartment (corresponding to a search means).

A function (corresponding to tracking means) of tracking the position of the eye in the pupil search area on the screen so that the eye falls within the area.

A function of learning a driver's blink habit from a pupil image in the pupil search area at a predetermined learning time and determining a reference blink as a reference for estimating arousal level.

After learning, a function of estimating the driver's arousal level from the image information of the CCD camera 4 based on the reference blink, using a map (not shown), for example (corresponding to arousal level estimating means).

A function of activating the alarm device 6 connected to the control unit 5 when the estimated arousal level becomes equal to or lower than a predetermined value.

When the driver's arousal level is detected during driving and the driver's arousal level decreases, the alarm device 6 notifies the driver of the fact.

In addition to the above, the control unit 5 has a signal indicating that the operation panel unit 2a of each air conditioner or the operation panel unit 2b of the navigation device has been operated in order to detect a change in the driving posture of the driver. Is input (corresponding to detection means for detecting changes in driving posture).

Each preset operation panel section 2a,
Setting data for the location directions of the operation panel units 2a and 2b of various in-vehicle devices based on the driver's face position for each input signal of 2b.

With the input of the signal, a function of expanding the pupil search area in which the pupil is detected in the above-mentioned location direction determined according to the source of the input signal, for example, the operation panel units 2a and 2b are operated. Then, of the current pupil search area, a function of extending the end portion on the side where the operation panel units 2a and 2b are present obliquely downward (corresponding to an area enlarging means).

Is set, and the operation panel units 2a, 2
When the position of the driver's face changes, for example, obliquely downward in accordance with the operation of b, the size of the search area itself is enlarged in the changing direction accordingly. Note that, here, the end of the search area on the opposite side to the changing direction is limited to be smaller than the enlargement amount.

By the function of expanding the pupil search area, the act of operating the in-vehicle device is accepted as it is, and the blink detection can be continued.

The control of such a wakefulness detecting device is shown in the flow chart of FIG.

Explaining the same control, for example, it is assumed that the driver now gets into the automobile, sits on the front seat 3, and starts driving.

Along with this, the control unit 5 controls the CCD camera 4
Then, the imaging of the driver's face is started as shown in FIG.

Then, as shown in FIG. 3, the control unit 5 searches the face image of the driver for the nostril α at the center of the face using the nostril search area A1 (step S1).
Then, the control unit 5 starts the search of the left eye β that is the target detection target site based on this nostril position by the pupil search area A2 (step S2), and puts the left eye in the pupil search area A2.

After that, the control section 5 goes through step S3 and detects blinks using the left-eye screen in the pupil search area A2 (step S5). Next, the reference blink of the driver is set in order to learn the individual differences of the driver, and the awakening degree of the driver is estimated in consideration of the reference blink (step S7). Then, if the estimated arousal level is equal to or lower than the predetermined value, it is determined that the driver is in a sleepy state, an alarm is issued from the alarm device 6, and the fact is notified to the driver (steps S8 and S9).

When the driver operates an in-vehicle device (air conditioner, navigation device, etc.) while the awakening level is being detected, the driver's driving posture suddenly changes, and the driver's face direction changes. Suddenly changes drastically, and the left eye deviates from the pupil search area A2 that has been stored in the periphery of the pupil, and the pupil cannot be tracked.

If left as it is, the left eye may be lost, but this can be suppressed by the enlargement control of the pupil search area.

That is, first of all, the control unit 5 controls the in-vehicle device,
For example, when the operation panel unit 2a of the air conditioner is operated, an operation signal emitted from the operation panel unit 2a at that time, for example, an ON signal, a switching signal, etc., is detected, and the in-vehicle device, here the air conditioner is operated. The determination is made (step S3). Next, the control unit 5 reads out the preset direction data based on the face of the driver with respect to the transmission source set in advance. From this reading, it is determined which direction the driving posture of the driver has changed in accordance with the operation of the in-vehicle device, that is, which direction the face of the driver has changed.

After that, the control unit 5 determines the end of the pupil search area A2 on the center side of the passenger compartment as shown by the solid double arrow K in FIG. 3 according to the location data.
Extend in the direction of a. That is, the pupil search range A2 is expanded in the direction in which the posture of the driver changes.

Here, since the left eye, which cannot be tracked completely, operates the air conditioner, it can be estimated that the left eye is moving in the direction in which the operation panel unit 2a is present. Therefore, the pupil search range A
By expanding 2 in the direction in which the operation panel unit 2a is present, the pupil is not lost or is immediately detected even if it is lost, and blink detection, reference blink setting, and awakening level estimation are continued. .

Therefore, good awakening level detection can always be continued without interruption. This is particularly effective for detecting the arousal level by detecting blinks that are easy to lose sight of.

FIG. 4 shows a second embodiment of the present invention.

In the present embodiment, as a means for detecting a change in the driver's driving posture, the steering wheel is operated by paying attention to the fact that the driver's posture changes and the face direction changes in accordance with the steering operation to the left or right. It is intended to enlarge the pupil search area in the direction.

Specifically, in this embodiment, the steering wheel angle sensor 10 is provided as shown by the chain double-dashed line in FIG.
Structure for detecting whether the handle 11 is operated leftward or rightward, and step S in the flowchart of FIG.
When the handle 11 is operated as shown at 10 and S11, the handle 1 indicated by the two-dot chain line arrows L and R in FIG.
1 is combined with the control for enlarging the pupil search area A2 in the operated direction, and even if the direction of the driver's face is changed by the steering wheel operation, the left eye is not lost and the blink detection, the standard blink setting, and the awakening degree are performed. This is so that the estimation can be continued.

Even with such a method, good awakening level detection can always be continued without interruption.

Since the second embodiment is the same as the first embodiment except for the above-mentioned features, the same reference numerals are given and the description thereof is omitted.

FIG. 5 shows a third embodiment of the present invention.

In this embodiment, as means for detecting a change in the driver's driving posture, attention is paid to the fact that the driver's posture changes and the face direction changes in accordance with the left or right blinker operation. On the other hand, it is intended to expand the pupil search area.

Specifically, in this embodiment, as shown in FIG. 1, it is detected whether the winker lever 13 is operated leftward or rightward from a signal of a winker switch (not shown) of the winker lever 13. If the turn signal is operated as shown in the structure and steps S13 and S14 of the flowchart of FIG. 5, the turn signal shown by the two-dot chain line arrows L and R in FIG. 3 is operated, as in the second embodiment. In combination with the control to expand the pupil search area A2 in the direction, even if the direction of the driver's face changes due to the blinker operation,
It is designed so that blink detection, reference blink setting, and arousal level estimation can be continued without losing sight of the left eye.

Even with such a method, good awakening level detection can always be continued without interruption.

Since the third embodiment is the same as the first embodiment except for the above-mentioned features, the same reference numerals are given and the description thereof is omitted.

FIG. 6 shows a fourth embodiment of the present invention.

In the present embodiment, as means for detecting a change in the driving posture of the driver, the seat position of the front seat is changed by noting that the posture of the driver changes and the face direction changes in accordance with the change in the sitting posture of the front seat. This is an attempt to expand the pupil search area in the direction in which is changed.

Specifically, in this embodiment, as shown in FIG. 1, for example, the structure in which the direction in which each position of the front seat 3 changes is detected from the control signal of the electric unit 3a, and the flowchart of FIG. Steps S16 and S1
When the seating posture of the front seat 3 is changed as shown in 7, the control for enlarging the pupil search area A2 in the direction in which the height changes as shown by the two-dot chain line arrows U and D in FIG. In combination, even if the driver's face orientation is changed by seat adjustment operation, the blink detection, the reference blink setting, and the alertness estimation can be continued without losing sight of the left eye.

Even with such a method, good awakening level detection can always be continued without interruption in the middle.

Since the fourth embodiment is the same as the first embodiment except for the above-mentioned features, the same reference numerals are given and the description thereof is omitted.

FIG. 7 shows a fifth embodiment of the present invention.

The present embodiment focuses on the fact that when the driver operates the steering wheel or blinker in response to a leftward or rightward course instruction given by the navigation device, the posture of the driver changes and the direction of the face changes. Thus, the pupil search area is expanded in the direction of the path indicated by the navigation device.

Specifically, this embodiment is shown in a structure for inputting route instruction information from the main body of the navigation device 2b as shown in FIG. 1 and steps S19 and S20 of the flowchart of FIG. When the course direction is instructed as described above, as in the case of the previous embodiment, in combination with the control for expanding the pupil search area A2 in the directions indicated by the two-dot chain line arrows L and R in FIG. This is so that the blink detection, the reference blink setting, and the alertness estimation can be continued without losing sight of the left eye.

Even with such a method, good awakening level detection can always be continued without interruption in the middle.

Since the fifth embodiment is the same as the first embodiment except for the features described above, the same reference numerals are given and the description thereof is omitted.

Of course, the techniques of the above-described first to fifth embodiments may be combined appropriately.

[0061]

As described above, according to the present invention, since the search area is enlarged in accordance with the change of the driver's posture, it is easy to track the detection target portion, and even if the driver's face is in the handle direction. Even if it changes due to operation, blinker operation, operation of in-vehicle device, operation of sitting posture, etc., the driver's detection target site is not lost, or even if it is once lost, it can be found immediately and the awakening level detection can be continued.

Therefore, good awakening level detection can always be continued without interruption in the middle.

In particular, the expansion of the search area is effective for detecting the arousal level by detecting the blink, which often causes the eyes to lose sight.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic configuration of a driver alertness detection device according to a first embodiment of the present invention.

FIG. 2 is a flowchart for explaining control of the device.

FIG. 3 is a diagram for explaining tracking of a pupil in the same control.

FIG. 4 is a flowchart for explaining control of the driver's awakening degree detection device according to the second embodiment of the present invention.

FIG. 5 is a flowchart for explaining control of a driver's awakening degree detection device according to a third embodiment of the present invention.

FIG. 6 is a flowchart for explaining control of the driver's awakening degree detection device according to the fourth embodiment of the present invention.

FIG. 7 is a flowchart for explaining control of the driver's awakening degree detection device according to the fifth embodiment of the present invention.

[Explanation of symbols]

2a, 2b ... Operation panel section of air conditioner, operation panel section of navigation device. 3 ... Front seat 4 ... CCD camera (imaging device) 5 ... Control unit (searching means, tracking means, awakening degree estimating means, area expanding means)

Continued front page    F term (reference) 3D037 FA05 FA23 FA26 FB09                 5B057 AA16 CA12 CB20 CC03 DA08                       DA15 DB02 DC16 DC36                 5C086 AA23 BA22 CA28 CB36 DA01                       DA33 EA11 EA41 FA01 FA11                 5L096 BA04 CA02 DA02 DA03 FA69                       HA05 JA18 LA11

Claims (2)

[Claims] 1. Search means for searching a detection target part from an image of a driver's face, and storing the searched detection target part in a search area to track the detection target part by capturing the characteristics of the detection target part. Tracking means, an awakening degree estimating means for estimating the awakening degree of the driver from the condition of the detection target part in the search area, a detecting means for detecting a change in the driving posture of the driver, and a change in the driving posture of the driver And an area enlarging means for enlarging the search area in the direction in which the posture of the driver changes, the arousal level detecting device for the driver. 2. The driver alertness detection device according to claim 1, wherein the search area of the tracking means is formed by an area for detecting the position of the driver's pupil.