JP4706917B2 - Driver monitoring system - Google Patents

Description

  The present invention relates to a driver monitoring system including a determination unit that determines a state of the driver using an image of the driver taken by a camera installed on a steering wheel of a vehicle.

  An example of this type of system is described in Patent Document 1 below. In the driver monitoring system described in Patent Document 1, a driver's face image is obtained by a camera installed in a vehicle, and the change of the driver is detected by collating with the registered facial feature data. Alternatively, it can be determined whether or not the driver is a valid driver (an example of the driver's state). Furthermore, in one embodiment, since the camera is disposed at the center portion (boss portion or pad portion) of the steering wheel, the steering wheel is steered as compared with the case where the camera is disposed on the instrument panel of the vehicle. Problems such as steering wheel spokes obstructing imaging due to the corners have been solved.

Japanese Patent Laying-Open No. 2003-256813 (paragraph numbers 0024 to 0027, FIGS. 2 and 8)

  However, in the driver monitoring system described in Patent Document 1, the camera is disposed on the boss portion of the steering wheel, and the camera rotates with the steering wheel during driving. The image could not be detected well, and as a result, the state of the driver could be determined smoothly only while the vehicle was stopped.

  SUMMARY OF THE INVENTION In view of the above-mentioned drawbacks of the above-described prior art driver monitoring system, an object of the present invention is to provide driver monitoring that can smoothly determine the state of the driver regardless of whether the vehicle is stopped or driving. To provide a system.

A first characteristic configuration of the present invention is a driver monitoring system including a determination unit that determines the state of the driver using image information of the driver imaged by a camera installed on a steering wheel of the vehicle. And
Steering angle detection means for detecting the steering angle of the steering wheel and providing it as steering angle information, angle information giving means for giving the steering angle information to an inclination image captured by the camera , and steering the inclination image Inclination correction means for correcting to an erect image using angle information, and the determination unit is configured to perform determination using the erect image .

In the driver monitoring system according to the first characteristic configuration of the present invention, the determination unit does not determine the state of the driver based only on the tilt image captured by the camera , but does not match the information on the steering angle at the time of imaging. it can be determined using the tilt image became, for example, it is possible to know the inclination angle of the object occurring during image based on the camera during operation rotates with the steering wheel.

Therefore, the program provided in the determination unit for analyzing the feature amount of the face based on the obtained image can easily compare image information sequentially captured through the camera, and the vehicle is stopped. The driver's state can be accurately determined regardless of whether the vehicle is driving.

Further, in this configuration, since the image is taken with the steering wheel being steered, even if the obtained image is tilted, the determination unit converts the tilted image into an upright image based on the steering angle information at the time of imaging. Judgment can be performed in the corrected state. That is, since the determination unit can use an erect image in this way, for example, face detection can be performed using existing face detection software created in principle with the symmetry axis of the face image being vertical. Thus, effects such as easy extraction of facial feature quantities for driver authentication and arousal evaluation performed after face detection can be obtained. Further, even when face detection software corresponding to a face image whose symmetry axis is not vertical is used, face detection can be performed more quickly, and the load on a computer used for face detection or the like is reduced.

A second characteristic configuration of the present invention is a driver monitoring system including a determination unit that determines the state of the driver using image information of the driver imaged by a camera installed on a steering wheel of the vehicle. And
Steering angle detection means for detecting the steering angle of the steering wheel and providing it as steering angle information, and adding the steering angle information to a perspective image captured by the camera from the front of the driver as the steering wheel rotates. Angle information providing means, and the determination unit registers in advance a number of perspective images obtained by capturing a legitimate driver at various viewing angles, and the steering image is captured by the camera. The point is that the authentication is performed in comparison with the registered perspective image of the viewing angle that matches the viewing angle value of the camera obtained based on the angle information .

  With this configuration, for example, a large number of perspective images obtained by capturing a legitimate driver at various viewing angles are registered in advance, and the perspective image captured by the camera on the steering wheel from the front of the driver is displayed as angle information. By comparing with the registered perspective image of the viewing angle that matches the viewing angle value of the camera obtained based on the above, it is possible to cause the determination unit to perform a stricter authentication test than the authentication based only on the face image from the front Can do. Of course, a perspective image captured by a camera on such a steering wheel may be compared with an upright registered perspective image after being corrected to an upright image based on steering angle information at the time of imaging. The determination operation by the unit is simplified.

A third characteristic configuration of the present invention is that the camera is installed on a steering pad of the steering wheel at a position where the camera does not interfere with deployment of an airbag mounted in the steering pad.

  With this configuration, a camera based on steering is used because the camera is installed at a position closest to the axis of the steering wheel in the radial direction within a range that does not interfere with the deployment of the airbag even in the region on the steering wheel. Minimal movement in the vertical and horizontal directions. Therefore, since the driver's face is imaged from the front as much as possible, it is possible to obtain a captured image from which feature amounts can be easily extracted from the face image. Further, when the correction based on steering of the steering wheel is performed, it is only necessary to correct the inclination of the face image with respect to the vertical line, and the process of converting the perspective image into the image from the front is substantially unnecessary, so the determination unit performs The amount of calculation processing is relatively small.

A fourth characteristic configuration of the present invention is that the camera is installed in a grip portion or a spoke portion of the steering wheel.

  With this configuration, the camera is installed in a position that is radially separated from the axis of the steering wheel, even in the area on the steering wheel. A large number of perspective images taken from various angles other than images can be obtained in abundance, and a more strict authentication test can be performed.

A fifth characteristic configuration of the present invention is that the camera is provided at an upper end position when the steering wheel is in a neutral posture on the grip portion.

  With this configuration, the camera is less likely to be covered with the driver's hand, particularly in the region of the grip portion of the steering wheel, as compared to the left and right ends and the lower end of the grip portion, and the front of the driver. It is possible to capture an image that can be easily determined by the determination unit.

A sixth characteristic configuration of the present invention is that the camera is installed such that its optical axis is inclined toward the rotation axis of the steering wheel.

  With this configuration, even when the camera is installed at a position radially separated from the rotation axis of the steering wheel, such as a grip portion, the driver's face image is displayed as the steering wheel rotates due to steering. The problem of deviating from the imaging range is less likely to occur.

The best mode for carrying out the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 is a schematic configuration diagram showing a driver monitoring system 100 according to the first embodiment of the present invention.
The driver monitoring system 100 includes a video camera 10 attached to the steering wheel 2 of the vehicle 1 and a steering angle sensor 6 installed on the steering shaft 2a of the steering wheel 2 in order to detect the steering angle of the steering wheel 2. (An example of a steering angle detection means).
A light receiving unit 10a (an example of a camera, including an imaging lens unit and a light receiving element unit) of the video camera 10 is an area belonging to the pad unit 4 of the steering wheel 2 and air mounted in the pad unit 4 A portion that does not interfere with the development of the bag (not shown), specifically, is installed on the upper end surface of the pad portion 4. The main body 10b of the video camera 10 is installed behind the imaging lens unit 10a. At least the light receiving unit 10a rotates integrally with the steering wheel 2 based on steering by the driver D. Further, the optical axis L1 of the imaging lens unit 10a is not parallel to the steering shaft 2a of the steering wheel 2, but is slightly inclined (θ1) near the rotational axis of the steering wheel 2.

In addition, the driver monitoring system 100 includes a control unit 50 including a CPU, a memory, and the like for processing an image near the driver's seat taken by the video camera 10.
The control unit 50 temporarily stores image information in the vicinity of the driver's seat including the head of the driver D captured by the video camera 10 in the memory 40, and angle detection sent from the steering angle sensor 6a. A steering angle calculation unit 22 that calculates a steering angle of the steering wheel 2 based on the signal and stores it in a memory as steering angle information is provided.

  The control unit 50 is provided with a determination unit 30 that determines the state of the driver D using image information captured by the video camera 10. The determination unit 30 detects only the face part from the image information to which the steering angle information is added by the angle information adding unit 24, and extracts the face image extraction unit 25 as an upright image in which the symmetry axis of the face extends vertically. Then, the facial feature amount suitable for the person authentication is extracted from the erect image of the face obtained by the face image extraction unit 25, and compared with the reference data regarding the legitimate driver registered in advance, the driver D's The feature amount of the face suitable for arousal level evaluation is extracted from the face image obtained by the driver authentication unit 26 and the face image extraction unit 25 that perform authentication, and the driver D awakens from the change rate of the feature amount. A wakefulness evaluation unit 28 for evaluating the degree is included.

In the face image extraction unit 25, a “face detection” operation for always detecting a face image from image information in the vicinity of the driver's seat regardless of the tilt of the video camera 10 based on the shaking of the head of the driver D or the steering by the driver D. Then, as shown in FIG. 2, the face image that is an inclined image because it is captured by the light receiving unit 10 a that is inclined based on steering is upright using the steering angle information provided by the angle information providing unit 24. An “inclination correction” operation for correcting an image is simultaneously performed by existing software.
That is, since the light receiving unit 10a of the video camera 10 is installed on the pad unit 4 close to the axis of the steering wheel 2, the face image detected by the face detection operation from the image captured by the video camera 10 is the steering amount. Regardless, the face image is basically obtained by imaging the driver D from substantially the front. However, as illustrated in FIG. 2, when the steering wheel 2 is steered by + α °, the symmetry image S is an inclined image F1 that is inclined to the right by α ° with respect to the vertical line. In the “tilt correction” operation, the tilted image F1 is corrected to an upright image F2 using the steering angle information “+ α °” detected by the steering angle sensor 6a and applied by the angle information providing unit 24.

The driver authentication unit 26 can use, for example, a ratio between the distance between the pupils and the distance from the eyebrow to the lips as the facial feature amount. If the driver authentication unit 26 does not authenticate the driver D as a legitimate driver, the driver authentication unit 26 locks the starter 7 of the engine (not shown) via the starter lock driver, thereby making it impossible to start the engine.
On the other hand, the arousal level evaluation unit 28 can use the blinking frequency of the eyes detected based on a sudden change in the area of the pupil as the facial feature amount. The arousal level evaluation unit 28 drives the buzzer 8 in the driver's seat via the buzzer driver when it is evaluated that the awakening level of the driver D is insufficient.
The driver authentication unit 26 and the arousal level evaluation unit 28 smoothly perform authentication and evaluation by using the face image information corrected to the upright state using the steering angle information provided by the angle information providing unit 24. be able to.

(Second Embodiment)
FIG. 3 is a schematic configuration diagram illustrating a driver monitoring system 200 according to the second embodiment of the present invention.
The driver monitoring system 200 includes a video camera 20 attached to the steering wheel 2 of the vehicle 1, and a steering angle sensor 6 installed on the steering shaft 2 a of the steering wheel 2 in order to detect the steering angle of the steering wheel 2. (An example of a steering angle detection means).
The light receiving unit 20a of the video camera 20 (which is an example of a camera and includes an imaging lens unit and a light receiving element unit) is located at the upper end position on the grip unit 3 of the steering wheel 2 when the steering wheel 2 is in a neutral posture. is set up. The main body 20b of the video camera 20 is installed on the side surface of the pad portion 4 of the steering wheel 2 or the like. At least the light receiving unit 20a rotates integrally with the steering wheel 2 based on steering by the driver D. Similarly to the first embodiment, the optical axis L2 of the imaging lens unit 20a is not parallel to the steering shaft 2a of the steering wheel 2, but is inclined by θ2 closer to the rotational axis of the steering wheel 2. Yes.

Since the configuration of the control unit is basically the same as that of the first embodiment, detailed description thereof is omitted.
However, in the second embodiment, since the light receiving portion 20a of the video camera 20 is installed in the grip portion 3 instead of the pad portion 4 of the steering wheel 2, it is detected by a face detection operation from the image captured by the video camera 10. The face image thus obtained becomes a face image obtained by capturing the driver D from diagonally forward depending on the steering amount. That is, for example, in a state where the steering wheel 2 is steered by + α °, as illustrated in FIG. 4, the symmetry axis S is inclined to the right by α ° with respect to the vertical line, and the driver D is tilted forward. The image is an inclined / perspective image F3. In the “tilt correction” operation, this tilt / perspective image F3 is detected by the steering angle sensor 6a, and using the steering angle information “+ α °” given by the angle information giving means 24, the upright / perspective image F4. To correct.

  In the second embodiment, feature amounts extracted from a plurality of perspective images obtained by photographing a valid driver from various photographing angles belonging to obliquely ahead are stored as reference data together with corresponding photographing angles. deep. The driver authentication unit 26 provided in the determination unit 30 of the control unit 50 extracts the facial feature amount from the upright / perspective image F4 obtained by the face image extraction unit 25, and detects it by the steering angle sensor 6a. Then, the driver D is authenticated in comparison with the reference data corresponding to the steering angle information “+ α °” given by the angle information giving means 24. Here, a large number of perspective images (still images) obtained by photographing a legitimate driver at various viewing angles are registered in advance in the memory 40 together with the viewing angle information as reference data, similar to the face image from the front. deep. Then, the driver authentication unit 26 calculates the viewing angle value of the light receiving unit 20a at the time of imaging (how many times the driver D was imaged) from the angle information provided by the angle information providing unit 24, and this calculation is performed. A registered perspective image corresponding to the viewing angle value is extracted from the reference data and compared with the upright / perspective image F4 obtained by the face image extraction unit 25. This makes it possible to perform a stricter authentication test than the authentication based only on the face image from the front.

  Incidentally, for both the first and second embodiments, in order to reduce the required memory capacity, for example, after being authenticated as a valid driver by the driver authentication unit 26, the steering angle of the steering wheel 2. The controller 50 may be configured to perform arithmetic processing only when the angle is within a range of a predetermined angle (for example, ± 15 °).

  Furthermore, as the determination unit 50, an armpit monitoring unit (not shown) that monitors the driver's armpit while driving can be provided. The side-view monitoring unit extracts a facial feature quantity from the face image obtained by the face image extraction unit 25, and evaluates the driver D's side-view driving degree from the change rate of the feature quantity. In the armpit monitoring section, the driver's D face is tilted with respect to the traveling direction of the vehicle 1 during the armpit look, and is tilted around the vertical axis with respect to the video cameras 10 and 20. Or the number of detectable pupils can be used as the feature amount. When the shift gear is being input to the forward side, the buzzer 8 in the driver's seat is turned on via the buzzer driver 29 when the pupil interval is less than the threshold value or the look-ahead state where the number of pupils is one continues for a certain time. It is configured to drive. Also here, smooth evaluation can be performed by using the face image information corrected to the upright state using the steering angle information provided by the angle information providing unit 24.

[Another embodiment]
<1> When the airbag is not attached to the pad portion of the steering wheel, the light receiving portion 10a described in the first embodiment is provided not at the upper end surface of the pad portion 4 but at the center of the pad portion 4. Is also possible.

<2> The light receiving portion 20a described in the second embodiment can be provided not on the grip portion 3 of the steering wheel 2 but on the spoke portion 5 of the steering wheel 2.

<3> The steering angle detection means 6 does not use the steering angle sensor 6a installed on the steering shaft 2a of the steering wheel 2, but the light receiving unit 10a (20a) installed on the steering wheel 2, and the light receiving unit. It is also possible to configure with arithmetic means for calculating the steering angle from the position, the amount of inclination, etc. in the image of the fixed object that becomes the mark in the vehicle imaged by 10a (20a).

  The present invention relates to a driver monitoring system including a determination unit that determines a state of the driver using image information of the driver taken by a camera installed on a steering wheel of a vehicle, and relates to a steering angle of the steering wheel. Steering angle detecting means for detecting and providing as steering angle information and angle information providing means for adding steering angle information to image information obtained by the camera are provided, and the determination unit uses the image information to which the angle information is added. It can utilize for making into the driver | operator monitoring system which can perform determination with a high precision by the determination part.

It is a block diagram which shows the driver | operator monitoring system in 1st Embodiment of this invention. It is explanatory drawing which shows the inclination correction operation of face image information using steering angle information. It is a block diagram which shows the driver | operator monitoring system in 2nd Embodiment of this invention. It is explanatory drawing which shows the inclination correction operation of face image information using steering angle information.

D Driver L1, L2 Camera optical axis 1 Vehicle 2 Steering wheel 3 Grip part 4 Pad part 5 Steering wheel spoke part 6 Steering angle sensor (steering angle detection means)
7 Starter 8 Buzzer 10, 20 Video camera 10a, 20a Light receiving part (camera)
10b, 20b Camera body 21 Image input unit 22 Steering angle calculation unit 24 Angle information giving means 25 Face image extraction unit 26 Driver authentication unit 27 Starter lock driver 28 Arousal level evaluation unit 29 Buzzer driver 30 Determination unit

Claims (6)

A driver monitoring system including a determination unit that determines the state of the driver using image information of the driver imaged by a camera installed on a steering wheel of the vehicle,
Steering angle detection means for detecting the steering angle of the steering wheel and providing it as steering angle information, angle information giving means for giving the steering angle information to an inclination image captured by the camera , and steering the inclination image And a tilt correction unit that corrects an erect image using angle information, and the determination unit is configured to make a determination using the erect image . A driver monitoring system including a determination unit that determines the state of the driver using image information of the driver imaged by a camera installed on a steering wheel of the vehicle,
Steering angle detection means for detecting the steering angle of the steering wheel and providing it as steering angle information, and adding the steering angle information to a perspective image captured by the camera from the front of the driver as the steering wheel rotates. Angle information providing means, and the determination unit registers in advance a number of perspective images obtained by capturing a legitimate driver at various viewing angles, and the steering image is captured by the camera. A driver monitoring system configured to authenticate against a registered perspective image of a viewing angle that matches a viewing angle value of the camera obtained based on angle information . 3. The driver monitoring system according to claim 1, wherein the camera is installed on a steering pad of the steering wheel at a location that does not interfere with deployment of an airbag mounted in the steering pad . The driver monitoring system according to claim 1 or 2 , wherein the camera is installed in a grip portion or a spoke portion of the steering wheel . The driver monitoring system according to claim 4 , wherein the camera is provided at an upper end position when the steering wheel on the grip portion is in a neutral posture . The driver monitoring system according to claim 3 , wherein the camera is provided such that an optical axis thereof is inclined toward a rotation axis center of the steering wheel .

Images