JP2006213146A - Face direction determination device of driver - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face direction determination device of a driver capable of determining a face direction by acquiring a face picture image of a proper size by restraining erroneous determination with a distance to a portion except for a face as a distance to the face. <P>SOLUTION: This face direction determination device of the driver is furnished with a second camera (for example, room mirror built-in camera 2) arranged at a position different from a face direction camera 1 in a car room and to acquire the picture image of the driver 7 from a different direction and devised to be capable of acquiring the face picture image of the driver 7 of the proper size by the face direction camera 1 by finding a face position of the driver 7 by picture image processing at a picture image processing part 32 from the picture image acquired by the second camera 2, computing a distance between the face of the driver 7 and the face direction camera 1 by a distance determining part 34 and adjusting a zoom ratio of the face direction camera 1 by a camera adjusting part 35 in accordance with it. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a driver's face orientation determination device that is mounted on a vehicle, acquires a driver's face image with a camera, determines the face orientation, and detects a side-view driving.

  2. Description of the Related Art As a device that detects a side-view driving of a vehicle driver, a device that acquires a driver's face image and determines the face direction by image processing is known. In this type of device, for example, a face image is acquired by a camera arranged on a steering column or an instrument panel, but the distance relationship between the driver's face and the camera depends on the difference in the driver's physique and its operation posture. Since it fluctuates, the size of the face in the acquired face image also differs. In addition, due to the change in the projection relationship, the shape of the face image itself obtained by projecting a three-dimensional face on a plane also differs.

Therefore, it is conceivable to acquire the same driver's face image even when the distance is different between the cameras. The technique of Patent Document 1 relates to a so-called camera-equipped mobile phone, and measures the distance to the subject and changes the focal length of the lens based on the distance information to photograph the subject at an optimal size. It is.
Japanese Patent Laid-Open No. 2002-27306

  By the way, if control is to be performed based on the distance between the camera and the driver's face as the subject, it is necessary to directly measure or indirectly estimate the distance from the camera to the driver's face. However, it is difficult to directly detect the position of the driver's face in the passenger compartment. For example, when a distance sensor is provided in addition to the camera that acquires the face image, a position other than the driver's face can be detected by a steering operation or the like. There is a possibility of false detection by measuring the distance to the part. In addition, there is a method of making a determination based on the acquired image. In this case, however, there is a possibility that an image other than the face is erroneously determined as a face, or an erroneous determination is made depending on the face orientation.

  In view of this, the present invention provides a face orientation determination device for a driver capable of performing a face orientation determination by acquiring a face image of an appropriate size while suppressing such an erroneous determination in a face orientation determination apparatus mounted on a vehicle. It is an object to provide an apparatus.

  In order to solve the above-mentioned problems, a driver's face orientation determination device according to the present invention includes (1) a first camera that is disposed in a passenger compartment and acquires a driver's face image, and (2) the first camera. The camera is arranged at a different position in the vehicle interior and acquires a driver's image from different angular directions; and (3) the driver's face by image processing from the acquired image of the second camera. Distance determining means for determining the distance between the first cameras; (4) adjusting means for adjusting the face size in the face image acquired by the first camera based on the result of the distance determining means; and (5) adjustment. Determining means for determining a face orientation using a later face image.

  The first camera and the second camera acquire the driver image from different angles. The first camera is disposed at a position where the driver's face image can be obtained from substantially the front, for example, on the steering column or in the instrument panel. On the other hand, the 2nd camera is incorporated in the rearview mirror, for example, and acquires a driver picture from the vehicle center direction ahead. At this time, not only the driver but also an object (hereinafter referred to as a reference object) image whose positional relationship with the surrounding first cameras is known may be acquired at the same time. If there is a driver's face position or reference object in the image acquired by the second camera in this way, the distance determination means further determines the driver's face and the first camera by image processing based on the positional relationship with the reference object. Find the distance to. Based on this distance relationship, the adjustment means adjusts the image acquired by the first camera or the size of the face image in the acquired image to a predetermined size. The determination means determines the driver's face orientation based on a face image of a predetermined size.

  This adjustment means may be adjusted by adjusting the zoom ratio of the first camera or by changing the reduction ratio of the image acquired by the first camera. The former is to change the optical zoom ratio, and the latter is to change the so-called compression ratio.

  According to the present invention, the image of the driver is acquired by the second camera from a direction different from the first camera arranged substantially in front of the driver, and the distance between the first camera and the driver's face is obtained. Therefore, even if there is an obstacle (driver's hand, arm, etc.) due to the driver's steering operation between the driver's face and the first camera, the driver's face The distance from the first camera can be accurately determined, and a face image of an appropriate size can be acquired based on this information.

  If this face image is adjusted by adjusting the zoom ratio of the first camera, an image including an appropriately sized face image can be easily obtained. On the other hand, if adjustment is performed by changing the reduction ratio of the image acquired by the first camera, there is an advantage that a camera with a fixed focal length can be used. In this case, it is necessary to use a high-definition camera compared to the resolution of the image used at the time of determination.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the description, the same reference numerals are given to the same components in the drawings as much as possible, and duplicate descriptions are omitted.

  FIG. 1 is a schematic diagram showing a configuration of a face orientation determination apparatus according to an embodiment of the present invention, and FIG. 2 is a block configuration diagram thereof. Here, a right-hand drive vehicle will be described as an example. A face-facing camera (first camera) 1 is disposed on the steering column 4, and a room mirror built-in camera (second camera) 2 is disposed in the rear mirror 5 at the front center in the vehicle interior. Both the cameras 1 and 2 are directed to the driver 7 who is seated in the driver's seat 6 from different directions, that is, the front and the diagonally left front.

  Images acquired by the face-facing camera 1 and the room mirror built-in camera 2 are converted into digital signals having a predetermined number of pixels by the A / D converters 11 and 21 and input to an ECU (Electric Control Unit) 3. Is done. The ECU 3 includes a CPU, a ROM, and a RAM, and performs predetermined image processing based on a built-in program. The ECU 3 includes a first image processing unit 31 that processes an acquired image of the face-facing camera 1, a second image processing unit 32 that processes an acquired image of the camera 2 with a built-in mirror, and a processing result of the first image processing unit 31. A face direction determination unit 33 that determines the driver's face direction, a distance determination unit 34 that determines the distance between the driver's face and the face-facing camera 1 from the processing result of the second image processing unit 32, and a distance determination unit A camera adjustment unit 35 that adjusts the zoom ratio of the face-facing camera 1 based on the determination result 34, and an alarm unit 36 that issues an alarm instruction to the alarm device 8 based on the processing result of the face direction determination unit 33. Yes. The warning device 8 issues a warning to the driver by, for example, sound from a speaker, display on an instrument panel, a display device, or the like.

  Next, the operation of this embodiment will be specifically described. When the vehicle is turned on, a front image (see FIG. 3) of the driver 7 is acquired by the face-facing camera 1, and an image (see FIG. 4) from the front left of the driver 7 by the camera with built-in rearview mirror 2. ) Is acquired. These acquired images are acquired at, for example, a television frame rate (about 33 ms), converted into digital signals by the A / D converters 11 and 21, and transferred to the respective image processing units 31 and 32 of the ECU 3.

  In the acquired image of the rearview mirror built-in camera 2 sent to the image processing unit 32, it is preferable that the driver 7 and the face-facing camera 1 are captured at the same time, but the face-facing camera 1 itself is not reflected. However, as shown in FIG. 4, it is preferable that the steering column 4 and the steering wheel 41 whose positional relationship with the face-facing camera 1 is known are imaged simultaneously. Hereinafter, when the face-facing camera 1 is reflected in the image, the position of the face-facing camera 1 in the image is the same as that of the other face-facing cameras 1 where the face-facing camera 1 is not reflected. When an object having a known positional relationship is reflected, the position of the known object in the image is referred to as a reference object position.

  The second image processing unit 32 obtains the driver's face position and reference object position from the acquired image by image processing. This position detection is performed by combining edge detection and pattern recognition. The distance determination unit 34 obtains the distance from the face-facing camera 1 to the face of the driver 7 based on the positional information, the position of a known reference object, and the face-facing camera 1. The camera adjustment unit 35 adjusts the face image of the driver 7 to an appropriate size in the acquired image by adjusting the zoom ratio of the face-facing camera 1 according to the obtained distance information.

  By controlling the zoom ratio in this way, an image including the face image of the driver 7 of an appropriate size is sent to the first image processing unit 31. The image processing unit 31 detects the center position (line passing through the nose) A and both end positions B and C shown in FIG. 5 by combining known methods such as edge detection from the acquired image.

The face orientation determination unit 33 calculates the face orientation angle from the face end positions B and C and the face center position A. Specifically, when the distance between AB is L _r , the distance between AC is L ₁ , and L ₁ + L _r is L, the face orientation angle θ (positive when facing right) is: Assuming that the face is the cylindrical model shown in FIG.

  The warning unit 36 determines the obtained face orientation angle θ, and determines that the side is in a side-view state if the absolute value of the angle continues for a predetermined time or more. Here, the frequency of the face of the driver 7 being blocked by the steering wheel 41 or the arm of the driver 7 is increased during the steering, as viewed from the face-facing camera 1, and the direction other than the front, such as the course direction, is during the steering. It is better not to determine the side-viewing state because there is a high possibility of facing away from the front for direction confirmation. And when it determines with a side-view state, it instructs the alarm device 8 to issue an alarm to the driver. According to the instruction, the alarm device 8 instructs the driver to turn in the traveling direction by emitting a sound from a speaker or displaying an alarm on an instrument panel or a display.

  According to the present embodiment, the face orientation in the image acquired by the face orientation camera 1 is adjusted in advance so as to have an appropriate size, and the face orientation determination process is performed based on the adjusted image. The determination accuracy of the face angle can be maintained at an appropriate level. For this reason, it is possible to suppress the variation in the accuracy of determination of the face orientation angle due to the difference in face size due to the difference in the driver's physique and driving posture, and to perform accurate determination.

  In addition, since the positional relationship (distance) between the driver and the face-facing camera is obtained by performing image processing on an image acquired using a camera with a built-in mirror, when measurement is performed using a distance sensor provided in the face-facing camera 1 In comparison, the distance can be accurately measured regardless of the operation of the driver. Although the driver's seating position can be detected by detecting the seat position, in this case, a change due to the driver's driving posture cannot be detected. On the other hand, according to the present embodiment, there is an advantage that the face position can be accurately determined without being affected by the difference in the posture or the physique of the driver.

  In the above description, the second camera is a camera with a built-in room mirror. However, the present invention is not limited to this, but the ceiling of the passenger compartment (in the center of the passenger compartment or directly above the driver or the door side of the driver's seat) or driving It may be arranged at the door on the seat side. The second camera may share the one used for driver authentication.

  In the above description, the example in which the optical zoom ratio of the first camera is changed has been described. However, the resolution of the image acquired by the first camera is sufficiently higher than the resolution required for image processing in face orientation determination. In this case, the size of the face image in the image used for the image processing for determination processing may be adjusted to an appropriate level by changing the target area when converting the resolution. For this, a so-called digital zoom method can be applied.

  Here, an example in which alarm processing is performed based on the determination result of side-view driving has been described. However, the determination result can be used in a collision avoidance program or a driving support program.

It is the schematic which shows the structure of the face direction determination apparatus which is one Embodiment of this invention. It is a block block diagram of the apparatus of FIG. It is a schematic diagram which shows the example of the acquired image by the face direction camera of the apparatus of FIG. It is a schematic diagram which shows the example of the acquired image by the camera with a built-in room mirror of the apparatus of FIG. It is a figure explaining face orientation determination using a face image. It is a figure explaining the principle of face direction angle calculation using a face image.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Face-facing camera, 2 ... Camera with built-in room mirror, 3 ... ECU, 4 ... Steering column, 5 ... Room mirror, 6 ... Driver's seat, 7 ... Driver, 8 ... Alarm device, 11, 21 ... A / D conversion 31 ... 1st image processing part, 32 ... 2nd image processing part, 33 ... Face direction determination part, 34 ... Distance determination part, 35 ... Camera adjustment part, 36 ... Alarm part, 41 ... Steering wheel.

Claims (3)

A first camera disposed in the passenger compartment to obtain a driver's face image;
A second camera which is arranged at a different position in the vehicle interior from the first camera and obtains an image of the driver from different angular directions;
Distance determining means for determining a distance between the driver's face and the first camera by image processing from an acquired image of the second camera;
Adjusting means for adjusting the face size in the face image acquired by the first camera based on the result of the distance determining means;
Determining means for determining the face orientation using the adjusted face image;
A device for determining a face orientation of a driver, comprising:   The face orientation determination apparatus according to claim 1, wherein the adjustment unit performs adjustment by adjusting a zoom ratio of the first camera.   The face orientation determination apparatus according to claim 1, wherein the adjustment unit performs adjustment by changing a reduction ratio of an image acquired by the first camera.

Images