JP2005242428A - Driver face imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driver face imaging device that takes face images suited for image recognition, by effectively avoiding or preventing the influence of reflected light on the front and back of eyeglasses lenses. <P>SOLUTION: The driver face image taking device has an image taking means for taking images of a vehicle driver's face; an eye detection means for detecting the position of the driver's eyes from each face image; and a near infrared illumination means for reflecting near infrared light at a roof part to indirectly apply the light to the driver's face. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a driver's face image capturing apparatus, and more particularly to a face image capturing apparatus suitable for recognizing a driver's face wearing spectacles.

  Conventionally, for the purpose of detecting driving conditions such as a driver's sideways and dozing, and for examining the driver's point of interest while driving, the driver's face image is taken and the resulting image is processed. Many devices for extracting feature points (such as eyes) of a driver's face have been proposed.

For example, an optical filter is placed on the optical axis of a camera that captures a face image, and a near-infrared illuminator that illuminates the driver's face with near-infrared light in the wavelength range that passes through the optical filter is used as the optical axis of the camera A face image that can capture a driver's face image even when the driver is wearing spectacles by suppressing the influence of reflected light on the lens surface of the spectacles as much as possible by arranging it at a position that is at a predetermined angle or more with respect to An imaging device is known (see, for example, Patent Document 1).
JP-A-9-21611

  However, because the near-infrared illuminator is a point light source and the degree of freedom in the interior layout of the vehicle is small, even if it is possible to suppress the reflection of the front scenery on the lens surface of the glasses, , Specular reflection of the point light source of the near-infrared illuminator occurs. The specular reflection that occurs on the back of the lens becomes a high-intensity point on the photographed image, so the reflection of the light emission point of the near-infrared illuminator that occurs on the back of the lens is an obstacle to image recognition. End up. Further, when the high luminance point overlaps the eye, it becomes difficult to correctly capture the shape of the eye even on the original image.

  The feature of the present invention is that image photographing means for photographing a driver's face image of the vehicle, eye detection means for detecting the position of the driver's eye from the face image, and reflecting near infrared light at the roof portion of the vehicle. The gist of the present invention is a driver's face image photographing device having a near-infrared illumination means for indirectly irradiating the driver's face.

  According to the present invention, it is possible to provide a driver's face image capturing device that captures a face image suitable for image recognition by effectively suppressing or preventing the influence of reflected light on the front and back surfaces of the spectacle lens. it can.

  Embodiments of the present invention will be described below with reference to the drawings. In the description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

  As shown in FIG. 1, a driver's face image capturing device according to an embodiment of the present invention includes a camera 1 as an example of an image capturing unit that captures a face image of a vehicle driver 6, and the camera 1 captures images. The processing device 2 that detects the driving state of the driver 6 from the face image of the driver 6 and the near infrared that indirectly reflects the near infrared light on the roof portion 5 of the vehicle and irradiates the face of the driver 6 indirectly. An illumination unit (near infrared illuminator) 3 and an alarm device 4 that receives a detection result from the processing device 2 and alerts the driver 6 are provided. The processing device 2 includes an eye detection unit 10 that detects the position of the eyes of the driver 6 from the face image, an eye tracking unit 11, and an open / close / sleeping determination unit 12.

  The near-infrared illuminator 3 does not directly irradiate the face portion of the driver 6 with near-infrared light, but directly irradiates the vehicle roof portion 5 and reflects the near-infrared light reflected by the vehicle roof portion 5 to the driver. 6 face part is irradiated. That is, the near-infrared illuminator 3 uses the reflection on the vehicle roof portion 5 to avoid the handle 8 and the like that hinder direct near-infrared light irradiation, and indirectly controls the face portion of the driver 6. Is irradiated with near infrared light.

  Next, with reference to FIGS. 2 (a) to 2 (d), a case will be described in which face image recognition of a driver wearing glasses is performed using the face image photographing device of FIG. As shown in FIG. 2 (a), when the spectacle lens part is reflected in a uniform state over the entire eye, the parameters for image preprocessing are set as shown in FIG. 2 (b). By adjusting, the feature amount of the eye can be captured with high accuracy. However, when the forward scenery is reflected only in the lower part of the eye as shown in FIG. 2C, the feature amount of the eye is used in the adjustment of the image preprocessing parameters as shown in FIG. It is difficult to capture accurately. The phenomenon in which a front scene is reflected only in the lower part of the eye as shown in FIG. 2C is affected by the radius of curvature of the lens surface of the spectacles, the front and rear of the driver's face, and the vertical position. The boundary line of the reflection that overlaps the eye is the upper end line of the windshield of the vehicle. The lower part of the spectacle lens part where the reflection is occurring shows the front scenery seen through the front window, and the upper part of the spectacle lens part where the reflection is not reflected is the roof surface of the vehicle .

  Therefore, even when the phenomenon of the front scene reflection as shown in FIG. 2C occurs, the vehicle roof portion 5 is irradiated with near-infrared illumination by using the face image photographing device shown in FIG. By doing so, it is possible to indirectly illuminate the face portion and to brighten the upper surface of the roof, so that the contrast difference divided into two at the lens center portion can be made uniform. Therefore, it is possible to obtain an advantageous state for capturing the eye feature amount by pre-processing by image recognition. As described above, the face image capturing device shown in FIG. 1 can effectively suppress or prevent the influence of the reflected light from the spectacle lens surface even in a driver wearing spectacles, thereby obtaining an image suitable for image recognition. The ability to detect the driver's condition such as falling asleep can be improved by enabling photographing.

  Next, the flow of processing operations of the face image photographing device of FIG. 1 will be described with reference to the flowchart of FIG.

  (A) First, in step S01, the face of the driver 6 is photographed by the camera 1, and the face image is input to the processing device 2. In step S02, it is determined whether an eye tracking area is set. Since the eye tracking area is not set immediately after the process is started (No in step S02), the process proceeds to step S03, and the eye detection unit 10 in FIG. 1 detects the position of the eye from the entire face image.

  Various methods for detecting the position of the eye from the entire face image in step S03 have been devised so far, and an example thereof will be described with reference to FIGS. A point where the change state of the density value in the vertical direction of the image shown in FIG. 4 satisfies a predetermined condition is extracted on each vertical line (Xa, Xb,...). The result is shown on the two-dimensional screen as shown in FIG. As shown in FIG. 6, the extracted points are grouped according to the approaching vertical direction in adjacent vertical lines. The groups G1 to G6 shown in FIG. 6 capture density characteristics that are black and long horizontally. In a relatively simple method, the right eyebrow: G1, the left eyebrow: G2, the right eye: G3, and the left eye: It can be detected as G4, nose: G5, and mouth: G6. Thereafter, as shown in FIG. 7, the feature amount of the face captured as continuous data is divided into several zones (ZONE: L, ZONE: C, ZONE: R) where each feature amount appears in the vertical direction. The eye position is detected by checking the relative positional relationship.

  (B) In step S04, the eye tracking unit 11 stores the eye position coordinates detected by the continuous data selection, and sets a small area including the eyes as the eye tracking area with reference to the eye position coordinates. .

  (C) In step S05, the eye tracking unit 11 detects the position of the eye within the eye tracking area. Specifically, continuous data extraction is performed in the same manner as when the eye position is detected in step S03. However, the extraction of continuous data in the eye tracking process is limited to the tracking area 20a of the eye 21a shown in FIG. Further, in this process, since it is necessary to detect the eye position with a small area with high accuracy compared to the case of detecting the eye position, the scanning interval density of the vertical lines (Xa, Xb,...) Shown in FIG. Carry out higher.

  (D) In step S06, the eye tracking unit 11 checks whether or not the eye is correctly tracked. Eye tracking has failed, that is, the case where the target object as an eye cannot be detected in step S05 or the shape characteristics of the continuous data of the detected target do not match. If not (No in step S06), the process proceeds to step S07, the eye tracking area 20a is cleared, and the process returns to the image input in step S01. At this time, since it is determined in step S02 that there is no eye tracking region, the process proceeds to step S03, and eye position detection is performed again from the entire image. If eye tracking is performed correctly (Yes in step S06), the process proceeds to step S08.

  (E) In step S08, the eye tracking means 11 updates the eye tracking area. Specifically, the center position of the eye identified in step S05 is stored as a reference, and the eye tracking area is updated based on the position. The tracking area 20a of the eye 21a shown in FIG. 8A is the eye tracking area set in step S05, and the center position 21a of the eye and the center position of the tracking area 20a coincide. After that, a new face image is captured in step S01, and if there is a change in the position of the eye in the image, as shown in FIG. 8 (b), the darkened eye 21b is replaced with the newly captured image. The position of the eye, and the thin eye 21a is the position of the eye in the front frame. The update of the eye tracking region in step S08 is a process of moving the eye tracking region around the eye position 21b in the newly captured image shown in FIG. 8B. This is shown in FIG. 8 (c). In other words, the eye position 21b in the newly captured image and the center position of the new eye tracking area 20b coincide. The eye tracking can be continued by continuously updating the eye tracking region positions 20a and 20b based on the center coordinates of the eyes detected in the eye tracking regions 20a and 20b. In addition, the size of the eye tracking areas 20a and 20b is faster when the required area is set in consideration of the amount of movement of the eye in the image capture interval, and the eye tracking performance is excellent. Can be processed.

  (F) When the eye of the driver 6 can be tracked as described above, in the next step S09, the open / close / slumber determining means 12 determines the open / closed state of the eye in the eye tracking areas 20a, 20b. In step S10, the drowsiness state of the driver 6 is determined by determining a closed eye occurrence pattern within a predetermined time. If it is determined in step S10 that the driver 6 is in a dozing state, the alarm device 4 outputs an alarm that calls the driver 6 in step S11.

(Modification)
As shown in FIG. 9, the driver's face image photographing apparatus according to the modification is further arranged in the near infrared region arranged on the optical axis of the camera 1 with respect to the driver's face image photographing apparatus shown in FIG. 1. An optical filter 9 that includes light and transmits light in a predetermined wavelength range is included. The other configuration is the same as that of the face image photographing apparatus shown in FIG. 1, and the processing operation thereof is also the same as that shown in FIG.

  In order to suppress the reflection on the lens surface of the glasses 7, it is desirable to use near infrared light that does not cause the driver 6 to feel dazzling. However, as shown in FIG. 10, the wavelength range of near-infrared illumination is narrower than the visible light range, so that the output of the near-infrared illuminator 3 needs to be large in order to counter the light in the entire wavelength range of the sun. It becomes. Therefore, in general, a near-infrared illuminator 3 having a smaller output can be obtained by setting a band-pass filter or a low-pass filter that transmits only light in the wavelength band of the near-infrared illuminator 3 on the optical axis of the camera 1. Thus, the reflection of the spectacle lens portion can be reduced. However, when the spectacles 7 subjected to antireflection processing are targeted, the effect is mainly in the range of visible light, so that in the photographed image under near infrared light, the reflection on the spectacle lens part becomes strong. Tend. This reflection can be suppressed to some extent by irradiating with strong near-infrared light, but the contrast of the eye itself is also weakened. Absent. Therefore, by changing the transmission characteristics of the optical filter 9, it is possible to improve the ability to cope with the glasses 7 having the antireflection film. That is, it is possible to efficiently reduce the reflection of the front scenery with respect to the spectacle lens to which the antireflection film (multicoat, AR coat) is applied.

  As shown in FIG. 11 (c), a low-pass filter (optical filter) 9 having a transmission characteristic that hardly transmits visible light (wavelength: 380 nm to 780 nm) and transmits near-infrared light is disposed on the optical axis of the camera 1. Consider the case. In this case, when the driver 6 wearing the glasses 7 with the antireflection film is photographed by the camera 1, as shown in FIG. 11 (a), the white reflector placed in front of the vehicle is directed to the spectacle lens surface. Reflection becomes extremely strong, and it becomes difficult to understand the shape of the eye even on the image, so naturally it is difficult to detect the position of the eye in image recognition. Even in such a state, as shown in FIG. 11B, the shape of the eye can be made to appear by irradiating near infrared light having a certain intensity.

  Next, an example in which the shape of the eye is further easily captured by image recognition by changing the characteristics of the optical filter 9 under the face image capturing conditions shown in FIG. 11A will be described with reference to FIGS. This will be described with reference to f). As shown in FIG. 11 (f), a low-pass filter (optical filter) 9 that transmits light in the red wavelength band and near-infrared light of visible light is disposed on the optical axis of the camera 1. Think about the case. In this case, when the driver 6 wearing the glasses 7 with the antireflection film is photographed by the camera 1, as shown in FIG. 11D, light in the red wavelength band is transmitted in addition to near-infrared light. By doing so, the reflection of the white reflector placed in front of the vehicle on the spectacle lens surface can be reduced as compared with FIG.

  This is due to the reflection characteristics by wavelength of the antireflection film of the glasses shown in FIG. The reflection characteristic of this antireflection film is about 0.3% or less in the entire wavelength band of visible light, and it can be seen that there is a considerable antireflection effect. The reflectance of non-coated glass and plastic on which no antireflection film is formed is about 4%. Further, the reason why the reflected light of the lens surface of the glasses that are normally subjected to antireflection treatment appears green is that the wavelength band where the reflectance is high is about 540 nm. That is, as shown in FIG. 12, the light in the red wavelength band near 650 nm whose reflectance is close to 0% does not become the reflected light of the spectacle lens part, but becomes the transmitted light that reaches the eye part through the lens. Therefore, the effect of reflection can be suppressed. Therefore, when the near-infrared light having the same output as that shown in FIG. 11B is irradiated, the shape of the eye can be further easily captured as shown in FIG.

  Note that the reflection characteristics shown in FIG. 12 are those when the incident angle to the lens surface is 0 °, and the reflected light has a characteristic that the wavelength band changes depending on the incident angle. Since the change in the wavelength band tends to become shorter as the incident angle becomes larger, for example, the light in the green wavelength band having a strong reflection intensity changes in the direction of the purple wavelength band. Even if the incident angle to the surface is about 20 °, no problem occurs by using the optical filter 9 that transmits light in the red region. In this way, for the spectacles that have been subjected to the antireflection processing that is generally used, the optical filter 9 that transmits light in the red wavelength band that is less affected by the spectacle lens portion is used. The reflection can be further effectively reduced.

(Other embodiments)
As described above, the present invention has been described by way of one embodiment and modifications thereof. However, it should not be understood that the description and drawings constituting a part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

  The near-infrared illuminator 3 shown in FIGS. 1 and 9 can be arranged at various locations in the in-vehicle layout. An example of the position of the illuminator that can indirectly irradiate near-infrared light using the roof portion of the vehicle will be described with reference to FIGS. FIG. 13 shows an example in which a near-infrared illuminator 31 is attached to the upper surface of an instrument panel centered on a meter hood. Since it is indirect illumination using the roof part 5, it is possible to attach a lighting device with a very high degree of freedom without the possibility of shadowing the handle. FIG. 14 shows an example in which a near-infrared illuminator 32 is attached to the upper surface of the center console. FIG. 15 shows an example in which the near-infrared illuminator 33 is mounted on the front pillar (A pillar). FIG. 16 shows an example in which a near-infrared illuminator 34 is attached to the upper surface of the door trim. As shown in the examples of FIGS. 14 to 16, the range in which indirect illumination using the roof portion is possible is wide, and it is possible to correspond to various vehicle layouts. FIG. 17 shows an example in which the near-infrared illuminator 35 is attached to the upper surface of the seat back of the driver's seat. In this example, due to the difference in the physique of the driver 6, it is possible to always maintain the same irradiation condition even if the seat position changes.

  Thus, it should be understood that the present invention includes various embodiments and the like not described herein. Therefore, the present invention is limited only by the invention specifying matters according to the scope of claims reasonable from this disclosure.

  As described above, according to the first aspect of the present invention, it is possible to achieve both suppression of reflection of the front scenery on the spectacle lens surface and prevention of reflection of the light emitting point of the near-infrared illuminator on the spectacle lens back surface. it can.

  According to the second aspect of the present invention, it is possible to efficiently reduce the reflection of the front scenery with respect to the spectacle lens subjected to antireflection processing (multi-coating, AR coating).

  According to the inventions of claims 3 to 7, the installation location of the near infrared illumination means is the upper surface of the instrument panel, the upper surface of the center console, the front pillar portion, the upper surface of the door trim, or the upper surface of the driver's seat back. Accordingly, it is possible to suppress the front scenery from being reflected on the spectacle lens surface, satisfy the vehicle interior layout conditions, and prevent the light emission point of the near-infrared illuminator from being reflected on the back surface of the spectacle lens.

(Reflection of light emission point of near-infrared illuminator on spectacle lens surface)
With reference to FIGS. 18 to 20, the reflection of the light emission point of the near-infrared illuminator on the front and back surfaces of the spectacle lens that occurs when the conventional method is used will be described. Reflection of the near-infrared illuminator 3 on the spectacle lens surface may occur on the lens surface or on the lens back surface. The curvature radius of the lens surface is in the range of about 80 to 400 mm, and even if the curvature radius is the smallest 80 mm, the photographing apparatus 1 and the illumination in the passenger compartment are relatively easy so that the light emitting point is not reflected on the lens surface. A vessel 3 can be arranged. As shown in FIG. 18, no reflection of the light emission point of the illuminator 3 on the lens surface (surface reflection point) occurs unless the driver 6 looks under the camera 1.

  On the other hand, the reflection of the light emitting point of the illuminator 3 on the back surface of the lens (back surface reflection point) is the relationship between refraction when near infrared light travels from the air layer to the glass layer or plastic layer and the radius of curvature of the lens back surface. As shown in FIG. 19, it occurs near the center of the lens. The position of the reflection that occurs on the rear surface of the lens usually varies depending on the power of the glasses when the glasses for correcting myopia are worn even during driving. The power of the glasses for correcting myopia is adjusted by the ratio between the radius of curvature of the lens surface and the radius of curvature of the lens back surface. Since the ratio is larger as the degree of spectacles is higher, the radius of curvature on the back surface of the lens tends to be smaller. Therefore, as shown in FIGS. 20 (a) to 20 (c), the back surface reflection point of the light emitting point of the near-infrared illuminator 3 is generated near the center of the lens as the spectacles are more powerful. It is possible to eliminate the reflection of the light emission point of the near-infrared illuminator on the back of the lens, including glasses with such a high frequency, or to move the lens to the periphery of the lens. It becomes difficult to cope with this arrangement.

It is a schematic diagram showing a driver's face image photographing device according to an embodiment of the present invention. FIG. 2A is a face image of the driver wearing the glasses photographed by the camera of FIG. 1, and the face when the front scene is reflected in a uniform state on the entire eye. An example of an image is shown, FIG. 2 (b) is a diagram showing parameter adjustment of the pre-processing of the face image of FIG. 2 (a), and FIG. 2 (c) is wearing glasses taken by the camera of FIG. FIG. 2D shows an example of a face image of a driver who has a frontal scene reflected only in the lower half of his eyes. FIG. 2D shows the face image of FIG. It is a figure which shows the parameter adjustment of pre-processing. It is a flowchart which shows the flow of a processing operation of the face image photographing device of FIG. It is explanatory drawing (the 1) regarding the detection method of an eye position, and is a figure which shows the state which divided | segmented the face image into the several vertical line. It is explanatory drawing (the 2) regarding the detection method of an eye position, and is a figure which shows the extraction point in the vertical line of FIG. It is explanatory drawing (the 3) regarding the detection method of the position of an eye, and is a figure which shows the state which grouped the extraction point which continues in an adjacent vertical line. It is explanatory drawing (the 4) regarding the detection method of an eye position, and is a figure which shows the state which divided | segmented the location where a group appears into several zones in the vertical direction. FIG. 8A to FIG. 8C are explanatory diagrams regarding the eye tracking method. It is a schematic diagram which shows the face image imaging device concerning a modification. It is a graph for comparing the width of the wavelength band of visible light and the wavelength band of near infrared light. FIG. 11 (a) shows an operation of wearing spectacles with an antireflection film photographed when a low-pass filter that hardly transmits visible light and transmits near-infrared light is disposed on the optical axis of the camera. A person's face image is shown. FIG.11 (b) shows the face image at the time of increasing the light intensity of a near-infrared illuminator on the same conditions as Fig.11 (a). FIG. 11C is a graph showing the transmission characteristics of the low-pass filter used in the imaging of FIGS. 11A and 11B. FIG. 11D shows spectacles with an antireflection film taken when a low-pass filter that transmits light in the red wavelength band and near-infrared light of visible light is disposed on the optical axis of the camera. The face image of the driver who wears is shown. FIG.11 (e) shows the face image at the time of increasing the light intensity of a near-infrared illuminator on the same conditions as FIG.11 (d). FIG. 11 (f) is a graph showing the transmission characteristics of the low-pass filter used in the imaging of FIGS. 11 (d) and 11 (e). It is a graph which shows an example of the reflection characteristic of the antireflection film given to the lens surface of spectacles. It is a schematic diagram which shows the example which installed the near-infrared illuminator on the upper surface of the instrument panel. It is a schematic diagram which shows the example which installed the near-infrared illuminator on the upper surface of the center console. It is a schematic diagram which shows the example which installed the near-infrared illuminator in the front pillar part. It is a schematic diagram which shows the example which installed the near-infrared illuminator on the upper surface of the door trim. It is a schematic diagram which shows the example which installed the near-infrared illuminator on the upper surface of a driver | operator's seat back. It is a figure for demonstrating the reflection of the light emission point of the near-infrared illuminator on the spectacle lens surface which generate | occur | produces when the conventional method is used. It is a figure for demonstrating the reflection of the light emission point of a near-infrared illuminator on the spectacle lens back surface which generate | occur | produces when the conventional method is used. 20 (a) to 20 (c) are diagrams showing how the position of the reflection changes depending on the frequency of the glasses in the reflection on the back surface of the spectacle lens shown in FIG.

Explanation of symbols

1 ... Shooting device (camera)
DESCRIPTION OF SYMBOLS 2 ... Processing apparatus 3, 31-35 ... Near-infrared illuminator 4 ... Alarm 5 ... Roof part 6 ... Driver 7 ... Glasses 8 ... Handle 9 ... Optical filter 10 ... Eye detection means 11 ... Eye tracking means 12 ... Opening / closing / Dozing determination means 20a, 20b ... tracking areas 21a, 21b ... eyes

Claims (7)

Image photographing means for photographing a face image of a driver of the vehicle;
Eye detection means for detecting the position of the driver's eyes from the face image;
A near-infrared illumination unit that reflects near-infrared light on the roof of the vehicle and indirectly irradiates the driver's face.   2. The driver's face image photographing apparatus according to claim 1, further comprising an optical filter that transmits light in a predetermined wavelength range including near infrared light disposed on an optical axis of the image photographing means.   The driver's face image photographing apparatus according to claim 1 or 2, wherein the near-infrared illumination means is installed on an upper surface of an instrument panel of the vehicle.   The driver's face image photographing device according to claim 1, wherein the near-infrared illumination means is installed on an upper surface of a center console of the vehicle.   3. The driver's face image photographing apparatus according to claim 1, wherein the near infrared illumination means is installed on a front pillar of the vehicle.   The driver's face image photographing device according to claim 1 or 2, wherein the near-infrared illumination means is installed on an upper surface of a door trim of the vehicle.   3. The driver's face image capturing apparatus according to claim 1, wherein the near infrared illumination means is installed on an upper surface of a seat back of a driver's seat.