KR19980047573A - Facial Image Imaging Device - Google Patents

Abstract

SUMMARY OF THE INVENTION An object of the present invention is to obtain a face image pickup apparatus capable of capturing the face image of a driver by suppressing the influence of the reflected light of the lens surface of the glasses as much as possible even if the driver is wearing the glasses. Eye detection means for detecting the eye of the detection subject based on the face image of the detection subject imaged by the two-dimensional imaging means, and illuminating at least the face of the detection subject with near infrared light passing through the optical filter, Near-infrared illumination means arranged so that the angle formed between the optical axis and the near-infrared light axis is equal to or greater than a predetermined angle, and excitation means for exciting the near-infrared illumination means when the eye detection means detects the eye of the detection subject.

Description

Face image pickup device

BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates to a face image image pickup apparatus for a person, and more particularly, to a face image image pickup device for use in a person state detection device that detects a state of a person to be detected from a state of a feature region of a face image by image processing a face image of a person to be detected will be.

Prior art

Conventionally, as described in Japanese Patent Application Laid-Open No. Hei 6-68500, the driver's face is photographed by a camera installed in the vehicle room, and the obtained face image is processed to extract the eye, which is the feature point of the face, and the eye of the vehicle driver's eye from the open / closed state A driver state detection device for detecting a driving state such as selling or drowsy driving is specified. The driver's face image pickup device illuminates the driver's face by placing an infrared strobe or LED near a camera having a near-infrared wavelength range. The camera which picks up the driver's face by the imaging device, such as CCD which provided the filter which cut the visible light in the front, is specified.

25 to 28 show such a conventional example, FIG. 25 is a configuration diagram of a driver state detection device including a conventional face image pickup device, FIG. 26 is a spectral reflectance characteristic diagram of various spectacle lenses, and FIG. 27 is a face image. The spectral transmittance characteristic of the visible light cut filter used for an imaging device is also an example of the face image at the time of wearing a bright state image | photographed with the conventional face image imaging device. Hereinafter, this conventional example will be described using the above drawings.

In Fig. 25, reference numeral 100 denotes a face image pickup unit, and 101 denotes an example in which a CCD is used as a two-dimensional image pickup device. 102 is an image signal processing circuit, 103 is an imaging lens, 104 is a visible light cut filter disposed on the front optical axis of the imaging lens 103, and as shown in FIG. 27, 50% transmission wavelength is 700 nm and less than this. The visible light cut filter which cuts the light of wavelength is shown.

The CCD 101, the image signal processing circuit 102, the imaging lens 103, and the visible light cut filter 104 form a camera unit 110. 105 is a light control circuit to which the contrast output of the image signal processing circuit 102 is connected, 106 is a light source in which a plurality of high-intensity near-infrared LEDs are arranged in a near-infrared light source, a halogen lamp or a light source in which a visible light cut filter is installed in front of a xenon lamp The illumination control circuit 105 and the near infrared light source 106 are integrally arranged separately from the camera unit 110. Reference numeral 120 denotes a face image processor, 121 denotes an input I / F to which the CCD imaging timing signal of the image signal processing circuit 102 is connected, 122 denotes an A / D converter to which the image output of the image signal processing circuit 102 is connected, 123 is a gate array to which the output of the A / D converter 122 is connected, or an image processing hardware (H / W) 124 which is composed of a digital signal processor (SDP) is an image memory connected to an image processing H / W, and 125 is a CPU. 126 is a ROM, 127 is a RAM, and 128 is an output interface (I / F). The A / D converter 122, the image processing H / W 123, the image memory 124, the ROM 126, the RAM 127, and the output I / F 128 are the CPU 125 and the pass 129. Is connected. Reference numeral 130 denotes a driver, and 131 denotes glasses worn by the driver.

Next, the operation will be described. The visible light component of 700 nm or less is cut by the visible light cut filter 104, and the reflected light by external light from the head of the driver 130 is cut, and the light of the near-infrared region is condensed by the imaging lens 130, and the driver 130 ) Is imaged on the CCD 101 controlled by the video signal processing circuit 102. The image signal processing circuit 102 outputs the face image of the driver 130 formed on the CCD 101 as an image signal to the A / D converter 122, and calculates the average luminance on the CCD 101. The brightness around the face of 130 is obtained, and the contrast state signal is output to the illumination control circuit 105.

Here, in the state where there is no illuminance by sunlight, such as at night or in a tunnel, imaging of the driver 130 becomes difficult. Thus, when the dark state signal is output from the video signal processing circuit 102, the illumination control circuit 105 lights up the near infrared light source 106 and illuminates the face of the driver 130.

In addition, during the day, the face is sufficiently photographed by the near-infrared component of sunlight. For this reason, the illumination control circuit 105 turns off the near-infrared light source 106 when a bright state signal is output from the video signal processing circuit 102.

The video signal of the face image of the driver 103 is A / D-converted by the A / D converter 122 to be converted into a digital harmonic image, and the image processing H / W 123 performs fine noise components through an appropriate smoothing filter. After elimination, the image is converted into a bivalent image and stored in the image memory 124. Next, the binary image of the image memory 124 is accessed using the partial image processing H / W 123 to extract the eye from the binary face image, detect the open / closed state of the eye, and open or close the eye. The drowsiness state of the driver 130 is determined from the above, and when it is determined that the drowsiness state is determined, an alarm signal is sent to the outside by the output I / F 128 to alert the driver 130. These series of operations are controlled by the CPU 125 in accordance with instructions stored in the ROM 126 in accordance with the CCD imaging timing signal, and the RAM is used for storing temporary data during control and calculation.

However, such a conventional apparatus has a problem that the eyes of a driver equipped with the glasses 131 during the day become impossible to image due to reflection from the lens of the glasses 131.

Fig. 26 shows examples of the spectral irradiance characteristics of the glass lens and the plastic lens of the spectacles. Recently, the spectacle reflectance of the spectacle lens surface is almost anti-reflective coating, and the spectroscopic reflectance of the coated spectacle lens is rapidly increased when the near-infrared region becomes as shown in comparison with the case without the coat. While driving a vehicle, the driver 130 usually runs slightly upward. In such a case, a white cloud or a part of the external scenery is reflected on the glasses 131 of the driver 130, and the face image of the driver 130 captured by the conventional imaging device 100 can be seen in glasses without a coat. Although it is difficult to see, especially in the case of coated glasses, the eye is not seen at all due to the specular lens surface reflection as shown in FIG. 28 due to the high spectral reflectance in the near infrared region of the spectacle lens.

Therefore, there is a drawback that the driver's drowsiness, one eye selling or the like cannot be detected.

This invention is made | formed in order to solve the said subject, and an object of this invention is to make it possible to capture the image of a driver's face by suppressing the influence of the reflected light of the lens surface of the glasses as much as possible even if a driver wears glasses.

It is also an object of the present invention to provide a face image image pickup device having a long device life while reducing time average power consumption.

A face image image pickup apparatus according to the present invention includes eye detection means for detecting an eye of a person to be detected based on a face image of a person to be detected captured by two-dimensional imaging means, and at least a face of the person to be detected passing through an optical filter. Infrared light illuminating means arranged so that the angle between the optical axis of the two-dimensional imaging means and the optical axis of the infrared light is equal to or greater than a predetermined angle while illuminating with infrared light, and the infrared illuminating means when the eye detecting means detects the eye of the subject to be detected. It is provided with an excitation means to excite.

Furthermore, the face image image pickup apparatus according to the present invention has a light and dark detection means for detecting the brightness of the surrounding or near the face of the detection subject and detecting whether the surrounding or the face near the detection subject is in a bright or dark state. The excitation means is to excite the infrared light illuminating means when the contrast detecting means detects a bright state and the eye detecting means does not detect the eye of the subject to be detected.

In addition, the face image image pickup apparatus according to the present invention determines a light state or a dark state on the basis of whether or not the volume of the image including the face of the person to be detected captured by the two-dimensional imaging means is equal to or greater than a predetermined luminance. Contrast detection means is provided.

Furthermore, the face image image pickup apparatus according to the present invention includes excitation means for stopping the infrared light illuminating means once when the predetermined time has elapsed since the infrared light illuminating means is excited.

Further, the face image image pickup apparatus according to the present invention has eyeglass detecting means for detecting the presence or absence of eyeglasses attached to the detection subject, wherein the excitation means includes the eyeglass detection means detects the glasses and the eye detection means detects the eye of the subject. In this case, the infrared illuminating means is excited.

Further, the face image image pickup apparatus according to the present invention has a dark state infrared light illuminating means for illuminating the face of the subject to be detected with infrared light passing through the optical filter when the contrast detecting means detects a dark state, The infrared light illuminating means and the infrared light illuminating means are separated from the infrared light illuminating means which are excited when the eye of the subject is detected.

In addition, the face image image pickup apparatus according to the present invention includes two-dimensional image pickup means for picking up a predetermined area including a face of a person to be detected, a first pass band for passing visible light through a predetermined wavelength range, and a predetermined wavelength or more. An optical filter having a second pass band through which external light passes and disposed on the optical axis of the two-dimensional imaging means.

In the face image pickup apparatus according to the present invention, the second pass band of the optical filter passes only infrared light in a predetermined wavelength range.

Further, the face image image pickup apparatus according to the present invention has a first pass band passing through visible light in a predetermined wavelength range and a second pass band passing infrared light over a predetermined wavelength, and at the same time the optical axis of the two-dimensional imaging means. An optical filter disposed thereon, eye detection means for detecting the eye of the detection subject based on the face image of the detection subject captured by the two-dimensional imaging means, and at least the face of the detection subject with infrared light passing through the optical filter; Infrared illumination means arranged so that the angle between the optical axis of the two-dimensional imaging means and the optical axis of the infrared light is equal to or greater than a predetermined angle, and the infrared illumination means when the eye detection means detects the eye of the subject to be detected. Excitation means is provided.

In addition, the face image image pickup device and the optical filter according to the present invention are constituted as a first optical filter having a first pass band and a second optical filter having a second pass band. Contrast detecting means for detecting whether the light is in a bright or dark state around or near the detection subject, and the first optical filter on the optical axis of the two-dimensional imaging means when the light detecting means detects the bright state. And the two-dimensional imaging means includes a filter replacement means for arranging the second optical filter on the optical axis when the contrast detection means detects the dark state.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram of a driver state detection apparatus including a face image pickup apparatus according to the first embodiment.

2 is a flowchart of driver state detection in Embodiment 1. FIG.

3 is a schematic diagram of a face image of a driver wearing glasses.

4 is an explanatory diagram of eye setting area setting.

5 is an explanatory diagram of light reflection on the face of a driver;

Fig. 6 is an explanatory diagram of a face image at the time of wearing glasses in a bright state picked up by the face image pickup device according to the first embodiment;

7 is a flowchart relating to eye extraction.

8 is an X-axis bar graph of the eye presence region.

9 is an X-axis bar graph of the candidate region.

10 is a block diagram of a video signal processing circuit.

11 is a control characteristic diagram of an automatic gain control circuit.

12 is a flowchart of driver state detection in Embodiment 4. FIG.

Fig. 13 is a flowchart of eyeglass detection of Example 4;

Fig. 14 is a block diagram of a driver state detection device including the face image pickup device according to the fifth embodiment.

15 is an explanatory diagram of a face image in a dark state.

Fig. 16 is a block diagram of a driver state detection device including the face image pickup device according to the sixth embodiment.

17 is an external perspective view of the image capturing unit (a) of the sixth embodiment;

18 is a sectional view of an image pickup section a in Example 6. FIG.

19 is a spectral transmittance characteristic diagram of a composite optical filter.

20 is a spectral transmittance characteristic diagram of the composite optical filter of Example 7. FIG.

21 is a block diagram of a driver state detection device including the face image pickup device according to the tenth embodiment;

Fig. 22 is a sectional view of an image capturing section a in Example 10;

Fig. 23 is a perspective view of the filter replacer of the tenth embodiment;

24 is a spectral transmittance characteristic diagram of one optical filter of Example 10;

Fig. 25 is a configuration diagram of a driver state detection device including a conventional face image pickup device.

26 is a spectral reflectance characteristic diagram of various spectacle lenses.

Fig. 27 is a spectral transmittance characteristic diagram of a visible light cut filter used in a conventional face image pickup apparatus.

Fig. 28 is an explanatory diagram of a face image at the time of wearing glasses in a bright state photographed by a conventional face image pickup apparatus;

* Description of the symbols for the main parts of the drawings *

a: face image imaging unit b: face image processing unit

c: Camera unit g: Glasses

p: Driver 1: CCD

2: Image signal processing circuit 3: Image lens

4: visible cut filter 5: lighting control circuit

6: near-infrared light source 7: illuminance sensor

8: near-infrared LED 9: LED control circuit

10: pressure I / F 11: A / D converter

12: Image processing H / W 13: Image memory

15: CPU 16: ROM

17: RAM 20: CCD control circuit

21: Gain amplifier 22: AGC control circuit

23: amplifier circuit 24: buffer

25, 26: Non-church 27: Electronic shutter control circuit

26: first optical filter 47: second optical filter

60: filter replacement means 101: CCD

102: video signal processing circuit 103: imaging lens

104: visible cut filter 105: light control circuit

106: near-infrared light source 121: input I / F

122: A / D converter 123: Image processing H / W

124: image memory 125: CPU

126: ROM 127: RAM

127: output I / F

129: Pass

The face image image pickup apparatus according to the present invention excites an infrared light illuminating means to illuminate the face of the detecting subject when the eye detecting means detects the eye of the detecting subject.

Further, the face image image pickup apparatus according to the present invention excites and detects infrared light illuminating means when the contrast detecting means detects a bright state and the eye detecting means does not detect the eye of the subject to be detected. Illuminate the subject's face.

Further, the face image image pickup apparatus according to the present invention determines the bright state or the dark state based on whether or not the luminance of the image including the face of the subject to be imaged by the two-dimensional image pickup means is equal to or higher than a predetermined luminance.

Further, the face image image pickup apparatus according to the present invention stops the infrared light illuminating means once when the predetermined time elapses after the infrared light illuminating means is excited.

Further, according to the present invention, the face image image pickup apparatus excites infrared light illuminating means when the eyeglass detecting means detects the eyeglasses and the eye detecting means does not detect the eye of the subject to be detected and the face of the subject to be detected. To illuminate.

Further, the face image image pickup apparatus according to the present invention has a dark state infrared light illuminating means for illuminating the face of the subject to be detected with infrared light passing through the optical filter when the contrast detecting means detects a dark state, and the dark state The infrared light illuminating means and the infrared light illuminating means which are excited when the eye detecting means detects the eye of the detection subject function separately.

In addition, the face image image pickup device according to the present invention passes visible light in a predetermined wavelength range and infrared light having a wavelength greater than or equal to an image, and picks up a face image of a person to be detected.

Further, the face image image pickup apparatus according to the present invention picks up the face image of the person to be detected by only visible light in a predetermined wavelength region and infrared light in a predetermined wavelength region.

In addition, the face image image pickup apparatus according to the present invention captures the face image of the person to be detected by the visible light having a predetermined wavelength and the infrared light having a predetermined wavelength or more, and the eye cannot be detected from the face image. The external light luminaire is excited to illuminate the face of the subject.

In addition, the face image image pickup apparatus according to the present invention arranges the first optical filter on the optical axis of the two-dimensional image pickup means when the contrast detection means detects a bright state, and simultaneously detects the dark state of the contrast detection means. When present, the second optical filter is disposed on the optical axis of the two-dimensional imaging means.

Example 1

1 to 9 show one embodiment of the face image pickup apparatus of the present invention, FIG. 1 is a configuration diagram of a driver state detection apparatus including a face image pickup apparatus, FIG. 2 is a flowchart of driver state detection, and FIG. 4 is a schematic diagram of a face image of a driver wearing glasses, FIG. 4 is an explanatory diagram of setting an eye presence area, FIG. 4 is an explanatory diagram of light reflection on the face of the driver, and FIG. 6 is an image captured by the face image imaging apparatus of the present invention. An example of a face image when wearing glasses in a bright state, FIG. 7 is a flowchart relating to eye extraction, FIG. 8 is an X-axis bar graph of an eye present region, and FIG. 9 is an X-axis bar graph of a candidate region. Hereinafter, this embodiment is described as FIG. 1.

In Fig. 1, a denotes a face image pickup unit, b denotes a face image processing unit, and c denotes a camera unit. In the figure, 1 denotes a total of 380,000 pixels of 768x493 in a CCD as a two-dimensional imaging means for imaging a predetermined area including the driver's face. 2 is an image signal processing circuit for processing signals from the CCD 1, 3 is an imaging lens provided on the optical axis in front of the CCD 1, 4 is an imaging lens provided on the optical axis in front of the imaging lens 3; As a visible light cut filter as an optical filter for cutting visible light incident on (3), the visible light cut filter 4 has characteristics as shown in FIG. The camera unit c, which is composed of the CCD 1, the image signal processing circuit 2, the imaging lens, and the visible light cut filter 4, is disposed on a driver's seat or on an instrument panel unit. The predetermined area | region containing the driver's face is image | photographed in the direction from which the face vertical direction becomes 768 pixels from the front. It is most advantageous for the eye area extraction to make the photographing angle at this time slightly inclined to the front of the face.

5 is an illumination control circuit serving as an excitation means for exciting or regularizing an infrared light source by receiving an output from a CPU to be described later, and 6 is excited by an illumination control circuit 5 to near-infrared light in a direction including the driver's face or its surroundings. It is a near-infrared light source as an infrared light illuminating means for irradiating light. In the near infrared light source 6, the specular reflected light r reflected by the lens of the driver's glasses is not directly incident on the meteorological lens 3 of the camera part c by the near infrared light irradiated from the near infrared light source 6. It is installed in a location that does not. Specifically, the angle between the optical axis of the near-infrared light irradiated by the near-infrared light source 6 and the optical axis of the camera part c is at least about 20 to 30 degrees or more so as to be at a distance of up, down, left and right from the camera part c. The near-infrared light source 6 is disposed at the position.

7 is an illuminance sensor as a contrast detecting means for detecting the brightness of the driver's surroundings or the liquid level to detect whether it is a bright state (bright state) or a dark state (dark state), wherein the illuminance sensor 7 is the environment or face of the driver. It is installed in places such as a dash port or the rear window bottom to detect the brightness of the light.

The face image processing unit b is composed of the following. In the drawing, reference numeral 10 denotes input I / F which receives various signals such as a CCD imaging timing signal or shutter signal of the image signal processing circuit 2 and an output signal from the illuminance sensor 7, and 11 denotes an image signal processing circuit. (2) A / D converter to which image output is connected, 12 is an image processing hardware (H / W) consisting of a gate array or digital signal processor (DSP) to which the output of A / D converter 11 is connected, 13 is an image memory connected to the image processing H / W 12, and 14 is an output I / F that outputs a command from the CPU 15 which performs arithmetic processing. The above is connected.

Reference numeral 16 denotes a ROM which stores various programs or numerical values, 17 denotes a RAM for temporarily storing a value being calculated, 18 denotes an output I / F, and is connected to various subsequent devices. In addition, the input I / F 10, the A / D converter 11, the image processing H / W 12, the image memory 13, the ROM 16, the RAM 17, and the output I / F 18 Is connected to the CPU 15 and the path 19. 2 is a program of driver state detection. Some face imaging apparatuses perform lighting control of a near-infrared light source in a dark state such as a night light, but in Embodiment 1, detailed description thereof will be omitted.

2, the operation of the first embodiment will be described. As the image input means of step ST 10, the video signal VOUT of the original image shown in Fig. 3 captured by the CCD 1 is converted into a digital gradation image signal and output to the image processing H / W 12, In the image processing H / W 12, the noise component is eliminated through an appropriate smoothing filter. As the floating binarization means in step ST20, the floating binarization is performed by the image processing H / W 12 and converted into a binary image, and the pixel length slightly longer than the width in the vertical direction of the eye is a predetermined length. In step ST21, the black block which is longer than the predetermined length in the face lengthwise direction is removed, and the bivalent image 201 in which the areas of the hair which differ greatly in size are individually removed as shown in Fig. 4 is obtained in step ST22. It is input to and stored in the memory 13. Next, the eye extracting means ST30 accesses the bivalent image 201 of the image memory 13 using the partial image processing H / W 12 to extract eyes from the face image. In step ST30, the eye extraction routine described later is performed. When the eye extraction routine ends in the eye extraction means as the eye detection means in step ST30, it is determined in step ST31 whether the eye extraction flag is ON or not.

Here, as shown in Fig. 28, the eye detection flag is in the OFF state because no eye is detected in the eye extraction routine ST30 in the face image where the eye is not visible at all by specular lens surface reflection. If the eye detection flag is OFF, the flow advances to step ST32 to read the light intensity signal of the illuminance sensor 7 from the input I / F 10, and detect whether or not the outer world is brighter than the light intensity signal. Here, when it is determined that the external light is bright, the reason why the eye is not extracted is the surface reflection of the spectacle lens, and at step ST40, the light for removing the specular reflection to the illumination control circuit 5 rather than the output I / F 14 at step ST40. The control signal is sent to illuminate and control the near-infrared light source 6 to illuminate the driver's face or the vicinity including the face.

Now, the reason for irradiating near-infrared light to the driver's face when the eye cannot be detected by the surface reflection of the spectacle lens will be described.

5 shows the luminous flux at the external object i; i) and the luminous flux from the near infrared light source 6 When L) is illuminating the face of the driver wearing the glasses g, it is a figure explaining each reflection and transmitted light beam in the lens of the eye e and glasses g, and the camera part c in the figure. The luminous flux entering) is expressed by the following equation.

[Equation 1]

= iR + iT + LT

From here iR is the speed of light Reflected light beam by the glasses g of I),

iT is the speed of light ( Diffused reflected light in the eye e of the driver p after transmission of the glasses g of I), LT is the luminous flux of the near-infrared light source 6 It is the diffuse reflecting light beam in the eye (e) as L).

As described above, the near-infrared light source 6 is disposed so that the angle formed by the optical axis of the irradiated light and the optical axis of the camera unit c becomes equal to or greater than a predetermined angle, and thus, the emission light beam of the near-infrared light source 6 as described above. ( Reflected light beam by the lens of the glasses g of L) LR does not enter the camera portion c.

When the image luminance on the CCD 1 is described by dividing it into the reflected light flux in the glasses g and the diffuse reflection light in the eyes c, the brightness due to the reflected light flux in the glasses g ( R), the luminance due to the diffuse reflected light flux in the eye e T) is the same as Equation 2, respectively.

[Equation 2]

R = ∑ _λ iR (λ) F (λ) R (λ)

T = ∑ _λ ( iT (λ) + LT) F (λ) R (λ)

Here, F (λ) is the spectral transmittance characteristic of the optical filter 4, and R (λ) is the spectral sensitivity characteristic of the CCD 1. Luminance due to the diffuse reflection light beam in the eye e within the entire luminance of the image ( The ratio K of T) is given by the expression (3).

[Equation 3]

K = T / ( R + T)

That is, the larger the ratio K, the better the eye e can be observed.

By the way, as apparent from Equation 3 herein, the reflection in the glasses g ( R) When the ratio with respect to T is large and an eye cannot see, it is by lighting of the near-infrared light source 6 By adding LT T's The ratio for R becomes large.

In other words, even if there is reflection from the spectacle lens, the application of near-infrared light K makes it possible to clearly observe the eye.

FIG. 6 shows an image of illuminating the face of the driver p by turning on the near-infrared light source 6 as described above in the image state of FIG. 28 in which eyes are not observed by the lens surface reflection of the glasses g. As shown, eyes are clearly observed by illumination by the near-infrared light source 6, as shown.

If it is determined in step ST32 that N is in the dark state, the process proceeds to step ST41 in this case and performs illumination control in the dark state.

By the way, after excitation of the near-infrared light source 6 in order to remove the influence of the reflection of the glasses g next, the main routine returns to step ST10 to perform the above-described processing sequentially. In the next step ST31, since the eye is detected, it is determined as Y, and the process proceeds to step ST33 this time.

Subsequent steps ST33 to ST35 are processes capable of stopping the excited near-infrared light source 6.

In other words, when the eye cannot be detected, the near-infrared light source 6 is excited at step ST40, whereby the eye can be detected irrespective of the reflection of the glasses g. By the way, the reflection of the glasses g does not continue for a long time, but is relatively short for a change in the surrounding situation.

Thus, in steps ST33 to ST35, the irradiation of infrared light is once stopped after a predetermined time after excitation of the near infrared light source 6, and when the eye is detected in the next step ST31, the reflection of the glasses g is reduced. It is judged that the disappearance of the near-infrared light source 6 can be continued, and if the eye cannot be detected, it is determined that the reflection of the glasses g is continued and the near-infrared light source 6 is continued in step ST40. Here again.

Specifically, when the eye detection plug is confirmed to be turned on at step ST31, it is determined at step ST33 whether or not the glasses reflection removing illumination of the near infrared light source 6 is performed, and the glasses reflection removing illumination is performed. If present, the elapsed time of illumination is checked at step ST35, and when the predetermined time has elapsed, the glasses reflection removing illumination is turned off at step ST35. Such predetermined time may be within several minutes in consideration of the duration of specular reflection. This method minimizes the use time of the near-infrared light source 6 in the state where the external light is bright.

When the driver's eyes can detect as described above, the driver's state is detected based on this.

The blinking means of step ST50 detects the blinking of the driver p from the opened / closed state of the extracted eye e of the driver p. In the subsequent step ST60, the drowsiness determining means determines the drowsiness based on the state of such blinking, and alerts the driver p by the warning means ST70 in accordance with the drowsiness state and wakes up.

By the way, the eye extraction routine of the eye extraction means ST30 omitted above will be described.

In Fig. 7, first, in step ST301, the binary levels of the binary image 201 are accumulated in the X-axis direction and the Y-axis direction, respectively, to show the image X-axis bar graph SUMX and Y-axis bar graph SUMY. Obtained as shown in Fig. 4, the center positions XFC and YFC of the bar graphs SUMY and SUMX are obtained, and this is set as the face center coordinates FC (XFC and YFC), and from the face center FC at step ST302. The candidate existence area 202 to the right of the X axis direction length ECAH and the Y axis direction length ECAW is set starting from the point PER separated from XECA in the X axis direction and YECA in the Y axis direction. The candidate presence area 202 on the left of the same size is set based on (PEL).

Subsequently, in step ST303, as shown in FIG. 8, the Y-axis bar graph SUMY in the candidate existence region 202 is obtained, and the region where SUMY is greater than or equal to the predetermined threshold value SHL is determined. )

In the drawing, the area band BER1 corresponding to the eyebrow, the area band BER2 corresponding to the eyeglass frame, and the area band BER3 corresponding to the eye are registered as the candidate area band 203. 8 shows one candidate present region 202, and of course, the other candidate present region 202 is similarly processed.

In step ST304, as shown in FIG. 9, the X-axis bar graph SUMX in each candidate region band 203 is obtained, and in ST305, an area equal to or greater than another predetermined threshold SHL such that SUMX is the same. It is set as the candidate area 204. In the drawing, an area BER11 corresponding to the eyebrow, an area BER21 corresponding to the eyeglass frame, and an area BER31 corresponding to the eye are registered as the candidate area 204. In step ST306, the maximum value SUMXMAX, the deviation from the maximum value SUMXMAX-SUMX, and the like are determined from the registered X-axis bar graph SUMX of each candidate area 204, and each candidate area ( In step 204, the eye evaluation function is calculated. As shown in the figure, the SUMX of the IC eye region has a feature that both the maximum value and the dispersion of the deviation are large compared with other regions.

Next, in step ST307, the candidate areas 204 are called out one by one, and in step ST308, it is determined whether the evaluation function is within a predetermined range shown by the eye, and the candidate area 204 is not determined to be eye. In step ST309, the candidate area 204 is incremented, and the same operation is performed on the next candidate area. If the candidate area 204 to be determined in step ST307 disappears, the eye detection flag is turned off in step ST312, and the processing returns to the main routine. If there is a candidate region 204 determined to be eye at step ST308, at step ST310, the candidate region 204 at the innermost lower side of the candidate region 204 determined to be eye is used as the eye. In the same manner, the eye detection flag is turned on in step ST311 to end the eye extraction routine.

As described above, according to this embodiment, the eye can be easily observed even when an external scenery is reflected and reflected on the glasses g worn by the driver p.

In addition, since the near-infrared light source 6 does not need to be used in the state in which the external view is not reflected on the glasses g, the use time of the near-infrared light source 6 is suppressed as much as possible, thereby reducing the time average power consumption. There is an advantage that the life of the near infrared light source 6 can be extended.

(Example 2)

In the first embodiment, the case where the illuminance sensor 7 is used as the contrast detecting means is shown. However, the contrast can be determined even when the illuminance sensor 7 is not used. That is, you may take the method of determining the contrast from the brightness | luminance of the vicinity of the driver p, ie, the brightness correspondence value of the face image of the driver p, without using the external contrast state for contrast detection.

FIG. 10 is a block diagram of the video signal processing circuit 2, and FIG. 11 is a control characteristic diagram of an auto gain control circuit.

First, the operation of the video signal processing circuit 2 will be described.

In Fig. 10, the CCD control circuit 20 controls the imaging timing and the image accumulation time (hereinafter referred to as shutter time) of the CCD 1 and simultaneously outputs the captured image as the video signal VOUT. The AGC control circuit 22 calculates the image average luminance L by integrating the output of the gain variable amplifier 21 and obtains the AGC control voltage VAGC according to the image average luminance L as shown in FIG. 21), the gain of the gain variable amplifier 21 is controlled so that the AGC control voltage VAGC becomes a predetermined target control voltage. The video signal VOUT whose image average brightness is controlled by the gain variable amplifier 21 is input to the A / D converter 11 of the face image processing section b and converted into a digital gradation image.

On the other hand, the AGC control voltage VAGC corresponds to the voltage V1 and the upper limit luminance Lh corresponding to the lower limit luminance L1 shown in FIG. 11 in the comparison circuit 25 and the comparison circuit 26, respectively. (Vh), each comparison result is input to the electronic shutter control circuit 27. The sun's altitude, the direction, the weather, the presence or absence of shadows, etc., changes the brightness of the outer world with respect to the vehicle, so that the image average luminance L of the face image of the driver p is controlled by the AGC control circuit 22 (V1). For example, when the luminance L is less than the lower limit luminance L, the electronic shutter control circuit 27 increases the shutter time by one step, and conversely, the upper limit luminance Lh. If the shutter speed is higher than 1, the shutter time of the CCD 1 in the CCD control circuit 20 is adjusted so that the AGC control is always established, that is, the AGC control voltage VAGC is within the range of V1 to Vh. While controlling from 1/60 to 1/10000 second in multiple layers, the shutter step of development is sent to the input I / F 10 of the face image signal processing unit b as the shutter signal SS.

Furthermore, the AGC control voltage VAGC corresponding to the image average luminance L is sent to the A / D converter 11 as the voltage signal VAGC through the amplifying circuit 23 and the buffer 24.

By the way, in Example 1, although the detailed description is abbreviate | omitted, illumination control in the dark state is performed as follows.

In Fig. 1, the CPU 15 reads the contrast signal of the illuminance sensor 7 and the shutter signal SS from the electronic shutter control circuit 27, rather than the input I / F 10, and at the same time the A / D converter ( 11), when the AGC control voltage (VAGC) corresponding to the image average brightness is read and it is determined that the external world is darker than the contrast signal, the predetermined time period in which the AGC control is within the control range is always in the range of 1/60 second. The amount of emitted light is calculated and the illumination control signal is sent from the output I / F 14 to the illumination control circuit 5 to light up the near-infrared light source 6.

Alternatively, in the case where it is determined that the outer world is darker than the contrast signal, although not shown, a control signal is sent from the CPU 15 to the electronic shutter control circuit 27 through the output I / F 14 to set the shutter time to 1 /. It is better to turn on the near-infrared light source 6 after fixing in 60 second.

However, since the video signal processing circuit 2 operates as described above, the illuminance sensor 7 is omitted using this.

For example, when the shutter speed becomes the minimum 1/60 second and the AGC control voltage (VAGC) cannot be controlled as the target control voltage of the AGC control voltage, and the deviation is large, the image accumulation for receiving light is accumulated. The longest time and the maximum amplification of the obtained image signal have not reached the target control voltage, and therefore it can be determined that the outer space is dark in this case.

In addition, the determination of the state where the external light is bright is made by the shutter speed being shorter than 1/60 second. In other words, this state allows the AGC control voltage VAGC to match the target control voltage even if the image accumulation time is shorter than 1/60 sec.

As another method of determining the bright state, the state in which the shutter speed is 1/60 second and the near infrared light source 6 is not lit may be determined as the bright state.

That is, the CPU 15 has described the purpose of illuminating the near-infrared light source 6 with an arbitrary light emission amount when the shutter speed is 1/60 second and the AGC control voltage VAGC does not reach the target control voltage. However, as described above, the shutter speed is 1/60 second and the state in which the near-infrared light source 6 is not turned on is aimed at the AGC control voltage VAGC even if the near-infrared light source 6 is not turned on. It is a state which can be matched with a voltage, and it can be determined that an external field is bright enough.

Therefore, also in Example 2, the same effect as Example 1 is acquired, and the special sensor which determines the brightness | luminance, such as the illumination intensity sensor 7, becomes unnecessary.

Example 3

The third embodiment relates to a face image image pickup device that eliminates the influence of reflection of the driver's glasses, and particularly, to obtain a simple face image image pickup device by eliminating the contrast detection means.

That is, in Example 3, without using the contrast detecting means, in step 2, when the eye is not detected in step ST31, step 32 is omitted so that the near-infrared light source 6 is turned on. As a result, near infrared light is always irradiated when the driver's eyes cannot be detected regardless of the external condition.

By the way, in the above-described embodiment, the near-infrared light source 6 is excited only when the driver's eyes cannot be detected in the bright state in daylight, and such control is performed in the dark state such as at night. In Example 3, the same control is performed even in a dark state such as at night.

In other words, in the above-described embodiment, since the contrast state is detected, the driver does not irradiate the driver with near-infrared light for removing the influence of the spectacle lens reflection when the driver is selling at a glance in the dark state. There is a demerit that irradiates near-infrared light if the eye cannot be detected, even in dark conditions without the doctor's concern.

However, by the configuration as described above, the device can be miniaturized and simplified, and thus can be inexpensive.

Therefore, according to the third embodiment, it is possible to obtain a simple face image pickup device which is not affected by spectacle lens reflection.

Example 4

The fourth embodiment is a modification of the first embodiment. When the eye cannot be detected, and the eyeglass frame is detected, it is determined that the eye cannot be detected by the surface reflection of the eyeglass lens, and the near infrared light is detected by the driver. Investigate on the face.

12 and 13 show a fourth embodiment of the present invention, FIG. 12 is a flowchart of driver state detection in the fourth embodiment, and FIG. 13 is a flowchart of glasses detection. Hereinafter, such an embodiment will be described with reference to the drawings, referring to FIGS. 1 and 9.

In Fig. 12, after executing from step ST10 to the eye extracting means of step ST30, it is determined whether or not the driver p is wearing glasses g by the eyeglass detecting means of step ST80.

The detecting method of the glasses by the glasses detecting means will be described later in detail using a flowchart.

Next, when the eye detection flag is checked in step ST31, and the eye detection flag is OFF, the eyeglass detection flag is subsequently checked in step ST81, and when the eye detection flag is ON, the driver wears glasses. This is because the eye cannot be detected, and the reason why the eye is not extracted is that the reflection of the surface of the spectacle lens causes the lighting control circuit 5 from the output I / F 14 to step ST40. The near-infrared light source 6 is turned on by sending an illumination control signal for removing specular reflection.

When the eye detection plug is ON in step ST31, steps ST33 to 70 are executed as in the first embodiment.

Next, the operation of the glasses detecting means will be described.

Eyeglass detection first calculates the spectacle frame evaluation function using the X-axis bar graph SUMX of the candidate region 204 of the first embodiment described above in step ST801 of FIG. As shown in Fig. 9, the SUMX of the spectacle frame has almost the same flat characteristics as the eyebrows, so that the dispersion of the deviation SUUXUAX-SUUX is much smaller than the eye. In addition, the width EAW of the candidate area corresponding to the spectacle frame is slightly longer than that of the eyebrow. The spectacle frame evaluation function is determined in consideration of these features. Next, in step ST802, it is determined whether or not the candidate area 204 other than the same as defined in step ST30 exists, and when the candidate area 204 exists, the step ST803 Based on the spectacle frame evaluation function, it is determined whether or not the candidate region 204 is the spectacle frame, and if it is determined that the spectacle frame is not the spectacle frame, the candidate region 204 is incremented in step ST804 to perform the same operation on the next candidate region. . If the candidate area 204 to be determined in step ST802 has disappeared, the glasses detection flag is turned off in step ST809, and the processing returns to the main routine.

Next, in step ST805 and step ST802, the number of candidate areas 204 determined to be glasses in the left and right candidate presence areas 202 is examined. If either of the glasses is a genuine pair of glasses, the glasses detection flag is turned on in step ST808. If the number of the eye candidate areas 204 determined to be glasses in the candidate existence area 202 is 1 or less, in order to distinguish them from the eyebrows, the SUMX and the EAW of the candidate area 204 in step ST806. The eyebrow evaluation function using the back and the like is calculated to determine whether or not the eyebrow evaluation function of the candidate region 204 is within a predetermined range at step ST807, and the glasses at step ST808 only when it is not within the predetermined range. If the detection flag is turned on and the eyebrow is likely to be in the predetermined range, the glasses detection flag is turned off in step ST809, and the processing is returned to the main routine.

This can determine whether or not the driver is wearing glasses.

Therefore, also in this embodiment, the same effect as in Example 1 is obtained, and by confirming the presence of the glasses, the reliably near-infrared light source 6 can be identified on the image only when it is thought that the eye cannot be seen by spectacle lens reflection. ) Can be turned on, and the eye can be clearly detected and the time average power consumption can be reduced.

Example 5

Conventionally, in order to compensate for the lack of illuminance in the dark state, there exist some near-infrared light illuminating means for a dark state.

On the other hand, in the above-described embodiment, the near-infrared light source 6 for removing the influence of spectacle lens reflection for reliably detecting the driver's eyes is provided, and the near-infrared light source 6 is a spectacle lens doctor. In addition to the near-infrared lighting means for removing the influence of the light, the infrared-light illuminating means for the dark state can also serve as changing the control program of the lighting control circuit 5.

However, there is a large difference between the amount of emitted light of infrared light for extracting eyes from a face image in a dark state and the amount of emitted light of infrared light for removing the influence of specular lens reflection in daylight. Therefore, when both are combined, the infrared light of the amount of emitted light is irradiated more than necessary in a dark state, and electric power is wasted.

Therefore, in Example 5, time average power consumption is reduced by separating and functioning both.

14 and 15 show a fifth embodiment, FIG. 14 is a configuration diagram of a driver state detection device including a face image pickup device, and FIG. 15 is an example of a face image in a dark state.

In FIG. 14, 8 is a coaxial near-infrared light source which is arranged so that the emission optical axis of near-infrared light is near to and close to the imaging optical axis of the camera part c near the camera part c, where light output as a whole is It has a low power of about 10mW or less and shows the case of using near-infrared LED of about 900nm in center wavelength. 9 is an LED control circuit, and the shutter signal SS and the AGC control voltage VAGC of the video signal processing circuit 2 shown in FIG. 10 are connected. In addition, the shutter signal SS is connected to the input I / F 10, and the video signal output VOUT and the AGC control voltage VAGC are connected to the A / D converter 11, and the illuminance sensor 7 is not used. Do not.

Next, the operation will be described.

When the outer field becomes dark and the luminance on the driver p decreases, the AGC control circuit 22 and the electronic shutter control circuit 27 in FIG. 10 work to decrease the shutter time. The LED control circuit 9 judges that it is dark when the shutter time becomes 1/60 second and the AGC control voltage becomes lower than or equal to a predetermined value, and turns on the near-infrared LED 8. Since the near-infrared LED 8 has almost the same coaxial axis as the imaging optical axis of the camera unit c, as shown in FIG. 15, only the pupil of the driver p can be imaged brightly. The pupil has the property of reflecting the incident light in almost the same direction, so that the brightness of the coaxial coarse brightness is remarkably large compared to other parts of the face, and thus the near-infrared LED is weak enough that the operation of the face cannot be captured as an image. Even when using (8), only the pupil is brightly captured as shown in the drawing. At this time, the luminance of the image is only about 7 to 8 times the maximum of the pupil of the size is large, the average luminance is hardly increased. Although not shown, in this dark state, since only a pair of pupils of the driver p are binarized as the back level in step ST20 of the main routine, the CPU omits step ST21 of FIG. The eye is blinked by detecting the pair of pupils based on a separate algorithm and evaluation function in the bright state in the eye extraction means (ST30), and hiding by the eyelids when the pupils blink. It detects and performs drowsiness judgment similarly.

In the present embodiment, as described in the above embodiment, it is determined that the state is bright when the shutter speed is shorter than 1/60 second or when the shutter speed is 1/60 second and the AGC control voltage VAGC is higher than or equal to a predetermined value. do.

Then, when the eye is not detected as in Example 1 in the image in the bright state, the near-infrared light source 6 is turned on or glasses are detected as in Example 4 in the image in the bright state and the eye is detected. If not, the near-infrared light source 6 is turned on.

Therefore, according to the fifth embodiment, the synchronism is effective with the above-described embodiment, and the near-infrared light 8 for low power consumption and the near-infrared light source 6 for removing the effects of spectacle lens reflections are usually used in a dark state. By dividing, the time-averaged power consumption of the entire imaging device can be further reduced, and the use of the near infrared light source 6 is limited, which makes it possible to prolong the life of the near infrared light source 6.

In the fifth embodiment, the near-infrared LED 8 captures the pupil of the driver p in a dark state, but the radiation light axis is close to the imaging light axis of the camera part c in close proximity to the camera part c. The pupil may be imaged using another near-infrared light source arranged so as to be close to each other.

Example 6

By arranging the optical filter on the optical axis of the camera of Example 6, the eye can be reliably detected even if opposition by the spectacle lens occurs.

16 to 19 show the embodiment of the sixth embodiment, FIG. 16 is a configuration diagram of a driver state detection device including the face image image pickup device according to the sixth embodiment, FIG. 17 is an external perspective view of the image pickup unit a; 18 is a cross-sectional view of the imaging unit a, and FIG. 19 is a spectral transmittance characteristic diagram of the composite optical filter.

In the figure, 41 and 42 are optical filters arranged on the front surface of the imaging lens 3 on the imaging optical axis of the camera unit c, and 50 are the CCD 1 of the imaging unit a, the imaging lens 3, and the optical. As a housing for supporting the filters 41 and 42, the near-infrared LED 8 is also supported in such a housing. 51 is a printed circuit board on which the CCD 1, the image signal processing circuit 2, and the LED control circuit 9 are arranged, 52 is the output lead of the imaging unit a, and the near-infrared LED 8 is the same as that of the fifth embodiment. As shown in Fig. 17, the LED having a central wavelength of about 900 nm is symmetrical to the rear side of the optical filter 41 near the aperture 31 of the lens and substantially parallel to the imaging optical axis of the camera portion c and the emission optical axis of near infrared light. It is installed in four places. However, the near-infrared LED 8 is different from that in FIG. 17 and may be disposed without the optical filter 41 inserted.

Next, the optical filter will be described.

As shown in FIG. 19, the optical filter 41 is a long wavelength transmission filter (LPF) as shown by the dotted line of the figure, and shows the example using LPF whose wavelength of 50% transmittance | permeability is about 500 nm here, and visible light And light having a wavelength of at least 500 nm including infrared light. In addition, the optical filter 42 is a band-rejection filter (BIRF), in which a wavelength of 50% of transmittance is set at 650 nm and 850 nm to remove the wavelength band between the 650 nm to 850 nm between the visible light and the near infrared light. An example using is shown. The BRF 42 uses a dielectric multilayer film filter in which a dielectric film is laminated in multiple layers on an optical transparent substrate such as glass or plastic. The LPP 41 uses the same dielectric multilayer film filter in which a dye is diffused into the optical transparent substrate. Inexpensive absorption filter is used.

The LPF 41 and BRF 420 constitute an overlapped composite optical filter 43 as shown in FIG. 18, and as shown in the solid line in the drawing, the wavelength of the coating having a high transmittance of the spectacle lens is around 400 to 700 nm, and High spectral transmittance is exhibited both around the center wavelength of the near-infrared LED 8 used for imaging in a dark state.

That is, the composite optical filter 43 has two pass bands of 400 to 700 nm as the first pass band and 850 nm or more as the second pass band.

By the composite optical filter 43, the near-infrared LED 8 is turned on in the dark state, and the circumferential wavelength around the center wavelength of the near-infrared LED 8 is high. The pupil is imaged to detect the driver's condition, that is, drowsiness.

In the bright state, the face image of the driver p is picked up by the components of the two wavelength regions in the external light with a spectral transmittance between the wavelengths of 500 to 650 nm and 850 nm or more.

By the way, when the conventional visible cut filter 4 is used, the spectroscopic reflectance of the spectacle lens in the blocking region of the filter 4 is low, and conversely, the spectroscopic reflectance of the spectacle lens in the translucent back melting of the filter 4 is high. Therefore, the value of K indicating the visible side of the eye becomes small, and as shown in FIG. 28, the eye can hardly be captured.

On the other hand, in the composite optical filter 43 of the present embodiment, since the high transmittance is given in the region where the spectral reflectance of the visible wavelength of the spectacle lens is low, the diffuse reflection light beam from the eye in the visible region ( iT) enters the CCD 1 efficiently, and the ratio K becomes large, so that eyes can be reliably captured as shown in FIG.

That is, in this embodiment, the compound optical filter 43 having a pass band in both the visible wavelength region with high transmittance of the spectacle lens and the near infrared wavelength region corresponding to the near infrared illumination for imaging the driver p in the dark state. By imaging the face of the driver p wearing the glasses g with the visible and near-infrared pass wavelength band components of the sunlight in a bright state, the surface reflection of the spectacle lens may There is an advantage in that the eye can be surely photographed without using illumination.

Example 7

The seventh embodiment is a modification of the sixth embodiment, which obtains a clearer face image than the sixth embodiment.

Fig. 20 shows the spectral transmittance characteristic diagram of the composite optical filter 45 of Example 7. The composite optical filter 43 of Fig. 19 of Example 6 has a short wavelength transmission filter (HPF) having a transmittance of 50% and a point of about 950 nm. , 44), and as shown, the transmittance of the coating of the spectacle lens is in the high wavelength region of about 400 to 700 nm, and can be used for imaging in the dark state and only around the center wavelength of the near-infrared LED 8 It is set as the composite optical filter which limited the transmission wavelength range.

The HPP 44 may use a dielectric multilayer filter like the BRF 42. Such a compound optical filter 45 is disposed in front of the imaging lens 3 of the camera unit c as in Example 6, and in the dark state, the near-infrared LED 8 is turned on to capture the pupil of the driver p. At the same time, in the bright state, the face image of the driver p is imaged only by the components between the wavelengths of 500 to 650 nm and 850 to 950 nm in the external light.

Here, the first passband is a wavelength of 500 to 650 nm.

According to the use of such a composite optical filter 45, the wavelength region of illumination of near-infrared light used for imaging in a dark state is diffused from the eye in the visible region. Since light of only two wavelength regions with visible wavelengths including iT) is transmitted, disturbed light in wavelength regions other than these can be removed. For this reason, it is possible to eliminate large reflections of the generator on the long wavelength side of the spectacle lens, that is, the wavelength region of 700 nm or more, and to capture the eye more clearly even if the surface reflection of the spectacle lens is present, and at the wavelength other than necessary. Since the light of the castle is cut, there is an advantage that the noise light existing in the surroundings does not invade the camera and the imaging S / N ratio of the driver p in the dark state can be improved.

Example 8

Embodiment 8 relates to the simplification of the face image device.

In Examples 6 and 7, the LPF 41, the BRF 42, and the HPF 44 were used as separate filters, respectively. For example, a dielectric multilayer film was laminated on one side of the substrate of the absorption type LPF 41 to form BRF. (42) or LPF 41 on one side and BIRF 42 on the other side by laminating a dielectric multilayer film on both sides of the transparent substrate, the spectral transmittance characteristic shown by the solid line of FIG. It is possible to form a composite optical filter 43 having a.

Further, when the dielectric multilayer film is laminated on both sides of the substrate of the absorption type LPF 41, and the BRF 42 is formed on one side and the HPF 44 on the other side, the spectral transmittance shown by the solid line in FIG. 20 on one substrate. A composite light source filter 45 having characteristics can be formed.

Therefore, according to these methods, the composite optical filter 45 can be thinned and manufactured at low cost, and the troublesome problem of alignment of a plurality of filters does not occur.

Example 9

The ninth embodiment relates to the combination of the above-described embodiments, and even if the surface reflection of the spectacle lens is relatively strong, it is said that it is not affected.

Although not shown, the composite optical filter 43 or the composite optical filter 45 shown in Examples 6 and 7 is replaced with the optical filter 4 of the camera unit c shown in Example 1 or Example 5. Therefore, the influence of spectacle lens reflection can be further reduced.

That is, after the driver p is imaged using the camera unit c using the composite optical filter 43 or the composite optical filter 45, the influence of the surface reflection of the spectacle lens is first reduced, and then, Example 1 As in the fifth embodiment, when the eye cannot be detected in any of the surrounding conditions or the bright state near the face of the driver p or the glasses of the driver p, the output I / The near-infrared light source 6 is lighted by sending an illumination control signal for removing specular reflection to the lighting control circuit 5 through the F 14.

According to this embodiment, the near-infrared light source 6 is turned on to ensure the influence of lens reflection even when the surface reflection of the spectacle lens is strong, such as a white cloud or a white wall reflected on the spectacle lens when it is clear. The eye can be clearly captured by removing it.

In addition, since the degree of surface reflection of a conventional spectacle lens can be clearly captured by only the compound optical filter 43 or the compound optical filter 45, the frequency of occurrence of a state in which the eye cannot be detected is reduced. As a result, the frequency of use of the near-infrared light source 6 in the bright state is reduced, and there is an advantage that the life can be extended.

Example 10

In Example 10, two optical filters are alternately used, whereby the eye can be clearly detected without providing the near-infrared light source 6 for removing the influence of specular lens reflection.

21 to 24 show a tenth embodiment, FIG. 21 is a block diagram of a driver state detection device including the face image pickup device according to the tenth embodiment, FIG. 22 is a sectional view of the image pickup unit a, and FIG. 23 is a filter. 24 is a perspective view of the spectral transmittance characteristic of one optical filter 46.

In the figure, reference numeral 46 denotes a first optical filter having a pass band only in a visible region having a high transmittance of the spectacle lens coating, and a band pass filter (BPF) having a 50% transmittance wavelength of 500 nm and 650 nm as shown in FIG. 47 is a second optical filter in which the transmission wavelength is combined with the wavelength region of the near infrared light source for illumination of the driver p in the dark state, and is, for example, a near infrared LED having a center wavelength of 900 nm (see Example 5 above) as a near infrared light source. In the case of using 8), a normal visible cut filter may be used. However, the use of a BPF with a half width of about 100 nm centered on a wavelength of 900 nm enables the removal of disturbance light components other than illumination light. The visible region having a high transmittance of the spectacle lens coating is the first pass band region, and the wavelength region of the near infrared light source for illumination of the driver p in the dark state is the second pass band.

60 is filter replacement means for replacing the first optical filter 46 and the second optical filter 47, and is incorporated in the housing 50 of the imaging section a. The first optical filter 46 and the second optical filter 47 may be disposed on the front side of the lens 3, but here, the first optical filter 46 and the second optical filter 47 may be disposed on the rear side of the lens 3 to reduce the size of the filter replacement unit 60. At the same time, the chromatic aberration of the lens 3 is corrected by changing the thickness of the filter substrate, the material, that is, the refractive index, in accordance with the transmission wavelength band, thereby changing the effective focal length of the lens 3. 48 is a transparent protective member such as glass with an antireflective coat.

The filter changing means 60 includes a motor 61 for generating a driving force for replacing the filter, a lead 62 for giving a signal to the motor 61, and a gear box 63 for reducing the rotation of the motor 61. ), A rotating cam plate 64 which rotates with the rotational force from the gear box 63 with the projection roller pin, and the L-shaped shape supports two filters on one side thereof and the roller on the other side. It consists of the filter support plate 65 which has a long hole in which a pin is inserted.

The operation of the tenth embodiment will be described.

First, when the device is reset, a first optical filter 46 for passing the visible region is disposed on the imaging axis, and in the bright state, the face image of the driver p passes through the filter through the first optical filter 46. It is imaged with component light in the wavelength region. In this case, since the transmission wavelength region of the first optical filter 46 is adapted to the wavelength region where the spectroscopic reflectance of the spectacle lens is low, the reflected light is reduced even if there is a lens surface reflection of the spectacles g worn by the driver p. Incident to the camera is prevented by the one optical filter 46. On the other hand, the diffuse reflecting light beam from the eye ( The wavelength component of the pass band of the first optical filter 46 in iT) is incident on the camera without being blocked by the first optical filter 46, whereby the eye can be clearly captured.

When the outer space becomes dark and the average brightness of the face image of the driver p decreases, the video signal processing circuit 2 controls the shutter to the open side at the same time as the AGC control is performed. In this state, the same contrast determination as in Example 5 is performed, and when it is determined that it is dark, the filter is replaced with the second optical filter 47.

When the contrast signal by the image signal processing circuit 2 on the printed circuit board 51 outputs a signal indicating a dark state, the CPU 15 gives a drive signal to the motor 61 through the lid 62 and rotates it. Rotation of the motor 61, the rotation speed is lowered from the gear box 62 connected to the motor 61, and rotates the rotary cam plate 64 half. This rotational motion is converted into a linear motion by the long hole of the filter support plate 65 and the roller pin of the rotary cam plate 64 inserted into the long hole, thereby moving the filter support plate 65 perpendicularly to the imaging axis. The first optical filter 46 is replaced with the second optical filter 47.

At this time, the near-infrared LED 8 is turned on to illuminate the driver's face to capture the pupil.

On the contrary, when the extraterrestrial is turned from a dark state to a bright state, that is, when it is morning from night, other solar light of the near infrared light of the near infrared LED 8 is applied. Therefore, the 2nd optical filter 47 passes the irradiation light of the near-infrared LED 8 and the component of the near-infrared light of sunlight. As a result, the average volume of the face image of the driver p increases, and the video signal processing circuit 2 determines that the video signal processing circuit 2 has become a bright state from a dark state on the basis of the increase of the brightness, indicating a bright state to the CPU 15. Output the signal. The CPU 15 receives the signal indicating the bright state, rotates the motor 61 in the opposite direction described above, changes the optical filter from the second optical filter 47 to the first optical filter 46, and The infrared LED 8 is turned off.

In this embodiment, in order to capture the driver p using only the visible wavelength region without the influence of the surface reflection of the spectacle lens in the bright state, the eye is clear without using special specular removal illumination even if the spectacle lens reflection exists. At the same time, the driver p is imaged only in the wavelength region of the near-infrared light source illuminating the driver p in the dark state, so that the disturbance light is eliminated and the image pickup S / N ratio of the driver p can be improved. There is an advantage that can be.

Example 11

Example 11 greatly reduces the influence of surface reflection of the spectacle lens by combining Example 1 and Example 10.

That is, in Example 11, the camera part c which has the filter replacement means 60 shown in Example 10, the contrast detection means, such as the illuminance sensor 7 demonstrated in Example 1, and the illumination control circuit 5 ) And a near-infrared light source 6 for removing the influence of spectacle lens reflection.

Therefore, after switching to the first optical filter 4 having the pass band only in the visible region where the transmittance of the spectacle lens coating is high in the bright state, the face image of the driver p is imaged, and after the influence of the lens surface reflection is first reduced, As in the first to fifth embodiments, when the surrounding environment or the vicinity of the face of the driver p is bright or in which state the eyeglasses of the driver p are detected and the eye cannot be detected. The near-infrared light source 6 is turned on by sending an illumination control signal for removing specular reflection to the illumination control circuit 5 through the output I / F 44.

According to this embodiment, even when the lens surface reflection is the same as in the ninth embodiment, the near-infrared light source 6 is turned on, so that the effect of lens reflection can be reliably eliminated, and the eye can be clearly captured, As the degree of lens reflection generated, the eye can be imaged clearly with only the first optical filter 46, so that the frequency of use is reduced in the bright state of the near-infrared light source 6 and the life can be extended.

In the sixth to eleventh embodiments, the pupil p of the driver p is imaged as the near-infrared LED 8 in a dark state. However, as in the first embodiment, the near-infrared light source such as a halogen lamp or a xenon lamp is used. The area or the near-infrared LED may be bundled to illuminate and image the driver's face with the near-infrared light source 6.

In addition, in each of the above embodiments, a face image image pickup apparatus for detecting a drowsiness or the like of a vehicle driver is shown. As a matter of fact, generally, the face image of a subject to be detected is image-processed to detect the eyes of the subject to be detected. It can be used in a person state detection device that detects a subject's condition and can be used as a face image pickup device.

Since this invention has the structure as mentioned above, it has the following effects.

According to the face image image pickup apparatus according to the present invention, when the eye detection means detects the eye of the detection subject, the infrared light illuminating means illuminates the face of the detection subject, so that the influence of the reflection of the glasses worn by the driver Can be reduced.

Further, according to the face image image pickup apparatus according to the present invention, when the contrast detecting means detects a bright state and the eye detecting means does not detect the eye of the detecting subject, the infrared illuminating means is excited to detect the subject. Since it illuminates the face, power consumption can be reduced.

Further, according to the face image image pickup apparatus according to the present invention, the bright state or the dark state is determined based on whether or not the luminance of the image including the face of the subject to be imaged by the two-dimensional image pickup means is equal to or higher than a predetermined luminance. The configuration of the device can be simplified.

In addition, the face image image pickup apparatus according to the present invention stops the infrared light lighting means once a predetermined time has passed since the infrared light lighting means is excited, so that the infrared light lighting means can be excited when necessary.

Further, according to the face image image pickup apparatus according to the present invention, when the eyeglass detecting means detects the eyeglasses and the eye detecting means does not detect the eye of the subject to be detected, the infrared light illuminating means is excited to Since the face is illuminated, power consumption can be reduced.

Further, according to the face image image pickup apparatus according to the present invention, when the contrast detecting means detects a dark state, it has a dark state infrared light illuminating means for illuminating the face of the subject to be detected by infrared light passing through the optical filter. Since the state volume infrared light illuminator and the eye detecting means separately function the infrared light illuminator excited when detecting the eye of the subject to be detected, power consumption can be reduced.

In addition, according to the face image image pickup apparatus according to the present invention, since the visible image of a predetermined wavelength region and infrared light having a predetermined wavelength or more are passed through to capture a face image of a subject to be detected, the driver wears glasses with a simple configuration. The influence of reflection can be reduced.

In addition, according to the face image image pickup apparatus according to the present invention, since the face image of the subject to be detected is captured by only visible light having a predetermined wavelength and infrared light having a predetermined wavelength region, a clearer image can be obtained.

Further, according to the face image image pickup apparatus according to the present invention, when a face image of a subject to be detected is captured by visible light in a predetermined wavelength range and infrared light having a predetermined wavelength or more, and eyes cannot be detected in the face image. Since the excitation of the infrared light illuminator illuminates the face of the subject to be detected, the influence can be reduced even when the reflection of the glasses worn by the driver is relatively strong.

Further, according to the face image image pickup apparatus according to the present invention, when the contrast detecting means detects a bright state, the first optical filter is disposed on the optical axis of the two-dimensional imaging means, and the contrast detecting means detects the dark state. Since the second optical filter is disposed on the optical axis of the two-dimensional imaging means when present, the eye can be clearly detected and the influence of the reflection of the glasses worn by the driver can be reduced.

Claims (7)

Two-dimensional imaging means for imaging a predetermined area including the face of the subject to be detected, an optical filter disposed on an optical axis of the two-dimensional imaging means with a pass band for passing infrared light in at least a predetermined wavelength region, and the two Eye detection means for detecting the eye of the detection subject based on the face image of the detection subject captured by the dimensional imaging means, and illuminating at least the face of the detection subject with infrared light passing through the optical filter; An infrared light illuminating means arranged such that an angle formed between the optical axis of the dimensional imaging means and the optical axis of the infrared light becomes a predetermined angle or more, and the infrared light illuminating means when the eye detecting means does not detect the eye of the subject to be detected. A face image image pickup apparatus comprising an excitation means for exciting. 2. The apparatus according to claim 1, further comprising contrast detection means for detecting the brightness of the surroundings or near the face of the subject to be detected in a bright or dark state. And an infrared illuminating means excited when the contrast detecting means detects a bright state and the eye detecting means does not detect the eye of the person to be detected. Two-dimensional imaging means for imaging a predetermined area including the face of the subject to be detected, a first pass band for passing visible light in a predetermined wavelength region, and a second pass band for passing infrared light having a predetermined wavelength or more; And an optical filter arranged on the optical axis of the dimensional imaging means. The face image pickup device according to claim 3, wherein the second pass band of the optical filter passes only infrared light in a predetermined wavelength range. Two-dimensional imaging means for imaging a predetermined area including the face of the subject to be detected, a first pass band for passing visible light in a predetermined wavelength region, and a second pass band for passing infrared light having a predetermined wavelength or more, and simultaneously An optical filter disposed on the optical axis of the imaging means, eye detection means for detecting the eye of the detection subject based on the face image of the detection subject captured by the two-dimensional imaging means, and at least the face of the detection subject; Infrared light illuminating means arranged so that each part formed between the optical axis of the two-dimensional imaging means and the optical axis of the infrared light is equal to or more than a predetermined angle while illuminating with infrared light passing through the optical filter, and the eye detecting means is the object to be detected. And an excitation means for exciting the infrared light means when no eye is detected. The image pickup device. 4. The optical filter of claim 3, wherein the optical filter is configured as a first optical filter having a first pass band and a second optical filter having a second pass band, and detecting brightness of the surroundings or near the face of the subject to be detected. Contrast detecting means for detecting whether the surrounding or near face is a bright state or a dark state, and when the contrast detecting means detects a bright state, the first optical filter is disposed on the optical axis of the two-dimensional imaging means. And filter replacement means for disposing the second optical filter on an optical axis of the two-dimensional imaging means when the contrast detecting means detects a dark state. The optical filter of claim 5, wherein the optical filter is configured as a first optical filter having a first pass band and a second optical filter having a second pass band, and detecting brightness of the surroundings or the vicinity of the face of the detection subject. Contrast detecting means for detecting whether the surrounding or near face is in a bright state or a dark state, and when the contrast detecting means detects a bright state, the first optical filter is disposed on the optical axis of the two-dimensional imaging means. And filter changing means for arranging said second optical filter on the optical axis of said two-dimensional imaging means when said contrast detecting means detects a dark state.