JP2022072293A - Driver monitoring device - Google Patents

Abstract

To provide a driver monitoring device capable of reducing effects of sunlight on imaging.SOLUTION: A driver monitoring device includes: an infrared light irradiation unit 12 which irradiates a driver of a vehicle with infrared light according to lighting of an infrared LED; a camera 13 which periodically images a driver using infrared light reflected on the driver; a control unit 16 which periodically turns on the light of the infrared LED in synchronization with imaging of the camera 13; a filter which makes infrared light of a prescribed wavelength range incident on the camera 13; and a thermistor which detects ambient temperature of the infrared LED. If the temperature detected by the thermistor is lower than a reference temperature, the control unit 16 changes lighting time per one cycle of the infrared LED so that it is longer compared to when a detected temperature is equal to or higher than the reference temperature.SELECTED DRAWING: Figure 3

Description

The present invention relates to a driver monitoring device that captures an image of the driver of a vehicle and monitors the state thereof.

A head-up display device that creates and displays a virtual image by the display light reflected by the reflected light-transmitting member, which is superimposed on the actual scene (scenery in front of the vehicle) that passes through the reflected light-transmitting member such as the front windshield or combiner of the vehicle. Some have the function of a driver monitoring device (driver monitoring system) that irradiates the driver with infrared light to image the driver and monitors the driver's condition.

For example, the head-up display device described in Patent Document 1 reflects visible light emitted from a display means toward a driver by a combiner member to form a display image, and irradiates the driver with infrared rays. The infrared irradiation means and the mirror member that reflects the visible light emitted from the display means toward the combiner member and transmits the infrared rays reflected by the driver and the combiner member, and the infrared rays transmitted through the mirror member are sensed. A plurality of imaging means for imaging the driver from different directions and an image processing means for calculating the position of the driver's eyes based on the images captured by the imaging means are provided, and the calculation result of the position of the driver's eyes is provided. Can be used to determine the driver's condition such as looking aside or taking a nap.

Japanese Unexamined Patent Publication No. 2008-126984

By the way, in the head-up display device described in Patent Document 1, infrared rays are emitted from the infrared LED of the infrared irradiation means, which are reflected by the driver's face or the like, pass through a cold mirror, and are incident on the image pickup means. Since the emission wavelength of the LED changes depending on the ambient temperature (it becomes shorter when the temperature is low and becomes longer when the temperature is high), the transmission wavelength range of the filter (cold mirror) is used to suppress the incident of sunlight on the imaging means. If it is narrowed, the infrared wavelength required for imaging may be cut by the filter, and there is a problem that the influence of sunlight on imaging cannot be sufficiently reduced.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a driver monitoring device capable of reducing the influence of sunlight on imaging.

In order to solve the above problems, the driver monitoring device according to the present invention uses lighting means for irradiating the driver of a vehicle with infrared light by lighting an infrared LED and infrared light reflected by the driver. An image pickup means that periodically images the driver, a control means that periodically turns on the infrared LED in synchronization with the image pickup of the image pickup means, and an infrared light of a predetermined wavelength band incident on the image pickup means. The control means includes a filter means for detecting the ambient temperature of the infrared LED, and the control means per cycle of the infrared LED when the detection temperature of the temperature detection means is lower than the reference temperature. The lighting time of the light is changed so as to be longer than when the detected temperature is equal to or higher than the reference temperature.

When the detection temperature is lower than the reference temperature, the control means may change the lighting time according to the detection temperature.

Further, the driver monitoring device according to the present invention transmits light in the wavelength band of visible light on the imaging optical path through which the infrared light reflected by the driver passes by the time it is incident on the filter means, and is infrared. A wavelength selection optical means for reflecting light in the wavelength band of light may be provided.

At this time, the filter means is a long-pass filter that does not transmit light having a wavelength lower than the first threshold value among the light in the wavelength band of the infrared light, but transmits light having a wavelength higher than the first threshold value. Even if, or, light having a wavelength lower than the second threshold of the light in the wavelength band of the infrared light is not transmitted, and the wavelength is higher than the second threshold and lower than the third threshold. A bandpass filter that transmits light having a wavelength and does not transmit light having a wavelength higher than the third threshold value may be used.

Further, the control means may change the lighting time during a period during which the image pickup means is not performing an image pickup.

According to the present invention, there is an effect that the influence of sunlight on imaging can be reduced.

It is explanatory drawing which shows the vehicle provided with the head-up display device which built in the driver monitoring device which concerns on embodiment for carrying out the invention. It is explanatory drawing which shows the transmission characteristic of the front windshield of FIG. It is a block diagram which shows the main composition of the driver monitoring apparatus of FIG. It is explanatory drawing which shows the detailed structure of the infrared irradiation unit and a camera of FIG. It is explanatory drawing which shows the transmission characteristic of the filter of FIG. It is explanatory drawing which shows the image pickup synchronization signal of the driver monitoring apparatus of FIG. 1, and the operation timing of a camera and an infrared LED. It is a flow chart which shows the process by the control part of FIG. It is explanatory drawing which shows the other example of the transmission characteristic of a filter. It is explanatory drawing which shows the other example of the driver monitoring apparatus. It is explanatory drawing which shows the control example of the lighting time according to the detection temperature. It is explanatory drawing which shows the other control example of the lighting time according to the detection temperature.

A mode for carrying out the present invention will be described with reference to the drawings.

As shown in FIG. 1, the head-up display device (HUD) 1 incorporating the driver monitoring device according to the present embodiment is provided inside the instrument panel 4 below the front windshield 3 of the vehicle 2. , _The display light L1 which is visible light is projected on the front windshield 3. The display light L ₁ is reflected by the front windshield 3 to generate a virtual image V, and the virtual image V is superimposed on the actual view transmitted through the front windshield 3 to the driver D sitting in the driver's seat, which is not shown behind the steering wheel 5. To be visually recognized.

Further, in order to monitor the state of the driver D, the HUD 1 irradiates the infrared light L ₂ and reflects it on the front windshield 3, illuminates the driver D, and is reflected by the driver D. The driver D is imaged by the infrared light L ₂ .

Specifically, as shown in FIG. 2, the front windshield 3 transmits light in the wavelength band of visible light (for example, light having a wavelength of less than 800 nm (800 nanometers)) to transmit light in the wavelength band of infrared light. A half mirror film (not shown) is provided at a portion of the front windshield 3 where the display light L1 is projected, which has _a characteristic of reflecting light without transmitting (for example, light having a wavelength of 800 nm or more). ..

The HUD 1 is separated from the outside by being covered with a housing 6 molded from black ABS resin or the like to prevent the intrusion of outside light, and the housing 6 has an opening to which a transparent cover glass 7 made of polycarbonate or the like is attached. 8 is formed. Inside the housing 6, a display unit 9, a plane mirror 10, a concave mirror 11, an infrared light irradiation unit 12, a camera 13, a filter 14, a shielding piece 15, and a control unit 16 are held and housed. The control unit 16 is connected to the display unit 9, the infrared light irradiation unit 12, and the camera 13 as shown in FIG.

The display unit 9 includes a display 17 including a light source and a liquid crystal panel, and is controlled by the control unit ₁₆ to project and display visible light (display light L1) from the display unit 17. The plane mirror 10 is formed by depositing a metal such as aluminum or a dielectric multilayer film on glass formed so as to have a flat portion, and simply reflects visible light. The concave mirror 11 is formed by depositing a metal such as aluminum on a resin such as polycarbonate molded so as to have a concave portion, and magnifies and reflects visible light.

As shown in FIG. 4, the infrared light irradiation unit 12 includes an infrared LED 18 as an infrared light source, a thermistor 19, and a substrate 20 provided with the infrared LED 18 and the thermistor 19, and a cover glass in the housing 6. It is provided below 7. The infrared LED 18 is controlled to be lit by the control unit 16 to emit near infrared rays (infrared light L ₂ ), the thermistor 19 detects the temperature around the infrared LED 18, and the detected temperature is controlled via the substrate 20. It is output to the unit 16. The infrared light irradiation unit 12 irradiates the infrared light L ₂ toward the front window shield 3 through the shield piece 15 and the cover glass 7.

The camera 13 is provided in the housing 6 under the cover glass 7 side by side with the infrared light irradiation unit 12, and particularly has an image pickup element 21 having sensitivity to light in a wavelength band of about 950 nm, and an infrared light L ₂ ( Alternatively, it includes a lens 22 that transmits light in the wavelength band that passes through the filter 14 of the infrared light L ₂ and forms an image on the image pickup element 21. When the infrared light L ₂ emitted from the infrared light irradiation unit 12 is reflected by the front windshield 3 and reflected by the driver D and the front windshield 3 and is incident on the camera 13, the camera 13 is close to the driver D. Infrared images are periodically imaged at a predetermined frame rate.

The filter 14 incidents infrared light in a predetermined wavelength band on the image pickup element 21 of the camera 13. Here, as shown in FIG. 5, the threshold value λ ₁ = 900 nm of the light in the wavelength band of infrared light. A long-pass filter that does not transmit light with a lower wavelength than that and transmits light with a wavelength higher than the threshold value λ ₁ is used. In relation to the filter 14, it can be said that the front windshield ₃ is on the imaging light path through which the infrared light L2 reflected by the driver D passes by the time it is incident on the filter 14.

The shielding piece 15 shields visible light and transmits infrared light, and is provided above the infrared light irradiation unit 12 and the camera 13 in the housing 5.

The control unit 16 acquires an image pickup synchronization signal from the camera 13, and as shown in FIG. 6, periodically turns on the infrared LED 18 in synchronization with the image pickup of the camera 13 (ON of the shutter of the camera 13). At this time, the control unit 16 acquires the detection temperature T of the thermistor 19, and when the detection temperature T is lower than the predetermined reference temperature T1, the detection temperature T determines the lighting time _per cycle of the infrared LED 18. Change so that t ₁ is longer than t when T ₁ or more and less than T ₂ , and when the detection temperature T is the reference temperature T ₂ or more, the lighting time per cycle of the infrared LED 18 is shorter than t t. Change to ₂ . In other words, in the case of this example, it can be said that the temperature of the infrared LED 18 is controlled by the lighting time (large or small) of the infrared LED 18. It should be noted that this change in lighting time (prolonged or shortened) is performed during a period during which the camera 13 is not performing imaging (a period during which the shutter of the camera 13 is OFF).

In the HUD 1, the display light L ₁ from the display unit 9 is reflected by the plane mirror 10, then enlarged and reflected by the concave mirror 11, passes through the cover glass 7, and is projected onto the front windshield 3. The display light L1 projected on the front windshield ₃ is magnified and reflected toward the driver D by the half mirror film, and a virtual image V is generated and superimposed on the actual scene transmitted through the front windshield 3 by the driver D. Display on.

On the other hand, the infrared light L ₂ emitted from the infrared light irradiation unit 12 passes through the shield piece 15 and the cover glass 7 and is projected onto the front windshield 3, and is reflected by the front windshield 3 toward the driver D. And illuminate driver D. Then, a part of the infrared light L ₂ reflected by the driver D follows the reverse path, passes through the cover glass 7, the shielding piece 15, and the filter 14 and is incident on the camera 13, and the driver D is imaged. To.

At that time, as shown in FIG. 7, when the image pickup synchronization signal is input from the camera 13 (step 1 (described as “S.1” in FIG. 7; the same applies hereinafter)), the control unit 16 of the thermistor 19 The detection temperature T is acquired to determine the lighting time per cycle of the infrared LED 18 (steps 2 and 3), and the infrared LED 18 is turned on (step 4). Then, when the determined lighting time elapses, the infrared LED 18 is turned off (steps 5 and 6). As described above, the shutter of the camera 13 is turned on / off between the time when the infrared LED 18 is turned on and the time when the infrared LED 18 is turned off.

The driver monitoring device according to the present embodiment is reflected by the infrared light irradiation unit 12 that illuminates the driver D of the vehicle ₂ by illuminating the driver D of the vehicle 2 by lighting the infrared LED 18, and the driver D. A camera 13 that periodically takes an image of the driver D by the infrared light L ₂ , a control unit 16 that periodically turns on the infrared LED 18 in synchronization with the image taken by the camera 13, and the camera 13 having a predetermined wavelength band. A filter 14 for incident infrared light and a thermista 19 for detecting the ambient temperature of the infrared LED 18 are provided, and the control unit 16 of the infrared LED 18 has a detection temperature T lower than the reference temperature T ₁ when the detection temperature T of the thermista 19 is lower than the reference temperature T 1. Since the lighting time per cycle is changed so that the detection temperature is longer than when the detection temperature is T ₁ or higher, the emission wavelength of the infrared LED is generally shorter when the temperature is low (wavelength shift occurs), but the infrared By lengthening the lighting time of the LED 18, the temperature can be raised (so to speak, by the self-heating of the infrared LED 18), and such wavelength deviation can be suppressed. As a result, the transmission wavelength range of the filter 14 can be narrowed to reduce the influence of sunlight on the image pickup.

Here, the control unit 16 changes the lighting time per cycle of the infrared LED 18 to be shorter when the detection temperature T of the thermista 19 is equal to or higher than the reference temperature T ₂ than when the detection temperature is less than the reference temperature T ₂ . Therefore, when the ambient temperature of the infrared LED 18 is high, the lighting time of the infrared LED 18 is shortened to lower the ambient temperature to suppress the wavelength shift, and the transmission wavelength range of the filter 14 is narrowed to the long wavelength side for imaging. The influence of sunlight can be further reduced.

Further, the filter 14 is a long-pass filter that does not transmit light having a wavelength lower than the threshold value λ ₁ of the infrared light L ₂ reflected by the front windshield 3 and transmits light having a wavelength higher than the threshold value λ ₁ . Is used, so that it is possible to cut the component of sunlight having a wavelength lower than that of infrared light having a wavelength of about 950 nm, which is necessary for imaging, and the influence of sunlight on imaging is further reduced.

Further, since the control unit 16 changes the lighting time of the infrared LED 18 during the period when the camera 13 is not performing imaging, it is possible to suppress the wavelength shift and keep the illumination brightness constant during imaging at the same time. ..

FIG. 8 shows light in the wavelength band of infrared light that does not transmit light having a wavelength lower than the threshold λ ₂ = 900 nm, and emits light having a wavelength higher than the threshold λ ₂ and a wavelength lower than the threshold λ ₃ = 1000 nm. The transmission characteristics of the filter 14 are shown when a bandpass filter that transmits light and does not transmit light having a wavelength higher than λ ₃ = 1000 nm is used for the filter 14. According to this filter 14, it is possible to cut the components of sunlight having a lower wavelength and the components of sunlight having a higher wavelength than the infrared light having a wavelength of about 950 nm required for imaging, and the influence of sunlight on imaging is further increased. Reduce.

Although the embodiments for carrying out the present invention have been illustrated above, the embodiments of the present invention are not limited to those described above, and may be appropriately modified without departing from the spirit of the invention.

For example, as shown in FIG. 9, light in the visible light wavelength band is transmitted on the optical path of the display light L1 between the display unit 9 and the plane mirror ₁₀ to reflect light in the infrared light wavelength band. The hot mirror 23 is provided, and the infrared irradiation unit 12, the camera 13, and the filter 14 are provided on the opposite side of the concave mirror 11 with the hot mirror 23 interposed therebetween, and the infrared light L2 emitted from the infrared light irradiation unit ₁₂ is provided. , _The infrared light L2 emitted from the cover glass 7 after passing through the hot mirror 23, the plane mirror 10, the concave mirror 11 and the shield piece 15, and incident from the cover glass 7, is the shield piece 15, the concave mirror 11, the plane mirror 10, and the hot mirror. It may be configured to pass through the 23 and the filter 14 and enter the camera 13.

Further, when the detection temperature T is lower than the reference temperature T ₁ , the control unit 16 may change the lighting time according to the detection temperature T instead of uniformly changing the lighting time to t ₁ . In FIG. 10, when the detection temperature T is T ₅ ≤ T <T ₄ and T ₃ ≤ T <T ₁ , the lighting time changes continuously, and when the detection temperature T is T ₄ ≤ T <T ₃ , the lighting time changes continuously. Shows an example in which the lighting time does not change, and FIG. 11 shows a lighting time in each range where the detected temperature T is T ₅ ≤ T <T ₄ , T ₄ ≤ T <T ₃ , and T ₃ ≤ T <T ₁ . Is shown as an example in which is changed step by step. Such a method of changing the lighting time may be adopted even when the detected temperature T is equal to or higher than the reference temperature T ₂ .

Further, the filter 14 may be provided inside the camera 13 instead of outside, and the thermistor 19 is provided outside the infrared light irradiation unit 12 instead of inside the infrared light irradiation unit 12 as the ambient temperature of the infrared LED 18. The ambient temperature of the may be detected.

1 Head-up display device (driver monitoring device)
2 Vehicle 3 Front windshield (wavelength selection optical means)
12 Infrared light irradiation unit (illumination means)
13 Camera (imaging means)
14 Filter (filter means)
16 Control unit (control means)
18 infrared LED
19 Thermistor (Temperature detection means)
D Driver L ₂ Infrared light T Detection temperature T ₁ Reference temperature λ ₁ Threshold (first threshold)
λ ₂ threshold (second threshold)
λ ₃ threshold (third threshold)

Claims (6)

Lighting means that irradiates the driver of the vehicle with infrared light by lighting the infrared LED,
An imaging means that periodically images the driver with infrared light reflected by the driver, and
A control means for periodically turning on the infrared LED in synchronization with the image pickup of the image pickup means,
A filter means for incident infrared light in a predetermined wavelength band on the image pickup means, and a filter means.
A temperature detecting means for detecting the ambient temperature of the infrared LED is provided.
When the detection temperature of the temperature detecting means is lower than the reference temperature, the control means changes the lighting time per cycle of the infrared LED so as to be longer than when the detection temperature is equal to or higher than the reference temperature. A driver monitoring device characterized by. The driver monitoring device according to claim 1, wherein the control means changes the lighting time according to the detection temperature when the detection temperature is lower than the reference temperature. Frequency selection optics that transmits light in the visible light wavelength band and reflects light in the infrared light wavelength band on the imaging light path through which the infrared light reflected by the driver enters the filter means. The driver monitoring device according to claim 1 or 2, wherein the driver monitoring device comprises means. The filter means is a long-pass filter that does not transmit light having a wavelength lower than the first threshold value among the light in the wavelength band of the infrared light and transmits light having a wavelength higher than the first threshold value. 3. The driver monitoring device according to claim 3. The filter means does not transmit light having a wavelength lower than the second threshold of the light in the wavelength band of the infrared light, and has a wavelength higher than the second threshold and a wavelength lower than the third threshold. The driver monitoring device according to claim 3, further comprising a bandpass filter that transmits light and does not transmit light having a wavelength higher than the third threshold. The driver monitoring device according to any one of claims 1 to 5, wherein the control means changes the lighting time during a period during which the image pickup means is not performing an image pickup.

Images