JP3855439B2 - Night driving visibility support device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a night driving vision support device, and more particularly to a night driving visibility support device (night driving) which is displayed by a head-up display (hereinafter abbreviated as HUD) device in a vehicle cabin via an infrared camera as a vehicle forward visibility support device. Vision).
[0002]
In order to achieve safe driving at night, there is a need for a night driving vision support device that can recognize in advance the presence of pedestrians and the like that cannot be confirmed with visible light.
[0003]
[Prior art]
As a conventionally known vehicle forward visibility support device, for example, Japanese Patent Laid-Open No. 7-257228 is cited. Here, in particular, a technique for detecting the driver's line of sight has been proposed. An eyeball image of the driver is detected using an infrared camera, the detected image is processed, and navigation information (target data using map data and GPS is used). The arrow information when performing automatic guidance to the ground, etc.) is displayed virtually superimposed on the road surface image by the HUD device.
[0004]
That is, navigation arrow information indicating the traveling direction is displayed by the HUD device on a straight line (line of sight) connecting the intersection position with respect to the guidance route on the road ahead of the vehicle and the eyeball position of the driver. This eliminates the need to move the line of sight by looking for the intersection position after viewing the HUD display.
[0005]
On the other hand, as a conventionally known night driving vision support device, a technology (night vision) has already been proposed which uses an infrared camera to photograph a front vision at night when there is little visible light, and presents the image to the driver. ing.
[0006]
[Problems to be solved by the invention]
In the prior art disclosed in Japanese Patent Laid-Open No. 7-257228 described above, the driver's line of sight that constantly fluctuates in order to display an image captured from the driver's viewpoint superimposed on the front field of view, and the HUD display position is adjusted accordingly. It is essential to change
For this purpose, the driver's line of sight must be measured, and the HUD display position must be appropriately calculated and displayed accordingly.
[0007]
Here, in order to superimpose and display the HUD device on the front view in real time, the load on the computer becomes very large, and there is a possibility that the calculation process cannot be completed when vehicle motion such as turning is intense.
[0008]
On the other hand, even with the currently proposed gaze measurement technology, the gaze of the driver cannot always be measured, and gaze measurement becomes impossible when the driver's head shakes greatly or the light environment conditions are bad. Under some circumstances, it becomes impossible to display the HUD display superimposed on the front view.
[0009]
Therefore, such a vehicle front vision assistance device is not sufficiently reliable, and is particularly dangerous when applied to the night driving vision assistance device as described above.
Therefore, the present invention is to enable an appropriate safe driving without constantly measuring the driver's line-of-sight position in a night driving vision support device that displays an image of an infrared camera that captures the front of the vehicle on a head-up display device. With the goal.
[0010]
[Means for Solving the Problems]
In order to achieve the above object, a night driving vision support device according to the present invention includes a face photographing camera for photographing a driver's eyeball, an image processing unit for processing an image photographed by the face photographing camera, and the image processing. A pupil position detection unit that detects the pupil position of the eyeball from the processing data of the unit, and a predetermined range in which the pupil position obtained by the pupil position detection unit appears to overlap with the actual landscape in the image displayed by the HUD device A pupil position determination unit that determines whether or not the pupil position is within, and a control device that drives the HUD device only when the pupil position detection unit determines that the pupil position is within the predetermined range. It is characterized by.
[0011]
In short, in the present invention, the HUD display is not always performed, but the HUD display is performed only when the eyeball position (line-of-sight position) of the driver is at a specific position.
[0012]
That is, in the present invention, the eyeball position (line-of-sight position) of the driver is measured by the face photographing camera. An image processing unit performs image processing on the video output from the face photographing camera.
[0013]
Then, the pupil position detection unit detects the driver's pupil position (eyeball position or viewpoint position) based on the data output from the image processing unit.
Then, the control unit determines whether or not the pupil position is within a predetermined range that appears to overlap the actual scenery in the video displayed on the HUD device, and controls the HUD display device based on the determination result.
[0014]
Therefore, when the pupil position is within the predetermined range, the control unit drives the HUD device (ON), and the video from the infrared camera 1 is matched with the front view from the specific viewpoint position of the driver to display the HUD. Let However, when there is no pupil position within the predetermined range, the HUD device is not driven (OFF) and the HUD display is not performed.
[0015]
By doing so, it is not necessary to always measure the driver's line of sight, and it is not necessary to automatically adjust the HUD display position in accordance with the driver's line of sight.
[0016]
Furthermore, when the driver wants to consciously check the situation in front, if he moves his head and aligns his viewpoint with a specific viewpoint position, he can actively obtain a HUD display that overlaps the front field of view. If they are shifted, it is possible to consciously prevent HUD display.
[0017]
It should be noted that, from the viewpoint position of the specific driver as described above, it is desirable to adjust the initial setting so that the front view and the HUD display coincide.
[0018]
The face photographing camera is a near-infrared camera composed of a camera and an infrared LED provided on the optical axis at the tip of the camera.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 shows an embodiment of a night driving vision support device according to the present invention. In the figure, 1 shows an infrared camera used for a surveillance camera, for example, and an output video of the infrared camera 1 is a HUD display device. Always sent to 2.
[0020]
Reference numeral 3 denotes a near-infrared camera for photographing a face, and particularly measures the driver's line-of-sight position (eyeball position). The near-infrared camera 3 is connected to an image processing unit 4 that converts the video into digital signal data and performs image processing as preprocessing on the data. The image processing unit 4 is connected to a pupil position detection unit 5 that detects a driver's pupil position (eyeball position or viewpoint position). The pupil position detection unit 5 has a detected pupil (eyeball) position within a predetermined range. It is connected to the control unit 6 that determines whether or not it exists.
[0021]
The control unit 6 is connected to the HUD display device 2 and controls the HUD display device 2 to drive the HUD display or not to drive the HUD display.
FIG. 2 schematically shows an embodiment when the night driving vision support device according to the present invention shown in FIG. 1 is actually mounted on a vehicle. In the figure, 10 indicates a cab of a commercial vehicle. 11 is a driver in the cab 10.
[0022]
As shown in the figure, the HUD display device 2 includes a projector 21 provided on a ceiling portion of the vehicle cab 10 and a combiner 22 attached to a window glass. The projector 21 is connected to the infrared camera 1 and captures an output image thereof. The near-infrared camera 3 for face photography is attached to the dashboard of the vehicle cab 10, and an infrared LED 30 is provided at the tip portion thereof.
[0023]
FIG. 3 shows a modification of the embodiment shown in FIG. 2, in which a projector 21 is provided inside the dashboard, and an output image of the projector 21 is combined via a reflector 23. 22 is different in that it is configured to be projected onto the screen 22.
[0024]
Note that the infrared camera 1 is used for monitoring, and an object in which the shape of an object can be seen as an image by a specific radiant heat even when there is almost no visible light of several lux is used.
[0025]
Here, as is well known, when the HUD display device 2 is driven, as shown in FIG. 4, the virtual video plane VP overlaps the front view and the driver 11 can see the output video from the infrared camera 1. It is what you want to do.
[0026]
FIG. 5 is a flowchart showing the operation of the nighttime driving vision support device according to the present invention shown in FIG. 1, and the operation of the above embodiment will be described below with reference to this flowchart.
[0027]
First, when a key switch (not shown) is turned on, the system starts its operation and this control program is executed.
Next, the control part 6 determines whether it is driving | running | working (step S1). For example, the vehicle speed may be detected by using a vehicle speed sensor (not shown) to determine whether the vehicle is running, or may be determined from a pedal operation, a steering wheel operation, or the like.
[0028]
Next, the control part 6 determines whether it is night (step S2). This can be determined by whether or not the headlight is turned on, or can be determined by using the automatic iris function of the near-infrared camera for face photography.
[0029]
Here, an example of HUD display by the infrared camera 1 will be described with reference to FIG.
Fig. 1 (1) shows a road image during the daytime. At night in this state, the upper half of the image becomes dark as shown in Fig. 2 (2). I can't get the field of view. This is because there are many cases where it is necessary to travel with a low beam when there is an oncoming vehicle.
[0030]
Therefore, if the HUD display is superimposed on the front field of view using the image from the infrared camera 1 as shown in FIG. 3 (3), the front field of view similar to the high beam state can be obtained.
[0031]
Returning to FIG. 5, a face image is acquired by the near-infrared camera in step S3.
FIG. 7 shows the face image of the driver 11 obtained in this way.
[0032]
Further, such a face image is subjected to image processing by the image processing unit 4 to detect the pupil position of the eyeball of the driver 11 (step S4).
[0033]
Here, a method for detecting the pupil position (eyeball position) of the driver will be described.
One way to detect the driver's pupil position is to apply a similar principle to the eyes of a subject such as a person who shines red at night when taking a picture with a strobe. .
[0034]
That is, as shown in FIG. 8, by using a light source that is very close to the optical axis of the near-infrared camera 3, a reflected image from the retina of the driver's eyeball 110 can be captured by the camera. This makes it easy to extract the pupil when performing image processing.
[0035]
At this time, if it is a light source of visible light, it will become dazzling light to the driver and hinder its driving, so near-infrared rays that are not perceptible to human eyes (light rays in the wavelength range from 760 nm to 800 nm) ) Using an infrared LED (light emitting diode) 30 as a light source and using a CCD camera with high sensitivity up to the near infrared region, a retina reflection image of the eyeball 110 can be obtained without dazzling the driver.
[0036]
FIG. 9 shows such a pupil reflection image. In the case shown in FIG. 9 (1), an infrared LED (light source) 30 not on the optical axis of the camera 3 is used. An image in which the pupil looks dark (dark pupil reflection image) is obtained, but when the light source 30 is on the optical axis of the camera 3, a retina reflection image (bright pupil) in which the pupil part shines brightly as shown in FIG. A reflection image) is obtained.
[0037]
Therefore, the position of the near-infrared camera 3 shown in FIGS. 2 and 3 needs to be set in consideration of the principle of FIGS.
[0038]
As a result, the pupil position EP shown in FIG. 10 can be extracted by performing image processing such as binarization processing on the face image of the driver shown in FIG.
[0039]
Returning to FIG. 5, after the pupil position is detected in this way (step S4), it is determined whether or not the pupil position is within a predetermined range (step S5).
[0040]
In this step, it is determined whether or not the forward visual field based on the pupil position (viewpoint) of the driver and the HUD display image by the infrared camera 1 are exactly located at a specific viewpoint position (within a predetermined range). As shown in FIG. 6, it can be determined by whether or not there is a pupil position EP in the portion SP indicated by a black frame.
[0041]
At this time, two pupil positions detected as pupils are detected. If any of the pupil positions is in the position of the black frame, it is determined that the driver's viewpoint is within the predetermined range.
[0042]
Here, the predetermined range as shown in FIG. 11 needs to be adjusted by the driver itself in accordance with the individual difference of the driver to perform the initial setting. Such an initial setting is, for example, that the driver fixes his / her head at a position where the HUD display image should be seen to overlap the front view, and adjusts the projector 21 while viewing the HUD display image so that the HUD display image is in the front view. Make sure they overlap. Then, the predetermined range is stored in, for example, the control unit 6 when the adjustment of the projector 21 is completed.
[0043]
Once stored, if the same driver is operated next, the initial setting need not be performed unless the seat position is shifted.
[0044]
When it is found that the pupil position is within the predetermined range in this way, the control unit 6 turns on the HUD device 2 and displays the front image by the infrared camera 1 in HUD.
[0045]
However, if it is not within the predetermined range, the HUD device 2 is turned off and the forward video display by the infrared camera 1 is not performed (step S7).
[0046]
【The invention's effect】
As described above, according to the nighttime driving vision support apparatus according to the present invention, the image from the near-infrared camera is processed to detect the driver's pupil position, and is this pupil position present at a predetermined position? Since it is configured to drive the HUD display device only when it is present and display the night vision of the night by the output image of the infrared camera, the driver's line of sight need not always be measured, There is no need to automatically adjust the HUD display device in accordance with the line of sight, and the HUD display of the front field of view by the infrared camera can be displayed superimposed on the driver's front line of sight. Furthermore, it is possible to reflect the driver's awareness of whether or not to perform HUD display.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of a night driving visibility support device according to the present invention.
FIG. 2 is a diagram schematically showing an embodiment (1) when the night driving visibility support device according to the present invention is mounted on an actual vehicle.
FIG. 3 is a diagram schematically showing an embodiment (2) when the nighttime driving vision support device according to the present invention is mounted on an actual vehicle.
FIG. 4 is a diagram showing an example of HUD display by the HUD display device.
FIG. 5 is a flowchart showing the operation of the nighttime driving visibility support device according to the present invention.
FIG. 6 is a diagram showing an example of HUD display (night vision) by an infrared camera.
FIG. 7 is a view showing a driver's face image taken by the night driving vision support device according to the present invention;
FIG. 8 is a diagram illustrating the principle of photographing a pupil reflection image in the nighttime driving vision support device according to the present invention.
FIG. 9 is a diagram for explaining a pupil reflection image in the nighttime driving vision assistance device according to the present invention.
FIG. 10 is a diagram illustrating a driver's pupil position extracted in the nighttime driving vision support apparatus according to the present invention.
FIG. 11 is a diagram showing a relationship between a pupil position and a specific position in the nighttime driving vision support apparatus according to the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Infrared camera 2 HUD (head up display) display device 3 Near-infrared camera for face photography 4 Image processing part 5 Pupil position detection part 6 Control part 10 Cab 11 Driver 21 Projector 22 Combiner 30 Infrared LED
110 In the eyeball diagram, the same reference numerals indicate the same or corresponding parts.

Claims (3)

In the night driving vision support device that displays the image of the infrared camera that images the front of the vehicle on the head-up display device,
A facial camera that shoots the driver's eyeballs;
An image processing unit that processes an image captured by the facial camera;
A pupil position detection unit that detects the pupil position of the eyeball from the processing data of the image processing unit;
A pupil position determination unit that determines whether or not the pupil position obtained by the pupil position detection unit is within a predetermined range that appears to overlap with an actual landscape in an image displayed by the head-up display device;
A control device that drives the head-up display device only when the pupil position detection unit determines that the pupil position is within the predetermined range;
A night driving visibility support device characterized by comprising: In claim 1,
The night driving field of view assistance device, wherein the predetermined range is initially set so that the forward field of view from the viewpoint position of the driver and the display by the HUD device coincide with each other. In claim 1,
A night driving vision support device, wherein the face photographing camera is a near-infrared camera comprising a camera and an infrared LED provided on the optical axis at the tip of the camera.

Images