JP2009096323A - Camera illumination control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera illumination control device capable of realizing the light flooding quantity of a light source optimum for pupillary extraction while suppressing an increase of the cost. <P>SOLUTION: The camera illumination control device has an output calculation unit 2 for performing the processing of Step S1 of calculating the optical axis of a near infrared ray camera unit 61, the processing of Step S2 of calculating the body axis of a driver, the processing of Step S3 of computing the position of a face of the driver, the processing of Step S4 of computing the distance between the near infrared ray camera unit 61 and the position of the face of the driver, the processing of Step S5 of calculating the intensity of the external light based on information obtained from a solar radiation sensor 8, and the processing of Step S6 of computing the output of the near infrared ray floodlight 3 based on the image pickup distance and the intensity of the external light. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention belongs to the technical field of a camera illumination control device for a camera used for detecting a driver's line of sight in a vehicle.

Conventionally, when capturing a pupil reflection image, a dark pupil reflection image is captured using a light source that is not on the optical axis of the camera, and a bright pupil reflection image is captured by placing a light source on the optical axis of the camera, The driver's line of sight is detected by processing these two images (see, for example, Patent Document 1).
In some cases, the pupil position is detected from a difference image between the bright pupil image and the dark pupil image without using a marker or the like (see, for example, Patent Document 2).
Japanese Patent Laid-Open No. 11-263145 (page 2-5, full view) JP 2005-348832 A (page 2-9, full view)

  However, in the conventional camera illumination control device, it is not sufficient to make the light projection amount of the light source appropriate.

  The present invention has been made paying attention to the above-mentioned problems, and an object of the present invention is to provide a camera illumination control device that can suppress the increase in cost by making the light source projection amount optimal for pupil extraction. There is to do.

  In order to achieve the above object, in the present invention, in order to detect the driver's line of sight, camera illumination that projects near-infrared light from the vicinity of the camera that captures the bright and dark pupil images of the driver pupil to the driver's pupil In the camera illumination control device that controls the camera, a camera optical axis calculation unit that calculates the optical axis of the camera based on information obtained from the vehicle device, and a driver that calculates the body axis of the driver based on information obtained from the vehicle device Body axis calculation means, driver face position calculation means for calculating the driver's face position from the camera optical axis and the driver body axis, imaging distance calculation means for calculating the distance between the camera and the driver face position, Outside light intensity calculation means for calculating the light intensity based on information obtained from the vehicle device, and calculating the output of the camera illumination based on the imaging distance and the outside light intensity Characterized by comprising a force calculating means.

  Therefore, in the present invention, it is possible to make the light source projection amount optimal for pupil extraction while suppressing an increase in cost.

  Hereinafter, an embodiment for realizing the camera illumination control device of the present invention will be described based on Example 1 corresponding to the first, second, and third aspects of the invention.

First, the configuration will be described.
FIG. 1 is a diagram illustrating a block configuration of the camera illumination control apparatus according to the first embodiment.
The camera illumination control device 1 according to the first embodiment includes an output calculation unit 2, a near infrared projector 3, a steering adjustment mechanism unit 4, and a seat adjustment mechanism unit 5 as main components.
The output calculation unit 2 calculates the body axis of the driver from the state of the power seat in the driver's seat, the steering position, and the positional relationship between them, and calculates the distance from the camera to the driver. Then, the external light intensity of solar radiation is obtained, and the optimum infrared light projection amount is calculated and output.
The near-infrared projector 3 performs near-infrared light projection in the pupil detection camera according to the calculated calculated value of the optimum infrared light projection amount. Details will be described later.

The steering adjustment mechanism unit 4 includes a tilt angle output unit 41 and a telescopic unit 42, and outputs information on the steering position to the output calculation unit 2.
The tilt angle output unit 41 outputs a tilt angle adjustment position to be adjusted as a steering position to the output calculation unit 2.
The telescopic unit 42 outputs to the output calculation unit 2 the adjustment position of the steering front-rear position that is adjusted as the steering position.

The seat adjustment mechanism unit 5 includes a seat front / rear position output unit 51 and a reclining angle output unit 52, and outputs information on the state of the power seat to the output calculation unit 2.
The seat front / rear position output unit 51 outputs information on the seat front / rear position to the output calculation unit 2 as adjustment of the power seat.
The reclining angle output unit 52 outputs information on the reclining angle of the seat to the output calculation unit 2 as adjustment of the power seat.

Next, the schematic structure of a camera using a near infrared projector will be described.
FIG. 2 is an explanatory diagram showing a schematic structure of the near-infrared camera.
The near-infrared camera unit 6 that is controlled by the camera illumination control device 1 according to the first embodiment is a camera that combines a near-infrared camera unit 61 and two near-infrared light sources.
In the near-infrared camera unit 61, a plurality of LEDs that emit near-infrared light are arranged so as to surround the periphery of the camera portion provided with a visible light cut filter. Thereby, the inner LED portion 31 is formed. The inner LED unit 31 performs light projection coaxially with the near-infrared camera unit 61. That is, it becomes a light source for capturing a bright pupil image.

Next, the two near-infrared light sources are light sources provided separately from the near-infrared camera unit 61, and a plurality of LEDs are arranged on each of them to constitute the outer LED units 32a and 32b. The outer LED units 32 a and 32 b perform light projection on another optical axis that is separated from the near-infrared camera unit 61. That is, it becomes a light source for capturing a dark pupil image.
Note that the output image of the near-infrared camera unit 61 is based on the NTSC system, and the line-of-sight detection device 12 that receives the image processes the image and performs line-of-sight detection.

Next, vehicle installation states such as a near-infrared camera and a near-infrared projector will be described.
FIG. 3 is an explanatory diagram showing the installation state of the vehicle.
The seat on which the driver h is seated is a so-called power seat 7, and the position adjustment is performed electrically. The power seat 7 includes the above-described seat adjustment mechanism portion 5.
A solar radiation sensor 8 is provided on the upper surface of the instrument panel 9 as a sensor used by the air conditioner system.
The arrangement portion of the steering wheel 10 includes the steering adjustment mechanism 4 described above for adjustment. A near-infrared camera unit 6 is disposed on the upper surface of the steering column 11 so as to capture an image in the vicinity of the eyes of the driver h through a portion penetrating the steering wheel 10 before and after.

  Here, the position and state information regarding the power seat 7, the solar radiation sensor 8, and the steering wheel 10 are originally used in other devices, and the output calculation unit 2 of the first embodiment does not increase the cost. Can get information.

The operation will be described.
[Camera lighting control for driver gaze detection]
FIG. 4 is a flowchart showing the flow of the camera illumination control process for driver gaze detection executed by the output calculation unit of the camera illumination control apparatus according to the first embodiment. Each step will be described below.

  In step S1, the camera optical axis of the near-infrared camera unit 61 is calculated from the tilt angle information and telescopic information of the steering adjustment mechanism unit 4.

  In step S2, the center of the driver for determining the driver's line of sight is set based on the body axis of the driver, that is, the body to the head, based on the seat front-rear position information and the reclining angle information of the seat adjustment mechanism unit 5. Calculate the line.

  In step S3, the position of the driver's face is calculated from the camera optical axis of the near-infrared camera unit 61 and the driver body axis.

  In step S4, the distance between the position of the driver's face and the position of the near-infrared camera unit 61 is calculated.

  In step S5, the external light intensity is detected from information from the solar radiation sensor 8.

  In step S6, the intensity of the near infrared ray projected by the near infrared projector 3 is calculated and output.

[Pupil detection effect]
FIG. 5 is an explanatory diagram of a bright pupil state and a dark pupil state. FIG. 6 is an explanatory diagram of each state of pupil detection.
In the gaze detection of the driver using the camera illumination control device of the first embodiment, images of the bright pupil state and the dark pupil state are used.
The bright pupil state (bright pupil phenomenon) is a state in which the imaged pupil (cornea) appears to shine by disposing the inner LED unit 31 as a light source near the optical axis of the near-infrared camera unit 61 (FIG. 5 (a)). This is the same generation principle as the so-called red-eye phenomenon when shooting with a visible light camera. The image captured in this way is referred to as a bright pupil image 101.

  On the other hand, in the dark pupil state (dark pupil phenomenon), the outer LED portions 32 a and 32 b are arranged as light sources at positions away from the optical axis of the near infrared camera portion 61. As a result, the pupil (cornea) to be imaged appears to appear dark (see FIG. 5B). The image captured in this way is referred to as a dark pupil image 102.

In the driver's line-of-sight detection of the first embodiment, the difference image 103 between the bright pupil image 101 and the dark pupil image 102 is captured by the near-infrared camera unit 6 at approximately the same time and at the same position, and input to the line-of-sight detection device 12.
The line-of-sight detection device 12 generates a difference image 103 in which a difference between the data is extracted from the bright pupil image 101 and the dark pupil image 102, that is, a change (difference) is extracted like an image in which the pupil is shining and an image in which the pupil is dark. To do.
The difference image 103 is binarized with a predetermined threshold value to generate a binarized image 104. Then, since the difference between the bright pupil image 101 and the dark pupil image 102 is the main difference between the two images, pupil candidates are detected. From here, the pupil position is detected by performing processing such as making only two adjacent points into pupils. The line of sight is determined from the relationship between the fixed camera position and the pupil position. As a result, for example, it is possible to detect a driver's looking away.

[Camera lighting control function]
This gaze detection, called the bright / dark pupil method, is relatively simple. In this line-of-sight detection, it is necessary that an appropriate amount of infrared light is projected.
For example, if the amount of light emitted by infrared rays is too strong, the captured image becomes an image that becomes too white (flys white), and the pupil cannot be extracted. On the other hand, if the near-infrared light projection amount is too weak, the near-infrared light projection is lost to external light (disturbance), and the bright pupil state (bright pupil phenomenon) cannot be obtained.

FIG. 7 is an explanatory diagram of the driver's face position. 8 and 9 are explanatory diagrams regarding the calculation of the driver's face position.
The near-infrared camera unit 6 is fixed to the upper surface of the steering column 11. Therefore, the camera position moves together with the adjustment of the steering adjustment mechanism section 4.
That is, if the driver adjusts the steering adjustment position to an easy-to-handle position, the camera position is also adjusted to a position that matches the driver.

Therefore, in the first embodiment, the tilt angle information (tilt angle α) from the tilt angle output unit 41 of the steering adjustment mechanism unit 4 and the steering front / rear position information (adjustment position A) from the telescopic unit 42 are obtained. And the camera optical axis 201 of the near-infrared camera part 61 of the near-infrared camera part 6 is calculated from this information (refer to the process of step S1, FIG. 8, FIG. 9).
Since the near-infrared camera unit 61 is fixed to the upper surface of the steering column 11, the angle of the camera optical axis is the same as the tilt angle α. When the angles are different, the difference is added to or subtracted from the adjusted tilt angle α.
In other words, in the first embodiment, the camera optical axis 201 is calculated while suppressing the cost from the tilt angle and the front-rear position detected for storing the steering adjustment position or controlling the position change.

Next, information on the seat front / rear position (adjustment position B) from the seat front / rear position output unit 51 of the seat adjustment mechanism unit 5 and the reclining angle β from the reclining angle output unit 52 is obtained.
Then, the driver's body axis 202 is calculated from this information (step S2, see FIGS. 8 and 9).

  The driver's body axis 202 and the driver's face position change as shown in FIGS. 7A and 7B depending on the seat front-rear position and the reclining angle of the power seat. In the first embodiment, a power seat that is widely used stores the driver position, and obtains the seat longitudinal position detected for the position change control and the reclining angle β to reduce the cost of the driver's body axis 202. To calculate.

Next, as shown in FIG. 6, the intersection position of the camera optical axis 201 and the driver body axis 202 is obtained and set as the driver face position 203 (see step S3, FIG. 9).
Then, a distance C between the driver face position 203 and the near-infrared camera unit 61 is calculated (step S4).
Furthermore, the inner LED unit provided as the near-infrared projector 3 from the external light intensity (step S5) from the solar radiation sensor 8 provided for the air conditioning system and the distance C between the driver face position 203 and the near-infrared camera unit 61. 31, the near-infrared light projection intensity of the outer LED portions 32a and 32b is calculated.

  Thereby, the state C of the external light and the distance C to the driver's face are accurately obtained, and the near-infrared light projection intensity is determined, so that the bright pupil image and the dark pupil image can be surely obtained. Furthermore, the cost is greatly reduced by the information obtained from existing devices.

Further explanation will be given.
When the distance between the driver face position 203 and the near-infrared camera unit 61 is short, the output of the near-infrared projector 3 is reduced. This adjusts the output level so that the light reaches the driver's face. The output is lowered so that the image is not overexposed (a white image) due to strong light projection due to the close distance.

  Next, when the distance between the driver face position 203 and the near-infrared camera unit 61 is long, the output of the near-infrared projector 3 is increased. This adjusts the output level so that the light reaches the driver's face. Since the light is attenuated due to the long distance, the output is increased accordingly.

  Next, when the external light intensity is weak, the output of the near-infrared projector 3 is lowered. l This is to adjust the output level so that the light is not lost to the outside light and the light is not blocked by the outside light and a bright pupil image is not generated. That is, in this case, the output is lowered because the light projection is not covered by outside light.

Next, when the external light intensity is high, the output of the near-infrared projector 3 is increased. This is to adjust the output level so that the light is not lost due to the external light and the bright light image is not generated due to the light being blocked by the external light. That is, in this case, the output is increased in order to prevent the light projection from being covered with outside light.
Note that these output adjustments are performed within a range that takes into consideration the influence on the human body.

Consider providing another sensor for distance measurement to the camera illumination control apparatus of the first embodiment. Then, for example, if the user is crouching for the passenger's baggage, the distance to the headrest is recognized as the distance to the face.
In contrast, in the first embodiment, it is assumed that the driver body axis 202 is at an appropriate position as shown in FIGS. 7A and 7B with respect to the sheet. Then, the distance to the headrest is not recognized as the distance to the face.

Further, consider that the camera illumination control apparatus according to the first embodiment performs automatic correction from the brightness of a captured image. Then, when the image of the camera is blocked and the screen becomes completely dark, correction is necessary each time, and the fluctuation of the image becomes large.
In the first embodiment, since automatic correction is not performed in this way, even if the image of the camera is interrupted, there is no great variation in the near infrared ray that is calculated and output.

  In such automatic correction, when the camera position is set as in the first embodiment, the support portion from the rotating shaft of the steering wheel 10 frequently blocks the camera during the steering operation. In Example 1, there is no problem even if such a phenomenon occurs.

  Further, consider a feedback method in which the camera illumination control apparatus according to the first embodiment is calculated from the brightness of the current captured image and the projection output is changed in the next frame image. In the feedback method, the output is weakened when the image is bright, and the output is strengthened when the image is dark. In this feedback method, it takes time until the optimum output value is reached in order to adjust the output step by step for each frame.

Further, when the camera is mounted on the steering column, the camera field of view is often obstructed by the steering operation and becomes dark. In addition, when the hand being operated comes near the near-infrared projector, it frequently occurs that the screen is completely white due to a strong projection on the hand. It takes time to perform simple feedback control when returning to an ordinary image from such an extremely bright image.
In the first embodiment, since the optimum value of the light projection amount is obtained in advance from the distance to the face and the external light intensity, the return to the normal state becomes very fast.

Next, the effect will be described.
In the camera illumination control device of the first embodiment, the effects listed below can be obtained.

  (1) Control of the near-infrared projector 3 for projecting near-infrared light from the vicinity of the near-infrared camera unit 61 that captures the bright pupil image and dark pupil image of the driver pupil to the driver's pupil in order to detect the driver's line of sight In the camera illumination control device 1 that performs the processing, the process of step S1 for calculating the optical axis of the near-infrared camera unit 61 based on the information obtained from the steering adjustment mechanism unit 4, and the information obtained from the seat adjustment mechanism unit 5 for the driver's body axis The process of step S2 calculated based on the above, the process of step S3 of calculating the driver's face position from the camera optical axis and the driver body axis, and the step S4 of calculating the distance between the near-infrared camera unit 61 and the driver face position. The process, the process of step S5 for calculating the external light intensity based on the information obtained from the solar sensor 8, and the near-infrared projector 3 based on the imaging distance and the external light intensity. It may due to an output calculation unit 2 which performs processing in step S6 for calculating the force to be the optimal source projection amount to the pupil extraction, while suppressing an increase in cost.

  (2) The near-infrared camera unit 61 is attached to the steering column 11, and the processing of step S1 by the output calculation unit 2 includes the tilt angle from the tilt angle output unit 41 of the steering adjustment mechanism unit 4 and the steering from the telescopic unit 42. Since the camera optical axis is calculated based on the front and rear positions, the cost can be suppressed by the information from the existing steering adjustment mechanism unit 4, and the light source projection amount optimal for pupil extraction can be obtained.

  (3) The process of step S2 by the output calculation unit 2 is performed based on the seat longitudinal direction from the seat longitudinal position output unit 51 of the power seat adjustment mechanism unit 5 and the reclining angle from the reclining angle output unit 52. Therefore, the cost can be suppressed by the information from the existing sheet adjustment mechanism unit 5, and the light source projection amount optimal for pupil extraction can be obtained.

As described above, the camera lighting control device of the present invention has been described based on the first embodiment, but the specific configuration is not limited to these embodiments, and the invention according to each claim of the claims is not limited thereto. Design changes and additions are allowed without departing from the gist.
The camera optical axis and driver body axis may be calculated by obtaining information from another vehicle device. In that case, the vehicle device is preferably an existing device.

It is a figure which shows the block configuration of the camera illumination control apparatus of Example 1. FIG. It is explanatory drawing which shows schematic structure of a near-infrared camera. It is explanatory drawing which shows the installation state of a vehicle. It is a flowchart which shows the flow of the camera illumination control process for driver gaze detection performed by the output calculation part of the camera illumination control apparatus of Example 1. FIG. It is explanatory drawing of a bright pupil state and a dark pupil state. It is explanatory drawing of each state of pupil detection. It is explanatory drawing of the face position of a driver. It is explanatory drawing regarding calculation of the face position of a driver. It is explanatory drawing regarding calculation of the face position of a driver.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera illumination control apparatus 2 Output calculation part 3 Near-infrared light projector 31 Inner LED part 32a Outer LED part 32b Outer LED part 4 Steering adjustment mechanism part 41 Tilt angle output part 42 Telescopic part 5 Seat adjustment mechanism part 51 Seat front-and-rear position output part 52 Reclining angle output unit 6 Near-infrared camera unit 61 Near-infrared camera 7 Power seat 8 Solar radiation sensor 9 Instrument panel 10 Steering wheel 11 Steering column 12 Gaze detection device 101 Bright pupil image 102 Dark pupil image 103 Difference image 104 Binary image 201 Camera Optical axis 202 Driver body axis 203 Driver face position A (steering) adjustment position B (sheet) adjustment position C (camera and face) distance h Driver α Tilt angle β Reclining angle

Claims (3)

In the camera illumination control device for controlling the camera illumination for projecting near infrared light from the vicinity of the camera that captures the bright pupil image and the dark pupil image of the driver pupil to the driver pupil to detect the driver's line of sight,
Camera optical axis calculation means for calculating the optical axis of the camera based on information obtained from the vehicle device;
Driver body axis calculation means for calculating the body axis of the driver based on information obtained from the vehicle device;
Driver face position calculating means for calculating a driver's face position from the camera optical axis and the driver body axis;
Imaging distance calculation means for calculating the distance between the camera and the driver face position;
An external light intensity calculating means for calculating the external light intensity based on information obtained from the vehicle device;
Output computing means for computing the output of the camera illumination based on the imaging distance and the external light intensity;
A camera illumination control device comprising: The camera illumination control device according to claim 1,
The camera is attached to the steering column,
The camera optical axis calculation means is
Calculate the camera optical axis based on the tilt angle from the steering adjustment device and the front and rear position of the steering,
A camera illumination control device characterized by that. In the camera illumination control device according to claim 1 or 2,
The driver body axis calculation means includes
Calculate the driver body axis based on the seat front and back position and reclining angle from the power seat,
A camera illumination control device characterized by that.