JP2016049262A - Lighting imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a lighting imaging device which suppresses attenuation of reflected light from the eye by keeping efficiency for light utilization high and which comprises an irradiation part making an optical axis coaxial with an optical axis of an imaging member.SOLUTION: A lighting imaging device for detecting an object person's sight line on the basis of an image acquired by an imaging member includes an irradiation part that applies detection light to the object person's eye, and the imaging member that acquires the image including the object person's eye. The irradiation part is arranged in such a manner that an optical axis overlaps an optical axis of the imaging member, in an area that is positioned on the optical axis of the imaging member and in which the image is not formed on the imaging member.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an illumination imaging apparatus for detecting a line of sight of a subject.

  The human eye has a recursive characteristic in the reflection of light by the retina, and light incident from the pupil is strongly reflected in the direction of the light source. Therefore, in the gaze detection device, when photographing a human face, an image with a bright pupil is obtained by arranging a light source coaxially with the optical axis of the photographing camera, and a pupil image is extracted from this image. It has been proposed to detect the line of sight.

  As an apparatus for taking an image in this way, an annular light source is provided in a ring shape, and the center of gravity (virtual optical axis) of the luminance distribution of the entire light source is arranged to be the same as the optical axis of the camera. There are known a half mirror system (for example, Patent Document 1) in which a half mirror that reflects incident light on the optical axis of a camera is disposed.

  Here, as a characteristic of the eye, in order to obtain a pupil image, the illumination light needs to reach the region of the retina photographed by the camera, but the region of the retina photographed because the pupil is contracted It has been found that a pupil image is difficult to obtain when the eye is small or when the eye is in focus at a distance approximately equidistant from the camera.

Japanese Patent No. 3848953

  However, although the ring system is simple in structure, it is difficult to align the virtual optical axis of the light source and the optical axis of the camera exactly on the same axis, and the pupil is contracted in such an arrangement relationship. When the eyes are focused in the vicinity of the imaging camera, it is difficult to obtain a pupil image with a certain brightness or higher.

  Further, in the above half mirror method, when the light from the light source is reflected by the half mirror and when the incident light from the eyes is transmitted through the half mirror, the light amount is attenuated to ½ or less. The light utilization efficiency is lowered, and the incident light amount to the camera is small with respect to the emitted light amount from the light source, so that the SN ratio is likely to be lowered, and the accuracy of line-of-sight detection is lowered.

  Therefore, the present invention provides an illumination imaging apparatus having an irradiation unit that maintains high light utilization efficiency, suppresses attenuation of reflected light from the eyes, and has an optical axis that is coaxial with the optical axis of the imaging member. Objective.

In order to solve the above-mentioned problem, an irradiating unit that provides detection light to the eye of the subject and an imaging member that acquires an image including the eye, and the line of sight of the subject based on the image acquired by the imaging member In the illumination imaging device for detecting
The irradiating unit is arranged in a region on the optical axis of the imaging member and not imaged on the imaging member.

  With this configuration, even when the pupil of the subject's eyes is contracting or when the subject's eyes are focused on a range approximately equidistant from the imaging member, the pupil image is reliably acquired. This makes it possible to accurately detect the direction of the subject's line of sight regardless of the conditions during imaging.

  In the illumination imaging apparatus of the present invention, the irradiation unit is a reflection diffuser, a light source that provides light to the reflection diffuser is provided, and this light is reflected by the reflection diffuser to become the detection light. Can be configured.

This makes it possible to suppress the attenuation of the detection light in the process from the reflection diffuser to the subject side and the reflection light in the process from the subject side to the imaging member. High efficiency can be maintained. Therefore, a sufficient amount of light incident on the imaging member can be secured, so that the SN ratio can be prevented from being lowered, and pupil image extraction and gaze direction detection can be performed with high accuracy.
Alternatively, the irradiation unit may be a light source that emits the detection light.

  By using a light source as the irradiating unit, the number of constituent members can be reduced as compared with a case where a light source and a reflective diffuser are combined. Furthermore, when a laser light source that emits highly convergent light is used as the light source, the rate of incidence on the subject's eyes is higher than that of diffused light, so that the light utilization efficiency can be increased.

  In the illumination imaging apparatus of the present invention, a lens for collecting imaging light is provided on the imaging member, and the irradiation unit is fixed to the surface of the lens.

  Alternatively, a translucent plate is provided in front of the imaging member, and the irradiation unit is fixed to the surface of the transparent plate.

  Alternatively, a diaphragm is provided in front of the imaging member, and the irradiation unit is provided in front of the diaphragm.

  The illumination imaging apparatus of the present invention can be configured to include a bright pupil image extraction unit that extracts a bright pupil image from an image acquired by the imaging member.

  According to the present invention, it is possible to provide an illumination imaging apparatus including an irradiation unit that maintains high light utilization efficiency, suppresses attenuation of reflected light from the eyes, and has an optical axis that is coaxial with the optical axis of the imaging member. it can.

It is a figure which shows schematic structure of the illumination imaging device which concerns on the 1st Embodiment of this invention. It is a figure which shows schematic structure of the illumination imaging device which concerns on the 2nd Embodiment of this invention. It is a figure which shows schematic structure of the illumination imaging device which concerns on the 3rd Embodiment of this invention. It is a figure which shows schematic structure of the illumination imaging device which concerns on the 4th Embodiment of this invention. It is a block diagram which shows the structure of the gaze detection apparatus containing the illumination imaging device which concerns on embodiment of this invention. It is a top view which shows the relationship between the direction of the eyes | visual_axis of a human eye, and an illumination imaging device. It is explanatory drawing for calculating the direction of eyes | visual_axis from the pupil center and the center of corneal reflected light.

Hereinafter, an illumination imaging apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. Hereinafter, the illumination imaging device used for the gaze detection device for detecting the gaze of the subject will be described.

  FIG. 1 is a diagram showing a schematic configuration of an illumination imaging apparatus according to the first embodiment of the present invention, and FIG. 5 is a block diagram showing a configuration of a line-of-sight detection apparatus including the illumination imaging apparatus according to the embodiment. It is.

  As shown in FIG. 1, the illumination imaging apparatus 10 according to the present embodiment includes a light source 11, an imaging member 12, a reflection diffuser 13 as an irradiation unit, and a lens 14 (imaging optical system) as an optical member. Prepare.

  Further, as shown in FIG. 5, the line-of-sight detection device according to the present embodiment includes the illumination imaging device 10 and a calculation control unit CC, and is, for example, an upper part of an instrument panel or a windshield in a vehicle cabin. It is installed so as to face the driver's face as the subject.

  For example, an LED (light emitting diode) or a laser is used as the light source 11. The light source 11 is disposed outside the optical path of the light beam incident on the lens 14 and emits detection light toward the reflection diffuser 13. When the light source 11 is an LED, a condensing lens for condensing light on the reflection diffuser 13 is provided.

  The light source 11 includes two light emitting sources (light emitting chips) that emit light having two wavelengths, and infrared light having a wavelength of 850 nm (first wavelength) and infrared light having a wavelength of 940 nm (second wavelength) are used as detection light. Can be selectively emitted. Selection of the wavelength of the detection light emitted from the light source 11 and lighting / non-lighting of the light source 11 are controlled by the light source controller 21. Here, infrared light (near-infrared light) having a wavelength of 850 nm or a wavelength close thereto is not easily absorbed by moisture in the eyeball, and the amount of light that reaches and is reflected by the retina located in the back of the eyeball is large. Become. On the other hand, infrared light of 940 nm is easily absorbed by moisture in the eyeball of human eyes. Therefore, the amount of light that reaches the retina located behind the eyeball and is reflected is reduced. Note that light having a wavelength other than 850 nm and 940 nm may be used as the detection light.

  The imaging member 12 is, for example, an imaging device included in a camera, and is configured by a CMOS (complementary metal oxide semiconductor), a CCD (charge coupled device), or the like. The imaging member 12 acquires an image of a face that enters through the lens 14 and includes the eyes of the driver. In the image sensor, light is detected by a plurality of pixels arranged two-dimensionally.

  The lens 14 is a biconvex lens arranged so that its optical axis 14C overlaps the optical axis 12C of the imaging member 12, and collects incident light from the subject and emits it to the imaging member 12 side. Note that the imaging optical system is not limited to one lens 14, and the incident light from the subject may be collected by an optical system including a plurality of lenses.

  The reflection diffuser 13 is made of, for example, glass or plastic provided with a reflection diffusion surface directed toward the subject. The reflection diffusion surface has, for example, a shape or a function for diffusing incident light on the surface or inside, and further reflects incident light by a metal coating or the like. The reflection diffuser 13 is on the optical axis 12C of the imaging member 12 (the optical axis 14C of the lens 14), and is located between the subject and the lens 14 in an area near the lens 14 that does not form an image on the imaging member 12. Has been placed. In the embodiment shown in FIG. 1, the reflection diffuser 13 is directly fixed to the surface of the lens 14.

The reflection diffuser 13 has a sufficiently small shape with respect to the effective light beam diameter of the lens 14 in the radial direction of the lens 14 so as not to form an image on the imaging member 12, and the optical axis is the optical axis 12 of the imaging member 12.
It is arranged so as to overlap with C. That is, the reflection diffuser 13 and the imaging member 12 have the same optical axis. The reflection diffuser 13 reflects and diffuses the detection light incident from the light source 11 toward the subject side with the direction of the optical axis 12C of the imaging member 12 as the center, thereby providing detection light to the subject's eyes.

  The arithmetic control unit CC is composed of a CPU and a memory of a computer, and the function of each block shown in FIG. 5 is calculated by executing software installed in advance.

  The arithmetic control unit CC includes a light source control unit 21, an image acquisition unit 22, a pupil image extraction unit 30, a pupil center calculation unit 33, a corneal reflection light center detection unit 34, and a gaze direction calculation unit 35. It has been.

  The image acquired by the imaging member 12 is acquired by the image acquisition unit 22 for each frame. The image acquired by the image acquisition unit 22 is read into the pupil image extraction unit 30 for each frame. The pupil image extraction unit 30 includes a bright pupil image detection unit 31 and a dark pupil image detection unit 32.

  FIG. 6 is a plan view schematically showing the relationship between the direction of the line of sight of the eye 40 of the subject and the illumination imaging apparatus 10. FIG. 7 is an explanatory diagram for calculating the direction of the line of sight from the center of the pupil and the center of the corneal reflected light. 6A and 7A show a case where the line-of-sight direction VL of the subject is along the optical axis 12C of the imaging member 12, and FIGS. 6B and 7B show the line-of-sight direction VL. This is a case where is deviated from the optical axis 12 </ b> C of the imaging member 12.

  The eye 40 has a cornea 41 at the front, and a pupil 42 and a crystalline lens 43 are positioned behind the cornea 41. And the retina 44 exists in the last part.

  Since the detection light having a wavelength of 850 nm is likely to be reflected by reaching the retina 44, when the light source 11 is turned on, in the image acquired by the imaging member 12 coaxial with the light source 11, the infrared light reflected by the retina 44 is in the pupil. 42 is detected and the pupil 42 appears bright. This image is extracted as a bright pupil image by the bright pupil image detection unit 31. On the other hand, since the detection light having a wavelength of 940 nm is easily absorbed in the eyeball before reaching the retina 44, infrared light from the retina 44 is transmitted from the retina 44 in the image acquired by the imaging member 12 when the light source 11 is turned on. It is hardly reflected and the pupil 42 looks dark. This image is extracted as a dark pupil image by the dark pupil image detection unit 32.

  On the other hand, both the detection light with a wavelength of 850 nm and a wavelength of 940 nm is reflected by the surface of the cornea 41, and the reflected light is detected by both the bright pupil image detection unit 31 and the dark pupil image detection unit 32. In particular, in the dark pupil image detection unit 32, since the image of the pupil 42 is dark, the reflected light reflected from the reflection point 45 of the cornea 41 is detected as a bright spot image.

  The pupil image extraction unit 30 subtracts the dark pupil image detected by the dark pupil image detection unit 32 from the bright pupil image detected by the bright pupil image detection unit 31 and the pupil image signal in which the shape of the pupil 42 is brightened. Is generated. This pupil image signal is given to the pupil center calculation unit 33. In the pupil center calculation unit 33, the pupil image signal is subjected to image processing and binarized, and an area image corresponding to the shape and area of the pupil 42 is calculated. Further, an ellipse including this area image is extracted, and the intersection of the major axis and the minor axis of the ellipse is calculated as the center position of the pupil 42.

The dark pupil image signal detected by the dark pupil image detection unit 32 is given to the corneal reflection light center detection unit 34. The dark pupil image signal includes a luminance signal by reflected light reflected from the reflection point 45 of the cornea 41 shown in FIGS. 6 and 7. The reflected light from the reflection point 45 of the cornea 41 forms a Purkinje image. As shown in FIG. 7, a spot image with a very small area is acquired on the imaging element of the imaging member 12. In the corneal reflection light center detection unit 34, the spot image is subjected to image processing, and the center of the reflected light from the reflection point 45 of the cornea 41 is obtained.

  The pupil center calculation value calculated by the pupil center calculation unit 33 and the corneal reflection light center calculation value calculated by the corneal reflection light center detection unit 34 are given to the gaze direction calculation unit 35. The line-of-sight direction calculation unit 35 detects the direction of the line of sight from the pupil center calculated value and the corneal reflection light center calculated value.

  In the case shown in FIG. 6A, the line-of-sight direction VL of the human eye 40 coincides with the optical axis 12C of the imaging member 12 of the imaging member 12. At this time, as shown in FIG. 7A, the center of the reflection point 45 from the cornea 41 coincides with the center of the pupil 42. On the other hand, in the case shown in FIG. 6B, the line-of-sight direction VL of the human eye 40 is directed in a direction different from the optical axis 12C of the imaging member 12. At this time, as shown in FIG. 7B, the center of the pupil 42 and the center of the reflection point 45 from the cornea 41 are displaced.

  The line-of-sight direction calculation unit 35 calculates a linear distance α between the center of the pupil 42 and the center of the reflection point 45 from the cornea 41 (FIG. 7B). Further, an XY coordinate having the center of the pupil 42 as an origin is set, and an inclination angle β between the line connecting the center of the pupil 42 and the center of the reflection point 45 and the X axis is calculated. Further, the line-of-sight direction VL is calculated from the linear distance α and the inclination angle β.

  In order to calculate the gaze direction VL with high accuracy in the gaze direction calculation unit 35, it is necessary to detect the center coordinates of the pupil 42 and the center coordinates of the reflection point 45 with high accuracy.

With the configuration described above, the following effects are achieved according to the above embodiment.
(1) Since the reflective diffuser 13 is arranged so that the optical axis is coaxial with the optical axis 12C of the imaging member 12, the pupil of the subject's eyes is contracted, or the subject's eyes are in contact with the imaging member 12. Even when focusing on a substantially equidistant range, pupil images (bright pupil images, dark pupil images) can be reliably acquired, so that the target person can be obtained regardless of the conditions at the time of imaging. It is possible to accurately detect the line-of-sight direction.

(2) The light emitted from the light source 11 is reflected and diffused toward the subject side by the reflection diffuser 13 and is provided as detection light. Further, the light from the subject's eyes is in a region where no image is formed on the imaging member 12. The light is collected by the lens 14 without being influenced by the reflection diffuser 13 disposed and forms an image on the imaging member 12. For this reason, the attenuation of the detection light in the process from the reflection diffuser 13 to the subject side and the reflection light in the process from the subject side to the imaging member 12 can be suppressed, and the light emitted from the light source 11 can be suppressed. High utilization efficiency can be maintained. As a result, a sufficient amount of light incident on the imaging member 12 can be secured, so that the SN ratio can be prevented from being lowered, and pupil image extraction and gaze direction detection can be performed with high accuracy.

Other embodiments will be described below.
(1) In the second embodiment shown in FIG. 2, a translucent plate 15 serving as a lens cover is provided in front of the lens 14, and a reflective diffuser 13 as an irradiation unit is provided on the surface of the translucent plate 15. It is fixed. The translucent plate 15 is a glass plate or a plastic plate.

(2) In the third embodiment shown in FIG. 3, a diaphragm 17 is provided between the imaging member 12 and the lens 12 so that the depth of focus of an image detected by the imaging member 12 is deep. In this case, the reflection diffuser 13 as an irradiation unit is disposed in front of the diaphragm 17. For example, a translucent plate 16 is disposed between the lens 14 and the diaphragm 17, and the reflective diffuser 13 is disposed on the translucent plate 16.

(3) In the fourth embodiment shown in FIG. 4, instead of the light source 11 and the reflective diffuser 13, a light source 111 that emits detection light as an irradiation unit is disposed in the vicinity of the lens 14. In FIG. 4, the light source 111 is fixed to the surface of the translucent plate 15.

  For example, an LED (light emitting diode) or a laser is used as the light source 111. The light source 111 is disposed on the optical axis 12 </ b> C of the imaging member 12 and in a region near the lens 14 that does not form an image on the imaging member 12 between the subject and the lens 14. The light source 111 has a shape that is sufficiently small with respect to the effective light beam diameter of the lens 14 in the radial direction of the lens 14, and is disposed so that the optical axis overlaps the optical axis 12 </ b> C of the imaging member 12. That is, the light axes of the light source 111 and the imaging member 12 are coaxial with each other. The light source 111 emits detection light in the direction of the optical axis 12C of the imaging member 12 and provides detection light to the subject's eyes.

  The light source 111 has, for example, two light sources (light emitting chips) that emit light of two wavelengths, and selectively emits infrared light with a wavelength of 850 nm and infrared light with a wavelength of 940 nm as detection light. Selection of the wavelength of the detection light emitted from the light source 111 and lighting / non-lighting of the light source 111 are controlled by the light source controller 21.

  By using the light source 111 as the irradiating unit, it is possible to reduce the number of constituent members as compared to the above embodiments in which the light source 11 and the reflective diffuser 13 are used. Furthermore, when a laser light source that emits highly convergent light is used as the light source 111, the ratio of light incident on the subject's eyes is higher than that of diffused light, so that the light utilization efficiency can be increased.

  Although the present invention has been described with reference to the above embodiment, the present invention is not limited to the above embodiment, and can be improved or changed within the scope of the purpose of the improvement or the idea of the present invention.

  As described above, the illumination imaging device according to the present invention is useful for illumination and imaging in the visual line detection device for detecting the visual line of the subject.

DESCRIPTION OF SYMBOLS 10 Illumination imaging device 11 Light source 12 Imaging member 12C Optical axis 13 Reflection diffuser 14 Lens 14C Optical axis 15, 16 Transmissive plate 17 Aperture 30 Pupil image extraction part 31 Bright pupil image detection part 32 Dark pupil image detection part 33 Pupil center calculation Unit 34 cornea reflected light center detection unit 35 line-of-sight direction calculation unit 60 eyes 111 light source

Claims (7)

In an illumination imaging apparatus that includes an irradiation unit that supplies detection light to the eyes of a subject and an imaging member that acquires an image including the eyes, and that detects the line of sight of the subject based on an image acquired by the imaging member.
The illumination imaging apparatus, wherein the irradiation unit is disposed in a region on the optical axis of the imaging member and not imaged on the imaging member.   The illumination imaging apparatus according to claim 1, wherein the irradiating unit is a reflection diffuser, a light source that provides light to the reflection diffuser is provided, and the light is reflected by the reflection diffuser to become the detection light.   The illumination imaging apparatus according to claim 1, wherein the irradiation unit is a light source that emits the detection light.   The illumination imaging apparatus according to any one of claims 1 to 3, wherein a lens that collects imaging light is provided on the imaging member, and the irradiation unit is fixed to a surface of the lens.   The illumination imaging device according to any one of claims 1 to 3, wherein a translucent plate is provided in front of the imaging member, and the irradiation unit is fixed to a surface of the transparent plate.   The illumination imaging apparatus according to claim 1, wherein a diaphragm is provided in front of the imaging member, and the irradiation unit is provided in front of the diaphragm.   The illumination imaging apparatus according to claim 1, further comprising a bright pupil image extracting unit that extracts a bright pupil image from an image acquired by the imaging member.

Images