JP6360752B2 - Illumination imaging device - Google Patents

Description

  The present invention relates to a line-of-sight detection device capable of detecting the line-of-sight direction of a driver of a car and other subjects, a lighting imaging device used for a monitoring device, and the like.

  Patent Document 1 describes a method of detecting a gazing point using an illumination imaging device. In this gaze point detection method, two cameras are directed to the eyes of the subject. Two types of light sources are concentrically arranged around each camera, a light source that emits light having a center wavelength of 850 nm is arranged along an inner circle near the camera, and a center wavelength is arranged along the outer circle. A light source emitting 950 nm light is disposed. By irradiating with 850 nm light, a bright pupil image can be acquired with a camera, and by irradiating with 950 nm light, a dark pupil image is acquired.

  In this gazing point detection method, a pupil image is acquired based on the bright pupil image and the dark pupil image, and the corneal reflection point of the light source is acquired from the dark pupil image. Based on this image, a vector from the subject's cornea reflection point to the pupil on a plane perpendicular to the reference line connecting the camera and the pupil is calculated, and based on this vector, the subject's line of sight with respect to the reference line of each camera Is calculated using a predetermined function.

International Publication 2012/020760

  In the gaze point detection method described in Patent Document 1, two types of light sources are arranged around the camera. However, it is preferable as an appearance of the apparatus to arrange the camera and the two types of light sources so that they are exposed in space. It is preferable to provide a light guide cover plate that covers the front in the imaging direction of the camera and the front in the light emission direction of the light source.

  However, when such a cover plate is provided, a part of the light emitted from the light source is reflected inside the cover plate and reaches the camera, and a part of the image acquired by the camera due to this light component is overexposed. May be. Light having a wavelength of 850 nm emitted from a light source near the light source has high light intensity. Therefore, when this light is acquired by the camera, a part of the image is clearly overexposed.

  Further, when the optical axes of the lenses of the two cameras and the optical axis of the light source are arranged obliquely with respect to the inner surface of the glass plate so as to face the human face or eye, the optical axis and the inner surface form an obtuse angle. The light emitted from the light source arranged on the side is reflected inside the cover plate and easily irradiates the lens of the camera.

  The present invention solves the above-described conventional problems, and provides an illumination imaging apparatus that is provided with a cover plate that covers the camera and the light source in common, and that can prevent the camera from being overexposed due to reflection of the cover plate. The purpose is to do.

The present invention provides a camera that acquires an image of an imaging target, a plurality of light sources that provide detection light to the imaging target, and a translucent cover plate that covers in common the imaging direction front of the camera and the light emission direction front of the light source. In an illumination imaging device having:
A plurality of the light sources are arranged so as to sandwich the camera, and an optical axis of the camera and an optical axis of each of the light sources extend in parallel to each other, and the optical axis of the camera and the optical axis of the light source are the cover plate Is obliquely incident on the inner surface of
A shielding member is provided that partitions the front of the camera in the imaging direction and the light emission direction of the light source, and the shielding member is inclined toward the inner surface of the cover plate with an opening angle with respect to the optical axis of the camera. Extending,
On the inner surface of the cover plate, a recess is formed at the boundary between the camera in the imaging direction and the light source in the emission direction, and the recess is located on or near the extension of the shielding member. It is characterized by this.

In the illumination imaging device of the present invention, it is preferable that an extension line of the inner surface of the shielding member on the camera side and an inner surface of the concave portion are located on the same line .

  The illumination imaging device according to the present invention is provided with a recess on the inner surface of the cover plate, so that even if light emitted from the light source enters the cover plate, it can be prevented from propagating in the direction of the camera. Thus, the camera can be prevented from being overexposed.

In the present invention, it is preferable that the cross-sectional shape of the recess is a shape in which the opening width gradually decreases from the inner surface toward the outer surface of the cover plate .

  In the present invention, it is preferable that the concave portion is filled with a light absorbing material that absorbs light emitted from the light source.

Illuminating the imaging apparatus of the present invention is effective in those previous SL light source is located closer to the inner surface than the lens of the camera.

  In the illumination imaging apparatus, even when the optical axis of the camera and the optical axis of the light source are arranged obliquely with respect to the inner surface of the cover plate, the light from the light source can be restricted from being acquired by the camera via the cover plate. It becomes like this.

  The present invention includes a first light source that emits light having a wavelength that is easily reflected by the retina of a human eye, and a second light source that emits light having a wavelength that is less likely to be reflected by the retina than the first light source. The concave portion is formed at a boundary portion between the light emitting direction front of the first light source and the imaging direction front of the camera.

  Since the light from the first light source is high in energy and easily affects the camera, the influence of the light from the first light source on the camera can be reduced by forming a recess at the boundary between the first light source and the camera.

  In the present invention, a plurality of the cameras are provided, and the light sources are arranged at positions close to the respective cameras, and the cover plate has a common front in the imaging direction of all the cameras and a front in the emission direction of all the light sources. Can be configured as covering.

  By providing a cover plate that covers all cameras and all light sources in common, the appearance of the apparatus can be improved.

  Even if the illumination imaging device of the present invention is provided with a cover plate that covers the front of the camera and the light source, the light emitted from the light source can be prevented from propagating through the cover plate and reaching the camera. Can be prevented from entering the camera directly through the cover plate.

  Further, even when the optical axis of the camera and the optical axis of the light source are arranged obliquely with respect to the inner surface of the cover plate, it is possible to restrict the light from the light source from entering the camera. Therefore, even if the two cameras and the light source adjacent to the two cameras are arranged obliquely so that the optical axis thereof faces the object to be imaged, all the cameras and all the light sources are covered with a flat cover plate. And the appearance of the apparatus can be improved.

(A) is a top view of the illumination imaging device which concerns on the 1st Embodiment of this invention, (B) is the front view, The expanded sectional view which cut | disconnected the illumination imaging device shown in FIG. 1 (B) by the II-II line | wire, Sectional drawing which expands and shows a part of FIG. (A) (B) (C) is an enlarged sectional view showing a modified example of the recess formed in the cover plate, (A) (B) is explanatory drawing which shows typically the positional relationship of an illumination imaging device and a subject's eyes, (A) (B) is explanatory drawing which shows calculation of a gaze detection, Block diagram showing the structure of a line-of-sight detection device using an illumination imaging device, (A) is the front view of the illumination imaging device which concerns on the 2nd Embodiment of this invention, (B) is the side view,

  Hereinafter, an illumination imaging apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment, the illumination imaging device is used as a line-of-sight detection device for in-vehicle use.

<Structure of illumination imaging device>
As shown in FIGS. 1 and 2, the illumination imaging apparatus 1 according to the embodiment of the present invention has a case 2. The case 2 has a cubic shape elongated in the left-right direction, and an opening 3 is formed in the front. The shape of the opening 3 is rectangular, and this opening is covered with a single cover plate 4. The cover plate 4 can transmit infrared light, but is colored so that the light transmittance is reduced in the visible light wavelength band. Alternatively, the cover plate 4 is translucent and is configured by laminating colored sheets that reduce the light transmittance. Therefore, in the front view of FIG. 1B, the internal structure of the case 2 cannot actually be seen through the cover plate 4.
A first image receiving device 10 and a second image receiving device 20 are housed inside the case 2.

  As shown in FIG. 1, the optical axis 13c of the first camera 13 provided in the first image receiving device 10 and the optical axis 23c of the second camera 23 provided in the second image receiving device 20 are separated by a predetermined distance L1. Are arranged to be. The cameras 13 and 23 have an image sensor such as a CMOS (complementary metal oxide semiconductor) or a CCD (charge coupled device), and acquire a face image including the driver's eyes. In the image sensor, light is detected by a plurality of pixels arranged two-dimensionally.

  As shown in FIG. 1, the first image receiving device 10 includes a first camera 13, first light sources 11a and 11b, and second light sources 12a and 12b. A plurality (two) of the first light sources 11a and the first light sources 11b are provided, and the first light sources 11a and the first light sources 11b are arranged on both sides in the left-right direction (X direction) with the first camera 13 interposed therebetween. Yes. A plurality (two) of second light sources 12a and second light sources 12b are also provided. The 2nd light source 12a and the 2nd light source 12b are arrange | positioned at both right and left sides so that both sides of the 1st camera 13 may be pinched | interposed.

  The distance in the left-right direction (X direction) between the optical axis O1 of the first camera 13 and the optical axis of the first light source 11a is L11a, and the distance in the left-right direction between the optical axis O1 and the optical axis of the first light source 11b is L11b. The distance in the left-right direction (X direction) between the optical axis O1 of the first camera 13 and the optical axis of the second light source 12a is L12a, and the distance in the left-right direction between the optical axis O1 and the optical axis of the second light source 12b is L12b.

  As shown in FIG. 1, the second image receiving device 20 includes a second camera 23, first light sources 21a and 21b, and second light sources 22a and 22b. A plurality (two) of the first light sources 21a are provided, the distance to the optical axis O2 of the second camera 23 is L11a, a plurality (two) of the first light sources 21b are provided, and the optical axis O2 of the second camera 23. The distance to is L11b. A plurality of (two) second light sources 22a are provided, and the distance to the optical axis O2 of the second camera 23 is L12a. A plurality (two) of the second light sources 22b are provided, and the distance to the optical axis O2 of the second camera 23 is L12b.

  In the first image receiving device 10, the distances L11a and L11b between the first light sources 11a and 11b and the optical axis O1 of the first camera 13 are the distances L12a between the second light sources 12a and 12b and the optical axis O1 of the first camera 13, Shorter than L12b. In the second image receiving device 20, the distances L11a and L11b between the first light sources 21a and 21b and the optical axis O2 of the second camera 23 are the distances L12a between the second light sources 22a and 22b and the optical axis O2 of the second camera 23, Shorter than L12b.

  Here, the optical axis distances L11a and L11b between the first camera 13 and the first light sources 11a and 11b and the optical axis distances L12a and L12b between the first camera 13 and the second light sources 12a and 12b are illumination imaging. Considering the distance between the device 1 and the driver as the imaging target, the first light sources 11a and 11b and the second light source are sufficiently short with respect to the distance L1 between the optical axes of the first camera 13 and the second camera 23. The optical axes 12a and 12b can be considered to be substantially coaxial with respect to the first camera 13. Similarly, the distance L11a, L11b between the optical axes of the second camera 23 and the first light sources 21a, 21b and the distance L12a, L12b between the optical axes of the second camera 23 and the second light sources 22a, 22b are the same as those of the first camera 13. The first light sources 21a and 21b and the second light sources 22a and 22b are substantially coaxial with respect to the second camera 23 because the distance between the optical axes L1 and the second camera 23 is sufficiently short. Can be considered.

  On the other hand, since the distance L1 between the optical axes of the first camera 13 and the second camera 23 is sufficiently long, the optical axes of the first light sources 11a and 11b, the second light sources 12a and 12b, and the first camera 13; The first light sources 21a and 21a, the second light sources 22a and 22b, and the optical axes of the second camera 23 are not substantially coaxial.

  The first light sources 11a and 12a and the first light sources 21a and 21b are LED light sources, and emit infrared light (near infrared light) having a wavelength of 850 nm (first wavelength) as detection light. Is arranged so that it can be given to the eyes of the subject. The second light sources 12a and 12b and the second light sources 22a and 22b are also LED light sources, and emit infrared light having a wavelength of 940 nm (second wavelength) as detection light. Arranged to be able to give to.

  850 nm is a wavelength that has a low absorption rate in the eyeball of a human eye and is easily reflected by the retina, and 940 nm is a wavelength that has a high absorption rate in the eyeball and is not easily reflected by the retina. Further, the light amount (light energy) of infrared light with a wavelength of 850 nm emitted from the first light sources 11a, 11b, 21a, and 21b is the light amount of infrared light with a wavelength of 940 nm emitted from the second light sources 12a, 12b, 22a, and 22b. Greater than (light energy).

  2 is a cross-sectional view taken along the line II-II in FIG. 1B, and shows the orientation of the first camera 13 and the first light sources 11a and 11b and the second light sources 12a and 12b provided in the first image receiving device 10. Show.

  The illumination imaging device 1 of this embodiment is used as a vehicle-mounted gaze detection device, and its installation location is obliquely forward with respect to the gaze direction of the driver that is the imaging target, such as a center console. May be placed. In such a case, as shown in FIG. 2, the optical axis O1 of the first camera 13 and the optical axes of the first light sources 11a and 11b and the second light sources 12a and 12b are not perpendicular to the inner surface of the cover plate 4, respectively. Without incident. The optical axis O1 of the first camera 13 and the optical axes of the first light sources 11a and 11b and the second light sources 12a and 12b are parallel to each other.

  The first light source 11a and the second light source 12a are located closer to the inner surface 4a of the cover plate 4 than the first camera 13, and the first light source 11b and the second light source 12b are also closer to the inner surface 4a than the first camera 13. In the position. Since the angle formed between the optical axis O1 and the inner surface 4a is an acute angle θ on the X2 side, the first light source 11b and the second light source 12b located on the X2 side are the first light source 11a and the second light source 11b located on the X1 side. It is closer to the first camera 13 than the light source 12a.

  As shown in FIG. 2, a shielding member 15 is provided between the first camera 13 and each of the first light sources 11 a and 11 b, and the shielding member 15 causes the first camera 13 to move forward in the imaging direction and the first light source. The light emission direction front of 11a, 11b is partitioned off. The shielding member 15 is made of a light-shielding and flexible material such as a rubber material or a foamed resin material. For example, the shielding member 15 is formed in a cone shape so as to continuously surround the lens front of the first camera 13.

  A recess 17 is formed on the inner surface 4a of the cover plate 4 on the front extension line of the shielding member 15, that is, at the boundary between the front of the first camera 13 in the imaging direction and the front of the first light sources 11a and 11b in the light emission direction. ing. When the shielding member 15 is formed so as to surround the lens of the first camera 13, the concave portion 17 is also formed so as to surround the front of the lens.

  Alternatively, as shown in FIG. 1B, the concave portion 17 is formed at the boundary between the first camera 13 and the first light source 11a and at the boundary between the first camera 13 and the first light source 11b. It may be formed continuously in the illustrated vertical direction, that is, formed linearly over the entire length in the direction transverse to the cover plate 4 in the width direction.

  In any case, the concave portion 17 is formed in the vicinity of the front extension line of the shielding member 15. The concave portion 17 is disposed at a position that does not obstruct light passing through the inside of the shielding member 15 as much as possible, so that the angle of view of the first camera 13 can be increased. Preferably, as shown in FIG. 2, the extension line of the camera-side inner surface of the shielding member 15 and the inner surface of the recess 17 are formed on the same line.

  As shown in an enlarged view in FIG. 3, the cross-sectional shape of the recess 17 is formed so that the opening width 17 a on the inner surface 4 a of the cover plate 4 is wide and the width dimension gradually decreases toward the outer surface 4 b of the cover plate 4. The cross-sectional shape is V-shaped.

  FIG. 3 illustrates the light emission state of the first light source 11a. Assuming that the concave portion 17 is not formed, a part of the light 33 emitted from the LED light source 31 of the first light source 11a is on the inner surface of a substantially spherical case 32 covering the front of the LED light source 31. The light enters the inside of the cover plate 4 obliquely, for example, by reflection. The light 34 that has entered the inside of the cover plate 4 is reflected by the outer surface 4 b of the cover plate 4, passes through the inner surface 4 a, and leaks to the X2 side from the shielding member 15. The leaked light 35 is acquired by the first camera 13, and a part of the image captured by the first camera 13 is overexposed, and the captured image cannot be acquired accurately.

  Therefore, as shown in FIGS. 2 and 3, a recess 17 is formed at the boundary between the front in the imaging direction of the first camera 13 and the front in the light emission direction of the first light source 11a, so that the inside of the cover plate 4 is propagated. The path of the light 34 to be performed can be blocked by the recess 17, and the light 35 can be prevented from being acquired by the first camera 13.

  Assuming that the optical axis of the first light source 11a is oriented perpendicularly to the inner surface 4a of the cover plate 4, many light components from the first light source 11a are perpendicular to the inner surface 4a of the cover plate 4. Since the light is incident at a close angle, the light component that is transmitted forward through the cover plate 4 increases. On the other hand, as shown in FIGS. 2 and 3, when the optical axis of the first light source 11 a extends obliquely with respect to the inner surface 4 a of the cover plate 4, the light emitted from the light source 31 is converted into the inner surface 4 a. , The probability that the light hits diagonally increases, and light enters the inside of the cover plate 4 and propagates easily. Therefore, as shown in FIG. 2, in the structure in which the optical axis O1 of the first camera 13 and the optical axis from the light source are oriented obliquely with respect to the inner surface 4a, the effect of providing the concave portion 17 is increased. The propagation path of the light 34 in the cover plate 4 can be blocked, and problems such as overexposure by the first camera 13 can be solved.

  This also applies to the boundary between the front of the X2 side first light source 11b shown in FIG. 2 and the front of the first camera 13 in the imaging direction. Since the optical axis of the first light source 11b is also obliquely directed toward the inner surface 4a of the cover plate 4, light emitted from the first light source 11b propagates through the cover plate 4 and easily enters the first camera 13. Become. Therefore, if a recess 17 is formed in front of or in the vicinity of the extended line of the shielding member 15 at the boundary between the first camera 13 and the first light source 11b, the light emitted from the first light source 11b is the first. It becomes easy to prevent getting into the camera.

  When comparing the first light source 11a and the first light source 11b, the influence of the first light source 11b on the X2 side on the first camera 13 is more than the influence of the first light source 11a on the X1 side on the first camera 13. Much bigger. The light sources 11a, 11b, 12a, and 12b need to be as close as possible to the inner surface 4a of the cover plate 4. This is because if the light sources 11a, 11b, 12a, and 12b are separated from the inner surface 4a, the light components that are reflected by the inner surface 4a and return to the internal space of the case 2 out of the light emitted from these light sources increase.

  As a result of bringing the light source closer to the cover plate 4 and orienting the optical axis O1 of the first camera 13 and the optical axis of each light source obliquely with respect to the inner surface 4a, the first light source 11b and the second light source 12b on the X2 side are The first light source 11a and the second light source 12a on the X1 side are arranged closer to the first camera 13. As a result, the possibility that the light emitted from the first light source 11b on the X2 side leaks to the X1 side from the shielding member 15 is higher than that of the first light source 11a on the X1 side. The partial overexposure phenomenon is likely to occur due to leakage light from the first light source 11b.

  Accordingly, the recess 17 is at least at the boundary between the first light source 11b and the first camera 13 located on the side (X2 side) where the angle θ formed by the optical axis O1 of the first camera 13 and the inner surface 4a is an acute angle. Preferably it is formed. However, it is more preferable that the recesses 17 are formed in both the boundary between the first light source 11a and the first camera 13 on the X1 side and the boundary between the first light source 11b and the first camera 13 on the X2 side.

  In order to obtain the bright pupil image by causing the first light source 11 a to emit light, the first light source 11 a is preferably disposed at a position close to the optical axis O <b> 1 of the first camera 13. In addition, the wavelength of the light emitted from the first light source 11a is 850 nm, and the energy of the light is larger than the light emitted from the second light sources 12a and 12b having a wavelength of 940 nm. Accordingly, a concave portion 17 is formed at the boundary between the first light source 11a and the first camera 13 to block the propagation of the light emitted from the first light source 11a in the cover plate 4, so that the portion of the first camera Overexposure can be prevented.

  The structure of the second image receiving device 20 arranged on the X1 side is symmetrical to the first image receiving device 10 in the left-right direction. The tilt direction of the optical axis O2 of the second camera 23 and the optical axes of the light sources 21a, 21b, 22a, and 22b is the same as the tilt direction of the optical axis O1 of the first camera 13. Also in the second image receiving device 20, a recess 17 is formed at the boundary between the front of the second camera 23 in the imaging direction and the front of the first light sources 21a and 21b in the light emission direction.

<Structure of gaze detection device>
FIG. 7 shows a circuit block diagram of the visual line detection device. The line-of-sight detection device includes the illumination imaging device 1 and a calculation control unit CC.

  The arithmetic control unit CC is composed of a CPU and a memory of a computer, and the function of each block is calculated by executing preinstalled software.

  The calculation control unit CC includes image acquisition units 44 and 45, a pupil image extraction unit 50, a pupil center calculation unit 53, a pupil diameter calculation unit 54, a corneal reflection light center detection unit 55, and a gaze direction calculation unit 56. And are provided.

  Illumination / non-lighting of the first light sources 11a, 11b, 21a, 21b and the second light sources 12a, 12b, 22a, 22b is controlled by a light source control unit (not shown), and images acquired by the cameras 13, 23 are frames. The image acquisition units 44 and 45 respectively acquire the images.

  The images acquired by the image acquisition units 44 and 45 are read into the pupil image extraction unit 50 for each frame. The pupil image extraction unit 50 includes a bright pupil image detection unit 51 and a dark pupil image detection unit 52.

  FIG. 5 is a plan view schematically showing the relationship between the direction of the line of sight of the eye 60 of the subject and the optical axes O1 and O2 of the cameras 13 and 23. FIG. As shown in FIG. 2, the optical axis O1 of the first camera 13 of the first image receiving device 10 and the optical axis of each light source are directed obliquely with respect to the cover plate 4, and the second camera also in the second image receiving device 20. The optical axis O <b> 2 of 23 and the optical axis of each light source are directed obliquely with respect to the cover plate 4. Therefore, even if the illumination imaging device 1 is disposed diagonally forward of the driver who is the imaging target like the center console, both the optical axis O1 of the first camera 13 and the optical axis O2 of the second camera 23 are illuminated. It is possible to face the subject's face and eyes 60 that are separated from the imaging apparatus 1 by a predetermined distance.

  FIG. 6 is an explanatory diagram for calculating the direction of the line of sight from the center of the pupil and the center of the corneal reflected light. 5A and 6A, the visual line direction VL of the subject is directed to the middle between the optical axis O1 of the first camera 13 and the optical axis O2 of the second camera 23. FIG. ) And FIG. 6B, the line-of-sight direction VL is directed toward the optical axis O1 of the second camera 23.

  The eye 60 has a cornea 61 in the front, and a pupil 62 and a crystalline lens 63 are located behind the cornea 61. The retina 64 is present at the end.

  When the image of the region including the eyes is acquired by the cameras 13 and 23, an image in which the brightness of the pupil 62 is different can be obtained by selecting a light source to be turned on. The bright pupil image and the dark pupil image are concepts that relatively represent the difference in brightness of the pupil 62 in the image, and when the two images are compared, the image with the brighter pupil 62 is the bright pupil image. The darker pupil image is the darker pupil image.

  The dark pupil image and the bright pupil image are performed by switching between the light emission of the first light sources 11a, 11b, 21a, and 21b and the light emission of the second light sources 12a, 12b, 22a, and 22b.

  The wavelength 850 nm of the first light sources 11a, 11b, 21a, 21b is easily reflected by the retina 64. Therefore, when the first light sources 11a and 11b of the first image receiving device 10 are turned on, infrared light reflected by the retina 64 is displayed in an image acquired by the first camera 13 that is substantially coaxial with the first light sources 11a and 11b. Is detected through the pupil 62, and the pupil 62 appears bright. This image is extracted as a bright pupil image by the bright pupil image detection unit 51. The same applies to an image acquired by the second camera 23 that is substantially coaxial with the first light sources 21a and 21b when the second image receiving device 20 is turned on.

  The wavelength 940 nm of the second light sources 12a, 12b, 22a, and 22b is difficult to be reflected on the retina 64. Therefore, when the second light sources 12a and 12b of the first image receiving device 10 are turned on, infrared light is almost reflected from the retina 64 in the image acquired by the first camera 13 substantially coaxial with the second light sources 12a and 12b. The pupil 62 appears dark. This image is extracted as a dark pupil image by the dark pupil image detection unit 52. The same applies to the image acquired by the camera 23 that is substantially coaxial with the second light sources 22a and 22b in the second image receiving device 20 when the second light sources 22a and 22b are turned on.

  By alternately repeating the imaging operation in the first image receiving device 10 and the second image receiving device 20, the bright pupil image and the dark pupil image can be separately acquired by the two cameras 13 and 23, and the pupil position is three-dimensionally determined. It becomes possible to measure.

  Alternatively, it is also possible to acquire a bright pupil image and a dark pupil image by switching the light source to be turned on as follows.

  As shown in FIG. 5A, the optical axis O1 of the image receiving device 10 and the optical axis O2 of the image receiving device are given to the eye 60 of the subject at different angles.

  When the first light sources 11a and 11b mounted on the first image receiving device 10 are turned on, an image acquired by the first camera 13 substantially coaxial with the first light sources 11a and 11b is reflected by the retina 64. Since the infrared light easily enters the camera 13, a bright pupil image in which the pupil 62 appears bright is obtained. This image is extracted as a bright pupil image by the bright pupil image detection unit 51. On the other hand, since the optical axis O2 of the second camera 23 provided in the image receiving device 20 is not coaxial with the optical axes of the first light sources 11a and 11b of the first image receiving device 10, the first light sources 11a and 11b are Even when the light is reflected by the retina 64 when the light is turned on, the light is not easily detected by the second camera 23. Therefore, the image acquired by the second camera 23 is a dark pupil image in which the pupil 62 is relatively dark. This image is extracted by the dark pupil image detection unit 52 as a dark pupil image.

  Conversely, when the first light sources 21a and 21b of the second image receiving device 20 are turned on, the light reflected by the retina 64 passes through the pupil 62 and is easily detected by the second camera 23 along the optical axis O2. An image acquired by the second camera 23 is a bright pupil image. At this time, the light reflected by the retina 64 is not easily detected by the first camera 13 located obliquely forward, and the image acquired by the first camera 13 becomes a dark pupil image.

  In other words, when detection light of the same wavelength is emitted, when light is emitted from a pair of light sources close to the camera (light source coaxial with the camera), the image acquired from the camera becomes a bright pupil image, When light is emitted from a pair of light sources remote from the camera (non-coaxial light source with the camera), an image acquired from the camera becomes a dark pupil image.

  The same applies to the combination of the second light sources 12a and 12b of the first image receiving device 10, the second light sources 22a and 22b of the second image receiving device, and the cameras 13 and 23.

  In the pupil image extraction unit 50 shown in FIG. 7, when the bright pupil image and the dark pupil image are acquired by both the image receiving device 10 and the image receiving device 20, the dark pupil image is obtained from the bright pupil image detected by the bright pupil image detection unit 51. The dark pupil image detected by the detection unit 52 is subtracted. As a result of this calculation, an image of the pupil 62 that appears brightly in the bright pupil image remains, and the other images are canceled and almost invisible.

  A pupil image signal indicating the shape of the pupil 62 is given to the pupil center calculation unit 53 and the pupil diameter calculation unit 54. In the pupil center calculation unit 53, the pupil image signal is subjected to image processing and binarized, and an area image corresponding to the shape and area of the pupil 62 is calculated. Further, an ellipse including this area image is extracted, and the intersection of the major axis and the minor axis of the ellipse is calculated as the center position of the pupil 62. The pupil diameter calculation unit 54 calculates the pupil diameter of the pupil 62 based on the pupil image signal, the center position of the pupil 62 calculated by the pupil center calculation unit 53, and the major and minor axes of the ellipse.

  Next, as shown in FIGS. 5A and 5B, when any one of the light sources is turned on, the light from the light source is reflected by the surface of the cornea 61 and the light is reflected on the first camera 13. And the second camera 23 and detected by the bright pupil image detection unit 51 and the dark pupil image detection unit 52. In particular, in the dark pupil image detection unit 52, since the image of the pupil 62 is relatively dark, the reflected light reflected from the reflection point 65 of the cornea 61 is bright and easily detected as a spot image.

  The dark pupil image signal detected by the dark pupil image detection unit 52 is given to the corneal reflection light center detection unit 55. The dark pupil image signal includes a luminance signal by reflected light reflected from the reflection point 65 of the cornea 61. The reflected light from the reflection point 65 of the cornea 61 forms a Purkinje image, and is acquired as a spot image with a very small area on the imaging devices of the cameras 13 and 23 as shown in FIG. The corneal reflection light center detection unit 55 performs image processing on the spot image, and obtains the center of the reflected light from the reflection point 65 of the cornea 61.

  The pupil center calculation value calculated by the pupil center calculation unit 53 and the corneal reflection light center calculation value calculated by the corneal reflection light center detection unit 55 are given to the gaze direction calculation unit 56. The line-of-sight direction calculation unit 56 detects the direction of the line of sight from the pupil center calculated value and the corneal reflection light center calculated value.

  In FIG. 5A, the line-of-sight direction VL of the human eye 60 is oriented in the intermediate direction between the optical axis O1 of the first camera 13 and the optical axis O2 of the second camera 23. At this time, the center of the reflection point 65 from the cornea 61 coincides with the center of the pupil 62 as shown in FIG. In contrast, in FIG. 5B, the line-of-sight direction VL of the human eye 60 is considerably directed to the left side. At this time, as shown in FIG. 6B, the center of the pupil 62 and the center of the reflection point 65 from the cornea 61 are displaced.

  The line-of-sight direction calculation unit 56 calculates a linear distance α between the center of the pupil 62 and the center of the reflection point 65 from the cornea 61 (FIG. 6B). Further, XY coordinates having the origin at the center of the pupil 62 are set, and an inclination angle β between the line connecting the center of the pupil 62 and the center of the reflection point 65 and the X axis is calculated. The line-of-sight direction VL is calculated by calculating the linear distance α and the tilt angle β based on images acquired by the two cameras 13 and 23.

<Modification>
A modification will be described below.
In the modification shown in FIG. 4A, the light absorbing material 71 is filled in the recess 17 formed in the cover plate 4. The light absorbing material 71 can absorb infrared light emitted from the first light source 11a, and preferably absorbs wavelengths of light from both the first light source 11a and the second light source 12b. The light absorbing material 71 is composed mainly of a carbon material, for example.

  By filling the concave portion 17 with the light absorbing material 71, reflection of light on the surface of the concave portion 17 can be prevented and propagation of light within the cover plate 4 can be easily regulated.

  In the modification shown in FIGS. 4B and 4C, the cross-sectional shape of the recess 117 formed in the cover plate 4 is substantially trapezoidal. The cross-sectional shape of the recess 117 also has a wide opening width on the inner surface 4a, and the opening width gradually decreases toward the outer surface 4b.

  In this way, the cross-sectional shape of the recesses 17 and 117 is formed such that the opening width on the inner surface 4a is wide and the opening width gradually decreases toward the outer surface 4b, so that the light propagating in the cover plate 4 is formed. When the light hits the boundary surface between the recesses 17 and 117, the light reflection direction is easily directed toward the outer surface 4b.

<Second Embodiment>
In the second embodiment shown in FIGS. 8A and 8B, the light sources 11 and 12 are arranged in a ring around the camera 13 in both the first image receiving device 10 and the second image receiving device 20. Yes. The light source 11 emits light having a wavelength of 850 nm, and the light source 12 emits light having a wavelength of 940 nm. A shielding member 15 extends in front of the camera 13, and a recess 17 is formed in the inner surface 4 a of the cover plate 4 at the boundary between the front of the camera 13 and the front of the light sources 11 and 12 in the light emission direction. In this case, the recess 17 is formed in an annular shape so as to surround the front of the camera 13.

4 Cover plate 4a Inner surface 4b Outer surfaces 10, 20 Image receiving devices 11a, 11b, 21a, 21b First light sources 12a, 12b, 22a, 22b Second light sources 13, 23 Camera 15 Shield member 17, 117 Recess 44, 45 Image acquisition unit 50 Pupil image extraction unit 53 Pupil center calculation unit 54 Pupil diameter calculation unit (determination information acquisition unit)
55 Cornea Reflected Light Center Detection Unit 56 Gaze Direction Calculation Unit 60 Eye O1, O2 Optical Axis

Claims (8)

Illumination having a camera that acquires an image of an imaging target, a plurality of light sources that provide detection light to the imaging target, and a translucent cover plate that covers in common the imaging direction front of the camera and the light emission direction of the light source In the imaging device,
A plurality of the light sources are arranged so as to sandwich the camera, and an optical axis of the camera and an optical axis of each of the light sources extend in parallel to each other, and the optical axis of the camera and the optical axis of the light source are the cover plate Is obliquely incident on the inner surface of
A shielding member is provided that partitions the front of the camera in the imaging direction and the light emission direction of the light source, and the shielding member is inclined toward the inner surface of the cover plate with an opening angle with respect to the optical axis of the camera. Extending,
On the inner surface of the cover plate, a recess is formed at the boundary between the camera in the imaging direction and the light source in the emission direction, and the recess is located on or near the extension of the shielding member. The illumination imaging device characterized by the above-mentioned. The illumination imaging apparatus according to claim 1, wherein an extension line of the inner surface of the shielding member on the camera side and an inner surface of the recess are located on the same line. 3. The illumination imaging apparatus according to claim 1, wherein a cross-sectional shape of the recess is a shape in which an opening width gradually decreases from the inner surface toward the outer surface of the cover plate . The illumination imaging apparatus according to claim 1, wherein the shielding member has a cone shape surrounding an optical axis of the camera. The illumination imaging apparatus according to claim 1, wherein the concave portion is filled with a light absorbing material that absorbs light emitted from the light source. Wherein the light source illuminating the image pickup device according to any one of 5 claims 1 is disposed at a position closer to the inner surface than the lens of the camera. The light source includes a first light source that emits light having a wavelength that is easily reflected by the retina of a human eye, and a second light source that emits light having a wavelength that is less likely to be reflected by the retina than the first light source. The illumination imaging device according to any one of claims 1 to 6 , wherein the concave portion is formed at a boundary portion between a light emitting direction front of the light source and a front of the camera imaging direction. A plurality of the cameras are provided, the light sources are arranged at positions close to the respective cameras, and the cover plate covers the front in the imaging direction of all the cameras and the front of the light emission directions of all the light sources in common. Item 8. The illumination imaging device according to any one of Items 1 to 7 .

Images