JP3179182B2 - Eye gaze detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a line-of-sight detecting device which detects the line-of-sight direction of a photographer and gives various instructions to a camera or the like based on the information on the line-of-sight direction.

[0002]

2. Description of the Related Art In recent years, for example, Japanese Patent Application Laid-Open No. 3-107909 discloses a technique of detecting a line of sight of an observer looking through a finder and giving an instruction to a camera based on the line of sight information. FIG. 6 is a diagram showing a configuration of the device disclosed in the publication.

[0003] As shown in the figure, in the observation optical system, a flip-up mirror 1 is placed on the optical path of the light passing through the taking lens 101.
02 is disposed on the optical path of the reflected light from the flip-up mirror 102, and the focusing plate 103 and the condenser lens 1
04, a pentaprism 105 is provided. Further, on the optical path of the light reflected by the reflection surface of the pentaprism 105, an eyepiece lens 14 serving as an optical member for detecting a line of sight is provided.

On the other hand, a line-of-sight detection optical system includes an illumination unit including an infrared light emitting diode (LED) 10 and a light projecting lens 11 which are light sources insensitive to an observer, and a line sensor 1.
5, a light receiving unit including a half mirror 12 and a light receiving lens 13, and is disposed above an eyepiece 14 having a beam splitter 14a including a dichroic mirror.

In such a configuration, in the observation optical system, the subject light that has passed through the taking lens 101 is reflected by the flip-up mirror 102 and forms an image on the focus plate 103. The subject light diffused by the focus plate 103 is condensed by the condenser lens 104 and the pentaprism 10.
5. The eye point 20 in the eyeball 200 of the observer via the eyepiece 14 having the beam splitter 14a.
It is led to 1.

On the other hand, in the visual axis detection optical system, the LED 1
The infrared light radiated from 0 becomes a parallel light flux by the light projecting lens 11 and is split by the beam splitter 14a of the eyepiece 14.
And irradiates the eyeball 200 of the observer.
Then, a part of the infrared light reflected by the eyeball 200 is reflected again by the beam splitter 14a, and the light receiving lens 13,
An image is formed on the line sensor 15 via the half mirror 12. FIG. 7 is a diagram showing a state of a detection image obtained by the line sensor 15.

[0007] As shown in FIG.
The pupil image greatly changes depending on the intensity of the visible light incident on the pupil. When the image is dark, a high-level fundus reflection image is detected, and when the image is bright, a low-level pupil image is detected. It should be noted that the figure shows the fundus reflected light, and the central processing unit (CPU) 9 calculates the line of sight of the observer based on such output signals.

In addition, for example, in Japanese Patent Application Laid-Open No. 2-5,
There is disclosed a technique of calculating the line of sight from the position of the reflected light (first Purkinje image) on the corneal surface and the center position of the iris image. According to the technique disclosed in the publication, as shown in FIG. 8, when the cornea 201 of the eyeball 200 is irradiated with a light beam 51 parallel to the optical axis l, an image at an optically infinite distance becomes the center of curvature of the cornea 201. An intersection generated at a midpoint 53 between the corneal vertex 54 and the corneal vertex is detected as a first Purkinje image.

[0009]

However, in the conventional apparatus as described above, the optical axis of the observation optical system and the optical axis of the light receiving optical system are coaxial between the beam splitter 14 and the eyeball. In other words, the light receiving optical system observes the eyeball image from almost the front.

Therefore, the image of the eyelashes of the observer as shown in FIGS. 9 (a) and 9 (b) overlaps the detected image, and FIG.
As shown in (1), it becomes difficult to detect a feature point such as a pupil edge from a detected image. Furthermore, when the observer narrows his eyes when gazing at the subject, or when he / she looks through the finder from above, such an effect becomes remarkable.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object the purpose of forming an optical system that is not easily affected by the eyelashes of the observer with a simple configuration, so that detection accuracy and reliability can be improved. It is an object of the present invention to provide a gaze detection device with improved performance.

[0012]

In order to achieve the above object, according to a first aspect of the present invention, there is provided a light projecting means for irradiating an eyeball of an observer looking through a finder with illumination light, and a light projecting means provided by the light projecting means. The reflected light from the eyeball of the illuminated light is received through the light receiving optical system, photoelectric conversion means for performing photoelectric conversion, and gaze direction detection means for detecting the gaze direction of the observer based on a signal from the photoelectric conversion means, Wherein the light receiving optical system is disposed between the optical axis of the light receiving optical system and the optical axis of the finder at a predetermined angle below the eyeball of the observer. A detection device is provided. According to a second aspect, in the first aspect, there is provided a visual line detection device, wherein the predetermined angle is an angle in a range of 5 to 30 °.

[0013]

That is, in the first aspect of the present invention, the illumination light is applied to the eyeball of the observer looking through the finder by the light emitting means,
The reflected light from the eyeball of the light projected by the light projecting means by the photoelectric converting means is received through the light receiving optical system, is photoelectrically converted, and is provided by the line-of-sight direction detecting means based on the signal from the photoelectric converting means. The direction of the line of sight is detected, and in particular, the light receiving optical system is disposed between the optical axis of the light receiving optical system and the optical axis of the finder at a predetermined angle below the eyeball of the observer. In a second aspect, in the first aspect, the predetermined angle is defined as an angle in a range of 5 to 30 °.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the configuration of the first embodiment of the present invention.

As shown in FIG. 1, a half mirror 2 is disposed on the optical path of the light projected from the light projecting unit 1, and an observation system optical axis L through which the light reflected by the half mirror 2 travels.
The eyeball 200 of the observer is located on 1. A light receiving unit 4 is provided via a light receiving optical system 3 on a light receiving system optical axis L2 through which the light reflected by the eyeball 200 travels, and an arithmetic unit 5 is connected to the light receiving unit 4. .

In this embodiment, in particular, the optical axis L1 of the observation system and the optical axis L2 of the light receiving system are not coaxial, and light is received by the observation optical system L1 in order to reduce the influence of eyelashes. The optical system L2 has an angle of about 5 to 30 °.

In such a configuration, the light projected from the light projecting unit 1 is reflected by the half mirror 2, and the reflected light travels on the optical axis L1 of the observation system and is guided to the eyeball 200 of the observer. Then, the reflected light from the eyeball 200 travels on the light receiving system optical axis L <b> 2 and forms an image on the light receiving surface of the light receiving unit 4 via the light receiving optical system 3. Then, the arithmetic unit 5 detects the line of sight of the observer based on the output from the light receiving unit 4. As described above, according to the present embodiment, the image disturbance due to the eyelashes is significantly reduced, so that the reliability and accuracy of the eye gaze detection are improved. FIG. 2 is a diagram showing the configuration of the second embodiment of the present invention.

As shown in FIG. 1, in the observation optical system, a mirror 1 is flipped up onto the optical path of light passing through the taking lens 101.
02 is disposed on the optical path of the reflected light from the flip-up mirror 102, and the focusing plate 103 and the condenser lens 1
04, a pentaprism 105 is provided. Further, on the optical path of the light reflected by the reflection surface of the pentaprism 105, an eyepiece lens 14 serving as an optical member for detecting a line of sight is provided.

On the other hand, the line-of-sight detection optical system includes an illumination unit including an infrared light emitting diode (LED) 10 and a light projecting lens 11 which are light sources insensitive to an observer, and an image sensor 1.
5, a light-receiving unit including a visible light cut filter 16, a half mirror 12, and a light-receiving lens 13, and a beam splitter 14 including a dichroic mirror.
a is disposed above the eyepiece 14 having a.

Here, the light receiving system optical axis L2 is arranged so as not to be coaxial with the observation optical system optical axis L1. In other words, the optical axis L2 is located below the eyeball of the observer with respect to the optical axis L1.
The angle is in the range of up to 30 °.

In such a configuration, in the observation optical system, the subject light that has passed through the photographing lens 101 is reflected by the flip-up mirror 102 and forms an image on the focus plate 103. The subject light diffused by the focus plate 103 is condensed by the condenser lens 104 and the pentaprism 10.
5. The eye point 20 in the eyeball 200 of the observer via the eyepiece 14 having the beam splitter 14a.
It is led to 1.

On the other hand, in the visual axis detection optical system, the LED 1
The infrared light radiated from 0 is converted into a parallel light beam through the light projecting lens 11, and becomes a beam splitter 14 of the eyepiece 14.
The light is reflected at a and irradiates the eyeball 200 of the observer. Then, a part of the infrared light reflected by the eyeball 200 is reflected by the half mirror 12, and the light receiving lens 13,
Image sensor 15 via visible light cut filter 16
Imaged on top. The visible light cut filter 16 removes an image due to light outside the detection range.

The information of the detected image obtained by the image sensor 15 is digitized via an interface (not shown), and then is taken into the CPU 9. Then, the CPU 9 extracts the characteristic coordinates of the detected image and calculates the line of sight of the observer in accordance with a predetermined algorithm described later. Hereinafter, the main sequence of the present embodiment will be described with reference to the flowchart of FIG.

When the line-of-sight detection sequence starts due to an interrupt such as a switch-on, first, an LED for illumination light is used.
10 is turned on (step S101), the image sensor 15 for detecting an eyeball image is reset, and light quantity integration is started (step S102). The image sensor 15 has a light amount monitoring means.
Tell

The CPU 9 waits for the completion of the sensor integration (step S103). After the completion of the sensor integration, the CPU 9 turns off the LED 10 (step S104), and subsequently reads an image signal from the image sensor 15 (step S105).

Then, feature coordinates of the Purkinje image position and the iris center position are detected from the obtained eyeball image (step S106). Here, noise removal processing is performed on the image data using the image data of only the noise light stored as necessary. In this way, the visual axis direction is calculated by adding the visual axis correction and the like (step S107), and one detection is completed.

As described above, in this embodiment, the detection accuracy can be improved by preventing the eyelashes from overlapping the Purkinje image, pupil image, iris image, etc. of the eyeball. Further, as a secondary effect, since the light receiving system and the light projecting system are not coaxial, there is no fear of noise light due to reflection at the coaxial portion.

Further, a one-dimensional line sensor has been described as an example of the image sensor, but a plurality of line sensors or area sensors may be used. Then, by selecting an optimal one line from a plurality of lines and detecting an image, more accurate information on the gaze direction can be obtained. FIG. 4 is a diagram showing the configuration of the third embodiment of the present invention.

As shown in FIG. 1, in the observation optical system, the mirror 1 is flipped up onto the optical path of the light passing through the taking lens 101.
02 is disposed on the optical path of the reflected light from the flip-up mirror 102, and the focusing plate 103 and the condenser lens 1
04, an M-shaped reflection prism 106 is provided.
On the optical path of the light reflected by the reflection surface of the M-shaped reflection prism 106, an eyepiece lens 14 serving also as an optical member for performing line-of-sight detection is provided.

On the other hand, the line-of-sight detection optical system is composed of an illuminating section comprising an LED 10 which is a light source insensitive to the observer and a light projecting lens 11, an image sensor 15, a visible light cut filter 16, and a light receiving lens 13. An illumination unit is provided above the eyepiece lens 14 having a beam splitter 14a formed of a dichroic mirror. The illumination unit is disposed below the eyeball of the observer with respect to the observation system optical axis L1. Optical axis L of the light receiving system having an angle within the range of 30 °
2, a light receiving unit is arranged.

In such a configuration, in the observation optical system, the subject light passing through the photographing lens 101 is reflected by the flip-up mirror 102 and forms an image on the focus plate 103. The subject light diffused by the focus plate 103 is condensed by the condenser lens 104 and the M-shaped reflection prism 1.
05, eyepiece 14 having beam splitter 14a
Via the eye point 2 in the eyeball 200 of the observer
It is led to 01.

On the other hand, in the visual axis detection optical system, the LED 1
The infrared light radiated from 0 is converted into a parallel light beam through the light projecting lens 11, and becomes a beam splitter 14 of the eyepiece 14.
The light is reflected at a and irradiates the eyeball 200 of the observer. Then, a part of the infrared light reflected by the eyeball 200 is formed on the image sensor 15 via the light receiving lens 13 and the visible light cut filter 16. The visible light cut filter 16 removes an image due to light outside the detection range.

The information of the detected image obtained by the image sensor 15 is digitized via an interface (not shown), and thereafter, is taken into the CPU 9. Then, the CPU 9 extracts the characteristic coordinates of the detected image and calculates the line of sight of the observer in accordance with a predetermined algorithm described later.

As described above, in this embodiment, the eyelash of the upper eyelid is prevented from overlapping the Purkinje image, pupil image, and iris image of the eyeball by shifting the optical axis of the light receiving system.
Further, the configuration of the M-shaped reflecting prism and the light receiving system disposed below the reflecting prism can reduce the scale of the device. FIG. 5 is a diagram showing the configuration of the fourth embodiment of the present invention.

As shown in the figure, in the observation optical system, the mirror 1 is flipped up onto the optical path of the light passing through the taking lens 101.
02 is disposed on the optical path of the reflected light from the flip-up mirror 102, and the focusing plate 103 and the condenser lens 1
04, an M-shaped reflection prism 106 is provided.
An eyepiece 17 is provided on the optical path of the light reflected by the reflection surface of the M-shaped reflection prism 106.

On the other hand, the line-of-sight detection optical system is composed of an illuminating part comprising an LED 10 which is a light source insensitive to the observer and a light projecting lens 11, an image sensor 15, a visible light cut filter 16, and a light receiving lens 13. And a light receiving unit, and is disposed below the eyepiece 17.

In such a configuration, in the observation optical system, the subject light passing through the photographing lens 101 is reflected by the flip-up mirror 102 and forms an image on the focus plate 103. The subject light diffused by the focus plate 103 is condensed by the condenser lens 104 and the M-shaped reflection prism 1.
05, through the eyepiece 17, is guided to the eye point 201 in the eyeball 200 of the observer.

On the other hand, in the visual axis detection optical system, the LED 1
The infrared light emitted from 0 becomes a parallel light flux through the light projecting lens 11, passes through the half mirror 12, and is emitted to the eyeball 200 of the observer. Then, a part of the infrared light reflected by the eyeball 200 is reflected by the half mirror 12 and forms an image on the image sensor 15 via the light receiving lens 13 and the visible light cut filter 16. The visible light cut filter 16 removes an image due to light outside the detection range.

The information of the detected image obtained by the image sensor 15 is digitized via an interface (not shown), and then is taken into the CPU 9. Then, the CPU 9 extracts the characteristic coordinates of the detected image and calculates the line of sight of the observer in accordance with a predetermined algorithm described later. As described above, in the present embodiment, the optical axis of the light receiving system is shifted to prevent the eyelashes of the upper eyelid from overlapping the Purkinje image, pupil image, and iris image of the eyeball. Furthermore, since the illumination light projection system was also deflected,
It can also be prevented that the illumination light is blocked by the eyelashes and a shadow is formed. As described above, the first to fourth embodiments of the present invention have been described. However, the present invention is not limited thereto, and various improvements,
Of course, changes are possible. For example, in consideration of the vertical position and the horizontal position of the camera, the light receiving system may be disposed below the optical axis of the finder. Further, in the description of the embodiments, a camera has been described as an example. However, the present invention is not limited to a camera, and can be widely applied to, for example, an optical device having an eyepiece.

As described in detail above, according to the present invention, the optical axis of the light receiving optical system is deflected from the optical axis of the observation optical system.
It is possible to prevent the eyelash image of the observer from being superimposed on the line-of-sight detection image and thus reducing the accuracy.

[0041]

According to the present invention, it is possible to provide an eye-gaze detecting device having improved detection accuracy and reliability by forming an optical system which is not easily affected by eyelashes of an observer with a simple structure. .

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 3 is a flowchart showing a main sequence of the second embodiment.

FIG. 4 is a diagram showing a configuration of a third exemplary embodiment of the present invention.

FIG. 5 is a diagram showing a configuration of a fourth exemplary embodiment of the present invention.

FIG. 6 is a diagram illustrating a configuration of a conventional visual line detection device.

FIG. 7 is a diagram illustrating a state of a detected corneal reflection image.

FIG. 8 is a diagram for explaining how to calculate the direction of the observer's line of sight from the position of the first Purkinje image, which is the reflected light of the corneal surface, and the center position of the iris image.

FIGS. 9A and 9B are diagrams showing an eye and an eyelash of an observer.

FIG. 10 is a diagram illustrating a state in which a detected image is distorted due to an influence of an image of an eyelash.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Projection part, 2 ... Half mirror, 3 ... Light receiving optical system, 4 ...
Light receiving unit, 5 ... calculation unit.

Claims (2)

(57) [Claims] A light projecting means for irradiating illumination light to an eyeball of an observer looking through a viewfinder; and a light reflected from the eyeball of the light projected by the light projecting means is received through a light receiving optical system and subjected to photoelectric conversion. A photoelectric conversion unit, and a line-of-sight direction detection unit configured to detect a line-of-sight direction of an observer based on a signal from the photoelectric conversion unit; and a line-of-sight detection device, comprising: an optical axis of the light-receiving optical system; Wherein the light receiving optical system is disposed at a predetermined angle below the eyeball of the observer. 2. The eye gaze detecting device according to claim 1, wherein the predetermined angle is an angle in a range of 5 to 30 °.