JP2894701B2 - Eye gaze detection device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a gaze detection device that detects a gaze position of an observer.

2. Description of the Related Art Conventionally, there is Japanese Patent Application Laid-Open No. Sho 61-172552 as a visual axis detection device that optically detects the visual axis (axial axis) of an observer.

In this method, the first Purkinje image, which is a reflection image from the anterior cornea, generated by irradiating the observer's eyeball with parallel light, and the position of the center of the pupil are detected.
Description will be made based on the drawings.

In the figure, 501 is a cornea, 502 is a sclera, 503 is an iris, 504 is a light source, 506 is a light projecting lens, 507 is a light receiving lens, 509 is an image sensor, and 510 is a half mirror. o 'is the center of rotation of the eyeball, o is the center of curvature of the cornea 501, a and b are the ends of the iris 503, c is the center of the iris, and d is the first Purkinje image generator. In the figure, the light receiving lens 507 is at the optical axis and coincides with the x image in the figure. A is the optical axis of the eyeball.

The light source 504 is an infrared emitting diode insensitive to an observer, and is disposed on the focal plane of the light projecting lens 506. Light source 504
The emitted infrared light becomes parallel light by the light projecting lens 506, is reflected by the half mirror 510, and illuminates the cornea 501. A part of the infrared light reflected on the surface of the cornea 501 forms an image at a position d ′ on the image sensor 509 by the light receiving lens 507 transmitted through the half mirror 510. The ends a and b of the iris 503 are half mirrors
510, an image is formed at positions a 'and b' on the image sensor 509 via the light receiving lens 507. With respect to the optical axis a of the light receiving lens 507. When the rotation angle θ of the optical axis a of the eyeball is small, and the z coordinates of the ends a and b of the iris 503 are z _a and z _b , the coordinate z _c of the center position c of the iris 503 is It is expressed as

The z coordinate of the first Purkinje image generation position d is z _d , and the distance between the center of curvature o of the cornea 501 and the center c of the iris 503 is ▲.
Assuming that ▼, the rotation angle θ of the eyeball optical axis a substantially satisfies the relational expression of ▼▼▼ sin θ ≒ z _c −z _d (1). Therefore, by detecting the position of each singular point (the first Purkinje image z _d ′ and the iris ends z _a ′, z _b ′) projected on the image sensor 509, the rotation angle θ of the optical axis a of the eyeball becomes apparent. Becomes At this time, equation (1) is I can change it. However, beta is a magnification determined by the distance l ₀ between the distance l ₁ between the light receiving lens 507 and the image sensor 509 and the light receiving lens 507 and the first Purkinje image generation position, usually takes a substantially constant value.

The direction of the line of sight can be detected based on the above principle.

[Problems to be Solved by the Invention] However, in this type of conventional visual axis detection device, the reflectance of the cornea is 2.5%, and for example, as shown in FIG. Although the position can be reliably detected, the reflectance of the iris is extremely small, and it is actually quite difficult to accurately detect the boundary between the iris and the pupil for determining the center position of the pupil.

An object of the present invention is to accurately detect an iris limbus which is a boundary between an iris and a pupil or a boundary between a sclera (white eye) and an iris (black eye),
An object of the present invention is to provide a gaze detection device capable of accurately detecting a gaze.

[Means for Solving the Problems] The gist for achieving the object of the present invention is as follows.
Illuminating means for illuminating the eye, and an image detecting means comprising a solid-state imaging device for detecting the Purkinje image position and the image position of other tissues of the eye with reflected light from the eye illuminated by the illuminating means,
A line-of-sight calculation unit that detects a line-of-sight direction from a relative relationship between the Purkinje image position detected by the image detection unit and an image position of another tissue of the eye; and an accumulation time control unit that controls the accumulation time of the solid-state imaging device. The image detecting and outputting means has a structure having an overflow drain function for storing image information stored in a certain amount or more and a peak value output function for outputting a peak value of image information of each pixel. The accumulation time is a value obtained by multiplying the time from the start of image accumulation by the image detection means to the time when the peak value of the image information reaches a certain value based on the ratio of the reflection characteristics of the cornea and the iris or sclera to the accumulation time. An eye-gaze detecting device is characterized in that, when it reaches, the image information sequentially stored is output.

[Effect] The gaze detecting device configured as described above can extract image information to be detected by multiplying the image information by controlling the accumulation time of the image and expanding the information respectively.

[Examples] Hereinafter, the present invention will be described in detail based on examples shown in the drawings.

FIG. 1 is an embodiment of an optical block of a camera having a visual line detection device according to the present invention.

1 is a photographic lens, 2 is a quick return mirror, 3 is a focus plate, 4 is a condenser lens, and 5 is a pentaprism, which forms a normal finder optical system.

6 is an eyepiece lens having a beam splitter for transmitting visible light and reflecting infrared light inside, 7 for a beam splitter, 8 for a light projecting lens, 9 for a light receiving lens, and 10 for infrared light for projecting.
The LED 11 is a photoelectric conversion element such as a linear or area type CCD, and forms a visual line detection device. 12 is the eye of the projector.

The light emitted from the infrared LED 10 is converted into a parallel light beam by the light projecting lens 8 and irradiated to the eye 12. The light reflected from the cornea of the eye and the iris is reflected by the beam splitter 7, and forms an image on the photoelectric conversion element (fixed imaging element) 11 via the light receiving lens 9.

The solid-state imaging device 11 has an overflow drain function that does not adversely affect adjacent pixels even if the signal of some pixels is saturated, and non-destructively real-time non-destructive image information stored in each pixel such as a floating gate. And a so-called real-time peak value output function for monitoring the maximum value of the monitor output. The configuration is shown in FIG.

FIG. 3 is a block diagram showing an example of the configuration of the solid-state imaging device 11.

In the figure, 31 is a photoelectric conversion storage unit for storing image information, 32
Is a charge transfer gate for transferring the information of the photoelectric conversion storage unit 31 to the analog shift register 33 for reading,
When a charge transfer pulse is input from the B terminal, information is transferred to the analog shift register 33, and the information is output from the terminal C. 34 is an overflow drain that discards further information when the information of the photoelectric conversion storage unit 31 reaches a certain value, and 35 detects non-destructively each information of the photoelectric conversion storage unit 31 and outputs the maximum value of them to the terminal. A is a real-time peak output circuit that outputs from A.

Incidentally, the eyeball 12 is irradiated with light from the infrared LED 10, the reflected image is formed on the solid-state imaging device 11, and the horizontal scanning signal B corresponding to the position of the eyeball when the center of the eyeball is horizontally scanned is As shown in FIG.

As can be seen from the figure, the corneal reflection image can be detected very strongly and accurately, but the contrast between the borders of other tissues is low, and the iris border between the iris and the pupil and the border between the iris and the iris of the iris can be accurately detected. It is quite difficult to detect well as described above.

In this embodiment, the high-precision detection of the boundary between the iris and the pupil and the boundary of the iris, which is the boundary between the iris and the iris, in the horizontal scanning signal B is performed by setting the accumulation time of the image accumulated in the solid-state imaging device 11 to the fourth time.
This is realized by controlling with the control device shown in FIG. 5A and obtaining, for example, a signal obtained by enlarging the iris portion as shown in FIG.

FIG. 4 is a block diagram of a control device for controlling the information accumulation time in the solid-state imaging device 11 and processing the output information.

Before explaining the block diagram of this control device, the basic principle of control will be described based on the structural characteristics of the solid-state imaging device 11.

When the solid-state imaging device 11 irradiates the reflected image from the infrared LED 10 reflected by the eyeball 12, a peak value is output from the output terminal A in real time, and the peak value is set to the second value.
As is apparent from the figure, the image is a corneal reflection image. When the peak value of the corneal reflection image reaches a certain level near the saturation level, a charge transfer pulse is input to the B terminal and the image is accumulated from the output terminal C. When the information is output, only the horizontal scanning signal B shown in FIG. 2 can be obtained. Therefore, even if the peak value of the corneal reflection image reaches a certain level near the saturation level, the charge transfer pulse is not input, and the reflection image is stored as it is. The information of the corneal reflection image, the information of the iris and pupil boundary, and the information of the iris limbus, which is the boundary of the iris and iris, are accumulated, and their values increase, and eventually the information of the corneal reflection image is saturated and overflows. However, since it has an overflow drain function, there is no adverse effect on adjacent pixels. Next, the information that reaches a certain level near the saturation level is the information of the iris portion as is clear from FIG. 2, but the information of the iris portion at this time is a value multiplied by the accumulation time ratio. ing.

Therefore, when accurate information on the iris limbus is required, the charge is required after a certain time based on the ratio of the reflection characteristics of the cornea and the white of the eye to the time required for the information of the corneal reflection image to reach a certain level near the saturation level. By inputting the transfer pulse, the information of the enlarged iris part can be extracted from the output terminal C.

When accurate information on the boundary between the iris and the pupil is required, the time required for the information of the corneal reflection image to reach a certain level near the saturation level is determined based on the ratio of the reflection characteristics of the cornea and the iris. It suffices to input a charge transfer pulse after a lapse of time.

 Next, the control device will be described.

42 and 43 the resistance of the reference voltage V ₀ for generating, 44 a comparator, 45 denotes a latch circuit, the counter 46, the first information unit 47, 48 and the second information unit, the selection switch 49, 50 is a multiplier, 51 is memory, 52 is magnitude comparator,
53 A / D conversion circuit, 54 and 55 the resistance of the upper-limit level voltages V ₁ for generating, 56, 57 the resistance of lower-level voltage V ₂ for generating, 58 in line-of-sight operation processing circuit, the first information unit 47 in the ratio of the reflection properties of the cornea and the white of the eye, i.e. the second view, information determined based on the ratio between the peak level a ₂ of the peak level a ₁ and the iris limbus of the cornea reflection image is input, the second information unit 48 the ratio of the reflection properties of the cornea and the iris, that is, in FIG. 2, information determined based on the ratio between the peak level a ₁ and the peak level a ₃ of the iris of the cornea reflection image is input. The details of the line-of-sight calculation processing circuit 58 will be described later.

Thus constituted control apparatus, the output of the peak of each pixel from the A terminal of the solid-state image sensor 11 is input in real time to the comparator 44, the comparator 44 output is inverted and its value reaches the reference voltage V _0, the solid The count value of the counter 46 that has started counting is latched by the latch circuit 45 at the same time as the start of accumulation of image information of the image sensor 11.

On the other hand, which of the iris information or the information of the boundary between the iris and the pupil is required is selected in advance by the selection switch 49, and the information latched by the latch circuit 45 and the information selected by the selection switch 49 are selected. Is multiplied by a multiplier 50, and the
Store in 51. Here, the magnitude comparator 52
Compares the count value of the counter 46 with the data stored in the memory 51 and stores the count value of the counter 46 in the memory.
When the data matches the data of 51, a match signal is generated, a charge transfer pulse is applied to the input terminal B of the solid-state imaging device 11, and image information starts to be output from the output terminal C.

That is, after the accumulation of the image information in the solid-state imaging device 11 is started and the time until the comparator 44 is inverted is multiplied by a fixed time based on the information from the first information section 47 or the second information section 48, the image is increased. Information will be output.

Image information from the output terminal C of the solid-state imaging device 11, in order to process the direction of the photographer gaze at gaze arithmetic processing circuit 58, that have been set A / D voltage level V _1, V ₂ of the upper and lower limit would be a / D converted by the conversion circuit 53, for example, when it selects the first information unit 47 for detecting the iris annulus, as shown in FIG. 5, the image signal V ₃ is a / D The optimum level is always set for the upper and lower limit voltage levels V ₁ and V _{2 of the} conversion. Note that the lower limit voltage level V ₂ is, for example, the solid-state imaging device 11.
May be set near the dark current signal level.

The gaze calculation processing circuit 58 detects the direction of the gaze of the photographer based on the corneal reflection image, the boundary between the iris and the pupil, information on the iris limbus, and the like, based on the detection information, an exposure control circuit of a camera (not shown), The focus detection circuit and the like are controlled so that the photographer adjusts the exposure and focus to what he or she wants to capture.

The line-of-sight calculation processing circuit 58 is configured as shown in FIG. 4 (B).

Reference numeral 101 denotes a pupil edge detection unit that detects a pupil edge based on an output signal from the A / D conversion circuit 53, and 102 denotes a pupil center detection unit that detects a pupil center from information output from the pupil edge detection unit 101. And 103, a corneal reflection image position detection unit for detecting a corneal reflection image position based on an output signal from the A / D conversion circuit 53, and 104, a corneal reflection image position (first Purkinje image) from the corneal reflection image position detection unit 103. ) And the pupil center information from the pupil center detection unit 102. This is a visual axis calculation unit that calculates the direction of the line of sight by the method shown in FIG.

In the above embodiment, among the output image signals from the solid-state imaging device 11, the signal of the corneal reflection image is saturated, and if the blur of the corneal reflection image is known in advance, the position of the corneal reflection image is determined from the saturated signal. Can be accurately detected.

The ratio of the reflection characteristics of the cornea and the iris or the white of the eye differs depending on the race, such as Westerners and Easterners, but information corresponding to the first and second information units 47 and 48 in FIG. 4 (A) is set. Just do it.

[Effects of the Invention] As described above, when the present invention is used, the boundary between the pupil and the iris and the iris of the iris can be detected with good S / N, so that there is a remarkable effect that high-precision gaze detection becomes possible.

In addition, even if the light power of the illumination means such as a light emitting LED fluctuates, the accumulation time of the element is automatically corrected via the peak output circuit of the solid-state imaging element, so that the iris and white eye always have the optimal output level. Is obtained, and there is also an effect that highly accurate gaze detection can be performed.

[Brief description of the drawings]

FIG. 1 is an optical block diagram of a camera having an embodiment of a gaze detecting device according to the present invention, FIG. 2 is a diagram showing a horizontal scanning signal of a solid-state image sensor corresponding to the position of an eyeball, and FIG. FIG. 4 (A) is a system block diagram showing an example of a control device of the gaze detection device, FIG. 4 (B) is a block diagram showing details of the gaze calculation processing circuit, FIG. 5 shows one of the output waveforms from the solid-state imaging device.
FIG. 6 is a view showing an example, and FIG. 6 is a schematic view of a conventional eye gaze detecting apparatus for detecting a gaze using a corneal reflection image and a pupil center. 7: Beam splitter, 8: Projecting lens, 9: Receiving lens, 10: Infrared LED, 11: Solid-state imaging device (linear or area type photoelectric conversion device), 12: Eye (eyeball) .

Continuation of the front page (56) References JP-A-57-109476 (JP, A) JP-A-55-110477 (JP, A) JP-A-59-105779 (JP, A) JP-A-57-93782 (JP, A) JP-A-62-178076 (JP, A) JP-A-63-37653 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) A61B 3/113

Claims (1)

(57) [Claims] 1. An illuminating means for illuminating an eye, and an image detecting means comprising a solid-state imaging device for detecting a Purkinje image position and an image position of another tissue of the eye by reflected light from the eye illuminated by the illuminating means. Line-of-sight calculating means for detecting a line-of-sight direction based on the relative relationship between the Purkinje image position detected by the image detecting means and the image position of another tissue of the eye, and storage time control means for controlling the storage time of the fixed image sensor Wherein the image detection and output means has a structure having an overflow drain function for storing image information stored in a certain amount or more and a peak value output function for outputting a peak value of image information of each pixel. The means sets the accumulation time to a value obtained by multiplying the time from the start of image accumulation by the image detection means to the time when the peak value of the image information reaches a certain value based on the ratio of the reflection characteristics of the cornea and the iris or the sclera to the accumulation time. A line-of-sight detection device that outputs sequentially accumulated image information when the time has arrived.