JP2995739B2 - Eye gaze detection device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a line-of-sight detection device, for example, an observer (photographer) on an observation surface on which a subject image is formed by a photographing system in an optical device such as a camera. The present invention relates to a gaze detection device that detects an axis in the direction of a gazing point, that is, a so-called gaze (viewing axis), using a reflection image formed when illuminating an eyeball of an observer. .

(Prior Art) Conventionally, various eye-gaze detecting devices for detecting a so-called eye-gaze (a visual axis) for detecting which position on an observation surface the observer is observing have been proposed.

For example, in Japanese Patent Application Laid-Open No. Sho 61-172552, a light beam from a light source is projected onto the anterior segment of the eye to be examined, and a visual axis (note) is formed by utilizing the image formation state of a reflected image based on light reflected from the cornea and the iris. Perspective).

FIG. 4 is an explanatory view of the principle of the eye gaze detection method proposed in the publication.

In the figure, reference numeral 4 denotes a light source such as a light emitting diode which emits infrared light insensitive to an observer, and is disposed on a focal plane of the light projecting lens 6.

The infrared light emitted from the light source 4 is converted into parallel light by the light projecting lens 6, reflected by the half mirror 10, and illuminates the cornea 1 of the eyeball 101. At this time, a part of the infrared light reflected on the surface of the cornea 1 passes through the half mirror 10, is condensed by the light receiving lens 7, and forms an image at the position d 'on the image sensor 9 (first Purkinje image). The ends a and b of the iris 3 are located at positions a 'and b' on the image sensor 9 via the half mirror 10 and the light receiving lens 7.
Image. If the rotation angle θ of the optical axis a of the eyeball with respect to the optical axis a of the light receiving lens 7 is small, and the Z coordinates of the ends a and b of the iris 3 are Za and Zb, the coordinate Zc of the center position c of the iris 3 is It is expressed as

Further, the Z coordinate of the generation position d of the first Purkinje image is Zd,
The distance between the center of curvature O of the cornea 1 and the center C of the iris 3 is ▲
Assuming that ▼, the rotation angle θ of the eyeball optical axis a substantially satisfies the relational expression of ▼▼ ・ sin θ ≒ Zc−Zd (1). Therefore, by detecting the position of each singular point (the first Purkinje image d and the ends a and b of the iris) projected on the image sensor 9, the rotation angle θ of the optical axis a of the eyeball can be obtained. At this time, equation (1) is …… (2) Can be rewritten. However, beta is made at a magnification determined by the distance ₀ between the distance ₁ and the light receiving lens 7 and the image sensor 9 of the generation position d and the light receiving lens 7 of the first Purkinje image, typically a substantially constant value.

As described above, by detecting the direction of the line of sight (point of sight) of the eye to be examined by the observer, for example, in a single-lens reflex camera, it is possible to know which position on the focus plane the photographer is observing.

This is, for example, in the case where distance measuring points are provided not only at the center of the screen but also at a plurality of positions in the screen in the automatic focus detection device, when the observer selects one of the distance measuring points and performs automatic focus detection, It is effective to save the trouble of selecting and inputting one of them and to regard the point observed by the observer as a ranging point, and to automatically select the ranging point to perform automatic focus detection.

(Problems to be Solved by the Invention) However, when the gaze detection device proposed in the above-mentioned Japanese Patent Application Laid-Open No. 61-172552 is applied to the gaze detection of an observer other than a finder of a camera, for example, an illuminating means and a light receiving means are required. And the relative positional relationship between the eye-gaze detection optical system and the eyeball of the observer are not constant but different. Therefore, for example, in FIG. 4, the distance ₁ between the light receiving lens 7 and the generation position d of the first Purkinje image does not become a predetermined value but differs, and the value of the magnification β obtained by Expression (2) does not become constant. As a result, there is a problem that an error occurs in the rotation angle θ of the optical axis of the eyeball.

According to the present invention, there is provided distance detecting means for detecting which position on the observation surface the observer is observing, that is, detecting the distance between the observer's line of sight to a predetermined surface of the line-of-sight detection device and a predetermined point of the observer's eyeball. Accordingly, it is an object of the present invention to provide a gaze detecting device that can always detect with high accuracy.

(Means for Solving the Problems) The gaze detection device of the present invention includes: (1-1) an illuminating unit that illuminates an eyeball with infrared light; and a light receiving unit that receives a first Purkinje image generated in the eyeball. Obtaining a distance from a predetermined surface of the illuminating unit or the light receiving unit to a first Purkinje image generated in the eyeball from an interval of the plurality of first Purkinje images formed on the image sensor surface included in the light receiving unit; It is characterized in that it has a calculating means for calculating the line of sight using the distance and the positions of the first Purkinje image and the iris image received by the light receiving means.

In particular, (1-1-1) the light receiving means has a pair of light receiving lenses disposed at a position shifted from the optical axis ahead of the light receiving surface with respect to the optical axis, and divides the first Purkinje image into a plurality. It is characterized by:

(1-2) Illuminating means for illuminating an eyeball with a plurality of infrared light sources, light receiving means for receiving a first Purkinje image generated in the eyeball, and image formation on an image sensor surface included in the light receiving means The distance between the plurality of first Purkinje images and the distance between the plurality of infrared light sources and the distance from the predetermined surface of the illuminating means or the light receiving means to the first Purkinje image generated in the eyeball are determined. Calculating means for calculating the line of sight using the positions of the first Purkinje image and the iris image received by the light receiving means.

(Embodiment) FIG. 1 is a schematic view of a main part of a first embodiment of the present invention. 2 (A) and 2 (B) are explanatory views when the lighting means and the light receiving means of FIG. 1 are developed.

In the figure, 101 is the eyeball of the subject (observer), 1 is the cornea of the subject's eyeball, 2 is the sclera, and 3 is the iris. O 'is the center of rotation of the eyeball 101, O is the center of curvature of the cornea 1, a and b are the ends of the iris 3, d and e are the positions where the first Purkinje image is generated based on the light sources 4 and 5 described later, respectively. . Reference numerals 4 and 5 denote light-emitting diodes and the like, each of which is a light source and emits infrared light insensitive to a subject. The light sources 4 and 5 are arranged near the focal plane of the light projecting lens. The light projecting lens 6 illuminates the surface of the cornea 1 via the half mirror 10 with the light beams from the light sources 4 and 5 as substantially parallel light beams. Here, the light source 4 is on the optical axis of the light projecting lens 6, and the light source 5 is arranged at a position away from the optical axis of the light projecting lens 6 by a distance S _{0 in the} −Z-axis direction.

Note that the light source 4 and the light projecting lens 6 constitute one element of the illumination means.

Reference numeral 7 denotes a light receiving lens which forms first Purkinje images d and e formed near the cornea 1 on the image sensor 9 surface.

The light receiving lens 7 and the image sensor 9 constitute one element of the light receiving means.

Numeral 11 denotes arithmetic means, which calculates the line of sight of the subject using an output signal from the image sensor 9 as described later.

The light source 5 and a part of the calculating means 11 constitute a distance detecting means described later.

A is the optical axis of the light receiving lens 7 and coincides with the X axis in the figure. A is inclined by an angle θ with respect to the X axis in the optical axis of the eyeball.

Next, with reference to FIGS. 2 (A) and 2 (B), a description will be given of the features of the visual line detection device of this embodiment and a method of detecting the distance from the subject's eyeball to a predetermined surface (light receiving lens 7) of the device. I do.

The half mirror 10 is omitted in FIGS. 2 (A) and 2 (B).

The infrared light emitted from the light sources 4 and 5 is
Illuminates the cornea 1 of the eyeball. At this time,
Distance ₄ on the corneal 1 side from the center of curvature O of the cornea 1 (≒ γ / 2:
(γ is the radius of curvature of the cornea 1), the first Purkinje images d and e are generated by the light beam reflected on the surface of the cornea 1. Here, the point d has the coordinates Z in the first Purkinje image for the light source 4 as well.
d, point e is the first Purkinje image for light source 5 and the Z coordinate is Z
e Becomes Here, ₂ is the focal length of the light projecting lens 6.
The first Purkinje images d and e are coordinated Zd 'on the image sensor 9 by the light receiving lens 7 as shown in FIG.
Projected to a position on Ze '. Here, the distance between the light receiving lens 7 and the position (d, e) where the first Purkinje image is generated is defined as
₁ , the distance between the image sensors 9 is ₁ , and the distance between the light receiving lens 7 and the image sensor 9 is _0. Becomes Here, the magnification β of the light receiving optical system is obtained by modifying equation (4). Is required.

In FIG. 1, the coordinates of the ends a and b of the iris 3 on the image sensor 9 are Za ′, Zb ′, and ≒ ▼ ₄ (C is the middle point between the ends a and b of the iris 3). , The rotation angle θ of the optical axis a of the eyeball is obtained from the equations (2) and (5). Becomes

In this embodiment, the rotation angle θ of the optical axis a of the eyeball is detected by the calculation means 11 using the calculation of the expression (6), and the visual axis of the eyeball is obtained, thereby detecting the line of sight of the subject.

As described above, in this embodiment, in addition to the normal light source 4 for illumination,
It is characterized in that a distance detecting light source 5 is provided, whereby the line of sight of the subject is detected with high accuracy.

FIG. 3 is a schematic view of a main part of a second embodiment of the present invention. In the figure, the same elements as those shown in FIG. 1 are denoted by the same reference numerals.

 In the figure, reference numeral 8 denotes a newly added light receiving lens.

In this embodiment, a pair of light receiving lenses 7 and 8 as distance detecting means are arranged at a certain distance from an axis A (X axis) which relatively matches the optical axis of the light projecting lens 6. It is characterized by.

The infrared light emitted from the light source 4 becomes parallel light by the light projecting lens 6 and is reflected by the half mirror 10 to illuminate the cornea 1. The first Purkinje image d is formed by the infrared light reflected on the surface of the cornea 1. The light beam from the first Purkinje image d passes through the half mirror 10 and forms images at the positions P and Q on the image sensor 9 by the light receiving lenses 7 and 8, respectively.

Here, the Z coordinate of the generation position d of the first Purkinje image is Zd,
The Z coordinate of the projection position p of the first Purkinje image d on the image sensor 9 surface by the light receiving lens 7 is Zp, the Z coordinate of the projection position q of the first Purkinje image d on the image sensor 9 surface by the light receiving lens 8 is Zq, Assuming that the shift amount of the light receiving lens 7 from the X axis is S ₂ and the shift amount of the light receiving lens 8 from the X axis is −S ₃ , the light receiving lenses 7 and 8 and the position where the first Purkinje image is generated d
The distance ₁ with Becomes Here, ₀ is the distance from the light receiving lenses 7 and 8 to the image sensor 9. Here, the magnification β of the light receiving optical system is obtained by modifying equation (7). Is required.

Now, assuming that the coordinates of the projected images of the ends a and b of the iris 3 on the image sensor 9 by the light receiving lens 7 are Za ′ and Zb ′, the rotation angle θ of the optical axis a of the eyeball is represented by the following equation (2). From the formula Becomes

In this embodiment, the calculation means 11 performs the calculation of the expression (9), detects the rotation angle θ of the optical axis a of the eyeball, obtains the visual axis of the eyeball, and detects the line of sight of the subject from this.

As described above, in the present embodiment, two light receiving lenses 7 and 8 are used, and a first Purkinje image is formed on each image sensor by each light receiving lens, and the coordinates and the coordinates of the first Purkinje image formed on the image sensor are determined. The method is characterized in that the line of sight of the subject is accurately detected using the distance ₁ from the first Purkinje image to the light receiving lenses 7 and 8.

(Effects of the Invention) (A-1) The invention described in claim 1 includes an illuminating unit that illuminates an eyeball with infrared light, a light receiving unit that receives a first Purkinje image generated in the eyeball, and a light receiving unit. A distance from a predetermined surface of the illuminating means or the light receiving means to a first Purkinje image generated in the eyeball is obtained from an interval of the plurality of first Purkinje images formed on the included image sensor surface, and the distance and the light receiving Calculating means for calculating the line of sight using the position of the first Purkinje image and the iris image received by the means, when the distance from a predetermined surface of the lighting means or the light receiving means to the eyeball is not constant. Also, it is possible to provide a gaze detection device capable of performing highly accurate gaze detection.

(A-2) The invention described in claim 2 is the invention according to claim 1, wherein the light receiving means is disposed at a position shifted from the optical axis forward of the optical axis with respect to the light receiving surface. By dividing the first Purkinje image into a plurality of parts, a plurality of first Purkinje images can be generated on the image sensor surface. Alternatively, a distance from a predetermined surface of the light receiving unit to a first Purkinje image generated in the eyeball can be obtained.

(A-3) The invention according to claim 3 is included in the illuminating means for illuminating the eyeball with a plurality of infrared light sources, the light receiving means for receiving the first Purkinje image generated in the eyeball, and the light receiving means. A distance between a plurality of first Purkinje images formed on the image sensor surface and a distance from a predetermined surface of the illuminating means or the light receiving means to a first Purkinje image generated on the eyeball from the distance between the plurality of infrared light sources; Calculating the line of sight by using the distance and the position of the first Purkinje image and the iris image received by the light receiving means, and thereby, from a predetermined surface of the lighting means or the light receiving means. It is possible to provide a gaze detection device capable of performing highly accurate gaze detection even when the distance to the eyeball is not constant.

[Brief description of the drawings]

FIG. 1 is a schematic view of a main part of a first embodiment of the present invention, FIGS. 2 (A) and 2 (B) are explanatory views of the illuminating means and light receiving means of FIG. 1, and FIG. FIG. 4 is a schematic view of a main part of a conventional eye gaze detecting apparatus. In the figure, 101 is an eyeball, 1 is a cornea, 2 is a sclera, 3 is an iris,
Reference numerals 4 and 5 denote light sources, 6 a light projecting lens, 7 and 8 light receiving lenses, 9 an image sensor, 10 a half mirror, and 11 an arithmetic means.

Continuation of front page (58) Field surveyed (Int.Cl. ⁶ , DB name) A61B 3/113

Claims (3)

(57) [Claims] An illumination unit configured to illuminate an eyeball with infrared light; a light reception unit receiving a first Purkinje image generated in the eyeball; and a plurality of image sensors formed on an image sensor surface included in the light reception unit. A distance from a predetermined surface of the illuminating means or the light receiving means to a first Purkinje image generated in the eyeball is obtained from an interval of one Purkinje image, and the distance and the first Purkinje image and the iris image received by the light receiving means are obtained. A gaze detecting device comprising: a calculation unit configured to calculate a gaze using the position. 2. The apparatus according to claim 1, wherein said light receiving means has a pair of light receiving lenses disposed at a position deviated from said optical axis ahead of said light receiving surface with respect to said optical axis, and divides said first Purkinje image into a plurality. The eye gaze detecting device according to claim 1. 3. An illumination means for illuminating an eyeball with a plurality of infrared light sources, a light receiving means for receiving a first Purkinje image generated in the eyeball, and an image formed on an image sensor surface included in the light receiving means. The distance between the plurality of first Purkinje images and the distance between the plurality of infrared light sources and the distance from the predetermined surface of the illuminating means or the light receiving means to the first Purkinje image generated in the eyeball are determined. A gaze detecting device comprising: a calculating unit configured to calculate a line of sight using the positions of the first Purkinje image and the iris image received by the light receiving unit.