JP2000229067A - Visual axis detecting device and optical instrument having same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a visual axis detecting device having a function of discriminating the attitude of a camera without particularly preparing an exclusive detector in the sight line detecting device and an optical instrument having it. SOLUTION: A visual axis detecting device is provided with an attitude discriminating means 1 for discriminating the attitude of an optical instrument, an illuminating means 3 for illuminating an eyeball of a user according to indication of the attitude discriminating means 1, a detecting means 4 for detecting an eyeball turning angle of the user by processing an eyeball image signal from a light receiving means and a calculating means 5 for calculating sight line information of the user from eyeball turning angle information detected according to indication of the attitude discriminating means 1. Thus, the attitude discriminating means 1 functions on the basis of a characteristic of the eyeball of the user.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a line-of-sight detecting device for an optical device such as a camera having a function of detecting the line of sight of a user, and an optical device having the same.

[0002]

2. Description of the Related Art Conventionally, a so-called line of sight (a visual axis) for detecting which position of an observer (user) is gazing on an observation surface, for example, a finder surface of a single-lens reflex camera, is detected. Various devices have been proposed.

For example, in Japanese Patent Application Laid-Open No. 1-241511 by the present applicant, an anterior eye of a user's eye illuminated by an infrared light emitting diode (hereinafter abbreviated as “IRED”) is used by an area sensor. The image signal is processed and the image signal is processed to detect the rotation angle of the user's eyeball, and then the line-of-sight coordinates of the user on the viewfinder are detected. A camera for selecting one of a (detection) area and a photometry area is disclosed. The outline of the technique described in the publication is as follows.

That is, since the cornea of the eyeball has a high reflectance, when illuminated with IRED, a corneal reflection image of the IRED is generated on the cornea of the eyeball. Therefore, when the eyeball rotates, the pupil, which is a part of the eyeball, also rotates with the eyeball, but the reflected image on the cornea shows a different motion from the pupil, and the relative positional relationship is a predetermined relationship with the rotation of the eyeball. Represents

Therefore, if the relative positional relationship between the corneal reflection image and the pupil in the eyeball image signal output from the light receiving means is known, the rotation angle of the eyeball can be calculated.

In addition, since the distance between the camera and the eyeball is also required to calculate the line-of-sight information accurately, two IREDs are usually used as a pair to detect the distance.

[0007] When the eyeball rotation angle is known, the gaze coordinates on the finder are calculated geometrically from this. However, since the correspondence between the eyeball rotation angle and the gaze coordinates differs from person to person, this is used. Practical gaze detection cannot be performed unless correction is made for each person. This correction is referred to as "individual difference correction".

The present applicant has proposed a camera having a "calibration mode" for performing the "individual difference correction" in Japanese Patent Application Laid-Open No. H4-242630.

According to the publication, a mode called a "calibration mode" is prepared for a camera separately from a normal camera operation for photographing a subject, and when a line of sight is detected, first, a user's individual difference is determined in this mode. Have the correction data registered.

More specifically, the user gazes at two specific targets in the viewfinder, and from the eyeball rotation angle at that time, the correspondence between the user's eyeball rotation angle and line-of-sight coordinates, that is, individual differences. The correction data is obtained and stored in a nonvolatile memory (for example, an EEPROM) built in the camera.

Thereafter, an eyeball image of the user is acquired in a normal camera operation, and a gaze detection is performed using the eyeball rotation angle obtained from the eyeball image and the set personal difference correction data.
Using the result of the line of sight as an input, focus detection, exposure and display of the camera are controlled.

FIG. 7 shows an example of an eyeball image signal output from an area sensor as light receiving means. The longitudinal direction in the figure is the reading direction of the sensor.

In the figure, 101 is a cornea reflection image of two IREDs, 102
Denotes a pupil portion, and 103 denotes an iris portion.

By the way, when the user holds the camera, the film feed direction is set to be horizontal, so-called "horizontal position" (also called "normal position").
There is a "vertical position" that can be rotated 90 degrees.

In the case of the normal horizontal position, the horizontal direction of the area sensor signal corresponds to the horizontal direction of the eyeball (human) as shown in FIG. 7, but in the case of the vertical position, it corresponds to the vertical direction of the eyeball (human). Corresponding to

Assuming that the horizontal component of the eyeball rotation angle detected by the eyeball rotation angle detecting means is θx and the vertical component is θy, the line-of-sight coordinates (X, Y) on the finder in the horizontal position are: X = a × θx + B Y = c × θy + d.

Here, X and x indicate the longitudinal direction of the camera (film feeding direction), and Y and y indicate the vertical direction.

A is a horizontal rotation angle coefficient of the user's eyeball, b is an offset value, c and d are values in the respective vertical directions, and both coefficients and offset values are different between the horizontal direction and the vertical direction.

The a, b, c, and d are individual difference correction values of a specific user acquired in the calibration operation, and generally differ for each user.

Considering the case where the camera is held in the vertical position, the horizontal component of the eyeball rotation angle detected by the camera is actually the vertical rotation angle of the user's eyeball.

Therefore, in order to calculate the line-of-sight coordinates on the finder, it is necessary to exchange the horizontal and vertical parameters such as X = c × θx + dY = a × θy + b.

Therefore, in order to correctly calculate the line-of-sight coordinates from the horizontal and vertical rotation angles of the eyeball corresponding to the horizontal position / vertical position, the posture of the camera with respect to the user is determined, and the above parameters are replaced. Otherwise, it is impossible to calculate a correct line-of-sight coordinate.

It is also important to change the arrangement of the IREDs that are lit in the horizontal / vertical positions.

In the example shown in FIG. 7, when the user changes the attitude of the camera from the horizontal position to the vertical position, the eyeball image signal becomes as shown in FIG. 8 (a).

Although it is possible to detect the line of sight in this state, if one of the two corneal reflection images is blocked by the eyelid 104 of the user as shown in FIG. 8B, only one IRED can be detected. , The distance between the camera and the eyeball cannot be detected,
Therefore, it is difficult to detect the line of sight.

To avoid this, the IREDs may be arranged as shown in FIG. 9, and the IREDs to be lit may be switched according to the attitude of the camera.

Four IREDs are arranged around the eyepiece 105.

IREDs 106 and 107 are a pair of horizontal position IREDs.

In the case of the vertical position, the IREDs 106 and 108 are used in the case of the vertical position in which the grip side with the release button 110 is on the top side.
Pair in the vertical position where the grip is on the ground side, IRED
Select 107 and 109 pairs respectively.

FIGS. 10A and 10B show examples of respective eyeball image signals.

In the same vertical position, the IRED illumination is always performed from below depending on whether the release button of the camera is on the top side or the ground side so that the upper eyelid of the user does not block the IRED corneal reflection image.

The corneal reflection images 111 and 112 in FIG. 10A are IREDs 106 and 108, respectively, and the corneal reflection images 11 and 112 in FIG.
Numerals 3 and 114 are reflection images of IREDs 107 and 109, respectively.

As described above, in order to mount the line-of-sight detection function on a device such as a camera used in both the horizontal position and the vertical position, it is essential to determine the posture of the device.

Regarding this point, Japanese Patent Laid-Open Publication No.
The details are described in JP-A-6-86758.

[0035]

Many of the conventional camera attitude determination means are based on the principle that an object (substance) moves according to gravity.
Alternatively, a photo interrupter or the like is used.

In the case of a photoelectric detector, a drive circuit and a detection circuit are also required, and the vertical position also needs to be distinguished between 90 ° left and 90 ° right. Therefore, two sets of these are required. This has been a factor in increasing the cost of the gaze detection function.

The above-mentioned means is based on the premise that the user is standing perpendicular to the gravity. When the user is holding the camera and looking straight down, as in the case of a projection, When it is inclined obliquely, it is difficult to determine the correct posture.

An object of the present invention is to provide a gaze detecting device,
In particular, an object of the present invention is to provide a line-of-sight detection device having a function of determining the attitude of a camera without preparing a dedicated detector, and an optical device having the same.

[0039]

In order to achieve the above-mentioned object, according to the present invention, at least a part of a characteristic pattern of an eyeball is detected by an eyeball image receiving means for receiving an eyeball image. Characterized in that it has a posture discriminating means for discriminating the posture of the gaze detecting device with respect to the user by comparing with an eyeball pattern stored in advance in accordance with the above.

Further, at least a part of the characteristic pattern of the eyeball is detected by the eyeball image light receiving means for receiving the eyeball image, and is compared with the eyeball pattern stored in advance in accordance with the attitude of the eyesight detecting device, so that the eyesight for the user is obtained. Attitude determination means for determining the attitude of the detection device, illumination means for illuminating the user's eye according to instructions from the attitude determination means, and detection of the user's eyeball rotation angle from an eyeball image signal from the eyeball image light receiving means An eyeball rotation angle detection unit, and a gaze calculation unit that calculates gaze information of a user from the posture determination information by the posture determination unit and the eyeball rotation angle information by the eyeball rotation angle detection unit.

Then, at least a part of the characteristic pattern of the eyeball is detected by the eyeball image receiving means for receiving the eyeball image,
Attitude determination means for determining the attitude of the gaze detection device with respect to the user by comparing with a prestored eyeball pattern according to the attitude of the gaze detection device, and illumination for illuminating the user's eyeball in accordance with instructions from the attitude determination means Means, eyeball rotation angle detection means for detecting a user's eyeball rotation angle from an eyeball image signal from the eyeball image light receiving means, storage means for storing a plurality of correction data, and attitude determination information by the attitude determination means, A gaze calculating unit configured to calculate gaze information of a user based on eyeball rotation angle information obtained by the eyeball rotation angle detection unit and specific correction data among a plurality of correction data obtained by the storage unit;

Further, the correction data is data for each individual representing the correspondence between the eyeball rotation angle and the line-of-sight information.

Further, the characteristic pattern of the eyeball used by the posture determining means is an iris pattern of the user.

In addition, the present invention is characterized in that the optical device is provided with the visual axis detecting device according to any one of the first to fifth aspects.

According to the above arrangement, the means for detecting the iris pattern from the eyeball image signal for line-of-sight detection and comparing the iris pattern with the previously recorded iris pattern in each posture is provided. Can be discriminated, and it is not necessary to prepare a special line-of-sight detector in particular, so that extra cost can be reduced, and erroneous detection in the line-of-sight detector using gravity can be reduced.

[0046]

DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) FIG. 1 is a block diagram appropriately representing a configuration according to a first embodiment of the present invention.

Reference numeral 1 denotes a posture discriminating means, which controls the illuminating means 3 to illuminate the eyeball based on the posture discrimination information, sends the posture discriminating information to the visual axis calculating means 5, and outputs the eyeball rotation angle detecting means 4. It is configured to calculate the line-of-sight coordinates 15 from the rotation angle information. Reference numeral 2 denotes an eyeball image receiving unit which receives reflected light from the illuminated user's eyeball and simultaneously transmits an eyeball image signal 12 to the posture determination unit 1 and the eyeball rotation angle detection unit 4. is there.

At first, the attitude determining means 1 first includes the lighting means 3
To illuminate the eyeball.

Signal 11 is an IRED selection signal. At this point, the posture of the camera is unknown, so for the time being IR for the horizontal position
Turn on the ED pair (IRED106, 107).

The eyeball image receiving means 2 comprising an area sensor or the like receives reflected light from the illuminated user's eyeball and outputs an eyeball image signal 12. FIG. 2 shows an example of the output image signal at this time.

The posture determining means 1 detects a part of the iris pattern in the area represented by the frame 200 from the eyeball image signal.

The iris pattern detected in this case is, for example, as shown in FIG.

Next, the iris pattern (b) at the user's horizontal position stored in advance, and the vertical position (90 ° leftward from the horizontal position)
The iris pattern (c) in the camera with the release button on the sky side with the camera rotated, and the vertical position (90 degrees to the right from the horizontal position).
The camera's release button is on the ground when rotated.)
Is compared with the iris pattern (a) just detected, and the closest pattern is detected.

For pattern comparison, for example, a correlation function may be used.

In this example, since the pattern (a) is closest to the horizontal position pattern (b), the current posture is determined to be the horizontal position, and the posture determination signal 13 representing the horizontal position is calculated by the line-of-sight calculation. To the means 5.

The eyeball rotation angle detecting means 4 outputs the eyeball image signal 1
2, the eyeball rotation angle 14 is detected, and
Calculates the line-of-sight coordinates 15 of the user. In this case, since the posture determination signal 13 indicates the horizontal position, the camera and the user can calculate the line-of-sight coordinates in the same posture.

The case where the camera is in the vertical position (rotated 90 ° to the left from the horizontal position) will be described. In this case, the eyeball image signal is as shown in FIG. 4 and the iris pattern is as shown in FIG. It is detected as shown in (a).

In this case, as in the case of FIGS. 2 and 3,
The iris pattern diagrams 5 (b), (c), and (d) of the user's horizontal position, the vertical position, and the vertical position, which are stored in advance, are compared with the current iris pattern diagram 5 (a). Position 1 pattern
Since (c) is the closest, it can be determined that the current posture is the vertical position 1.

Although the IRED used for the illumination is for the horizontal position, even if the determined posture is the vertical position, as shown in FIG. 4, if the two corneal reflection images of the IRED are not blocked,
The eyeball rotation angle can be detected with the eyeball image signal as it is.

However, as shown in FIG. 8 (b), when one or both of the two corneal reflection images are obstructed, switch to the IRED prepared for the vertical position and repeat the eyeball image. The gaze detection can be performed by redoing from the acquisition of.

The registration of the user's iris pattern diagrams 3 (b), (c), and (d) in each posture is performed by registering the user's iris pattern in advance before the actual use of the camera. You can get it.

As described above, it is effective to prepare a “calibration mode” for registering the user's line-of-sight characteristics in advance in order to accurately execute the line-of-sight detection. Also, the iris pattern of the user in each posture may be registered in advance at the time of this calibration.

More specifically, first, the user holds the camera in the horizontal position and looks into the finder, detects the iris pattern in the eyeball image of the user at that time, and detects the iris pattern as it is or by performing some conversion. The stored data is stored in a predetermined position in a nonvolatile memory (for example, an EEPROM) in the camera.

Next, the camera is held in the vertical position (rotated 90 degrees to the left from the horizontal position) and stored similarly. The same applies to the vertical position (a state rotated 90 ° clockwise from the horizontal position).

In this way, the iris pattern in each posture of the specific user can be stored in advance.

(Second Embodiment) FIG. 6 shows a block diagram of a second embodiment of the present invention. Note that parts having the same functions as those in FIG. 1 in the first embodiment are given the same numbers.

In the first embodiment, the description has been made by a single user.
The iris pattern may be collated to determine individual difference data for a plurality of users.

That is, the posture discriminating means 1 'does not discriminate the posture of only one user from the obtained iris pattern, but compares it with the iris patterns of a plurality of users to determine who the user is. Then, it is possible to determine the posture.

The posture discrimination signal 13 'is inputted to the personal difference data storage means 6 which stores the personal difference data, and the storage means 6 stores the current user's personal difference data in accordance with the user discrimination signal therein. 16 is output to the line-of-sight calculation means 5.

The line-of-sight calculation means 5 can calculate the line-of-sight coordinates of the current user based on the eyeball rotation angle signal 14, the posture determination signal 13 ', and the individual difference data 16.

(Other Embodiments) The above description has been given of an embodiment in which the present invention is applied to a single-lens reflex camera. However, the present invention is not limited to a single-lens reflex camera but is also effective for a lens shutter camera. It is clear that the present invention is also effective for optical devices such as video cameras, man-machine interface devices equipped with a line-of-sight detection device, and assistive devices for the disabled.

[0072]

As described above, the present invention provides a means for detecting an iris pattern from an eyeball image signal for gaze detection and comparing the iris pattern with the iris pattern in each of the previously recorded postures. , It is possible to determine the orientation of the camera, eliminating the need for special detectors, thereby reducing extra costs and reducing false detections with the gaze detector using gravity. Becomes

[Brief description of the drawings]

FIG. 1 is a block diagram according to a first embodiment of the present invention;

FIG. 2 is an iris pattern diagram at a horizontal position according to the first embodiment of the present invention;

FIG. 3 is a diagram illustrating a comparison between a lateral position iris pattern and a registered pattern according to the first embodiment of the present invention.

FIG. 4 is an iris pattern diagram in a vertical position according to the first embodiment of the present invention;

FIG. 5 is a diagram illustrating a comparison between a vertical position iris pattern and a registered pattern according to the first embodiment of the present invention.

FIG. 6 is a block diagram according to a second embodiment of the present invention;

FIG. 7 is a diagram showing an eyeball image signal at a horizontal position in a conventional example.

FIG. 8 is a diagram showing an eyeball image signal in a vertical position under IRED illumination for a horizontal position in a conventional example.

FIG. 9 is a diagram showing an example of an IRED arrangement in a conventional example.

FIG. 10 is a diagram showing an eyeball image signal in a vertical position under vertical position IRED illumination in a conventional example.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Attitude determination means 2 ... Eyeball image receiving means 3 ... Illumination means 4 ... Eyeball rotation angle detection means 5 ... Eye-gaze calculation means 6 ... Individual difference data storage means 11 ... IRED selection signal 12 ... Eyeball image signal 13 ... Attitude determination signal 14 … Eyeball rotation angle information 15… Gaze information 16… Individual difference data

Claims (6)

[Claims] An eyeball image receiving means for receiving an eyeball image detects at least a part of a characteristic pattern of an eyeball, and compares the feature pattern with an eyeball pattern stored in advance in accordance with a posture of the eyesight detection device to thereby provide an eye gaze to the user. A line-of-sight detection device, comprising: posture determination means for determining the posture of the detection device. 2. An eyeball image receiving means for receiving an eyeball image detects at least a part of a characteristic pattern of an eyeball and compares the feature pattern with an eyeball pattern stored in advance in accordance with a posture of the eyeball detection device to thereby provide an eye gaze to the user. Attitude determination means for determining the attitude of the detection device, illumination means for illuminating the user's eye according to instructions from the attitude determination means, and detection of the user's eyeball rotation angle from an eyeball image signal from the eyeball image light receiving means An eyeball rotation angle detection unit; and a gaze calculation unit that calculates gaze information of a user from posture determination information obtained by the posture determination unit and eyeball rotation angle information obtained by the eyeball rotation angle detection unit. Detection device. 3. An eyeball image receiving means for receiving an eyeball image detects at least a part of the characteristic pattern of the eyeball, and compares the feature pattern with an eyeball pattern stored in advance in accordance with the attitude of the eyesight detecting device to thereby provide an eyesight for the user. Attitude determination means for determining the attitude of the detection device, illumination means for illuminating the user's eye according to instructions from the attitude determination means, and detection of the user's eyeball rotation angle from an eyeball image signal from the eyeball image light receiving means Eyeball rotation angle detection means, storage means for storing a plurality of correction data, posture determination information by the posture determination means, eyeball rotation angle information by the eyeball rotation angle detection means, and a plurality of correction data by the storage means. A gaze detecting device comprising: gaze calculating means for calculating gaze information of a user from specific correction data. 4. The optical apparatus according to claim 3, wherein the correction data is data for each individual representing a correspondence between an eyeball rotation angle and line-of-sight information. 5. The eye-gaze detecting device according to claim 1, wherein the characteristic pattern of the eyeball used by the posture determining means is a user's iris pattern. 6. An optical apparatus comprising the line-of-sight detection device according to claim 1.