JPH1097376A - Operation device for line of sight - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform a sure operation of lines of sight in an easy posture of an operator and in a short time. SOLUTION: When the operation device is operated, various operation switches are shown on a display member 6. Then the eyes E of an operator watch one of those switches to operate it, and the position of the pupil P set on an image pickup surface is specified by calculation together with the direction and the distance of the eyes E set against a focus signal. At the same time, the direction of the line of sight is calculated from the relative position relation between the cornea reflection image formed by an infrared ray source 1 included in the pupil P. Thus, the switch that is viewed by the operator on the member 6 is specified. In such a way, the direction of the line of sight us recognized, various switches are inputted and these switch input signals are transmitted to an actuator. Then the operation device is operated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a line-of-sight operating device for detecting a line of sight and performing an operation based on the line of sight.

[0002]

2. Description of the Related Art Conventionally, gaze control devices have been used in airplane and automobile control devices, computers, word processors, welfare equipment, game machines, and the like.

[0003]

However, in the above-mentioned prior art, there is a problem that the subject cannot be changed in posture and is very tired because an error occurs if the position of the eye moves during the operation. In addition, when inputting, it is necessary to look in a predetermined direction for a certain period of time. Shortening the time of staring tends to cause a malfunction, and conversely, if it is long, it takes time to operate, and even if there is no intention of operation input, the gaze turns in that direction. Is entered.

An object of the present invention is to solve the above-mentioned problems,
It is an object of the present invention to provide a gaze control device capable of performing a reliable gaze operation in a short time in a comfortable posture.

[0005]

A gaze control apparatus according to the present invention for achieving the above object has an image pickup means for picking up an anterior segment of an eye, an illuminating means for illuminating the anterior segment of the eye, and an eye. Detecting means for detecting the position of the eye, and determining the gaze direction from the positional information of the eyes obtained from the detecting means and the positional relationship between the corneal reflection image of the illumination means and the pupil.

A gaze control device according to a second aspect of the present invention is a gaze control device that detects a gaze direction and performs an operation with the gaze, and detects that the gaze is directed a second time within a predetermined time and performs an input operation. Is performed.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the illustrated embodiment. FIG. 1 shows a configuration diagram of a line-of-sight operating device according to an embodiment. The line-of-sight operating device is located in front of an operator's eye, and includes an infrared light source 1 such as an LED that illuminates the periphery of the operator's eye E and a driving unit 2 A photographing lens 3 which is driven and automatically focused, an image pickup means 4 which is arranged behind the photographing lens 3 and comprises an infrared television camera for detecting blur of an image, calibration targets a and b and operation switches 5a; It comprises a display member 6 which is a CRT for displaying 5b, and an arithmetic control means 7 for calculating the position of the corneal reflection image. The arithmetic control means 7 includes a circuit for generating symbols to be displayed on the display member 6, a display control program thereof, a visual axis arithmetic program, a memory for the small screen M, and the like.

FIG. 2 and FIG. 3 show screens of the display member 6. FIG. 2 shows calibration targets a and b used for calibration of the line of sight before using the apparatus. In FIG. 4, various operation switches 5a, 5b, etc. for operating the apparatus are generated from the symbol generation circuit and displayed.
The small screen M which displays an anterior segment image as shown in FIG.

Before using the apparatus, the calibration targets a and b are sequentially displayed on the display member 6, and the operator's eye E sees them and calibrate the direction of the line of sight of the eye E in advance. Next, the operator sets the eye E
The user operates the apparatus by looking at the operation switches 5a and 5b displayed on the display member 6 with.

The luminous flux from the infrared light source 1 illuminates the periphery of the operator's eye E as shown by a dashed line, and a corneal reflection image by the infrared light source 1 is generated in the pupil P of the eye E. The anterior segment image of the eye E illuminated by the infrared light source 1 is captured by the imaging unit 4 via the photographing lens 3.
An image is detected by the arithmetic and control unit 7 based on the anterior eye image, and the image blur is detected, and the driving lens 2 drives the photographing lens 3 to automatically focus.
Then, the position of the taking lens 3 at the time of focusing is detected, and the distance to the eye E is calculated by the arithmetic and control unit 7.

The arithmetic control means 7 calculates the position of the pupil P and the position of the corneal reflection image of the infrared light source 1 from the video signal of the image pickup means 4. The image by the external light is distinguished and subtracted. Since the corneal reflected light is very strong, it is detected by setting a threshold value, and the pupil P is recognized as a dark round area around it. Infrared light source 1
Since the fundus of the eye E and the fundus of the eye E are almost conjugate, the fundus reflection light returns to the infrared light source 1 and does not enter the imaging means 4, so that the inside of the pupil P appears black, and the edge of the pupil P can be easily recognized. Can be. Further, if the position of the pupil P in the screen is known, the direction of the eye E viewed from the imaging means 4 can be known, so that the position of the eye E can be specified according to the distance of the eye E, and the position information of the eye E can be specified. Is used to calculate the gaze direction.

FIG. 5 shows the corneal reflection images 1a and 1b in the pupil P when the targets a and b in the display member 6 are viewed. Although a plurality of corneal reflection images 1a and 1b are drawn in the same pupil P, in practice, one corneal reflection image 1a and 1b is generated, and the calculation is sequentially performed to determine the center Po of the pupil P. A coordinate position on the imaging plane as the origin of the XY coordinates is obtained. The position of the corneal reflection image is the direction line L1 from the infrared light source 1 to the eye E and
It can be determined from the angles A and B between L2 and L3.

That is, by specifying the position of the eye E, the lines L1, L2, L3 in FIG.
If the direction of the line L1 connecting the eye and the eye E matches the line of sight, the reflected image 1k is generated near the center Po of the pupil P. Now, angles A and B formed by lines L2 and L3 and line L1 looking at optotypes a and b, respectively.
Is proportional to the distance from the image 1k to the corneal reflection images 1a and 1b of the infrared light source 1, and the angle difference B between the angles A and B is
From -A and the distance ab from the images 1a to 1b, the proportional constant C can be expressed as C = (BA) / (ab), and the position of the image 1k is calculated from the proportional constant C and the values of the angles A and B. Can be calculated.

The proportional constant C is related to the radius of curvature of the cornea and the depth of the anterior chamber, and the position of the image 1k is also individual. Therefore, calibration is performed for each operator before using the apparatus. At this time, when the astigmatism of the same eye E is strong, the diopter of the astigmatism differs depending on the direction. Therefore, it is preferable to calibrate the X direction and the Y direction separately. For this purpose, the optotypes a and b are arranged at positions vertically and horizontally separated on the screen of the display member 6, and the angle change B-A and the X of the distance ab are set.
The proportional constants Cx and Cy are calculated separately for the component and the Y component.

When operating the apparatus, as shown in FIG. 3, various operation switches 5a and 5b are generated and displayed on a display member 6 from a symbol generation circuit, and an operator's eye E is operated for operation. Looking at one of the switches 5b, the direction and distance of the eye E are specified by calculation from the position of the pupil P on the imaging surface and the focusing signal. At this time, the direction of the line of sight looking at the switch 5b is calculated from the relative positional relationship between the position of the corneal reflection image 5b 'by the infrared light source 1 and the image 1k and the proportional constant C. Is specified.

In this manner, the input of various switches 5a and 5b is performed by recognizing the direction of the line of sight, and the signal is transmitted to an actuator (not shown). When the device is a computer or a word processor, the actuator is a computing unit or a display unit, and in a mechanical device, the actuator is a driving unit such as a motor.

The image of the image pickup means 4 is displayed on the screen of the display member 6 as a small screen M, so that the operator can check during operation whether his or her own eye E is at an appropriate position. In addition, the operator sits at a position where the operator can easily operate before the operation, adjusts the direction of the imaging unit 4 so that the eye E is well reflected, and performs calibration with the targets a and b. Operations can be performed.

Further, if the face E moves within the range in which the eye E is reflected on the screen even if the face slightly moves during the operation, it is possible to detect the position of the pupil P in the imaging screen by calculation and specify its direction. . Furthermore, since the position of the eye E can be specified by detecting the distance to the eye E, the gaze direction can be calculated regardless of the distance. In a device in which the distance of the eye E is constant, only the direction may be detected. If the direction is constant, only the distance may be detected.

In the above description, the distance is obtained from the in-focus signal, but an optical system for measuring the distance may be provided separately. In this case, the light beam is projected from the vicinity of the imaging means 4 to the vicinity of the eye E, and measurement is performed using the principle of triangulation in which reflected light from the face is received and detected at a position away from the light source. The gaze detection is calculated from the measured distance as described above.

When the eye E looks at the switch 5b, the arithmetic control means 7
Recognizes this, the brightness of the switch 5b changes,
Notify the operator of the recognition. To input the switch 5b, once the line of sight is removed from the switch 5b,
The user turns his / her gaze again within a predetermined time, for example, within one second, and inputs the gaze when the second gaze is recognized. In this way, since it is not necessary to stare for a certain period of time, it is possible to perform a switch operation with a line of sight in a short time, and when there is no input intention, such an eye E does not move, so that there is no malfunction. .

The switches 5a and 5b are not limited to those displayed on the display member 6. If the operator performs a predetermined execution operation with the actuator when he sees a predetermined portion of the apparatus, the portion can be used as a switch. can do.
In this case, a voice is used to display to the operator that the arithmetic control means 7 has recognized the line of sight, and infrared light is emitted in front of the photographing lens 3 instead of the small screen M on the display member 6. A mirror that transmits and reflects visible light is attached to check the position of the eye E. This mirror is set to have a size that allows the area reflected by the imaging means 4 to be seen, and a mark serving as a target used for calibration is provided on the apparatus, or two parts of the apparatus that can be specified are used as calibration targets.

In the embodiment, the same image pickup means is used as the image pickup means for measuring the line of sight from the corneal reflection image on the pupil P and the image pickup means for specifying the direction of the eye E. However, separate image pickup means may be used. . In this case, the imaging range of the latter imaging means is widened so that the eye E can be reliably detected even if the face moves greatly, and the direction of the imaging means for detecting the pupil P is changed based on this. By driving in the direction of (1) and performing enlarged imaging, highly accurate detection can be performed.

[0023]

As described above, the gaze control device according to the first aspect of the present invention can detect the gaze direction with high accuracy regardless of the eye position by detecting the eye position information and determining the gaze direction. , Operation in a comfortable posture becomes possible.

The gaze control apparatus according to the second aspect of the present invention enables a reliable gaze operation in a short time by recognizing and inputting a second gaze within a predetermined time.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an embodiment.

FIG. 2 is an explanatory diagram of a calibration target for a display member.

FIG. 3 is an explanatory diagram of an operation switch of a display member.

FIG. 4 is an explanatory diagram of an anterior segment of an imaging unit.

FIG. 5 is an explanatory diagram of a corneal reflection image in a pupil.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Infrared light source 2 Driving means 3 Photographing lens 4 Imaging means 5a, 5b Operation switch 6 Display member 7 Arithmetic control means

Claims (3)

[Claims] An imaging device for imaging an anterior segment of an eye, an illuminating device for illuminating an anterior segment of the eye, and a detecting device for detecting a position of the eye, wherein an eye obtained from the detecting device is provided. A gaze direction is obtained from the positional information of the pupil and the positional relationship between the corneal reflection image of the illumination means and the pupil. 2. The gaze control device according to claim 1, wherein the position information is a distance or a direction of an eye. 3. A line-of-sight operating device that detects a line-of-sight direction and performs an operation with the line of sight, wherein an input operation is performed by detecting that the line of sight is directed a second time within a predetermined time. Operating device.