JPH11259225A - Device for inputting line of sight - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain stable line of sight input even under a solar light without being affected by any background noise from an arriving light. SOLUTION: A head mounted display is mounted on an operator. A picture for input instruction transmitted from a computer 5 to an LCD(liquid crystal display device) 1 is inputted through a mirror 4 and a half mirror A3 to the eye ball, and the infrared ray of an infrared light source 6 is turned on and off by a modulating circuit 10 so that the eye ball can be irradiated, and the eye ball picture at that time is captured through a half mirror B7 by a CCD camera(infrared camera) 8, and inputted to a line of sight detecting circuit 9. Then, a difference between the eye ball picture when the eye ball is irradiated and a picture when the eye ball is not irradiated is obtained by a difference picture storing means 12 of a line of sight detecting circuit 9, and a background noise is removed so that line of sight detection can be operated. Therefore, stable line of sight input can be attained even under a solar light.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a line-of-sight input device in a computer system using an HMD (head-mounted display).

[0002]

2. Description of the Related Art In order to find out what parts of an image appearing in the eye the person is interested in and what parts of the image he or she is looking at, look at the image further.
As a device that can indicate intention with gaze,
There is a line-of-sight detecting device, and there is a head-mounted display in which the line-of-sight detecting device is incorporated in order to reduce a positional deviation from a display image.

FIG. 4 shows a line-of-sight input device in which this head-mounted display and a computer are combined. This device consists of a head-mounted display and a gaze detection circuit
8 and a computer 5. The head-mounted display has a half mirror A3, a mirror 4, an optical system 2, a liquid crystal display LCD, and the like so that a wearer can observe an image.
And an infrared light source 6, a half mirror B7, and an infrared camera (CCD camera 8) for capturing an eyeball image.
And The input instruction image is displayed on the liquid crystal display device LCD1 from the computer 5, and enters the eye via the optical system 2, the mirror 4, and the half mirror A3. Then, low-level infrared light is emitted from the infrared light source 6 to the eyeball obliquely, and the eyeball image and the corneal reflection image are captured by the infrared camera (CCD camera 8) via the half mirror B7, and the eyeball image is obtained. The signal is taken into the line-of-sight detection circuit 18.

The gaze detection circuit 18 comprises an image pickup circuit 11, a binarization circuit 13, a pupil / corneal reflection image extraction circuit 14, a gaze detection operation circuit 15, a gaze detection storage circuit 16, and a light source power supply 17. I have. Since the eyeball image captured by the CCD camera 8 into the image capturing circuit 11 includes an intermediate grayscale tone, the binarizing circuit 13
The binarization is performed using two preset threshold values for pupil extraction and corneal reflection image extraction. And the pupil
The position of the reflected image of the cornea of the infrared light from the infrared light source 6 and the center position of the pupil image are extracted from the outline of the binarized image by the corneal reflection image extraction circuit 14. Is calculated.

[0005] The computer 5 moves the cursor on the image display of the liquid crystal display device LCD1 to the position of the line of sight, or instructs an input character, based on the detection result of the line of sight position information.

[0006]

The conventional eye-gaze input device is constructed as described above, irradiates the eyeball with low-level infrared light, captures an eyeball image by an infrared camera, and generates a pupil as a feature point. The reflection image of the infrared light source is extracted. However, when strong extraneous light (sunlight) enters the optical system of the apparatus, there is a problem that infrared light included in the extraneous light becomes noise and a line of sight is detected in an incorrect place.

The present invention has been made in view of such circumstances, and provides a gaze input device capable of performing stable gaze detection by removing background noise included in extraneous light. With the goal.

[0008]

In order to achieve the above object, a gaze input device according to the present invention is provided with a head mounted display for displaying at least an input instruction image in accordance with a command from a computer, and the head mounted display is provided with the head mounted display. A light source for eyeball illumination, a line-of-sight detection unit that detects a line of sight of a wearer of the display using an imaging optical system, and a calculation unit that calculates a gazing point of an image by the wearer of the head-mounted display from the detection result. The apparatus includes a modulation circuit for intensity-modulating a light source for eyeball illumination, and a mechanism for capturing an eyeball image in synchronization with the modulation signal.

A head-mounted display for displaying at least an input instruction image in accordance with a command from a computer, and a line of sight of a wearer of the display using an eyeball illumination light source and an imaging optical system provided on the head-mounted display. In a device comprising: a gaze detecting means for detecting the eye gaze; and a calculating means for calculating a gazing point of the image by the wearer of the head mounted display from the detection result, a modulation circuit for intensity-modulating a light source for eyeball illumination is provided. The image processing apparatus further includes a difference image storage unit that calculates and stores a difference between an eyeball image when the eyeball is not illuminated by the modulation and an eyeball image when the eyeball is illuminated.

The eye-gaze input device of the present invention is configured as described above. The eye-gaze input device captures an eyeball image in synchronization with an intensity modulation signal, and calculates and stores a difference between the eyeball image in the irradiation on state and the irradiation off state. The gaze detection can be performed without being affected by background noise due to extraneous light.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the gaze input device of the present invention will be described with reference to FIG.

This line-of-sight input device comprises a head-mounted display, a line-of-sight detection circuit 9 and a computer 5. The head mounted display includes a half mirror A3, a mirror 4, an optical system 2, and a liquid crystal display LCD1 for observation, and an infrared light source 6, a half mirror B7, and an infrared camera (CCD camera 8) for imaging. And Then, the input instruction image is displayed on the liquid crystal display device LCD1 from the computer 5, and enters the eyeball via the optical system 2, the mirror 4, and the half mirror A3. Then, low-level infrared light is radiated from the infrared light source 6 to the eyeball obliquely, and is transmitted to the infrared camera (CCD) via the half mirror B7.
The camera 8) captures the eyeball image and the corneal reflection image, and captures the eyeball image signal into the eye gaze detection circuit 9.

The line-of-sight detection circuit 9 includes an image pickup circuit 11 and
A modulation circuit 10, a difference image storage means 12, a binarization circuit 13, a pupil / corneal reflection image extraction circuit 14, a gaze detection operation circuit 15, and a gaze detection storage circuit 16 which are not provided in the conventional device. It is composed of

The infrared light source 6 is the modulation circuit 1 of the line-of-sight detection circuit 9.
By 0, the infrared light source intensity is modulated by a rectangular wave as shown in FIG. A synchronizing signal is sent from the modulation circuit 10 to the image pickup circuit 11 in synchronization with a half cycle of the infrared light source 6 driven by the modulation circuit 10, and an eyeball image illuminated by the infrared light source 6 is illuminated. Two types of images, that is, an eyeball image in a state where the eyeball is not performed, are input to the image capturing circuit 11 of the visual line detection circuit 9.

[0015] The background noise contained in the extraneous light is superimposed on the illuminated eyeball image, and the background noise and the reflected image from the eyeball are superimposed on the illuminated eyeball image. By subtracting the former image from the latter image of the sheets, a clear reflection image of the eyeball from which background noise from extraneous light has been removed can be obtained. This processing is performed by the difference image storage unit 12.

Next, the eyeball image that has been subjected to the above processing contains an intermediate grayscale tone, and is set in advance by the binarization circuit 13 in order to clarify the boundary of the outline of the image. Binarization is performed using two threshold values for extracting a pupil and a corneal reflection image. Then, the pupil / corneal reflection image extraction circuit 14 extracts the position of the reflection image of the cornea of the infrared light from the infrared light source 6 and the center position of the pupil image from the outline of the binarized image. Next, the line-of-sight detection calculation circuit 15 calculates the horizontal distance and the vertical distance between the center of the pupil and the center position of the corneal reflection image as viewed from the CCD camera 8 side, and is converted from the horizontal and vertical distances from a predetermined reference point. do it,
The gaze position of the wearer is calculated.

The computer 5 moves the cursor on the image display of the liquid crystal display device LCD1 to the position of the line of sight or instructs the input character according to the detection result of the line of sight position information.

Although the above-described infrared light source is modulated using a rectangular wave having a half cycle, the driving waveform of the modulation circuit 10 is not limited to a square wave as shown in FIG. Can also be used. In this case, the image may be captured at the timing of the upper and lower peaks of the sine wave.

[0019]

The eye-gaze input device of the present invention is configured as described above, and calculates the difference between two images of an eyeball image with infrared irradiation off and an eyeball image with irradiation on, thereby obtaining a difference from extraneous light. Since background noise can be removed and gaze detection can be performed without being affected by the background noise, stable gaze input can be performed even under sunlight.

[Brief description of the drawings]

FIG. 1 is a diagram showing an embodiment of a gaze input device of the present invention.

FIG. 2 is a diagram showing the infrared light source intensity and the timing of image capture.

FIG. 3 is a diagram showing the infrared light source intensity and another timing of image capture.

FIG. 4 is a diagram showing a conventional line-of-sight input device.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... LCD (liquid crystal display device) 2 ... Optical system 3 ... Half mirror A 4 ... Mirror 5 ... Computer 6 ... Infrared light source 7 ... Half mirror B 8 ... CCD camera (infrared camera) 9 ... Gaze detection circuit 10 ... Modulation circuit DESCRIPTION OF SYMBOLS 11 ... Image imaging circuit 12 ... Difference image storage means 13 ... Binarization circuit 14 ... Pupil / corneal reflection image extraction circuit 15 ... Gaze detection arithmetic circuit 16 ... Gaze detection storage circuit 17 ... Light source power supply 18 ... Gaze detection circuit

Claims (2)

[Claims] 1. A head mounted display for displaying at least an input instruction image in accordance with a command from a computer, a line of sight of a wearer of the display using an eyeball illumination light source and an imaging optical system provided on the head mounted display. Line of sight detection means for detecting the, and a device comprising a calculation means for calculating the gazing point of the image by the wearer of the head mounted display from the detection result,
A gaze input device comprising: a modulation circuit for intensity-modulating a light source for eyeball illumination; and a mechanism for capturing an eyeball image in synchronization with the modulation signal. 2. A head mounted display for displaying at least an input instruction image in accordance with a command from a computer, a light source for illuminating an eyeball provided on the head mounted display, and an image pickup optical system. In a device comprising a line-of-sight detecting means for detecting the line of sight, and a calculating means for calculating the gazing point of the image by the wearer of the head-mounted display from the detection result, a modulation circuit for intensity-modulating the light source for eyeball illumination is provided,
A gaze input device comprising: difference image storage means for calculating and storing a difference between an eyeball image when the eyeball is not illuminated by the intensity modulation and an eyeball image when the eyeball is illuminated.