JPH1014882A - Non-contact line-of-sight measuring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To measure the line of sight with high precision while the limitation of the operating range is relieved greatly by lessening the burden on a subject. SOLUTION: Two cameras 1A and 1B are furnished to photograph the head part of a subject, and using the output therefrom an eyeball position sensing means 31 senses the position of his eyeball from markers M1, M2, M3. On the basis of the obtained eyeball position, a tracking controlling means 32 seizes the eyeball E with an eyeball camera 2 and allows it to pursue eyeball. The position of the cornea reflection line and the center of the pupil are sensed from the output of the eyeball camera 2, and the line-of-sight vector is determined by a vector calculating means 33. A line-of-sight laying means 34 lays spatially the determined vector in the desired coordinates system.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an eye gaze measuring device, and more particularly to a non-contact eye gaze measurement method that allows a subject to wear only a simple position marker that does not burden the subject and that can detect the gaze with higher accuracy than before. Related to the device.

[0002]

2. Description of the Related Art Conventional gaze measuring apparatuses are mainly of a type in which a subject's head is fixed and a type in which a detecting device is attached to the subject's head. Both of these significantly impaired the subject's usual behavior and put a great burden on the subject in gaze measurement. In recent years, non-contact types have been made to reduce the burden on subjects.

[0003]

However, the measurement range is extremely limited just because the detection device is remote from the subject, and the measurement is practically performed only when the subject cannot move. Also, the accuracy was very poor.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and has as its object to reduce the burden on a subject, to allow the subject to operate at a normal operation speed in a range sufficient for performing work on a desk, and Is to provide a device that captures the image with sufficiently high accuracy.

[0005] In addition, operations on a desk, particularly operations using a computer, are performed by looking at a monitor or a desk.
The line of sight moves on a plurality of arbitrary planes. Accordingly, it is an object of the present invention to provide a device that can place a line of sight on an arbitrary plane and specify what the user is looking at.

[0006]

SUMMARY OF THE INVENTION According to the present invention, a plurality of head cameras detect the position and inclination of a head, and the eyeball position is constantly tracked by an eyeball camera which enlarges and shoots the eyeball from this information. A device that detects a line of sight from the eyeball image and places the line of sight on any coordinate axis,
Eyeball position detecting means for detecting the position of the head, the direction of the face, and the position of the eyeball from three or more markers attached to the head of the subject;
Tracking control means for always tracking the eyeball camera from the information to direct the eyeball at the eyeball; gaze vector calculation means for detecting the position of the corneal reflection image and the pupil center from the output of the eyeball camera to obtain a gaze vector; And a line-of-sight setting means for setting the line of sight in the space.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention relates to a gaze measuring apparatus capable of measuring gaze detection with extremely high accuracy for a long time, comprising at least three gazes attached around an eyeball for measuring the position of the head. By measuring the marker three-dimensionally with two or more head cameras, the position and inclination of the marker can be accurately measured, and a head coordinate system that can be converted to a desired coordinate system is configured.

By placing the marker such that the eye to be measured is located at the center of gravity of the marker, it can be assumed that the eye is present at the center of the head coordinate system with high accuracy. The center of gravity of the eyeball is calculated from the position of the marker, and tracking of the eyeball camera is set assuming that the center of the eyeball is located at the center of gravity. Is set to the eyeball center position. The head coordinate system is calculated accordingly.

The eyeball is tracked by the eyeball camera in accordance with the center of the three-dimensionally measured head coordinate system, so that the eyeball camera always keeps the eyeball in the field of view.

The eye camera captures the pupil image and the corneal reflection image, finds the center of the pupil from the pupil image, and corrects the coordinates of the center of the pupil even if there is no pupil at the center of the eye camera. And
A line-of-sight vector is obtained from the corneal reflection image and the corrected pupil center coordinates.

A line-of-sight vector is projected by calculation on an arbitrary predetermined plane, and the line of sight is laid on the plane. The line of sight laid on this arbitrary plane is presented as a model. This allows the observer to immediately know at any time where the subject is looking at the target object.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the present invention, a plurality of head cameras and one eyeball camera are used. First, an eyeball position detection method using a headcam will be described. FIG. 1 is a diagram in which three markers M1, M2, and M3 (hereinafter, M is used when it is not necessary to distinguish them) around an eyeball E to be measured. H indicates a head.

In FIG. 1, three markers M are attached to the spectacle frame, and the markers M are three-dimensionally measured by a plurality of head cameras 1A and 1B shown in FIG. Observe FIG. 2 shows a case where two head cameras are used. The distance from the position of the marker M to the midpoint of the head cameras 1A and 1B,
Also, the direction is determined by a plurality of head cameras 1A and 1B.
Are calculated by collating the marker information with each other. Thereby, the approximate position P of the center of the eyeball E can be measured. It is obtained by the image processing unit 3 using X, Y, and Z information.

Next, tracking of the eyeball E by the eyeball camera 2 will be described. FIG. 3 shows a mechanism in which the eyeball camera 2 that captures the eyeball E tracks the eyeball E based on the position information of the eyeball. In FIG. 3, for example, X, Y
Is shown by a method using a mirror for dedicated tracking. In the figure, 4 is an X-direction position adjustment mirror,
5 is a Y-direction position adjusting mirror, and 32 is a tracking control means. Z is adjusted by a focus function inside the eyeball camera 2. Thereby, the eyeball camera 2 can always correctly capture the eyeball E with accurate focus.

FIG. 4 is a block diagram showing the configuration of one embodiment of the present invention. In FIG. 4, reference numeral 31 denotes an eyeball position detecting means for processing the outputs of the head cameras 1A and 1B. 32
Is the above-described tracking control means, which performs tracking control of the eyeball E by the eyeball camera 2. 33 is a line-of-sight vector calculation means,
The output of the eyeball camera 2 is processed. Reference numeral 34 denotes a line-of-sight arrangement means for displaying the line of sight of the subject.

Next, the operation of the embodiment of FIG. 4 will be described mainly with reference to FIGS. Note that (S1) to (S12) in FIG. 5 indicate each step.

The eyeball position detecting means 31 calculates the distance from the position of the marker M to the midpoint of the head cameras 1A and 1B, the head H
, And the approximate position of the eyeball E is detected (S1). The center of gravity of the three markers M1 to M3 is obtained, and correction is performed (S2) described later, and it is assumed that the eyeball E is present here (S3). The focus of the eyeball camera 2 is set from the obtained eyeball position (S4), and the tracking control means 32 further sets the position adjustment mirrors 4 and 5. As a result, the eyeball camera 2 captures the eyeball E, and thereafter tracks the eyeball E (S5).

The line-of-sight vector calculating means 33 processes the image obtained from the eyeball camera 2 and extracts a pupil image (S6). Since the pupil image includes many images other than the pupil, those noise portions are removed (S7), and the center position is calculated (S8). Since it was assumed that the X, Y, and Z of the tracking information of the eyeball camera 2 were at the center of the head cameras 1A and 1B, these were corrected from the detected pupil center and X ', Y', Z Ask for '.

At the same time, the corneal reflection image is captured by the eyeball camera 2 (S9), and the pupil center position and the position of the corneal reflection image, X ′, Y ′, Z ′, and the eyeball diameter from a physiological viewpoint, A line-of-sight vector is created using the value of the eyeball center (S10), (S11). The created line-of-sight vector is geometrically mapped to an arbitrary plane, which has been measured in advance, and is placed on the plane as a line of sight (S1).
2).

FIG. 6 is a view for explaining the placement of the line of sight by the line of sight placement means 34. A personal computer PC and a keyboard KB are placed on a desk, and planes PL1 and PL2 preset as display planes are prepared. ing. In FIG.
The gaze E of the subject's eyeball E is directed to one of the keys on the keyboard KB. Therefore, the gaze vector and the plane PL
An intersection W2 with No. 2 is obtained, and this point W2 is displayed. This allows the observer to immediately know where the subject is looking.

When the subject looks at the display of the personal computer PC, an intersection W1 between the line-of-sight vector at this time and the plane PL1 is displayed. In this manner, the spatial arrangement of the line-of-sight vectors is performed.

In order to observe the line of sight, the user must place the camera on an arbitrary plane. At that time, when to show to the subject,
You may not show it. In the case of showing, for example, in the case of the plane PL1 in FIG. 6, it is conceivable that the line of sight is directly displayed on the monitor if it is limited to only on the monitor. The plane PL2 can be indicated by a laser pointer or the like. If it is not shown, it can be displayed by taking an arbitrary plane with a camera and superimposing it on the video. In the case of a display device such as a monitor, the image may be distributed and superimposed on a monitor that is invisible to the subject.

The marker M is set so that the eyeball E is located at the center of gravity of the marker. However, this is not necessarily accurate, and the number of the markers M does not need to be three. I just need more. However, considering the rotation of the head H, the head can be tilted only up to 45 degrees at four points, but can be up to 60 degrees at three points. Also,
Although the eyeball E may not be located at the position of the center of gravity, the correction calculation thereof will be described below. If the actual eyeball position is K and the center of gravity of the marker is G, if the markers are M1 to M3, K = G before correction

[0024]

(Equation 1) Can be corrected with. Here, P1, P2, and P3 are vectors of points obtained by three-dimensional measurement, and PG is a vector of the center of gravity. X and Y are horizontal and vertical correction values, respectively. Assuming that the spectacle frame does not largely move, the correction value may be obtained by obtaining the entire view of the eyeball E by image processing and capturing the center of gravity G of the eyeball E, or at the beginning of the operation of the apparatus.
May be determined by looking at the image of the eyeball camera 2 capturing the position of the eyeball. This provides a basis for chasing the eyeball E fairly accurately.

[0025]

As described above, according to the present invention, by attaching a simple marker to the vicinity of the eyeball, the approximate position of the eyeball can be grasped by three-dimensional measurement from a plurality of head cameras. An eyeball can be tracked and photographed by an eyeball camera having a magnification, and a gaze vector can be detected with considerably high accuracy. Moreover, the subject only attaches three or more simple markers and does not need any other constraint.

Also, since the line of sight can be placed at arbitrary coordinates, for example, when measuring the line of sight when operating a computer environment placed on a desk, not only the line of sight on the monitor but also the line of sight of the keyboard or other objects There is an effect that the gaze can be captured.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a head detection marker provided around an eyeball to be detected in the present invention.

FIG. 2 is a diagram showing an outline of the operation principle of head detection in the present invention.

FIG. 3 is a diagram showing an operation principle of tracking an eyeball position by an eyeball camera according to the present invention.

FIG. 4 is a block diagram showing a configuration of one embodiment of the present invention.

FIG. 5 is a flowchart for explaining the operation of the embodiment in FIG. 4;

FIG. 6 is a diagram showing a mechanism for disposing a line of sight on an arbitrary plane according to the present invention.

[Explanation of symbols]

 M1 to M3 Marker E Eyeball H Head PC Personal computer KB Keyboard PL1, PL2 Plane W1, W2 Intersection between line of sight vector and plane 1A, 1B Head camera 2 Camera for eyeball 3 Image processing unit 31 Eyeball position detecting means 32 Tracking control Means 33 Gaze vector calculation means 34 Gaze placement means

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Yoshinobu Masaya 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Japan Telegraph and Telephone Corporation (72) Inventor Takehiko Ohno 3--19, Nishishinjuku, Shinjuku-ku, Tokyo No. 2 Inside Nippon Telegraph and Telephone Corporation (72) Inventor Hisao Nojima 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Inside Nippon Telegraph and Telephone Corporation (72) Noriko Aragaki 3-19 Nishi-Shinjuku, Shinjuku-ku, Tokyo 2 Nippon Telegraph and Telephone Corporation (72) Takashi Tate 1-7-2 Nishiazabu, Minato-ku, Tokyo NAC Corporation (72) Inventor Shigeru Sakamoto 1-2-7 Nishiazabu, Minato-ku, Tokyo Shares Inside the company Nack (72) Inventor Shigetomo Mizuno 1-2-7 Nishiazabu, Minato-ku, Tokyo Tokyo, Japan Inside (72) Inventor Yoshinori Suzuki 1-2-7, Nishiazabu, Minato-ku, Tokyo No. Inside of Nack Co., Ltd.

Claims (1)

[Claims] 1. A non-contact eye-gaze measuring device using a plurality of head cameras and one high-magnification eye camera, wherein at least three heads attached to the head captured by the head camera are provided. Eyeball position detecting means for detecting the position of the head and the direction of the face from the markers to detect the position of the eyeball, and capturing the eyeball by the camera for the eyeball based on the detected eyeball position, A tracking control unit that constantly chase the eyeball by information from the head camera, a gaze vector calculation unit that detects a position of a corneal reflection image and a pupil center from an output of the eyeball camera, and obtains a gaze vector, A non-contact gaze measuring device, comprising: gaze placement means for converting the coordinate system into a desired coordinate system and spatially placing the gaze position.