JPS6253630A - Method and apparatus for measuring position of steady gaze point - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a gaze point position measuring method and apparatus for determining the gaze point position relative to surrounding space from a visual field image obtained from an eye camera and an eye mark signal.

[Conventional technology and its problems]

An eye camera uses an imaging device to detect the position of the reflected light from the protruding part of the eyeball in the X and Y directions by irradiating near-infrared light onto the protruding part of the eyeball, and detecting the position of the light relative to the direction of the head. In addition to outputting the eye mark signal as a digital value,
This is a device that displays a light spot in a visual field image taken by a camera fixed to the head. Therefore, the measured eye mark signal is given as the relative gaze point coordinates on the visual field image, so if the visual field image changes due to the movement of the subject's head, that change indicates the actual gaze position. It is output in the form of being added to the coordinate value.

FIG. 2 is a diagram schematically showing this situation.

A in the figure represents a visual field image at a certain point in time during measurement, and if the gaze point at this time is C, the eye mark signal (x',
y). Now, if the subject's head moves, the visual field image will change accordingly, as shown in B, for example, and even if the subject continues to gaze at the same position during this period, the eye mark signal will change. (x', y'). Therefore, in order to determine the absolute gaze point position in the surrounding space from the eye mark signal, some kind of processing is required to correct the movement of the head.

The conventional method is to fix the subject's head with a pie board or the like to fix the visual field image relative to the surrounding space, and directly determine the absolute position of the gaze point relative to the surrounding space from the eye mark signal. There is a way. However, this conventional method imposes large restrictions on the experimental posture of the subject, which has the disadvantage of not being able to perform measurements in a natural state or for long periods of time.

Another method is to record the measured visual field image on a videotape and have the experimenter analyze the reproduced image to determine the position of the gaze point in the surrounding space. However, this method requires visual analysis of each frame or several frames of the reproduced image.
The disadvantages are that real-time analysis is impossible and that analysis requires a great deal of effort and time.

[Purpose of the invention]

The purpose of the present invention is to eliminate such drawbacks of the prior art,
It is an object of the present invention to provide a method and apparatus for determining the position of a gaze point in the surrounding space in real time without restricting the head movement of an experiment subject by processing visual field images.

[Structure of the invention]

The first configuration of the present invention converts an eye mark signal indicating the direction of the eyeball in the X and Y directions relative to the direction of the head detected by the eye camera into an absolute gaze point position with respect to an object in the surrounding space. In the gaze point position measurement method, a plurality of feature points are set up in a measurement space so as to be distinguishable from others, and are photographed by the eye camera, and the positions of the plurality of feature points in the photographed visual field image are determined. The eye mark signal is originally corrected.

The second configuration of the present invention is a gaze point position measuring device that converts an eye mark signal output as a digital value from an eye camera into an absolute gaze point position with respect to an object in the surrounding space. a feature point supplying means for supplying feature points that can be identified from an image; and an AD conversion means for digitizing the visual field image from the eye camera;
a reference image storage means for storing the first frame of the digitized visual field image as a reference image; a binarization means for binarizing the digitized visual field image; and a binarized visual field image. a feature point coordinate extraction means for calculating the coordinates of the feature point from the reference image; a reference point coordinate storage means for storing the coordinates of the feature point of the reference image as reference point coordinates; A transformation matrix calculation means for calculating a transformation matrix of the visual field image with respect to the reference image from the coordinates of feature points, and a coordinate transformation means for transforming the eye mark signal into coordinates on the reference image using the transformation matrix. [Principle of the Invention] FIG. 3 is a schematic diagram showing an example of gaze point measurement possible according to the present invention. Figure 3 (a) is a reference image that is stored in advance before measuring the position of the gaze point.
, F' indicate this, and the same is true for FIGS. 3(b) to 3(d). FIG. 3(b) is a diagram showing one frame of the visual field image during measurement of the gaze point position, and an eye mark signal shown in G is displayed together with the object 1 light emitter. FIG. 3(c) shows the method according to the present invention shown in FIG.
The eye mark signal in b) is corrected and displayed on the reference image in FIG. 3 (a>).

With respect to the reference image in Figure 3 (a), the visual field image itself in Figure 3 (b) changes due to the movement of the subject's head (in this case, movement to the left), and this image The eye mark signal expressed in the coordinates cannot be expressed as is on the reference coordinates.Therefore, in the present invention, the changes in the two seated images are calculated from the changes in the respective positions of the corresponding light emitters, and using this, The coordinates of the eye mark signal are converted into the coordinates on the reference image, and the coordinates of the gaze point on the reference side @ (that is, the surrounding space) are determined. [Embodiment] Fig. 1 is a block diagram showing an embodiment of the present invention. In the figure, 1 is a feature point supply means, 2 is an imaging means, 3 is an AD conversion means, 4 is a reference image storage means, 5 is a binarization means, 6 is a feature point coordinate extraction means, and 7 is a feature point coordinate storage means,
8 is a reference point coordinate storage means, 9 is a transformation matrix calculation means, 10
1 is a coordinate conversion means, and 11 is an information processing means.

The feature point supplying means 1 supplies three light emitters (for example, LH'D) so as to provide feature points with high brightness in the field of view image.
It shall be installed in the measurement space at the position described below. The imaging means 2 is composed of an eye camera (for example, the eye mark recorder model ■ manufactured by NAC Corporation), and outputs a visual field image to the AD converting means 3 via a signal line 100, and outputs an eye mark signal to the coordinates via a signal line 111. It is sent to the conversion means 10. The AD conversion means 3 converts the sent analog visual field image into A
The image is D-converted and sent as a digital image f to the binarization means 5 and the reference image storage means 4 via the signal line 101.

The reference image storage means 4 stores the first frame of the sent digital visual field image as a reference image.

The binarization means 5 converts the entire area of the digital image into 2 as shown in the following equation (1) using a threshold value S sent from a control device (not shown) via a signal line 109. Value.

However, in equation (]), V(i, j) is the gray level of the (i, j) pixel of the digital image sent from the AD conversion means 3, and B (i.

j) represents the value of the (i, j) pixel of the binarized image.

This threshold value S is set when the area of the light emitting body on the image is "1".
, and other areas are set to "0". This binarized image is sent to the feature point coordinate extraction means 6 via the signal line 102.

The feature point coordinate extraction means 6 extracts this binarized image @(10
2) and Po1.2, which are the feature point coordinates of the previous frame sent from the next feature point coordinate storage means 7 via the signal line 104.
Poz, PO3, and the three regions H, J, and K (these are shown in Fig. 4 (a) (shown in Fig. 4(a)), the following equation (2) is performed for ), the obtained signal (Xi<, yk) (k-1+
2+3) are feature point coordinates p, , p2 . It is sent to the signal line 103 as p3. X here. k+! 10k represents the x and y coordinates of POk on the image, respectively.

(2) Pot, PO2, and PO3 are not given when calculating the feature point coordinates of the reference image, which is the first frame, but at this time, the initial values P'o+ and P as shown in FIG. 4(b) are given. '
02. P 03 is given, L' is set large, and the three light emitters are installed so that their image falls within each area, so that P1+
P2. P3 is required. Feature point coordinate storage means 7
stores the feature point coordinates, and changes the value each time a new feature point coordinate is sent by the feature point coordinate extraction means 6. Stored feature points PO1, PO2, PO
3 is then sent to the feature point coordinate extraction means 6 via the signal line 104 when extracting the feature points of the frame.

The reference point coordinate storage means 8 stores the feature point coordinates in the reference image photographed immediately before the start of the gaze point position measurement as the reference point coordinates (p, . . . ).

Psz, P, 3) is sent to the transformation matrix calculation means 9 via the signal line 105 for each frame when measuring the position of the gaze point.

The conversion matrix calculation means 9 calculates the reference point coordinates Ps□.

P3□ ps3 and feature point coordinates PI obtained from feature point coordinate extraction means 6 during measurement, p2. IF)3 to the following formula (
A 2×2 matrix A and a 2×1 matrix B that satisfy 3) are calculated. However, Pa1- and Pl, Pa2 and P2, pH3
and P3 respectively represent the coordinates of the image of the same light emitter.

P=+”AP t+B, P −z; AP z+
lB.

P −3=A P 3+B −(3>The coordinate conversion means 10 converts the eye mark signal E(x,y) sent from the imaging means 2 via the signal line 111 to the eye mark signal E(x,y) sent via the signal line 106. Coordinate transformation expressed by equation '(4) is performed using matrices A and B.

F=AE+IB (4) The resulting coordinates F (x, y) indicate the coordinates of the gaze point in the reference image. F is signal line 107
The information is sent to the information processing means 11 via.

The information processing means 11 connects the reference image storage means 4 to the signal line 1.
From the reference image sent via 08 and F sent via signal line 107, it is determined which object (area) in the reference image the gaze point is on.For example, as shown in FIG. 3(d), 2, it calculates which square the eye mark signal falls into when the reference image is divided into a grid pattern) and outputs it via the signal line 112.

〔Effect of the invention〕

As is clear from the above explanation, according to the gaze point position measuring device of the present invention, the gaze coordinates within the standard visual field image can be obtained in real time without fixing the head, so that the experiment subject can naturally Measurement can be performed in a stable state, and the data processing effort associated with gaze point analysis can be reduced.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of the gaze point position measurement method of the present invention, FIG. 2 is a schematic diagram showing an example of the output of an eye camera, and FIGS. FIG. 4 is a schematic diagram showing an example of the result of measuring the position of the gaze point, and FIG. 4 is a schematic diagram showing the feature point extraction method of this embodiment. 1... Feature point supply means, 2... Imaging means, 3...
AD conversion means, 4...Reference image storage means, 5...2
Value converting means, 6... Feature point coordinate extraction means, 7... Feature point coordinate storage means, 8... Reference point coordinate storage means, 9...
-Transformation matrix calculation means, 10...Coordinate transformation means, 11.
...Information processing means, 100-112...Signal line. horse-1

Claims (2)

[Claims] (1) A gaze point position measurement method that converts an eye mark signal indicating the direction of the eyeball in the X and Y directions relative to the head orientation detected by an eye camera into an absolute gaze point position with respect to an object in the surrounding space. In this step, a plurality of distinguishable features are set up in the measurement space and photographed by the eye camera, and the eye mark is determined based on the positions of the plurality of feature points in the photographed visual field image. A method for measuring the position of a gaze point, which is characterized by correcting a signal. (2) In a gaze point position measuring device that converts an eye mark signal output as a digital value from an eye camera into an absolute gaze point position with respect to an object in the surrounding space, a Feature point supply means for supplying feature points; AD conversion means for digitizing the visual field image from the eye camera; and reference image storage means for storing the first frame of the digitalized visual field image as a reference image. , a binarization means for binarizing the digitized visual field image, a feature point coordinate extraction means for obtaining the coordinates of the feature point from the binarized visual field image, and a coordinate of the feature point of the reference image. a reference point coordinate storage means for storing the reference point coordinates as reference point coordinates; and a transformation matrix calculation means for calculating a transformation matrix of the visual field image with respect to the reference image from the reference point coordinates and the coordinates of the feature point from the feature point coordinate extraction means. , coordinate conversion means for converting the eye mark signal into coordinates on the reference image using the conversion matrix.