JP3225270B2 - Gaze measurement method, gaze measurement device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a gaze measuring method and a gaze measuring device capable of accurately measuring the direction of a gaze of a person.

2. Description of the Related Art As a conventional line-of-sight direction measuring apparatus, for example, there is one as disclosed in Japanese Patent Application Laid-Open No. 9-238905. In such an apparatus, the gaze direction can be calculated by detecting the eyeball position and the eyeball rotation angle.

However, if the position of the eyeball deviates even if the same object is viewed, the eyeball rotation angle changes, and it becomes impossible to calculate an accurate gaze direction.
Therefore, conventionally, a calibration method as shown in FIG. 12 has been adopted. FIG. 12A shows a state in which the eyeball 1 is viewing the first target 3 for calibration from directly above. FIG.
(A) shows a second optotype 5 and a third optotype 7 in addition to the first optotype 3 respectively. When the eye is at the position of X _o in the figure, if viewed first target 3, the eyeball rotation angle is detected to θ = θ _o. Here, when detecting the eyeball position and the eyeball rotation angle, the head of the person is not always accurately located at the same position, and tends to slightly shift left and right with time, for example, the head moves to the left. When deviated eye position was in X _1, the eyeball rotation angle with respect to the first target 3 will be detected as theta = theta _1. Therefore, the same first
The eyeball rotation angle θ is detected differently while the target 3 has a line-of-sight direction, and the eyeball position _Xo and the eyeball rotation angle θ
There is a problem that when the gaze direction is calculated from ₁ , the calculated gaze direction is shifted. In such a conventional eye gaze measuring apparatus, a calibration data line 9 as shown in FIG.
Is created and the gaze direction is calibrated. This calibration method is performed as follows. For example, the eyeball 1 is the first
A plurality of eyeball rotation angles, such as eyeball rotation angles θ = θ ₁ , _θo, etc., corresponding to the movement of the head when looking at the target 3, and the detection is performed by the second target 5 and the third target. The same goes for the target 7,
A plurality of data, for example, two data are detected for each optotype as shown in FIG. A calibration data line 9 is obtained by applying the least square method based on these data. Then, for example, when the eyeball rotation angle theta ₁ is detected, the position in the lateral direction of 10 at the intersection between the eyeball rotation angle theta ₁ and the calibration data line 9 will represent calibrated viewing direction. In FIG. 12, the calibration for the movement of the head in the horizontal direction is shown, but the calibration for the movement of the head in the vertical direction is performed in the same manner. However, when the eyeball rotation angle theta = theta ₁ is detected, to deviate from the 12 first target 3 viewing direction by the calibration data line 9 is the original visual line direction as shown in (b) by a distance S Therefore, there is a problem that even if the calibration is performed while allowing the movement of the head, the calibration is not accurate. An object of the present invention is to provide a gaze measurement method and a gaze measurement device capable of measuring a gaze direction by more accurate calibration in consideration of an eyeball position.

SUMMARY OF THE INVENTION claims 1 invention, eyeball a physical quantity corresponding to the line-of-sight direction when gazing at the specific target multiple gaze measurable range in which the eye can move
The eyeball position determined for each of the specified positions
Plural sets determined, specific of determined for each of the plurality of sets of each set a plurality of physical quantities for each object more specific object at location
Based on a plurality of physical quantities at the selected eye position, a calibration data point at the selected eye position is determined for each set, and a calibration data line at the selected eye position is determined based on the plurality of calibration data points. Consider eyeball position
The gaze direction is obtained by using the considered calibration data line . The invention according to claim 2 is to determine a plurality of physical quantities corresponding to the direction of the line of sight when the eyeball is gazing at a specific target, and to obtain a plurality of sets of the plurality of physical quantities for each of the plurality of specific targets. For each of the plurality of sets, based on a plurality of physical quantities, a plurality of characteristics of a gaze measurable range in which an eyeball can move.
A plurality of sets of a plurality of physical quantities corresponding to the line of sight directions are estimated for each of the determined positions, and the identified and estimated sets are estimated for each of the plurality of sets .
Based on a plurality of physical quantity at eye position, determine the calibration data point in a selected eye position for each set, determine the calibration data lines in the selected eye position based on the calibration data points of said plurality, the eyeball position Taking into account
The gaze direction is obtained using the corrected calibration data line . According to a third aspect of the present invention, there is provided the eye gaze measuring method according to the first or second aspect, wherein the calibration data considering the eyeball position is used.
And obtaining the viewing direction and a physical quantity corresponding to the viewing direction in the selected eye position and Tarain. The invention according to claim 4 is an eyeball position measuring means for measuring an eyeball position of a subject, a physical quantity measuring means for measuring a physical quantity corresponding to a gaze direction of the subject, and a gaze direction when the eyeball is gazing at a specific target. the eyeball is moved a physical quantity corresponding to the
A plurality of physical quantities at the specified eyeball positions are obtained for each of a plurality of specified positions in the movable eye gaze measurable range , and a plurality of physical quantities at the specified eyeball positions are extracted and stored for each of the plurality of specific targets. Physical quantity extraction storage means, and a calibration data point calculation for calculating a calibration data point at a selected eyeball position for each set based on a plurality of physical quantities at a specified eyeball position extracted for each of the plurality of sets. Means,
Calibration de to a calibration data line calculating means for calculating the calibration data lines in the selected eye position based on the calibration data points of said plurality, considering the eyeball position
And obtaining the sight line direction using a Tarain. The invention according to claim 5 is an eyeball position measuring means for measuring the eyeball position of a subject, a physical quantity measuring means for measuring a physical quantity corresponding to the gaze direction of the subject, and a gaze direction when the eyeball is gazing at a specific target. Are obtained a plurality of times, and a plurality of sets of the plurality of physical quantities are obtained for each of the plurality of specified objects, and the eyeball is determined for each of the plurality of sets based on the plurality of physical quantities.
Multiple specified positions in the gaze measurable range where the user can move
Based on a plurality of physical quantity of a plurality of physical quantity and specific physical quantity estimation storage means for plurality of sets estimated and stored, in the eyeball position specified is estimated for each of the plurality of sets of each set corresponding to the viewing direction for each, Calibration data point calculation means for calculating the calibration data points at the selected eyeball position for each set,
Calibration de to a calibration data line calculating means for calculating the calibration data lines in the selected eye position based on the calibration data points of said plurality, considering the eyeball position
And obtaining the sight line direction using a Tarain. According to a sixth aspect of the present invention, there is provided the eye gaze measuring apparatus according to the fourth or fifth aspect, wherein a calibration data line considering the eyeball position is provided.
Characterized in that the emissions from the physical quantity corresponding to the viewing direction in is selected eyeball position measured by the eyeball position measuring means with a gaze direction calculating means for calculating a viewing direction.
According to a seventh aspect of the present invention, in the eye gaze measuring device according to any one of the fourth to sixth aspects, the eyeball position measuring unit includes an imaging unit configured to capture an image of a face including a subject's eyes, and the imaging unit. And an eyeball position calculating means for extracting a circular region from the face image according to (1) and using the position of the center of gravity of the circular region as an eyeball position. The invention according to claim 8 is the eye gaze measuring device according to claim 7, wherein the physical quantity measuring means includes an illuminating means for irradiating the face of the subject, a corneal reflection image based on the illuminating means, and the eyeball position calculation. And an eyeball feature amount measuring means for setting a distance from the center of gravity by the means to an eyeball feature amount corresponding to the physical quantity. According to a ninth aspect of the present invention, in the eye gaze measuring apparatus according to the eighth aspect, the eyeball feature amount measuring means sets a rectangular area around the circular area that exceeds the size of the cornea, and the rectangular area The brightest point within is defined as a corneal reflection image. According to a tenth aspect of the present invention, in the eye gaze measuring apparatus according to the fourth aspect, the specific physical quantity extracting and storing means includes a target projecting means for projecting a target as a specific target at a specific position; And an optotype control means for controlling the projection means to make the specific position a plurality. The invention of claim 11 is the invention of claim 5
The eye gaze measuring device according to claim 1, wherein the specific physical quantity estimation storage means includes: a target projection means for projecting a target at a specific position as the specific target; and a plurality of the specific positions by controlling the target projection means. And a target control means.

According to the first aspect of the present invention, a physical quantity corresponding to the gaze direction is obtained for each of a plurality of specified positions in the gaze measurement range where the eyeball can move, for example, for each limit position where the gaze measurement device can detect. Thus, since a calibration data line is created by obtaining a plurality of sets of the plurality of physical quantities for a plurality of specific objects, an accurate calibration data line can be obtained. In other words, the measurable limit position of the gaze measurement device is always a constant position, unlike the intermediate position, and the calibration data line is set based on the physical quantity corresponding to the gaze direction at the fixed position (eyeball position). When created, it is possible to obtain an accurate calibration data line in consideration of the eyeball position, and it is possible to obtain an accurate gaze direction using the calibration data line. According to the invention of claim 2, based on estimating a plurality of sets of a plurality of physical quantities corresponding to a gaze direction for each of a plurality of specified positions in a gaze measurable range in which an eyeball can move based on a plurality of detected physical quantities. Since the calibration data line is created, the physical quantity of the eyeball position specified similarly to the above can be used, and an accurate calibration data line in consideration of the eyeball position can be obtained, and the accurate gaze direction can be measured. According to the third aspect of the present invention, in addition to the effects of the first or second aspect, by applying a physical quantity corresponding to a line-of-sight direction at an eyeball position selected by measurement or the like to a calibration data line considering the eyeball position, An accurate gaze direction can be obtained. According to the fourth aspect of the present invention, calibration is performed based on a plurality of sets of a plurality of physical quantities corresponding to a line-of-sight direction for each of a plurality of specified positions in a measurable range in which an eyeball can move based on a plurality of detected physical quantities. Since the data line is created, it is possible to obtain an accurate calibration data line in consideration of the eyeball position, and it is possible to measure an accurate gaze direction. According to the fifth aspect of the present invention, calibration is performed based on a plurality of sets of a plurality of physical quantities corresponding to the gaze direction estimated for each of a plurality of specified positions in a measurable range in which the eyeball can move based on the plurality of detected physical quantities. Since the data line is created, it is possible to obtain an accurate calibration data line in consideration of the eyeball position, and it is possible to measure an accurate gaze direction. According to the sixth aspect of the invention, in addition to the effects of the fourth or fifth aspect, a physical quantity corresponding to the direction of the line of sight at the selected eyeball position measured and selected by the eyeball position measuring means is applied to the calibration data line considering the eyeball position. Thereby, the gaze direction calculating means can calculate the accurate gaze direction. According to the seventh aspect of the invention, in addition to the effects of any one of the fourth to sixth aspects, a face image including the eyes of the subject is captured, a circular region is extracted from the face image, and the center of gravity of the circular region is determined. To set the eyeball position,
The eyeball position can be accurately measured, and a more accurate gaze direction can be obtained. According to the eighth aspect of the invention, in addition to the effect of the seventh aspect, an eyeball feature quantity corresponding to a physical quantity can be measured based on a distance between the corneal reflection image and the position of the center of gravity. You can get directions. According to the ninth aspect of the invention, in addition to the effect of the eighth aspect, a rectangular area is set around the circular area specifying the eyeball position in consideration of the size of the cornea, so that a corneal reflection image is more accurately obtained. And a correct gaze direction can be obtained. According to the tenth aspect, in addition to the effects of the fourth aspect, the optotype can be accurately projected on each of the plurality of predetermined positions, and a specific physical quantity can be quickly and reliably extracted and stored. According to the eleventh aspect, in addition to the effect of the fifth aspect, the target can be accurately projected on each of the plurality of predetermined positions, and the specific physical quantity can be quickly and reliably estimated and stored.

FIG. 1 is a schematic block diagram of a gaze measuring apparatus according to an embodiment of the present invention. here,
As for the method of gaze measurement, a type in which a subject wears a goggle-shaped measuring device or a method such as “Senso Mot of Germany”
It can be applied to any measuring device such as a type that can measure the direction of the line of sight in a non-contact manner, such as a gaze point analyzing device of oric Instruments (Monte System of Japan Import Agency). However, in the present embodiment, a non-contact type eye gaze measuring device is taken as an example. The eye gaze measuring device of FIG.
It comprises an eyeball position measuring means 11, a physical quantity measuring means 13, a physical quantity estimating storage means 15, a calibration data creating means 17, a gaze direction calculating means 21, and a gaze direction output means 23.
The eyeball position measuring means 11 measures the eyeball position of the subject, and comprises an imaging means 25 and an eyeball position calculating means 27. The imaging means 25 is constituted by a CCD camera, has sensitivity in the near infrared region, and captures an image of a face including the eye position of the subject. The eyeball position calculation means 27 extracts the pupil of the subject from the face image obtained by the imaging means 25.
The physical quantity measuring means 13 measures a physical quantity according to the gaze direction of the subject, and includes an illuminating means 29 and an eyeball feature quantity measuring means 31. The illumination unit 29 is a near-infrared illumination, and illuminates the face including the eyeball. The illuminating means 29 may be turned on all the time or only when an image is input. The eye feature amount measuring unit 31 measures the ocular feature amounts corresponding distance [delta] _H of the pupil centroid position according to the corneal reflection image and the eyeball position calculating means 27 based on the illumination means _29, [delta] _{V (described} below) to the physical quantity. The physical quantity estimation storage means 15
1, the physical quantity corresponding to the direction of the line of sight when the eyeball is gazing at the target 33 in FIG. 1 is obtained a plurality of times, a plurality of sets are obtained for each of the plurality of specified targets, and each of the plurality of sets is obtained. A plurality of sets of a plurality of physical quantities corresponding to the line of sight directions are estimated and stored for each of a plurality of specified positions where the eyeball moves based on the plurality of physical quantities for each set. 37, a calibration data temporary storage means 39 and a part of the calibration data creation means 17. The target projection means 3
Reference numeral 5 denotes a projector configured to project the calibration target 33 to the specified position. The optotype control means 37 controls the optotype projection means 35 to determine a projection position, and sets a plurality of specific positions at which the optotype 33 is projected. The calibration data temporary storage unit 39 obtains a plurality of physical quantities corresponding to the direction of the line of sight when the eyeball gazes at a specific target, and obtains a plurality of sets of the plurality of physical quantities for each target at a plurality of specified positions. The obtained values, that is, pupil barycentric positions and δH and δV, which will be described later, acquired during calibration are temporarily stored. The calibration data generating means 17, the plurality of sets of [delta] _H as the plurality of physical quantities for each _set, a plurality of the identified for each position [delta] _H eyeball moves based on the [delta] _V, [delta] _V plural sets estimation And memorize. The plurality of specified positions where the eyeball moves will be described later. Further, the calibration data creating means 17 constitutes calibration data point computing means and calibration data line computing means. The calibration data point calculation means calculates a calibration data point corresponding to the eyeball position for each set based on a plurality of physical quantities δ _H and δ _V estimated for each of the plurality of sets. The calibration data line calculating means calculates a calibration data line at the selected eyeball position based on the plurality of calibration data points. Gaze direction calculation means 2
1 denotes the gaze direction of the subject from the calibration data line created by the calibration data creation unit 17 and δ _H and δ _V as physical quantities corresponding to the gaze direction at the selected eyeball position measured and selected by the eyeball position measurement unit 11. calculate. The line-of-sight direction output means 23 outputs the calculated line-of-sight direction, and is, for example, displayed on a screen. FIG. 2 shows a layout diagram including the coordinate system of the present apparatus.
In FIG. 2, the center position (X _o ,
Y _o ) indicates the line-of-sight direction, the lateral directions of the positions A, B, C, D, and E, and the positions F, G, H,
A state is shown in which the target 33 is switched and projected in the vertical direction of I and J, respectively. However, in FIG. 2, the optotypes 33 are shown at all positions for convenience. Then, at the eyeball position (X _o , Y _o ), the horizontal deviation angle of the line of sight when the gaze target 33 at the position A is gazed is θ _H, and the vertical deviation angle when the gaze target 33 at the position F is gazed is It is shown as θ _V. Further, the positions X _min , X at the ends of the sight line measurable range
_{max, Y max,} a plurality of specified positions of _{Y min} moves eyeball, that is, a special position. When the calibration data is created, the projection position of the target 33 is switched from A to J, and the horizontal declination θ _H and the vertical declination θ _V are physical quantities corresponding to the gaze direction a plurality of times for each position. eye feature quantity [delta] _H corresponding to the _measures as described below [delta] _V. Then, in the calibration, first, the lighting unit 29 is turned on, and the imaging unit 25 captures an image including the eyeball 1 of the subject. Next, the illumination unit 29 is turned off, the pupil is extracted from the input image by the eyeball position calculation unit 27, and the position of the center of gravity is measured. Details of the processing will be described later. Next, the distances δ _H and δ _V between the corneal reflection image based on the illumination means 29 and the pupil center of gravity position calculated by the eye position calculation means 27 are calculated as eye characteristic amounts corresponding to physical quantities by the eye characteristic amount measuring means 31. Details of the processing will be described later. Next, the calibration data temporary storage means 39 is provided with information indicating that the data is obtained when the target 33 is gazed at the position A in FIG. 2, and the pupil center-of-gravity position and δ _H and δ _V are stored. This memory is performed a plurality of times in accordance with the subject's head movement. That is, the eyeball 1
Has obtained and stored a plurality of physical quantities corresponding to the direction of the line of sight when gazing at the target 33 as a specific target. Further, the optotype projection means 35 is controlled by the optotype control means 37, and at the same time, a signal is also inputted to the calibration data temporary storage means 39, so that the pupil barycenter position and δ _H , and [delta] _V is stored. That is, a plurality of sets of a plurality of physical quantities are obtained for each of the plurality of specified objects. Further, the calibration data creating means 17 converts the eyeball 1 from the data held in the calibration data temporary storage means 39 into FIG.
X _min , which is the end of the measurable area 40 of
A plurality of pairs of δ _H and δ _V assuming that they are at X _max , Y _min , and Y _max are estimated and stored. That is, a plurality of physical quantities corresponding to the direction of the line of sight are estimated and stored for each of a plurality of specified positions where the eyeball moves based on the plurality of physical quantities for each of the plurality of sets for each object. Details of the processing will be described later. Next, the eye gaze measurement is performed in a configuration excluding the eye target control unit 37, the eye target projection unit 35, and the calibration data temporary storage unit 39 in FIG. While the subject is performing the work, the following processing is repeatedly executed. That is, first, the illumination unit 29 is turned on, and the imaging unit 25 captures an image including the eyeball position of the subject. Next, the illumination means 29 is turned off, the pupil is extracted from the input image by the eyeball position calculation means 27, and the position of the center of gravity is measured. Details of the processing will be described later. Next, δ _H and δ _V are calculated by the eyeball feature amount measuring means 31. Details of the processing will be described later. Next, the calibration data predicted at the selected eyeball position is estimated based on the calibration data created as described above using the pupil center-of-gravity position obtained by the gaze direction calculation means 21. That is, the configuration data points at the eyeball positions selected based on the plurality of physical quantities estimated for each of the plurality of sets are calculated for each set, and the configuration data lines at the eyeball positions selected based on the plurality of configuration data points are calculated. calculate. Details of the processing will be described later. Then, the line-of-sight direction calculating unit 21 obtains the line-of-sight direction using the calibration data line. That is, the eyeball feature amount of the subject's eyeball 1 is fetched, and the gaze direction is calculated by the calibration data line. The calculated line-of-sight direction is output by the line-of-sight direction output unit 23. Then, by using the output of the accurate line-of-sight direction, it can be used for visual environment design (design of the arrangement and size of instruments, etc.) typified by “monitor monitoring work of a factory plant”. In addition, on a meter panel display device of a car, the subject is shown the meter panel, evaluates the quality of the meter, puts information in and out according to the direction in which the eyes are looking, and focuses on the position where the photographer peeks. It is possible to obtain an operation effect such as matching.
Next, the operation of the gaze measuring method and the gaze measuring apparatus will be described with reference to FIGS. When the flow of FIG. 2 is executed, first, in step S1
, The eyeball position is detected. This eyeball position detection is performed by reading the eyeball position of the eyeball position calculating means 27 in FIG. 1 as described above. In step S2, an eyeball rotation angle is detected. This eyeball rotation angle detection is
The measurement is performed by reading the measurement result of the eyeball feature amount measurement unit 31 in FIG. The relationship between the eyeball rotation angle and the eyeball feature amount will be described later. In step S3, in order to consider the movement of the subject's head, it is determined whether the eyeball position and the eyeball rotation angle have been detected a plurality of times, and the determined number of times, for example, n = 5
If it has been detected twice, the process proceeds to step S4, and it is determined whether or not detection has been performed for a plurality of optotype positions. As described above, the optotypes at the plurality of positions are used as the optotype control means 37 in FIG.
The target 33 is projected at different specific positions A to J in FIG. 2, and the determination is made with respect to the target 33 at each of the specific positions. It is determined whether the eyeball position and the eyeball rotation angle have been detected. If detection is performed for a plurality of target positions, the process proceeds to step S5, where specific position data is predicted. The specific position is defined as X in FIG.
_min , _Xmax , _Ymin , and _Ymax . Then, a plurality of sets of eyeball rotation angles at the specific position are predicted and estimated for each of the optotypes 33 at different positions. Next, step S6
In, the eyeball position is read, and calibration data at the eyeball position is created. As described above, the calibration data is created by calculating the calibration data points in the calibration data creating means 17 and further calculating the calibration data line. The above steps S1 to S6 represent a calibration step. Next, in step S7, an eyeball rotation angle is detected. The detection of the eyeball rotation angle is performed by the eyeball feature value measuring means 31 measuring the eyeball feature value. In step S8,
The gaze direction is calculated using the calibration data and the eyeball rotation angle. Steps S7 and S8 are measurement steps. The detailed flow of the calibration data creation in step S6 is as shown in FIG. FIG. 4A shows the flow of creating calibration data in the horizontal direction, and FIG. 4B shows the flow of creating calibration data in the vertical direction. First (a), in eye position _{X i} in the lateral direction read in at step S31,
Calibration data points at eye position X _i is predicted in step S32. Step calibration data points differs for each optotype 33 by the position at S33 is performed is determined whether the predicted prediction calibration data lines in the eyeball position X _i in step S34 if it is predicted for each optotype position Is output. When the output of the lateral calibration data line is performed, is executed flow (b), step S35~S38 are executed by the same processing, prediction calibration data lines in the eye position Y _i is output . Therefore, by using the horizontal and vertical calibration data lines, the gaze direction can be accurately calculated by the gaze direction calculation unit 21 as described above, and the accurate gaze direction can be output by the gaze direction output unit 23. it can. FIGS. 5 to 7 show processes of the eyeball position detection in step S1 and the eyeball rotation angle detection in step S2 in the eyeball position calculation unit 27 and the eyeball feature amount measurement unit 31. First, step S111 in FIG.
This shows a state where the input face image is read. In step S112, binarization processing is performed, and an area having a certain brightness or less is left. In step S113, a circular area is extracted. This extraction is performed by template matching or region shape analysis. In the template matching, a template having a shape to be detected is prepared in advance, and the template is superimposed on all points on a target image to calculate a degree of coincidence. It can be seen that the same shape as the acted template exists at a location with a high degree of coincidence. In the case of the present embodiment, several types of circular or elliptical templates are prepared, and the degree of coincidence with those templates is counted. In the case of analyzing the shape of a region, a well-known method for measuring the circularity of the region is to evaluate the region based on the area and the perimeter of the region. This means that when the area of the region is S and the perimeter is L, the closer the target region is to a circle, the more L * L / S
Is close to 4π. Next, the center of gravity of the detected circular area is calculated. That is, the pupil extraction result is read in step S114 of FIG.
In step S212, a corneal reflection image search area is set. Since the corneal reflection image is always near the pupil, a rectangular area is set near the extracted pupil. That is, a rectangular area larger than the size of the cornea is set around the circular area.
In step S213, the brightest point P in the rectangular area set on the input image is searched for. This point P becomes a corneal reflection image. Next, in step S214, the feature amounts δ _H and δ _V of the eyeball that change according to the line of sight are calculated. This calculation is performed, for example, as shown in FIG. FIG. 7 shows the search result of step S213, in which the pupil 41,
The pupil center of gravity 43 and the corneal reflection image P are shown. This FIG.
, The positional deviation between the pupil barycenter 43 and the corneal reflection image P, that is, the horizontal and vertical distances δ _H and δ _V are calculated. The distances δ _H , δ _V are characteristic amounts of the eyeball that change according to the line-of-sight direction, and the characteristic amounts δ _H , δ _V of the eyeball are physical amounts corresponding to the line-of-sight direction, for example, the rotation angle θ of the eyeball. Yes, it is. FIG. 8 shows an example of the calibration method.
(A) shows the horizontal direction and (b) shows the vertical direction. The relationship between the physical quantity (for example, the rotation angle θ of the eyeball) corresponding to the line of sight and the feature quantity of the eyeball (for example, the positional shift δ between the pupil center of gravity and the corneal reflection image) that changes accordingly is as shown in FIG. 8, for example. . Obtaining this relationship is the calibration work. That is, (δ _Hi , θ _Hi ) (i =
, N) are obtained. In this case, consider only the horizontal direction. From FIG. 8, it is approximated that θ = aδ + b,
If the error is written in the square sum of the difference between the predicted line-of-sight direction (aδ _{Hi + b)} and the actual viewing direction (theta _Hi) and S,

(Equation 1) ★ Generally, calibration is performed by such a procedure (least square method). Since the line of sight has horizontal and vertical components, the above operation is performed in the horizontal direction in FIG.
8 (a) and the vertical direction FIG. 8 (b). FIG. 9 shows a plurality of sets of eyeball feature amounts δ for each index position A to J stored by the calibration data time storage means 39.
_H, shows [delta] _V. (A) is a horizontal direction, (b) is a vertical direction. The reason why a plurality of δ _H and δ _V are shown at each position is that the subject's head constantly moves as described above, and the eyeball position moves accordingly. Since the variations of δ _H and δ _V correspond one-to-one with the positions of the eyeballs, the eyeballs are positioned at special positions X _min , X _max , Y
It is possible to estimate δ _H , δ _V assuming that they are at _min , Y _max . The estimation can be performed by the aforementioned least squares approximation. For example, the n number of measurement values obtained when watching the visual target at positions A and repositioned as shown in FIG. 10 in accordance with the eye position X _i, applying the least-squares approximation to the graph, the resulting X = X _min and X _max are given to the approximate straight line to obtain corresponding δ _H1min and δ _H2max . That is, in the above equations (1), (2) and (3), θ _Hi may be replaced with X _i . Similarly, each position B, C,
If the same processing is performed on the data obtained when each of D and E is watched, the data shown in FIG. 11 is obtained. That is, the position X of the end of the visual line measurable range 39 described in FIG.
_min, ocular feature quantity [delta] _H in _{X max} is calculated. For example, FIG. 11 illustrates an eyeball position X =
Shows the data of X _o, for example _X = X max at index position A, _{X = X min} data calculated in 47, 49, δ _H1max, the δ _H1min. The horizontal deviation of the line of sight facing the target at the index position A is θ _H1 . Similarly, X = X _max , X = X
index position B in _min, C, D, the upper and lower points were respectively the eye feature quantity [delta] _H in E (51,
53), (55, 57), (59, 61), (63, 6)
5). In addition, the horizontal deviation of the line of sight at this time is also from θ _H2 to θ _H5 . And this X = X
Results for _{min, X =} result of _{X max} to calibration data points considering eyeball position shown in the figure as each subjected to least squares approximation 67,69,71,7
3, 75 and the calibration data line 77 can be obtained.
The same processing is performed for the vertical direction, and Y =
Estimate the calibration data line at Y _min , Y = Y _max . The created calibration data line is stored in the calibration data creating means 17. In addition, other eyeball positions X
The prediction of the calibration data line at _i is performed similarly.
Thus, the steps S31 to S34, S35 to
S38 is executed, and the calibration data line 77 and the like are stored in the calibration data creating means 17. Then the gaze direction calculation unit 21, when adapted eye feature quantity [delta] _H detected by the eyeball rotation angle detection S8 calibration data lines 77 in the X = X _o in Figure 11, the corresponding theta _{delta] H} eyeball position X = made in a horizontal direction of the line-of-sight in the X _o. The gaze direction finally calculated is described by two horizontal and vertical components, and (θ
_δH, it is represented by θ _δV). The calculated gaze direction is
The line-of-sight direction output means 23 displays or temporarily stores the coordinate data. After the measurement is completed, the result is read out and the movement of the line of sight during the work is analyzed. Such an analysis can be accurately performed by accurate calibration.
In the above embodiment, the eyeball feature at the end of the measurable range is determined by estimating the least squares approximation, but the eyeball of the subject is fixed at the end of the eyeball and the eyeball feature is actually calculated. It can also be determined by measuring. For example, four points (X _min , 0), (X _max , 0), (0,
The subject's head is fixed using a chin rest or the like so as to be fixed at (Y _min ), (0, Y _max ), and calibration is performed. Therefore, a physical quantity corresponding to the direction of the line of sight when the eyeball is gazing at a specific target is obtained for each of a plurality of specified positions where the eyeball moves, and the plurality of physical quantities are determined for each of the plurality of specific targets. Each time, a plurality of sets will be obtained. In this case, instead of the physical quantity estimation storage unit 15,
A physical quantity extraction storage unit including a chin rest and the like will be provided. Further, the specific positions X _min , X _max .
Is the end position of the measurable range. However, as long as the position can be specified spatially, data δ _H and δ _V at other positions can be measured and used.

[Brief description of the drawings]

FIG. 1 is a configuration diagram according to an embodiment of the present invention.

FIG. 2 is a layout diagram according to one embodiment.

FIG. 3 is a flowchart according to one embodiment.

FIGS. 4A and 4B are flowcharts of calibration data creation according to an embodiment, wherein FIG. 4A is a flowchart in a horizontal direction and FIG. 4B is a flowchart in a vertical direction.

FIG. 5 is a flowchart of image processing according to one embodiment.

FIG. 6 is a flowchart of image processing according to an embodiment.

FIG. 7 is an enlarged view showing an eyeball feature amount.

FIGS. 8A and 8B are diagrams for explaining least square approximation, in which FIG. 8A is an explanatory diagram in a horizontal direction and FIG. 8B is an explanatory diagram in a vertical direction.

FIG. 9 shows data of five eyeball feature amounts,
(A) is an explanatory view in the horizontal direction, and (b) is an explanatory view in the vertical direction.

FIG. 10 is an explanatory diagram showing calibration by least squares approximation.

FIG. 11 is an explanatory diagram in which a calibration data line is obtained by least squares approximation.

12A is a plan view showing a rotation angle of an eyeball when a specific target is viewed, and FIG. 12B is an explanatory view showing a calibration result.

[Explanation of symbols]

Reference Signs List 1 eyeball 11 eyeball position measurement means 13 physical quantity measurement means 15 physical quantity estimation storage means 17 calibration data creation means (calibration data point calculation means, calibration data line calculation means) 21 gaze direction calculation means 23 gaze direction output means 25 imaging means 27 eyeball position Calculation means 29 Lighting means 31 Eyeball feature quantity measurement means 33 Optotype

Claims (11)

(57) [Claims] 1. A sight gauge eyeball is the eyeball physical quantity corresponding to the line-of-sight direction when gazing at the particular subject may move
A plurality of physical quantities at the specified eyeball position are obtained for each of a plurality of specified positions in the measurable range , and a plurality of physical quantities at the specified eyeball positions are obtained for each of a plurality of specific objects. Based on a plurality of physical quantities at the specified specified eyeball position, calibration data points at the selected eyeball position are determined for each set, and at the selected eyeball position based on the plurality of calibration data points, determined calibration data line, line-of-sight measurement method characterized by determining the gaze direction by using the calibration data lines in consideration of the eyeball position. 2. A plurality of physical quantities corresponding to the direction of the line of sight when the eyeball is gazing at a specific target are obtained a plurality of times, a plurality of sets of the plurality of physical quantities are obtained for each of the plurality of specific targets, and the plurality of sets are obtained. Eyeball moves based on multiple physical quantities for each set of
A plurality of sets of a plurality of physical quantities corresponding to the line of sight directions are estimated for each of a plurality of specified positions in the gaze measurable range that can be measured, and at the specified eyeball position estimated for each of the plurality of sets, > based on a plurality of physical quantities, determine the calibration data point in a selected eye position for each set, determine the calibration data lines in the selected eye position based on the calibration data points of said plurality, considering the eyeball position A gaze measuring method, wherein a gaze direction is obtained using a calibration data line . 3. The eye gaze measuring method according to claim 1, wherein the gaze direction is obtained from a calibration data line considering the eyeball position and a physical quantity corresponding to the gaze direction at the selected eyeball position. Characteristic gaze measurement method. 4. An eyeball position measuring means for measuring a subject's eyeball position, a physical quantity measuring means for measuring a physical quantity corresponding to the subject's gaze direction, and corresponding to a gaze direction when the eyeball is gazing at a specific target. double the physical quantity of the line of sight measurement range which the eye can move
The number of specified positions is determined for each of the specified eyes.
A specific physical quantity extraction storage unit that extracts and stores a plurality of sets of a plurality of physical quantities at a spherical position for each of a plurality of specific objects for each of the plurality of specific objects; and a specified eyeball position extracted for each of the plurality of sets. based on the <br/> plurality of physical quantity, calibration in selected and calibration data points calculating unit that calibration data point in the eyeball position is calculated for each set, the selected eye position based on the calibration data points of said plurality and a calibration data line calculating means for calculating a data line, line of sight measuring device and obtaining the sight line direction using the calibration data lines in consideration of the eyeball position. 5. An eyeball position measuring means for measuring a subject's eyeball position, a physical quantity measuring means for measuring a physical quantity corresponding to the subject's gaze direction, and a gaze direction when the eyeball is gazing at a specific target. The obtained physical quantity is obtained a plurality of times, a plurality of sets of the plurality of physical quantities are obtained for each of the plurality of specified objects, and a gaze measurement range in which the eyeball can move based on each of the plurality of physical quantities for each of the plurality of sets .
Specific physical quantity estimation storage means for estimating and storing a plurality of sets of a plurality of physical quantities corresponding to the line-of-sight directions for each of the plurality of specified positions; and at a specified eyeball position estimated for each of the plurality of sets . a calibration data point calculating means for calculating a calibration data point at a selected eyeball position for each set based on a plurality of physical quantities; and calibration data at the selected eyeball position based on the plurality of calibration data points. and a calibration data line calculating means for calculating a line, line of sight measuring device and obtaining the sight line direction using the calibration data lines in consideration of the eyeball position. 6. The eye gaze measuring apparatus according to claim 4, wherein a calibration data line considering the eyeball position and a physical quantity corresponding to a gaze direction at the eyeball position selected and measured by the eyeball position measuring means. A gaze direction calculating unit for calculating a gaze direction from the gaze direction; 7. The eye gaze measuring device according to claim 4, wherein said eyeball position measuring means is configured to capture an image of a face including eyes of a subject, and a face by said imaging means. An eye-gaze measuring device, comprising: an eyeball position calculating unit that extracts a circular region from an image and uses the position of the center of gravity of the circular region as an eyeball position. 8. The eye-gaze measuring device according to claim 7, wherein the physical quantity measuring unit includes an illuminating unit that irradiates the face of the subject, a corneal reflection image based on the illuminating unit, and the eyeball position calculating unit. An eye gaze measuring apparatus, comprising: an eyeball feature amount measuring means for setting a distance from the position of the center of gravity to an eyeball feature amount corresponding to a physical quantity. 9. The eye gaze measuring apparatus according to claim 8, wherein the eyeball feature quantity measuring unit sets a rectangular area around the circular area that exceeds the size of the cornea, and sets the rectangular area within the rectangular area. An eye gaze measuring device wherein the brightest point is a corneal reflection image. 10. The eye gaze measuring device according to claim 4, wherein the specific physical quantity extraction storage means projects a target as a specific target at a specific position, and the target projection means. An eye-gaze measuring device, comprising: a target control means for controlling the target position and making the specific position plural. 11. The eye gaze measuring apparatus according to claim 5, wherein the specific physical quantity estimation storage means projects a target as a specific target at a specific position, and the target projection means. An eye-gaze measuring device, comprising: a target control means for controlling the target position and making the specific position plural.