WO2014188727A1 - 視線計測装置、視線計測方法および視線計測プログラム - Google Patents
視線計測装置、視線計測方法および視線計測プログラム Download PDFInfo
- Publication number
- WO2014188727A1 WO2014188727A1 PCT/JP2014/002707 JP2014002707W WO2014188727A1 WO 2014188727 A1 WO2014188727 A1 WO 2014188727A1 JP 2014002707 W JP2014002707 W JP 2014002707W WO 2014188727 A1 WO2014188727 A1 WO 2014188727A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- line
- visual
- axes
- optical axis
- sight
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Arrangement of adaptations of instruments
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Definitions
- the present invention mainly relates to a gaze measurement of a user who looks far away, for example, a gaze measurement of a user who looks far away from an observation deck, and a gaze measurement of a pilot such as a train, an automobile, and a ship.
- the present invention relates to a device that can perform calibration and perform line-of-sight measurement.
- This line-of-sight interface detects a user's line of sight as data using a camera and a light source, and operates an icon or the like on a computer screen using the detected line-of-sight data.
- the eyeball is imaged by irradiating the user's eyeball with light from a light source such as infrared rays, and direction data calculated from the distance between the reflected light such as infrared rays on the corneal surface of the photographed image and the pupil is obtained by the user. It is detected as estimated gaze data.
- An error that differs for each user occurs between the estimated line-of-sight data calculated by this technique and the actual line-of-sight data of the actual user.
- There are various factors that cause the error such as individual differences in eyeball shape, light refraction on the corneal surface, and individual differences in the position of the fovea. Therefore, in order to correct the error of the estimated line-of-sight data with respect to the actual line-of-sight data, a process called calibration is performed in which a correction parameter for each user is calculated in advance, and the calculated estimated line-of-sight data is corrected with this correction parameter. Done.
- the calibration process allows a user to gaze at a plurality of predetermined markers in order, detects estimated line-of-sight data when each marker is watched, and detects the estimated line-of-sight data and the actual position from the eyeball to each marker. This is done by using a correction parameter calculated from the difference from the direction data.
- the optical axis of the eyeball which is an axis connecting the center of curvature of the cornea and the center of the pupil of the pupil, is obtained from the eyeball image by photographing the reflected light and the pupil on the cornea surface of the light source with a camera. Then, a deviation (individual difference) between the optical axis of the eyeball and the visual axis (equivalent to the line of sight) is obtained by calibrating one point, and then the measured optical axis is accurately shifted by the amount of deviation. Seeking gaze.
- the position of the central fovea inside the eyeball cannot be photographed from the outside by a camera, and it is difficult to reduce the number of points to be watched during calibration from one point.
- the present inventor has already proposed a line-of-sight measurement device that does not require calibration processing by measuring the optical axes of both eyes and adding a constraint that the visual axes intersect on the display screen (patent) Reference 2).
- the eye gaze measuring apparatus proposed by the present inventor obtains an eyeball image in which light from a light source is reflected by a camera for a user looking at a display screen, and connects the center of curvature of the cornea and the pupil center of the pupil from the eyeball image.
- the optical axis that is the axis is calculated, and using the calculated optical axis, the deviation between the optical axis and the visual axis that is the axis connecting the fovea and the center of curvature of the cornea is calculated, and the optical axis and the visual axis are calculated. Based on the deviation from the axis, the optical axis is shifted to obtain the visual axis, and the gazing point on the user's screen is calculated as the intersection of the screen and the visual axis.
- a gaze measurement of a user who looks far away such as a gaze when driving a vehicle such as a car, a train, or a ship, or a user who looks far away from an observation deck is performed.
- the user is considered to be looking somewhere in front and far away most of the time.
- An arrow 1 in FIG. 1 indicates the line of sight of the car driver 2 in (1), the line of sight of the train driver 3 in (2), and the line of sight of the ship's captain 4 in (3).
- Such a user would be very ideal if the calibration was made naturally during a line-of-sight movement looking into the distance without looking elsewhere specifically.
- Patent Document 1 and Non-Patent Documents 1 to 3 have a limitation that it is necessary to watch at least one predetermined place. Even if there are such restrictions, if an individual can be identified, the first calibration is sufficient, but it is not suitable for a case where an unspecified number of people are to be measured.
- JP 2007-136000 A JP 2009-297323 A JP 2009-183473 A
- the present invention provides a line-of-sight measurement apparatus and method for performing line-of-sight measurement of users who look far away, including users who look far away from the observation deck, operators such as trains, automobiles, and ships, by automatic calibration.
- the purpose is to do.
- a visual line measuring device of the present invention is a visual line measuring device for a user who mainly looks far away, and is a light for calculating the optical axes of the left and right eyeballs when viewing at least two points in the distance.
- a user who mainly looks far away refers to a user who is looking far away at an observation deck or the like, or a pilot who is operating a moving means including a train, a car, and a ship.
- a pilot of a training simulation apparatus for driving training of vehicles such as trains, automobiles, ships, airplanes, and bicycles is also included.
- the present invention can be suitably used for measuring the line of sight of a driver who controls an actual vehicle and for measuring the line of sight of a pilot in a training simulation apparatus for driving training.
- the projection screen may be several meters away, but by using a device that displays an image at infinity with a special optical system, the simulator operator can be treated as a user looking far away There is sex. Further, it can be suitably used for measuring the line of sight of a user who looks far away on the observation deck.
- the line of sight of the user who looks far away at the observation deck and the operator (user) during the operation of trains, cars, ships, etc. is mainly focused on the distance.
- a constraint condition that the line of sight is parallel is used.
- the size of the outer product of the vectors of the visual axes of the left and right eyes or the inner product of the unit vectors is set up. Then, four variables expressing the deviation between the optical axis of the eyeball and the visual axis are calculated. Note that “at least” is calculated by extracting features from an image taken with a camera, for example, in order to calculate the optical axis of the eyeball. This is because the measurement accuracy can be improved by measuring points.
- the constraint condition is that the magnitude of the outer product of the vectors of the left and right eye visual axes is 0, or The inner product of the unit vectors of the visual axes of both the left and right eyes is 1.
- the constraint condition is the following formula (1) with respect to the optical axes of the left and right binocular eyes when viewing at least four points in the distance. Fulfill.
- c Ln ( ⁇ L , ⁇ L ) and c Rn ( ⁇ R , ⁇ R ) are unit vectors of the visual axis of the left and right eyeballs, respectively, and the horizontal shift angle ⁇ and the vertical shift from the optical axis. It is expressed as a function of the angle ⁇ .
- L is left
- R is right
- n is 0, 1,..., N (N is 3 or more).
- the constraint condition is that when the magnitude of the outer product of the vectors of the visual axes of the left and right eyes is 0, with respect to the optical axes of the left and right eyes when viewing at least four points in the distance, Satisfy (2).
- c Ln ( ⁇ L , ⁇ L ) and c Rn ( ⁇ R , ⁇ R ) are unit vectors of the visual axis of the left and right eyeballs, respectively, and the horizontal shift angle ⁇ and the vertical shift from the optical axis. It is expressed as a function of the angle ⁇ .
- L is left
- R is right
- n is 0, 1,..., N (N is 3 or more).
- the left side of the following formula (2) represents the magnitude of the vector (norm), it is surrounded by double vertical bars.
- the optical axis calculation means obtains an eyeball image that is an image of an eyeball for a user viewing a distance, and connects the center of curvature of the cornea or the rotation center of the eyeball and the pupil center of the pupil from the eyeball image.
- the optical axis is calculated.
- the eyeball image is preferably an eyeball image in which light from a predetermined light source is reflected.
- the line-of-sight measurement device of the present invention has different positions so as to acquire an eyeball image of both left and right eyes in front of the user. And at least two light source means arranged at different positions so that the reflected images of the light source on the eyeball are separated from each other.
- At least two Purkinje images are formed in a region of constant curvature radius of the cornea of the left and right eyeballs, the position of the Purkinje image of the light source means on the eyeball image picked up by any camera means, and the actual light source
- the positions of the means are associated with each other, and the optical axis of the eyeball is calculated from the positional relationship between at least two sets of the associated camera means and the light source means.
- the search range of the visual axis deviation angle with respect to the optical axes of the left and right eyes is ⁇ 7 ° or less in the horizontal direction and ⁇ 2 ° or less in the vertical direction. Converge the corners. This ensures the stability of the solution for the deviation angle.
- the horizontal direction is ⁇ 7 ° or less and the vertical direction is ⁇ 2 ° or less.
- the average of the optical axis of the eyeball and the line of sight seen by humans is about 5 ° in the horizontal direction and about 1 ° in the vertical direction. This is because it is known that there is a deviation.
- the magnitude and direction of the deviation differ from individual to individual, but there are few cases where the deviation in the horizontal direction is more than ⁇ 7 ° and in the vertical direction is more than ⁇ 2 °.
- a range of a predetermined deviation angle is set as a search range.
- the gaze measurement method of the present invention is a gaze measurement method for a user who mainly looks far away, and an optical axis calculation step for calculating the optical axes of the right and left eyeballs when viewing at least two points in the distance;
- a displacement calculating step for calculating a displacement between the optical axis of the left and right eyes and the visual axis, and a relationship between the optical axis and the visual axis, with the constraint that the visual axes (line of sight) of both the left and right eyes are parallel during the distant gaze
- the constraint condition is that the size of the outer product of the respective vectors of the left and right eye visual axes is 0 or the inner product of the unit vectors of the visual axes of both the left and right eyes is 1.
- the line-of-sight measurement program of the present invention is a line-of-sight measurement program for users who mainly look far away, and causes a computer to execute the following steps 1) to 3).
- a gazing point calculation step for calculating a user's gaze point based on the shift between the optical axis and the visual axis
- the present invention it is possible to perform gaze measurement by performing automatic calibration on the line of sight of a user who looks far away, including operators such as trains, cars, ships, etc., so that an unspecified number of people other than a specific user can be measured. There is an effect that can be done.
- there is no need for a user's burden because calibration is not required in which the user gazes at a specific marker in advance.
- the present invention by measuring the line of sight when a driver such as a car is mainly looking far away, it is possible to detect aside driving and activate a warning transmission or a safety device to prevent accidents. it can.
- the deviation angle between the optical axes of the left and right eyes and the visual axis is automatically obtained.
- the constraint is that the visual axes (line of sight) of both the left and right eyes are parallel during distant gaze. This constraint condition is used because, for example, it is assumed that most of the user's line of sight at the time of maneuvering a train, an automobile, a ship, etc. is a normal natural state of gazing far away in front.
- calibration can be automatically performed. Then, by obtaining the deviation between the optical axis and the visual axis, the subsequent gaze point of the user can be calculated, and the line of sight can be measured. Subsequent gaze points can be measured not only from distant gaze points, but also from nearby gaze points such as meter panels.
- the eyeball does not perform all the operations that can be performed in its operation, but normally operates according to a certain law. This constant law is called the law of listing.
- the law of listing is the law regarding the rotational movement and eye position of the eye, (a) any eye position of the eyeball takes only a position that can be reached with a single rotation from the primary position, and (B)
- the rotation axis of the rotation refers to a law relating to eye movements that exists in a plane (listing plane) perpendicular to the visual axis direction of the first eye position.
- the first eye position is a relative eye position with respect to the head satisfying the law of listing, and is a direction when the head is viewed straight in a horizontal position.
- FIG. 2 shows the relationship between the visual axis and the optical axis in the first eye position (Primary Position) before the eye movement and the eye position after the movement.
- a and b are the visual axis vector and optical axis vector of the eyeball in the first eye position
- c and d are the visual axis vector and optical axis vector after rotation. Both are unit vectors.
- A represents the center of corneal curvature of the eyeball.
- a, b, c, d, and A are written in bold to indicate that they are vectors.
- the capital letter (bold type) of the alphabet is properly used so as to indicate the position vector
- the small letter of the alphabet is the unit direction vector.
- the eye position after the operation includes rotation other than the position of the eyeball (second eye position) when the eyeball is rotated vertically or horizontally from the first eye position, and rotation from the first eye position vertically or horizontally. It includes the position of the eyeball (third eye position).
- the unit direction vector (first visual axis vector) of the visual axis in the first eye position is a
- the unit direction vector (first optical axis vector) of the optical axis in the first eye position is b
- after eye movement The unit direction vector (second visual axis vector) of the visual axis in the eye position is c
- the unit direction vector (second optical axis vector) of the optical axis in the eye position after the eyeball operation is d.
- the first visual axis vector a is a vector of a direction when a person stands almost straight and looks at the front, is a vector that varies with the position of the head, and is determined relative to the position of the head. is there.
- the first visual axis vector a is measured by some method such as approximating the front direction, and the direction is known.
- FIG. 3 shows the relationship between the visual axis and the optical axis in the first eye position and the eye position after movement based on the law of listing.
- the vectors a, b, c, and d in FIG. 2 are moved so that the starting points are collected in one place so that the rotation relationship can be easily understood.
- the rotation axis when the eyeball directly moves from the first eye position to the eye position after the operation is l.
- l is a unit direction vector along the rotation axis of the eyeball, and the eyeball is rotated from a, which is the first eye position, around an axis in the listing plane perpendicular to a (axis direction vector is l). Only take position.
- the first visual axis vector a rotates by an angle ⁇ to become the second visual axis vector c
- the first optical axis vector b Is rotated by an angle ⁇ to become the second optical axis vector d.
- the rotation axis l exists in a listing plane perpendicular to the first visual axis vector a. More specifically, the rotation axis l is perpendicular to the first visual axis vector a and the second visual axis vector c.
- the rotation axis 1 and the rotation angle ⁇ can be calculated using the following formulas (3) and (4), respectively.
- the second visual axis vector c can be calculated by rotating the first visual axis vector a by an angle ⁇ about the rotation axis l. That is, c can be obtained by rotating a as shown in the following formula (5).
- R ( ⁇ , l) is a matrix that rotates ⁇ around l.
- the line-of-sight measurement apparatus 100 includes five components, that is, a light source unit 10, a camera unit 12, an optical axis calculation unit 14, a deviation calculation unit 16, and a gaze point calculation unit 18.
- the optical axis calculation means is means for calculating the optical axis of the user's eyeball, but any means can be used as long as it can always calculate the optical axis of the eyeball using a known method.
- a method using two cameras and two light sources for one eye is used. As a result, the optical axis of the eyeball can be obtained accurately and quickly (in one frame).
- the LED light source irradiates infrared rays
- an infrared camera having sensitivity to infrared rays is prepared. Two cameras may be used to capture both eyes, but in order to take an eyeball image with high resolution, a total of four cameras, two for the right eye and two for the left eye, are used.
- the line-of-sight measurement apparatus 100 includes a CPU 211, a memory 212, a hard disk 213, a keyboard 214, a mouse 215, a display 216, an external memory 217, an LED 218, and a camera 219.
- the CPU 211 performs processing based on other applications such as an operating system (OS) and a line-of-sight measurement program recorded on the hard disk 213.
- the memory 212 provides a work area for the CPU 211.
- the hard disk 213 records and holds an operating system (OS), other applications such as a visual line measurement program, and measurement data obtained as a result of the visual line measurement.
- the keyboard 214 and the mouse 215 accept external commands.
- the display 216 displays the eyeball images of the subject captured by the two cameras 219 for the right eye and the two for the left eye for confirmation by the user of the line-of-sight measurement apparatus 100.
- the external memory 217 is a USB memory, for example, and reads data such as a line-of-sight measurement program.
- the LED 218 irradiates the subject whose line of sight is measured by the line-of-sight measuring apparatus 100 with light.
- the camera 219 captures an eyeball image of the subject. In the case of two cameras, they are configured as a stereo camera and are used to capture a binocular image. In the case of four cameras, two sets of stereo cameras are configured and used to capture images of the left and right eyes.
- the position of the Purkinje image of the light source on the eyeball image captured by any camera is associated with the actual position of the light source. Then, a surface including the corneal curvature center is calculated from the position of the camera (center of the lens) and the position of the light source associated with the position of the captured Purkinje image, and three or more surfaces are obtained, thereby calculating the curvature of the cornea. The center position is calculated. Then, the center position of the pupil is calculated from the eyeball image, and an optical axis that is an axis connecting the curvature center of the cornea and the pupil center of the pupil is calculated based on the center position of the curvature of the cornea and the center position of the pupil. For details, see the flow charts of FIGS. 6 and 7 and paragraphs 0019 to 0029 of Patent Document 2 described above.
- a deviation value between the optical axis of the eyeball and the visual axis is calculated in advance by, for example, gazing at a predetermined place by a calibration operation performed in advance by the user. Then, the visual axis, which is an axis connecting the fovea and the center of curvature of the cornea, is calculated from the calculated optical axis using the calculated deviation value.
- Calibration is to obtain the relationship between a and b in FIG.
- a is determined relatively to the position of the head
- the calibration is equivalent to estimating b by assuming that the front is approximated to the direction and is known.
- the deviation between a and b is represented by two variables. Assuming that the horizontal shift angle is ⁇ and the vertical direction is ⁇ , b can be expressed by the following equation (6) using ⁇ and ⁇ .
- c L and c R respectively the visual axis of the left and right eyes of the eye. If c L and c R are unit vectors and are parallel to each other, the inner product of c L and c R is 1 as shown in the following equation (7).
- a L is known, and b L can be expressed as a function of ⁇ L and ⁇ L from the above equation (6).
- d L is obtained from a conventional method. Therefore, from the above formulas (3) to (6), c L can be expressed as a function of ⁇ L and ⁇ L.
- c R can be expressed as a function of ⁇ R and ⁇ R. If there are four unknowns and the user looks at four different directions, four equations are established as shown in the following equation (8). By solving these four equations, four variables ⁇ L , ⁇ L , ⁇ R and ⁇ R which are unknown numbers can be obtained.
- the optical axis of the eyeball is calculated by extracting features from an image taken by a camera, but usually contains noise. Therefore, more points are measured, and solutions of the four variables ⁇ L , ⁇ L , ⁇ R , ⁇ R are determined so that the evaluation function F of the following formula (9) is minimized.
- the deviation between the optical axis of the eyeball and the visual axis When the deviation between the optical axis of the eyeball and the visual axis is large, the deviation can be accurately obtained when the present invention is used. However, when the deviation between the optical axis of the eyeball and the visual axis is small, the calculation for obtaining the deviation is performed. The influence of noise becomes large, the calculation result becomes unstable, and the deviation cannot be obtained accurately. However, when the deviation between the optical axis of the eyeball and the visual axis is small, it can be said that the optical axis can be approximated to the visual axis. In either case, there is a possibility that it can be judged by calculating several times and whether the result varies.
- FIGS. 9 and 10 an experimental system as shown in FIGS. 9 and 10 was constructed to evaluate the accuracy of the visual line measuring device.
- the subject 20 looks at a distant target through the front glass 21, and the state of the target is displayed by the four cameras 22 (Camera 0 to Camera 3) provided on the lower edge of the glass 21. It can be obtained.
- Two infrared LEDs (IR-LEDs) 23 are provided on the glass 21.
- a one-point calibration marker 24 for obtaining ⁇ L , ⁇ L , ⁇ R , ⁇ R is provided on the glass 21 by a one-point calibration method which is a conventional technique (one-point calibration is performed). (See Non-Patent Document 3 above for the method).
- the test subject looks at the nine targets ahead of the glass 21 by 8 m.
- the nine targets are arranged in a 3 m (horizontal direction) ⁇ 2 m (vertical direction) frame in front of the user.
- the optical axis of the eyeball of the right eye is calculated using a pair of two cameras 22 (Camera 0, Camera 1) among the four cameras provided on the lower edge of the glass 21, and the two cameras 22 (Camera 2, Camera 3) are calculated.
- the optical axis of the eyeball of the left eye is calculated using The camera used is a digital camera equipped with a CMOS image sensor equipped with a 50 mm lens and an infrared (IR) filter.
- IR infrared
- the experiment calculates the optical axis of both eyes while the subject is looking at one of the nine targets through the glass 21, and the inner product of the unit vectors of the visual axes of the left and right eyes is
- the displacement ( ⁇ L , ⁇ L , ⁇ R , ⁇ R ) between the optical axis and the visual axis was calculated with 1 being a constraint condition.
- the deviation between the optical axis and the visual axis was calculated using the conventional one-point calibration method, and the two were compared.
- Table 2 There are 3 participants in the experiment (1 boy and 2 girls).
- the calculation is a search range for the value of ⁇ in the horizontal direction of the left eye, from ⁇ 7 ° to 0 °, the search range of the value of ⁇ in the horizontal direction of the right eye is 0 ° to 7 °, and a search for the value of ⁇ in the vertical direction.
- the range was -2 ° to 2 ° for both left and right.
- ⁇ L , ⁇ L , ⁇ R , and ⁇ R are automatically obtained by looking at a distance, but these values can be used even when looking close. By just looking far away for a while, it becomes possible to know exactly where you saw a nearby meter.
- data necessary for calculation of ⁇ L , ⁇ L , ⁇ R , ⁇ R can always be obtained even after calibration. Unlike a normal gaze measurement device, always, It is possible to correct the calibration result. Since the calibration of the present invention is based on the premise that the user is looking far away, it is important to determine whether he is looking far or near.
- the optical axis of the eyeball is obtained within an error range of about ⁇ 5 °, so that it can be determined to some extent from the direction of the optical axis. From the direction of the optical axis alone, it is impossible to discern when looking at insects that are still on the glass. If you have a certain speed, such as a car, you can't think of seeing it for a long time. It is possible to determine the vehicle speed from the viewpoint of how often the convergence angle increases above a certain vehicle speed.
- Example 2 a line-of-sight measurement apparatus according to Example 2 will be described.
- a state when two distant points are viewed is captured, and at least two distant points are viewed with a constraint condition that the visual axes (gazes) of the left and right eyes of the user are parallel.
- the optical axis of the right and left eyeballs can be calculated and the deviation between the optical axis of the left and right eyes and the visual axis can be calculated.
- the unit vectors of the visual axes of the left and right eyes are equal.
- c L and c R are unit direction vectors of the visual axis of the left and right eyeballs, respectively, and ⁇ L , ⁇ L , ⁇ R , and ⁇ R represent the deviation between the optical axis of the eyeball and the visual axis. These represent the deviations in the horizontal direction, vertical direction, right eye horizontal direction, and vertical direction of the left eye, respectively.
- the restriction that the unit vectors of the visual axes of the left and right eyes are equal is the same as the constraint that the visual axes (line of sight) of the left and right eyes are parallel.
- the present invention is useful as a line-of-sight measurement device and a line-of-sight measurement method for operators of trains, cars, ships, and the like, and users looking far away from the observation deck.
Abstract
Description
そこで、実視線データに対する推定視線データの誤差を補正するために、ユーザ毎の補正用パラメタを予め算出しておき、算出された推定視線データをこの補正用パラメタで補正するキャリブレーションと呼ばれる処理が行われる。
キャリブレーション処理は、予め定められた複数のマーカを利用者に順に注視させ、それぞれのマーカが注視されたときの推定視線データを検出し、検出された推定視線データと眼球から各マーカへの実際の方向データとの差から算出される補正用パラメタを用いることにより行われる。
しかし、精度の高い視線データを検出するためには、補正用パラメタを生成する際に利用者に5点から20点程のマーカを注視させる必要があり、ユーザの負担が大きかった。このような状況下、キャリブレーション処理を1点のマーカにまで減少させる技術が開示されている(例えば、特許文献1,非特許文献1~3を参照。)。
本発明者の提案する視線計測装置は、ディスプレイ画面を見ているユーザについて、光源からの光が反射した眼球画像をカメラで取得し、眼球画像から角膜の曲率中心と瞳孔の瞳孔中心とを結ぶ軸である光軸を算出し、算出した光軸を利用して、光軸と、中心窩と角膜の曲率中心とを結ぶ軸である視軸との間のずれを算出し、光軸と視軸との間のずれに基づき、光軸をずらして視軸を求め、ユーザの画面上での注視点を画面と視軸の交点として算出するものである。
このようなユーザは、特別に指定されたどこかを見ることなしに、遠方を眺める視線動作中に自然にキャリブレーションがなされるならば、非常に理想的である。
なお、本明細書において、キャリブレーションとは、眼球の光軸と視軸のずれを求めることと同義である。
cLn(αL,βL)・cRn(αR,βR)=1 ・・・ (1)
||cLn(αL,βL)×cRn(αR,βR)||=0 ・・・ (2)
1)遠方の少なくとも2点を眺める際の左右両眼の眼球の光軸を算出する光軸算出ステップ
2)遠方注視の際、左右両眼の視軸(視線)が平行であることを拘束条件として、左右両眼の光軸と視軸との間のずれを算出するずれ算出ステップ
3)光軸と視軸との間のずれに基づき、ユーザの注視点を算出する注視点算出ステップ
また、本発明によれば、ユーザが予め特定のマーカを注視するというキャリブレーションが不要であるので、ユーザの負担がないといった効果がある。
本発明を用いて、車などの操縦者が主に遠方を見ている状況での視線を計測することにより、よそ見運転を検出して警告発信や安全装置を作動させ、事故防止を図ることができる。
左右両眼の光軸と視軸のずれ角を自動で求めるために、遠方注視の際に左右両眼の視軸(視線)が平行であることを拘束条件とする。この拘束条件を用いるのは、例えば、電車、自動車、船舶などの操縦時のユーザの視線の殆どは、前方で遠方を注視することが通常の自然な状態と想定されるためである。遠方の少なくとも4点を眺めた眼球の状態を捉えることにより、左右両眼の眼球の光軸を算出し、左右両眼の光軸と視軸との間のずれを算出する。
そして、光軸と視軸との間のずれが求まることで、ユーザのその後の注視点を算出でき、視線を計測できる。その後の注視点は、遠方の注視点のみならず、メータパネルなど近くの注視点も計測可能である。
眼球は、その動作において実行可能な全ての動作をするわけではなく、通常は、ある一定の法則に従って動作している。この一定の法則をリスティングの法則という。リスティングの法則は、眼の回転運動と眼位に関する法則であり、(a)眼球の任意の眼位は、第一眼位(Primary Position)から単一の回転で到達できる位置しかとらず、そして、(b)その回転の回転軸は、第一眼位の視軸方向に垂直な平面(リスティング平面)内に存在するという眼球動作に関する法則をいう。なお、第一眼位は、リスティングの法則を満たす頭部に対する相対的な目の位置であり、およそまっすぐ立ったときに水平に真正面を見たときの方向となる。
第一視軸ベクトルaは、人がおおよそ真っ直ぐ立って正面を見た時の向きのベクトルであり、頭部の位置とともに変動するベクトルであり、頭部の位置に相対的に決定されるものである。第一視軸ベクトルaは、正面の方向と近似するなど、なんらかの方法で計測し、向きは既知であるとする。
以下では、本発明の実施形態について、図面を参照しながら詳細に説明していく。なお、本発明の範囲は、以下の実施例や図示例に限定されるものではなく、幾多の変更及び変形が可能である。
ユーザの前方周囲に配置された2個のLED光源(LED光源は赤外光線を照射するものを用いる)と、前方で遠方を注視しているユーザについて、LED光源からの光が反射した眼球画像を取得するカメラ(LED光源が赤外光線を照射する場合、赤外線に感度をもつ赤外線カメラ(infrared camera))を用意する。
2台のカメラでそれぞれ両眼撮影すればよいが、高解像度で眼球画像を撮るためには、右目用2台、左目用2台の合計4台のカメラを使用する。
CPU211は、ハードディスク213に記録されているオペレーティング・システム(OS)、視線計測プログラム等その他のアプリケーションに基づいた処理を行う。メモリ212は、CPU211に対して作業領域を提供する。ハードディスク213は、オペレーティング・システム(OS)、視線計測プログラム等その他のアプリケーション、及び視線計測の結果得られた計測データを記録保持する。
キーボード214、マウス215は、外部からの命令を受け付ける。ディスプレイ216は、右目用2台、左目用2台のカメラ219で撮像した被験者の眼球画像を、視線計測装置100の使用者の確認のために表示する。外部メモリ217は、例えばUSBメモリなどであり、視線計測プログラム等のデータを読み取る。
LED218は、視線計測装置100によって視線を計測する被験者に対して、光を照射する。カメラ219は、被験者の眼球画像を撮影する。2台のカメラの場合、それらはステレオカメラとして構成され、両眼の画像を撮影するために用いる。また、4台のカメラの場合、2組のステレオカメラが構成され、左右の眼の画像を撮影するために用いる。
aとbのずれは2変数で表される。水平方向のずれ角度をα、垂直方向をβとすると、bは、α、βを用いて下記数式(6)のように表現できる。
これについて以下に説明する。ユーザが遠方を見ているとき、両眼の光軸と視軸の関係は図8のように示される。両目の視軸は平行になる。ここで、眼球の光軸と視軸のずれは、全部で4変数を用いて表現できる。4変数は、左目の水平方向、垂直方向、右目の水平方向、垂直方向を、それぞれαL,βL,αR,βRとする。
よって、上記数式(3)~(6)から、cLはαL,βLの関数として表現できる。同様に、cRはαR,βRの関数として表現できる。
未知数が4つあり、ユーザが異なる4方向を見るとすると、下記数式(8)のように式が4つ成立することになる。この4つの式を解くことにより、未知数である4変数αL,βL,αR,βRを求めることができる。
シミュレーションは、正解(True Value)のαL,βL,αR,βRを変化させた5ケースについて実施した。眼球の動きについては、水平、垂直方向、それぞれ10°刻みで、-20°~20°の範囲で計25方向について計算を行った。各方向について、標準偏差が0.3°となるように計測に伴う誤差を載せて60点のデータを作成した。計測誤差の大きさは、従来方法から眼球の光軸を計測した結果から、水平方向0.21°、垂直方向0.18°であったことから0.3°と決定した。
結果を下記表1に示す。
下記表1より、Case1~4の|α|が大きい場合(ずれ角が3°や5°の場合)、正確にαL,βL,αR,βRを計算でき、また、標準偏差が小さい。一方、Case5,6の|α|が小さい場合は(ずれ角が1°の場合)、αL,βL,αR,βRの値はあまり正確に求まらず、また標準偏差も大きくばらついていることがわかる。
実験システムは、被験者20に、前方のガラス21越しに遠方のターゲットを眺めてもらい、その状態を、ガラス21下縁部に設けた4台のカメラ22(Camera0~Camera3)で眼球部分の画像を取得できるようにしたものである。ガラス21上に赤外線LED(IR-LED)23を2台設けている。また、従来技術である1点キャリブレーション方法により、αL,βL,αR,βRを求めるための1点キャリブレーション用のマーカ24をガラス21上に設けている(1点キャリブレーションの方法については上述の非特許文献3を参照)。
ガラス21下縁部に設けた4台のカメラの内、2台のカメラ22(Camera0,Camera1)のペアーを用いて右目の眼球の光軸を算出し、2台のカメラ22(Camera2,Camera3)のペアーを用いて左目の眼球の光軸を算出することにしている。使用したカメラは、50mmレンズと赤外線(IR)フィルタを備えたCMOSイメージセンサを搭載したディジタルカメラである。
実験結果を下記表2に示す。実験参加者は3名(男子1名、女子2名)である。計算は、左目の水平方向のαの値の探索範囲を、-7°から0°、右目の水平方向のαの値の探索範囲は、0°から7°、垂直方向のβの値の探索範囲は、左右ともに-2°から2°とした。
光軸の向きのみからは、ガラス上に止まっている虫を見ている場合は判別できない。車など、ある程度速度が出ている場合、長時間近くを見ることは考えられない。車速を取得し、一定の車速以上で、どれだけの頻度で輻輳角が大きくなるのかという観点から判別が可能であろう。
実施例2の視線計測装置では、遠方の2点を眺める際の状態を捉えて、ユーザの左右両眼の視軸(視線)が平行であることを拘束条件とし、遠方の少なくとも2点を眺める際の左右両眼の眼球の光軸を算出し、左右両眼の光軸と視軸との間のずれを算出できることを説明する。
実施例1の視線計測装置とは異なり、遠方の2点を眺めることにより自動キャリブレーションを行う場合、左右両眼の視軸のそれぞれの単位ベクトルが等しいと制約を設ける。
すなわち、下記数式(10)が成立する。ここで、cLとcRはそれぞれ左目と右目の眼球の視軸の単位方向ベクトルであり、αL,βL,αR,βRは眼球の光軸と視軸のずれを表現するものであり、それぞれ左目の水平方向、垂直方向、右目の水平方向、垂直方向のずれを表す。
cL0(αL,βL)=cR0(αR,βR)
cL1(αL,βL)=cR1(αR,βR) ・・・(10)
10 光源手段
12 カメラ手段
14 光軸算出手段
16 ずれ算出手段
18 注視点算出手段
20 被験者
21 ガラス
22 カメラ
23 赤外線LED
24 1点キャリブレーション用のマーカ
100 視線計測装置
Claims (14)
- 主に遠方を眺めるユーザの視線計測装置であって、
遠方の少なくとも2点を眺める際の左右両眼の眼球の光軸を算出する光軸算出手段と、
遠方注視の際、左右両眼の視軸(視線)が平行であることを拘束条件として、左右両眼の光軸と視軸との間のずれを算出するずれ算出手段と、
光軸と視軸との間のずれに基づき、前記ユーザの注視点を算出する注視点算出手段と、を備えたことを特徴とする視線計測装置。 - 前記拘束条件は、左右両眼の視軸のそれぞれの単位ベクトルが等しいことを特徴とする請求項1に記載の視線計測装置。
- 前記光軸算出手段は、遠方の少なくとも4点を眺める際の左右両眼の眼球の光軸を算出し、
前記拘束条件は、左右両眼の視軸のそれぞれのベクトルの外積の大きさが0、もしくは、左右両眼の視軸のそれぞれの単位ベクトルの内積が1であることを特徴とする請求項1に記載の視線計測装置。 - 前記拘束条件は、左右両眼の視軸のそれぞれの単位ベクトルの内積が1である場合、遠方の少なくとも4点を眺める際の左右両眼の眼球の光軸に対して、下記数式を満たすことを特徴とする請求項3に記載の視線計測装置。
(数1)
cLn(αL,βL)・cRn(αR,βR)=1
但し、cLn(αL,βL)、cRn(αR,βR)は、それぞれ左右眼球の視軸の単位ベクトルであり、光軸からの水平方向のずれ角αと垂直方向のずれ角βの関数として表現されたもの。但し、Lは左、Rは右、nは0,1,・・・,N(Nは3以上)。 - 前記拘束条件は、左右両眼の視軸のそれぞれのベクトルの外積の大きさが0である場合、遠方の少なくとも4点を眺める際の左右両眼の眼球の光軸に対して、下記数式を満たすことを特徴とする請求項3に記載の視線計測装置。
(数2)
||cLn(αL,βL)×cRn(αR,βR)||=0
但し、cLn(αL,βL)、cRn(αR,βR)は、それぞれ左右眼球の視軸のベクトルであり、光軸からの水平方向のずれ角αと垂直方向のずれ角βの関数として表現されたもの。但し、Lは左、Rは右、nは0,1,・・・,N(Nは3以上)。 - 前記光軸算出手段は、
遠方を眺める際のユーザについて、眼球の画像である眼球画像を取得し、
前記眼球画像から、角膜の曲率中心もしくは眼球の回転中心と、瞳孔の瞳孔中心とを結ぶ軸である光軸を算出することを特徴とする請求項1~5のいずれかに記載の視線計測装置。 - 前記眼球画像が、所定の光源からの光が反射した眼球の画像であることを特徴とする請求項6に記載の視線計測装置。
- 前記ユーザの前方に
左右両眼の眼球画像を取得し得るように各々異なる位置に配置される少なくとも2個のカメラ手段と、
眼球における光源の反射像が互いに分離したものとなるように各々異なる位置に配置される少なくとも2個の光源手段と、
が設けられ、
左右両眼の眼球の角膜の曲率半径一定領域に少なくとも2個のプルキニエ像を形成させ、何れかのカメラ手段により撮像された眼球画像上における光源手段のプルキニエ像の位置と、実際の光源手段の位置とを対応付け、対応付けられた少なくとも2組のカメラ手段と光源手段の位置関係から眼球の光軸を算出する、
ことを特徴とする請求項6又は7に記載の視線計測装置。 - 前記ずれ算出手段において、
左右両眼の光軸に対する視軸のずれ角の探索範囲を、水平方向で±7°以下、垂直方向で±2°以下とする制約を設けることにより、ずれ角を収束させることを特徴とする請求項1~8のいずれかに記載の視線計測装置。 - 前記ユーザは、電車、自動車、船舶を含む移動手段の操縦時の操縦者であることを特徴とする請求項1~9のいずれかに記載の視線計測装置。
- 主に遠方を眺めるユーザの視線計測方法であって、
遠方の少なくとも2点を眺める際の左右両眼の眼球の光軸を算出する光軸算出ステップと、
遠方注視の際、左右両眼の視軸(視線)が平行であることを拘束条件として、左右両眼の光軸と視軸との間のずれを算出するずれ算出ステップと、
光軸と視軸との間のずれに基づき、前記ユーザの注視点を算出する注視点算出ステップと、を備えたことを特徴とする視線計測方法。 - 前記光軸算出ステップは、遠方の少なくとも4点を眺める際の左右両眼の眼球の光軸を算出し、
前記拘束条件は、左右両眼の視軸のそれぞれのベクトルの外積の大きさが0、もしくは、左右両眼の視軸のそれぞれの単位ベクトルの内積が1である請求項11に記載の視線計測方法。 - 主に遠方を眺めるユーザの視線計測プログラムであって、
コンピュータに、
遠方の少なくとも2点を眺める際の左右両眼の眼球の光軸を算出する光軸算出ステップと、
遠方注視の際、左右両眼の視軸(視線)が平行であることを拘束条件として、左右両眼の光軸と視軸との間のずれを算出するずれ算出ステップと、
光軸と視軸との間のずれに基づき、前記ユーザの注視点を算出する注視点算出ステップと、
を実行させる視線計測プログラム。 - 前記光軸算出ステップは、遠方の少なくとも4点を眺める際の左右両眼の眼球の光軸を算出し、
前記拘束条件は、左右両眼の視軸のそれぞれのベクトルの外積の大きさが0、もしくは、左右両眼の視軸のそれぞれの単位ベクトルの内積が1である請求項13に記載の視線計測プログラム。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/891,578 US10379609B2 (en) | 2013-05-22 | 2014-05-22 | Line-of-sight measurement device, line-of-sight measurement method and line-of-sight measurement program |
JP2015518087A JP6265348B2 (ja) | 2013-05-22 | 2014-05-22 | 視線計測装置、視線計測方法および視線計測プログラム |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013-108441 | 2013-05-22 | ||
JP2013108441 | 2013-05-22 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014188727A1 true WO2014188727A1 (ja) | 2014-11-27 |
Family
ID=51933294
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/002707 WO2014188727A1 (ja) | 2013-05-22 | 2014-05-22 | 視線計測装置、視線計測方法および視線計測プログラム |
Country Status (3)
Country | Link |
---|---|
US (1) | US10379609B2 (ja) |
JP (1) | JP6265348B2 (ja) |
WO (1) | WO2014188727A1 (ja) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017130514A1 (ja) * | 2016-01-27 | 2017-08-03 | ソニー株式会社 | 情報処理装置、情報処理方法、およびプログラムを記録したコンピュータ読み取り可能な記録媒体 |
JP2018026120A (ja) * | 2016-07-27 | 2018-02-15 | フォーブ インコーポレーテッド | 視線検出システム、ずれ検出方法、ずれ検出プログラム |
JP2020181281A (ja) * | 2019-04-24 | 2020-11-05 | 株式会社デンソーアイティーラボラトリ | 視線方向推定装置、視線方向推定装置の較正方法、およびプログラム |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6227996B2 (ja) * | 2013-12-18 | 2017-11-08 | 浜松ホトニクス株式会社 | 計測装置及び計測方法 |
US10682038B1 (en) | 2014-09-19 | 2020-06-16 | Colorado School Of Mines | Autonomous robotic laparoscope based on eye tracking |
JP6459421B2 (ja) * | 2014-11-17 | 2019-01-30 | セイコーエプソン株式会社 | 頭部装着型表示装置、頭部装着型表示装置を制御する方法、コンピュータープログラム |
DE102015204283A1 (de) * | 2015-03-10 | 2016-09-15 | Robert Bosch Gmbh | Verfahren und Vorrichtung zur Erfassung der Blickrichtung einer Person |
KR101745140B1 (ko) * | 2015-09-21 | 2017-06-08 | 현대자동차주식회사 | 시선 추적 장치 및 방법 |
EP3242228A1 (en) * | 2016-05-02 | 2017-11-08 | Artag SARL | Managing the display of assets in augmented reality mode |
EP3478557A4 (en) * | 2016-07-01 | 2020-03-11 | Eyesight Mobile Technologies Ltd. | DRIVER MONITORING SYSTEM AND METHOD |
JP6963820B2 (ja) * | 2016-08-12 | 2021-11-10 | 国立大学法人静岡大学 | 視線検出装置 |
EP3510562A1 (en) | 2016-09-07 | 2019-07-17 | Starship Technologies OÜ | Method and system for calibrating multiple cameras |
JP6776970B2 (ja) * | 2017-03-24 | 2020-10-28 | 株式会社Jvcケンウッド | 視線検出装置、視線検出方法及び視線検出プログラム |
JP2019021049A (ja) * | 2017-07-18 | 2019-02-07 | ソニー株式会社 | 情報処理装置、情報処理方法、及びプログラム |
TWI704473B (zh) | 2018-11-16 | 2020-09-11 | 財團法人工業技術研究院 | 視線向量偵測方向與裝置 |
TWI699671B (zh) * | 2018-12-12 | 2020-07-21 | 國立臺灣大學 | 減低眼球追蹤運算的方法和其眼動追蹤裝置 |
CN112561780B (zh) * | 2020-12-02 | 2022-04-15 | 武汉大学 | 一种附加多视线特征约束的城市场景网格模型优化方法 |
CN116027910B (zh) * | 2023-03-29 | 2023-07-04 | 广州视景医疗软件有限公司 | 一种基于vr眼动追踪技术的眼位图生成方法及系统 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007136000A (ja) * | 2005-11-21 | 2007-06-07 | Nippon Telegr & Teleph Corp <Ntt> | 視線検出装置、視線検出方法、および視線検出プログラム |
WO2012020760A1 (ja) * | 2010-08-09 | 2012-02-16 | 国立大学法人静岡大学 | 注視点検出方法及び注視点検出装置 |
WO2012077713A1 (ja) * | 2010-12-08 | 2012-06-14 | 国立大学法人静岡大学 | 注視点検出方法及び注視点検出装置 |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7533989B2 (en) * | 2003-12-25 | 2009-05-19 | National University Corporation Shizuoka University | Sight-line detection method and device, and three-dimensional view-point measurement device |
US8730266B2 (en) | 2008-11-13 | 2014-05-20 | Queen's University At Kingston | System and method for integrating gaze tracking with virtual reality or augmented reality |
US8553936B2 (en) | 2009-03-03 | 2013-10-08 | The Ohio State University | Gaze tracking measurement and training system and method |
ES2880475T3 (es) | 2009-04-01 | 2021-11-24 | Tobii Ab | Sistema de representación visual con iluminadores para el seguimiento de la mirada |
CN103402440B (zh) * | 2010-12-06 | 2016-03-02 | 国立大学法人冈山大学 | 用于确认是否出现痴呆症症状的装置 |
US8493390B2 (en) | 2010-12-08 | 2013-07-23 | Sony Computer Entertainment America, Inc. | Adaptive displays using gaze tracking |
US8510166B2 (en) | 2011-05-11 | 2013-08-13 | Google Inc. | Gaze tracking system |
US8885877B2 (en) | 2011-05-20 | 2014-11-11 | Eyefluence, Inc. | Systems and methods for identifying gaze tracking scene reference locations |
US8929589B2 (en) | 2011-11-07 | 2015-01-06 | Eyefluence, Inc. | Systems and methods for high-resolution gaze tracking |
US8710986B2 (en) | 2012-01-19 | 2014-04-29 | Utechzone Co., Ltd. | Gaze tracking password input method and device utilizing the same |
JP2014064784A (ja) * | 2012-09-26 | 2014-04-17 | Renesas Microsystem:Kk | 視線検出装置、視線検出方法及びプログラム |
KR101438948B1 (ko) | 2012-12-12 | 2014-09-11 | 현대자동차주식회사 | 시선 추적 제어 장치 및 방법 |
US9179833B2 (en) | 2013-02-28 | 2015-11-10 | Carl Zeiss Meditec, Inc. | Systems and methods for improved ease and accuracy of gaze tracking |
-
2014
- 2014-05-22 WO PCT/JP2014/002707 patent/WO2014188727A1/ja active Application Filing
- 2014-05-22 US US14/891,578 patent/US10379609B2/en not_active Expired - Fee Related
- 2014-05-22 JP JP2015518087A patent/JP6265348B2/ja active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007136000A (ja) * | 2005-11-21 | 2007-06-07 | Nippon Telegr & Teleph Corp <Ntt> | 視線検出装置、視線検出方法、および視線検出プログラム |
WO2012020760A1 (ja) * | 2010-08-09 | 2012-02-16 | 国立大学法人静岡大学 | 注視点検出方法及び注視点検出装置 |
WO2012077713A1 (ja) * | 2010-12-08 | 2012-06-14 | 国立大学法人静岡大学 | 注視点検出方法及び注視点検出装置 |
Non-Patent Citations (1)
Title |
---|
TAKASHI NAGAMATSU ET AL.: "Automatic user- calibration method for gaze tracking system by looking into the distance", IEICE TECHNICAL REPORT, vol. 113, no. 73, 23 May 2013 (2013-05-23), pages 65 - 68 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017130514A1 (ja) * | 2016-01-27 | 2017-08-03 | ソニー株式会社 | 情報処理装置、情報処理方法、およびプログラムを記録したコンピュータ読み取り可能な記録媒体 |
US10606351B2 (en) | 2016-01-27 | 2020-03-31 | Sony Corporation | Information processing apparatus, information processing method, and computer readable recording medium |
JP2018026120A (ja) * | 2016-07-27 | 2018-02-15 | フォーブ インコーポレーテッド | 視線検出システム、ずれ検出方法、ずれ検出プログラム |
JP2020181281A (ja) * | 2019-04-24 | 2020-11-05 | 株式会社デンソーアイティーラボラトリ | 視線方向推定装置、視線方向推定装置の較正方法、およびプログラム |
Also Published As
Publication number | Publication date |
---|---|
US10379609B2 (en) | 2019-08-13 |
US20160086338A1 (en) | 2016-03-24 |
JP6265348B2 (ja) | 2018-01-24 |
JPWO2014188727A1 (ja) | 2017-02-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6265348B2 (ja) | 視線計測装置、視線計測方法および視線計測プログラム | |
JP6340503B2 (ja) | 目追跡システム及び利き目を検出する方法 | |
CN110913751B (zh) | 具有滑动检测和校正功能的可穿戴眼睛跟踪系统 | |
CN109008944B (zh) | 视线计测装置、rom及视线计测方法 | |
JP6083761B2 (ja) | 瞳孔検出方法、角膜反射検出方法、顔姿勢検出方法及び瞳孔追尾方法 | |
US10902635B2 (en) | Line-of-sight detection device | |
JP6014931B2 (ja) | 視線計測方法 | |
WO2015190204A1 (ja) | 瞳孔検出システム、視線検出システム、瞳孔検出方法、および瞳孔検出プログラム | |
US20180133593A1 (en) | Algorithm for identifying three-dimensional point-of-gaze | |
JP2013024662A (ja) | 3次元範囲計測システム、3次元範囲計測プログラムおよび記録媒体 | |
JP7168953B2 (ja) | 自動キャリブレーションを行う視線計測装置、視線計測方法および視線計測プログラム | |
JP6948688B2 (ja) | 視線計測装置、視線計測方法および視線計測プログラム | |
JP2018099174A (ja) | 瞳孔検出装置及び瞳孔検出方法 | |
JP6452235B2 (ja) | 顔検出方法、顔検出装置、及び顔検出プログラム | |
CN106709398A (zh) | 凝视分析方法与装置 | |
EP3542308B1 (en) | Method and device for eye metric acquisition | |
Liu et al. | 3D model-based gaze tracking via iris features with a single camera and a single light source | |
JP2018101212A (ja) | 車載器および顔正面度算出方法 | |
CN106708249B (zh) | 交互方法、交互装置及用户设备 | |
JP2012029940A (ja) | 視線計測方法及び視線計測装置 | |
US20180089508A1 (en) | Visual line measuring device and visual line measuring method | |
JP7269617B2 (ja) | 顔画像処理装置、画像観察システム、及び瞳孔検出システム | |
Santini et al. | Depth perception in an anthropomorphic robot that replicates human eye movements | |
Kwon et al. | Selective attentional point-tracking through a head-mounted stereo gaze tracker based on trinocular epipolar geometry | |
JP2015123262A (ja) | 角膜表面反射画像を利用した視線計測方法及びその装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14801427 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2015518087 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14891578 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14801427 Country of ref document: EP Kind code of ref document: A1 |