CN116382473A - Sight calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction - Google Patents
Sight calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction Download PDFInfo
- Publication number
- CN116382473A CN116382473A CN202310224100.1A CN202310224100A CN116382473A CN 116382473 A CN116382473 A CN 116382473A CN 202310224100 A CN202310224100 A CN 202310224100A CN 116382473 A CN116382473 A CN 116382473A
- Authority
- CN
- China
- Prior art keywords
- screen
- point
- calibration
- scene image
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 55
- 238000012300 Sequence Analysis Methods 0.000 title claims abstract description 28
- 210000001508 eye Anatomy 0.000 claims abstract description 111
- 238000013507 mapping Methods 0.000 claims abstract description 34
- 238000005516 engineering process Methods 0.000 claims abstract description 16
- 230000004424 eye movement Effects 0.000 claims abstract description 16
- 210000005252 bulbus oculi Anatomy 0.000 claims abstract description 9
- 238000001514 detection method Methods 0.000 claims abstract description 8
- 238000011002 quantification Methods 0.000 claims abstract description 3
- 238000000034 method Methods 0.000 claims description 50
- 239000013598 vector Substances 0.000 claims description 24
- 230000003044 adaptive effect Effects 0.000 claims description 15
- 230000004397 blinking Effects 0.000 claims description 14
- 230000009466 transformation Effects 0.000 claims description 13
- 238000004458 analytical method Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 230000004438 eyesight Effects 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 4
- 235000002566 Capsicum Nutrition 0.000 claims description 3
- 239000006002 Pepper Substances 0.000 claims description 3
- 235000016761 Piper aduncum Nutrition 0.000 claims description 3
- 235000017804 Piper guineense Nutrition 0.000 claims description 3
- 235000008184 Piper nigrum Nutrition 0.000 claims description 3
- 238000013461 design Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 150000003839 salts Chemical class 0.000 claims description 3
- 230000004075 alteration Effects 0.000 claims description 2
- 244000203593 Piper nigrum Species 0.000 claims 1
- 238000012731 temporal analysis Methods 0.000 claims 1
- 238000000700 time series analysis Methods 0.000 claims 1
- 230000004886 head movement Effects 0.000 description 8
- 210000003128 head Anatomy 0.000 description 7
- 238000009499 grossing Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 238000000691 measurement method Methods 0.000 description 3
- 230000009897 systematic effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 241000722363 Piper Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000004418 eye rotation Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 1
- 210000004087 cornea Anatomy 0.000 description 1
- 230000004399 eye closure Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 238000001028 reflection method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G06T5/80—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/193—Preprocessing; Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
A sight line calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction specifically comprises the following steps: identifying color blocks and calibration point positions in a screen in a scene image by an image color identification technology, and roughly positioning the screen position by combining the actual size of the screen; performing distortion calibration based on the calibration point information to realize the position of the fine positioning screen and determining the mapping relation between the scene image and the fine positioning screen; performing eye control type nine-point calibration, determining a mapping model of eyeball physical information and the staring space position of a tester, and combining the mapping relation to obtain screen staring point coordinates; performing a sight line positioning accuracy quantification test; optimizing current sight line positioning and predicting future gaze point positions by a self-adaptive time sequence analysis prediction algorithm in combination with historical sight line information, and self-adaptively adjusting parameters in the time sequence analysis prediction algorithm according to real-time precision or jitter intensity; and detecting that the eyes are in the open-close state by a human eye state detection algorithm, and performing eye movement control.
Description
Technical Field
The invention relates to a sight line calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction, wherein the existing pupil-cornea sight line positioning algorithm only processes a single frame image, and the sight line positioning precision of the single frame image is not ideal due to the influence of factors such as algorithm identification errors, system random errors and the like, so that the sight line tracking in a video stream is extremely unstable, and the phenomenon of serious jittering of the position of a fixation point occurs. Aiming at the problem, the invention realizes automatic calibration through eye-control nine-point calibration, adopts a self-adaptive time sequence analysis and prediction method to ensure stable tracking of the motion sight, and simultaneously provides a quantized precision test method.
Background
In recent years, line-of-sight positioning has received a great deal of attention as a research hotspot, and line-of-sight positioning is applicable to a plurality of fields such as medical treatment, education, entertainment, and military.
Hutchinson (reference: hutchinson. Human-computer interaction using eye-size input. IEEE Trans. On System, man and Cybernetics,1989, 19 (6): 1527-1533) calculates the gaze point direction from the vector formed by the center of the spot and the center of the pupil obtained by infrared light irradiation of the eyeball, but is only applicable to a scene where the tester's head is stationary. Morimoto et al (ref: morimoto et al Keeping an eye for HCI, in: proc.on Computer Graphics and Image Processing, 1999.171-176) shows the relationship between the glint-pupil vector and the display gaze point position by a quadratic polynomial, and the user's sitting position is stable with good results, but the accuracy is poor in the case of head movements. Zhang Dengyin (reference: zhang Dengyin, a line-of-sight tracking and positioning method based on iris recognition, chinese patent application No. CN103136519 a) calculates the position of attention of the tester's line of sight from the current frame of human eye image acquired by the camera, without considering the influence of head movement under continuous eye movement recognition.
The method is mainly used for researching how to accurately acquire the gaze point position in a single frame image, but in engineering application, systematic errors of single frame gaze positioning can lead to randomness of gaze point position deviation, and when the head continuously moves to gaze at a target during long-time use, the gaze point position identified by a gaze positioning algorithm can shake randomly near the target, so that the effects of gaze positioning and eye movement control are seriously affected.
Different from the method, when the glasses type eye tracker is used for controlling the eye movement of the screen, the method can realize real-time screen position locking under indoor and outdoor multiple scenes through color positioning, and map the fixation point coordinates into screen coordinates; secondly, after model mapping is completed through nine-point calibration, precision testing is carried out, and real-time precision of line-of-sight positioning is quantitatively calculated; optimizing the current sight line positioning and predicting the future gaze point position by combining the historical sight line information through the self-adaptive time sequence analysis prediction algorithm, and self-adaptively adjusting parameters in the time sequence analysis prediction algorithm according to the real-time precision or the shaking intensity; and finally, acquiring eye control information through eye state identification, and realizing an eye movement control effect.
Disclosure of Invention
According to the invention, automatic calibration is realized through eye-controlled nine-point calibration, a stable tracking target of the fixation point in head movement is realized based on a self-adaptive time sequence analysis prediction algorithm, no equipment is required to be additionally arranged for automatically and quantitatively acquiring the sight line positioning precision in real time, the problems that the fixation point randomly shakes and the sight line positioning precision is difficult to quantitatively calculate in real time during head movement fixation are mainly solved, and the designed scheme has the characteristics of instantaneity, stability and reliability.
The technical scheme of the invention is as follows:
a sight line calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction comprises the following steps:
step 1: coarsely positioning the screen position: identifying color blocks and calibration point positions in a screen in a scene image by an image color identification technology, and roughly positioning the screen position by combining the actual size of the screen;
step 2: fine positioning of screen positions: performing distortion calibration based on the calibration point information to realize the position of the fine positioning screen and determining the mapping relation between the scene image and the fine positioning screen;
step 3: performing eye control type nine-point calibration, determining a mapping model of eyeball physical information and the staring space position of a tester, and combining the mapping relation of the step 2 to obtain screen staring point coordinates;
step 4: performing sight line positioning accuracy quantification test, and returning to the step 3 if the accuracy is not satisfied;
step 5: optimizing current sight line positioning and predicting future gaze point positions by a self-adaptive time sequence analysis prediction algorithm in combination with historical sight line information, and self-adaptively adjusting parameters in the time sequence analysis prediction algorithm according to real-time precision or jitter intensity;
step 6: and detecting that the eyes are in the open-close state by a human eye state detection algorithm, and performing eye movement control.
In step 1, the screen position is coarsely positioned, in order to adapt to a plurality of application scenes such as indoor and outdoor, bright field and dark field, the positions of color blocks and calibration points in the screen are identified in the scene image through an image color identification technology, the screen position is coarsely positioned by combining the actual size of the screen, and the positions of nine calibration points with special colors are positioned simultaneously;
in step 2, the screen position fine positioning mainly comprises 2 parts, firstly, the screen position fine positioning technology based on screen distortion calibration calculates distortion through nine calibration points which are arranged at equal intervals, and the accurate screen position is updated. And secondly, calculating the mapping relation between the scene image and the fine positioning screen through homography.
In step 3, the subject needs to observe the calibration points appearing at specific positions on the screen according to the system instructions. The coordinates of the calibration points on the screen coordinate system are known and fixed, and the coordinate positions of the calibration points on the screen image coordinate system can be obtained by combining the scene image-screen coordinate transformation relation obtained in the step 2, and the mapping relation between the pupil-cornea reflection vectors and the scene image coordinate system is obtained by mapping the pupil-cornea reflection vectors corresponding to the calibration points and the scene image calibration point coordinates. In the following actual fixation process, firstly, pupil-cornea reflection vectors corresponding to the fixation point are calculated, then, the coordinates of the fixation point on the scene image are calculated through the obtained mapping relation between the pupil-cornea reflection vectors and the scene image, and finally, the position of the fixation point on the display screen is obtained through the foreground image-display screen coordinate transformation relation.
In step 4, quantitatively measuring the sight line positioning precision, firstly generating a precision test picture which can be a test point or a test line; then sequentially gazing at the test pictures, acquiring coordinates of a gazing point in a screen by system software in the test process, and calculating the vertical distance between eyes and the screen only by a similar triangle principle of imaging characteristics without adding any other equipment; finally, the line of sight positioning accuracy is calculated in a statistics mode, and then the deviation angle of line of sight positioning is calculated by combining the distance between the target test point coordinates and the gaze point coordinates, and the root mean square error of the deviation angle is calculated as an actual error.
In step 5, the gaze point coordinate anti-shake mechanism based on the adaptive timing analysis prediction algorithm performs filtering by multiplying the historical gaze point coordinate by a weight, so that the current gaze point coordinate is smooth, and predicts the gaze point coordinate at the next moment. The self-adaptation is embodied in that the weight is automatically adjusted, the gaze point coordinates of the previous m time are stored in a queue mode, whether a target button exists is judged through the gaze point coordinates, if yes, the weight is changed through the error precision calculated in the step 4, and if no target button exists, the weight is changed through the intensity of gaze point shake.
In an eye movement control software system, the adaptive time sequence analysis prediction algorithm is mainly applied to the following two aspects: firstly, a pupil-cornea reflection sight line positioning algorithm is used for obtaining a fixation point coordinate under a scene image coordinate system; secondly, the gaze point coordinates in the screen coordinate system converted from the scene-screen coordinate system in the step 3 are obtained.
In step 6, the human eye state is detected, firstly, edge extraction is carried out, then, edge curve fitting is carried out to obtain a user eye contour curve, and finally, eye opening, eye blinking, eye closing and eye blinking ending states are selected according to the eye pixel area, so that the eye closing time length and the eye blinking times are calculated, and the design of subsequent eye movement control instructions is facilitated.
The invention has the beneficial effects that:
1. the invention provides an eye control type nine-point calibration method, which does not need extra personnel to assist a mouse to click a calibration point, and realizes eye movement calibration in a full-automatic and intelligent way;
2. the method is stable in real time, and can realize rapid and stable gaze point tracking and prediction during continuous head movement;
3. the invention provides a convenient and quick visual line positioning accuracy quantitative measurement method, which can detect the visual line positioning accuracy in real time without adding equipment and is beneficial to the optimization of a visual line positioning algorithm.
Drawings
The present invention is further described below with reference to the accompanying drawings for the convenience of understanding by those skilled in the art.
FIG. 1 is a flow chart of a line-of-sight calibration, tracking and testing method based on adaptive timing analysis prediction
Fig. 2 is an eye-controlled nine-point calibration screen
FIG. 3 is a schematic diagram of a distortion screen in a scene image
FIG. 4 is a schematic illustration of nine-point calibration and eye rotation
FIG. 5 is a diagram of a quantized line-of-sight positioning accuracy test point
FIG. 6 is a diagram of a quantized line-of-sight positioning accuracy test line
FIG. 7 is a diagram illustrating a method for calculating the accuracy of the positioning of a quantized line of sight
FIG. 8 is a schematic diagram of a blink sequence for the human eye
FIG. 9 is a schematic diagram of human eye state detection edge extraction
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the application implementation.
A flow chart of a vision calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction is shown in figure 1. Considering that the eyeglass type eye tracker is worn on the head of a user and moves along with the head, and the accuracy of the eyeglass type eye tracker is generally higher than that of a desktop type eye tracker, the invention is described by adopting the eyeglass type eye tracker. The glasses type eye tracker is generally equipped with 3 high-definition cameras, 1 scene camera obtains a picture of a user's field of view, and 2 eyeball cameras are responsible for obtaining left and right eye information.
First, the eye-glasses type eye-tracker device software can directly output the gaze point coordinates under the scene graph, but needs to be converted into screen coordinates to control the screen. Secondly, the conventional eye movement calibration requires additional personnel to assist in carrying out mouse click on the target calibration point position; the current sight line positioning accuracy measurement operation is complex and difficult to quantify, and a method for conveniently and quantitatively acquiring the sight line positioning accuracy in real time without adding equipment is lacked. Finally, the pupil-cornea reflection line of sight positioning algorithm of the current mainstream only aims at the point of regard at the current moment, and when the head moves, the point of regard shakes near the target when the target is stared due to systematic errors and random errors of the algorithm, so that the target is difficult to lock stably. In order to overcome the problems, a method of combining image recognition and time sequence analysis is adopted, a scene image color block is positioned on a screen, then automatic calibration is carried out through an eye control type nine-point calibration method, the sight line positioning precision is measured quantitatively, and finally the current gaze point position is optimized and the future gaze point position is predicted through a self-adaptive time sequence analysis prediction algorithm.
A sight line calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction comprises the following steps:
step 1: coarse positioning of the screen position, namely identifying color blocks and calibration point positions in the screen in the scene image through an image color identification technology, and coarse positioning the screen position by combining the actual size of the screen;
step 2: fine positioning of the screen position, performing distortion calibration based on the calibration point information, realizing the fine positioning of the screen position, and determining the mapping relation between the scene image and the fine positioning screen;
step 3: eye control type nine-point calibration is carried out, a mapping model of eyeball physical information and the staring space position of a tester is determined, and screen staring point coordinates are obtained by combining the information of the step 2;
step 4: quantitatively measuring the sight line positioning precision, and returning to the step 3 if the precision is not satisfied;
step 5: the anti-shake sight line positioning based on the self-adaptive time sequence analysis prediction algorithm optimizes the current sight line positioning and predicts the future gaze point position by combining the historical sight line information, and simultaneously, the parameters in the time sequence analysis prediction algorithm are self-adaptively adjusted according to the real-time precision or the shaking intensity, so that the gaze point position can be stably obtained when a target is focused in the head movement, and the shake is effectively prevented;
step 6: and detecting that the eyes are in the open-close state by a human eye state detection algorithm, and performing eye movement control.
The screen position coarse positioning technology based on image color recognition in the step 1 is as follows:
in order to adapt to a plurality of application scenes such as indoor and outdoor, bright field and dark field, a screen position coarse positioning technology based on image color recognition mainly comprises the following steps:
1) The hsv change range of the color block is selected, and chromatic aberration caused by light and dark change in a use scene is considered;
2) Converting the scene image into an hsv format, and extracting a designated color image through an hsv range;
3) Binarizing the image and removing salt and pepper noise;
4) Calculating the center position of the color block according to the binarized image;
5) And calculating pixel coordinates of four vertexes of the screen in the scene image by combining the actual size of the screen, and realizing coarse positioning.
Specifically, in step 1), a color block with a special color is selected to be placed at the upper left corner of the screen during design, and can also be an icon or logo. The hsv variation range of a special color is selected in consideration of the light and dark variation of the scene.
Specifically, in step 3), binarizing the image, removing salt and pepper noise through gaussian filtering or morphological filtering, and using a kernel function morphyodex function.
Specifically, in step 4), the pixel area s and the center position of the color block are calculated from the binarized image, and the cv2. Motion () function can acquire the image distance, and then calculate the centroid from the image distance.
Specifically, in step 5), the actual area S of the color block is combined with the actual area M of the screen, the pixel area of the screen is m=s×m/S, contour extraction is performed on the scene image, and the corner points near the pixel area of the screen can be regarded as the screen vertices, so that the pixel coordinates of the four vertices of the screen in the scene image can be obtained approximately, and coarse positioning is realized.
The position location of the nine calibration points is also a specific location in the scene image acquired by special color recognition.
The step 2 comprises 2 parts, namely a screen position fine positioning technology based on screen distortion calibration, and a mapping relation between a scene image and a fine positioning screen is calculated.
Screen position fine positioning technology based on screen distortion calibration:
the screen fine positioning step is completed under an eye-controlled nine-point calibration interface, as shown in fig. 2. Although the system is used in a manner that requires the user's head to be vertically facing the screen, it is not possible to be vertically facing the screen entirely due to the user's head movement, and therefore the screen must be distorted somewhat in the scene image, not a perfectly square rectangle. Screen distortion calibration is to calculate the distortion of the screen in the scene image by nine points arranged at equal intervals. The position location of the nine calibration points is also to acquire the specific positions of the nine calibration points in the scene image by adopting special color recognition, and then to calculate the distance between the subsequent calibration points. As shown in fig. 3, the actual distance L between the calibration point 1 and the calibration point 3 1 The pixel distance in the scene image is l 1 The actual size of the screen is M x N mm,the screen is l in length over pixels in the scene image 1 *M/L 1 . Similarly, the actual distance L between the calibration point 1 and the calibration point 7 2 The pixel distance in the scene image is l 2 The left width of the pixel in the scene image is l 2 *N/L 2 The actual distance L between the calibration point 3 and the calibration point 9 3 The pixel distance in the scene image is l 3 The right width of the pixel of the screen in the scene image is l 3 *N/L 3 The upper left vertex of the screen is positioned by the center of the color block, and the other three vertices of the screen in the scene image are positioned by the combination of coarse positioning and fine positioning, so that the position of the screen in the scene image is precisely positioned. When the subsequent screen does not use nine-point calibration images, the positioning of the screen adopts coarse positioning and real-time positioning, and the position of the screen in the scene image is obtained in real time by combining the parameters of fine positioning.
Mapping relation between scene image and fine positioning screen:
when the position of the screen can be precisely located, an image processing algorithm (homography) can be used to determine the coordinate transformation relationship between the screen and the scene image, and through the coordinate transformation relationship, the coordinate of the calibration point can be mutually converted in the screen coordinate system and the scene image coordinate system, and the point of regard coordinate on the scene image can be subsequently converted into the coordinate on the screen coordinate system. Wherein the homography is a two-dimensional projective transformation that maps points in one plane into another plane. The findHomographic () function in the OpenCV library can realize a function of calculating a homography matrix. Thus, a scene coordinate system oxy and a screen coordinate system o are obtained 1 x 1 y 1 The mapping relationship between them is shown in fig. 3.
The eye control type nine-point calibration technology in the step 3 is as follows:
the eye movement instrument software adopts a pupil-cornea reflection method to locate the line of sight, the software SDK interface outputs coordinates containing the pupil center and cornea reflection points, a mapping function between pupil-cornea vectors and scene image fixation points is found out through an eye control type nine-point calibration program, and then the line of sight position of a user in a screen is tracked in real time through detecting the variation of the pupil-cornea vectors. Because specific information of eyeballs of each person is different from a certain degree, the effect of using the same model parameter is poor, and eye control type nine-point calibration is needed to solve the problem. The eye control nine-point calibration model comprises a mapping relation between eye sight projection points and eye rotation angles, as shown in fig. 4.
The conventional nine-point calibration procedure is cumbersome and complicated, requires an additional operator to mouse click on a calibration point in the screen in the scene image, and is prone to introducing systematic errors due to mouse click inaccuracy. According to the eye-control nine-point calibration method provided by the invention, the eye-control nine-point calibration can be realized only through the relation between the scene image coordinate system and the screen coordinate system obtained in the step 2 without adding additional personnel or equipment.
In an eye-controlled nine-point calibration procedure, the subject needs to observe points appearing at specific positions on the screen according to the system instructions, which are called calibration points, typically using 9-point calibration, as shown in fig. 2. For each calibration point, the coordinates of the calibration point on the screen coordinate system are known and fixed, and the coordinate positions of the calibration points on the screen image coordinate system can be obtained by combining the scene image-screen coordinate transformation relation obtained in the step 2, and the mapping relation between the pupil-cornea reflection vector and the scene image coordinate system can be obtained by mapping the pupil-cornea reflection vector corresponding to the calibration points and the scene image calibration point coordinates. In the following actual fixation process, firstly, pupil-cornea reflection vectors corresponding to the fixation point are calculated, then, the coordinates of the fixation point on the scene image are calculated through the obtained mapping relation between the pupil-cornea reflection vectors and the scene image, and finally, the position of the fixation point on the display screen is obtained through the foreground image-display screen coordinate transformation relation.
The eye control nine-point calibration specifically comprises the following steps:
(1) Pupil-cornea vector V (V) can be obtained by processing the eye diagram according to pupil-cornea reflection sight line positioning algorithm x ,V y ) Coordinates P (P xi ,P yi ) I=1, 2, …,9, a mapping model between pupil-cornea vector and scene image is built：
Unknown a 0 ～a 5 And b 0 ～b 5 Is calculated by eye control type nine-point calibration.
(2) The tester looks at nine calibration points in the screen in turn to obtain pupil-cornea vector V i ＝(V xi ,V yi ),i＝1,2,…,9。
(3) Will V i And the gaze point coordinates (under the scene coordinate system) in the screen at each time are substituted into formula (1).
(4) From this, 9 sets of the following equations can be obtained
In order to make the eye control precision high, the minimum value is taken:
for R 2 Obtaining the parameter a by deviator 0 ～a 5 Similarly, let the parameter b 0 ～b 5 。
So far, a mapping model between pupil-cornea vectors and a scene image is obtained, in the subsequent use, the gaze point coordinates of a scene image coordinate system can be obtained according to the model only by carrying out image processing on each frame of eye pattern, and then the gaze point coordinates under the screen coordinate system are obtained through the coordinate transformation in the step 2.
The sight line positioning precision quantitative measurement technology of the step 4 comprises the following steps:
the quantitative measurement of the sight line positioning precision mainly comprises three steps:
1) Generating precision test pictures
2) Test data acquisition
3) Sight positioning accuracy statistical calculation
Step 1), generating test pictures for accuracy verification, wherein the test pictures are divided into two types, namely a definite fixation point and a fixation track, the two types of test pictures can be generated by using codes, and the key points and the real coordinates of the key track generated by the codes are more accurate. As shown in fig. 5, a test picture containing 8 keypoints and coordinates. As shown in FIG. 6, a test picture comprising 1 semicircle and 1 horizontal line segment
Step 2) test data acquisition.
After the system software is used for eye control type nine-point calibration, the test personnel automatically jumps to a sight line positioning precision measurement link, and the system generates test pictures and sequentially looks at key points, key line segments or key curves in the test pictures. The system software collects the coordinates of the gaze point in the screen during the test, and also calculates the vertical distance between the eyes and the screen.
The vertical distance between eyes and a screen is calculated, and a distance measuring device can be installed on the head of a tester by the traditional method, so that the influence of the distance measuring device on the eyes acquired images needs to be avoided, and the distance measuring device needs to be prevented from being arranged in a binocular data acquisition sensor and a scene camera image. When testing with the added distance measuring device, the distance measured by the distance measuring device needs to be ensured to be the distance D between eyes of the tester and the fixation plane. If a laser distance measuring device is used, it is necessary to ensure that the laser returns via the gaze plane. The method is more restricted and has weak practical operability, so a distance measurement method based on imaging characteristics is provided.
The distance measurement method based on imaging characteristics is mainly obtained through a similar triangle principle, the resolution of a screen in a scene image is M x N, the actual size of the screen is M x N mm, the focal length F of a camera is a known quantity, and the distance D=M x F/M of the screen from the scene camera;
and 3) calculating the line-of-sight positioning accuracy statistics. When test data are collected, the system software directly outputs corresponding fixation point coordinates, eye control calibration data recorded by the system software are used as calibration data of the eye control system for eye control calibration, binocular data recorded by the system software are used as input data of the eye control system, and the eye control system outputs the corresponding fixation point coordinates.
As shown in FIG. 7, the dot represents the actual output gaze point position coordinates (x 1, y 1) of the system, the target test point coordinates (x 2, y 2), and thus the actual distance between the gaze point position and the target test point is
The vertical distance D of the human eye from the screen is measured in mm in step 2), so the deviation angle of the line of sight positioning is α=tan -1 (L/D)。
Because the data collected in the test process is sequence data, the root mean square error of the deviation angle needs to be calculated as an actual error when accuracy calculation is carried out, the coordinate sequence of the point of gaze output by system software or an eye control system is recorded as Points, the corresponding deviation angle sequence is P, and the root mean square error of the accuracy can be calculated by the following formula:
the adaptive time sequence analysis and prediction technology in the step 5 is as follows:
the gaze point coordinate anti-shake mechanism based on the adaptive time sequence analysis prediction algorithm performs filtering by multiplying the historical gaze point coordinate by a weight, so that the current gaze point coordinate is smooth, and the gaze point coordinate at the next moment is predicted. The adaptation is embodied in that the change of the weights is automatically adjusted, following the intensity of the historical gaze point shake or the error accuracy of the historical gaze point coordinates and the buttons to be controlled.
There are various methods based on time series prediction, such as a simple translation method, a simple average method, a moving average method, a weighted moving smoothing method, a simple solution smoothing method, and a holter (holt) linear trend method. Taking a weighted movement smoothing method as an example for explanation, the whole eye control software system carries out a double-layer weighted movement smoothing method, firstly, a pupil-cornea reflection sight line positioning algorithm is carried out to obtain the fixation point coordinates under a scene image coordinate system; secondly, the gaze point coordinates in the screen coordinate system converted from the scene-screen coordinate system in the step 3 are obtained.
In using the weighted moving smoothing method, the value of the current moment is predicted by giving different weights to the historical moment gaze point coordinates. The gaze point coordinate (x i ，y i ) The calculation formula is as follows:
x i ＝ω 1 *x i-1 +ω 2 *x i-2 +ω 3 *x i-3 +…+ω m *x i-m
y i ＝ω 1 *y i-1 +ω 2 *y i-2 +ω 3 *y i-3 +…+ω m *y i-m
wherein, (x) i-1 ，y i-1 ) Is the gaze point coordinates of the moment immediately preceding the moment i, and so on (x i-m ，y i-m ) Is the gaze point coordinates, ω, of the m-th moment before the i-th moment 1...m Is the weight, omega, of the corresponding moment 1 +ω 2 +ω 3 +…+ω m =1. The initial value of the weight may be set to ω i ＝0.5 i The adaptation of the weights varies depending on the severity of the historical gaze point shake or the accuracy of the error of the historical gaze point coordinates with the target button. As shown in fig. 1, the gaze point coordinates of the previous m time are stored in a queue, whether a target button exists is determined by the gaze point coordinates, if yes, the weight is changed by the error precision calculated in the step 4, and if no target button exists, the weight is changed by the intensity of gaze point shake, specifically the weight changing method is as follows:
the intensity of the historical gaze point shake is calculated as follows:
wherein d i-1 Is the distance between the point coordinates of the point of gaze at the moment i-1 and the moment i-2, and so on, d i-m Is the distance between the point of regard coordinates at the moment i-m and the moment i-m-1. When all the shaking degrees alpha of the previous m time are obtained, the point of regard coordinates of the time with the maximum shaking degree are removed as wild values, namely the weight is reset to 0, and then the weight is redistributed, so that the weight of other times with small shaking degrees is improved.
The method for calculating the error precision of the historical gaze point coordinates and the target button comprises the following steps:
the method for calculating the error precision between the current gaze point coordinate and the target button is as in step 4, and only the precision verification point in step 4 is needed to be replaced by the target button. When all the precision errors of the previous m time are acquired, the point of regard coordinate of the time with the largest precision error is taken as a wild value to be removed, namely the weight is reset to 0, and then the weight is reassigned, so that the weight of other times with small precision errors is improved.
The human eye state detection technology in the step 6 is as follows:
after the stable fixation point coordinates are obtained, an eye movement control operation is required for the buttons on the screen, and then other information of eyeballs, such as an open state, a closed state, an eye closing time period, and a blink number, is required. But the original data of the information are obtained by detecting the human eye state.
The eyes of the user are photographed using an eyeball camera, and the sequence of blink pictures is a blink process, as shown in fig. 8. The complete blink sequence contains the following states: 1-open eyes, 2-start blinking eyes, 3-close eyes, 4-end blinking eyes. A complete blink may contain a number of identical states in succession, but must be a complete cycle of 1234 four states.
The technical effect of eye state detection is as shown in fig. 9, firstly edge extraction is carried out, and methods such as Canny operator, sobel operator or Robert operator can be adopted, then edge curve fitting is carried out to obtain a user eye contour curve, and finally eye opening, eye blinking starting, eye closing and eye blinking ending are carried out through eye pixel area selection. Taking the far and near changes of human eye pixels caused by head movement into consideration, carrying out normalization processing on the areas of human eye pixel points, and carrying out human eye state value:
wherein z is i The pixel area of human eyes and the size of human eye images are N multiplied by M.
When the human eye state value f mn Exceeding the threshold f of opening eyes 2 When the eyes are opened, the eyes are judged to be in an open eye state; when f mn Below the eye closure threshold f 1 When f, determining that the eyes are in a closed state mn At f 1 To f 2 In the above-mentioned case, it is necessary to determine whether to start blinking or end blinking in combination with the history information. If the last different state of the eyes is the open eye state, the current eye state is to start blinking; if the last different state of the human eye is the eye-closing state, the current human eye state is the eye-closing state ending blinking.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.
Claims (10)
1. A sight line calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction is characterized by comprising the following steps:
step 1: coarsely positioning the screen position: identifying color blocks and calibration point positions in a screen in a scene image by an image color identification technology, and roughly positioning the screen position by combining the actual size of the screen;
step 2: fine positioning of screen positions: performing distortion calibration based on the calibration point information to realize the position of the fine positioning screen and determining the mapping relation between the scene image and the fine positioning screen;
step 3: performing eye control type nine-point calibration, determining a mapping model of eyeball physical information and the staring space position of a tester, and combining the mapping relation of the step 2 to obtain screen staring point coordinates;
step 4: performing sight line positioning accuracy quantification test, and returning to the step 3 if the accuracy is not satisfied;
step 5: optimizing current sight line positioning and predicting future gaze point positions by a self-adaptive time sequence analysis prediction algorithm in combination with historical sight line information, and self-adaptively adjusting parameters in the time sequence analysis prediction algorithm according to real-time precision or jitter intensity;
step 6: and detecting that the eyes are in the open-close state by a human eye state detection algorithm, and performing eye movement control.
2. The method for calibrating line of sight, tracking motion and testing accuracy based on adaptive timing analysis prediction according to claim 1, wherein the step 1 is that
The method comprises the following specific steps:
1) The hsv change range of the color block is selected, and chromatic aberration caused by light and dark change in a use scene is considered;
2) Converting the scene image into an hsv format, and extracting a designated color image through an hsv range;
3) Binarizing the image and removing salt and pepper noise;
4) Calculating the center position of the color block according to the binarized image;
5) And calculating pixel coordinates of four vertexes of the screen in the scene image by combining the actual size of the screen, and realizing coarse positioning.
3. The method for calibrating sight, tracking motion and testing accuracy based on adaptive time sequence analysis and prediction according to claim 1, wherein the step 2 comprises 2 parts, namely a screen position fine positioning technology based on screen distortion calibration, wherein distortion is calculated through nine calibration points which are arranged at equal intervals, and the accurate position of a screen is updated; secondly, calculating the mapping relation between the scene image and the fine positioning screen through homography;
screen position fine positioning technology based on screen distortion calibration:
the screen distortion calibration is to calculate the distortion of the screen in the scene image through nine points which are arranged at equal intervals; the position location of the nine calibration points is to acquire the specific positions of the nine calibration points in the scene image by adopting special color recognition, and then to calculate the distance between the subsequent calibration points; the actual distance L between the calibration point 1 and the calibration point 3 1 The pixel distance in the scene image is l 1 The actual size of the screen is M x N mm, the length of the screen on the pixels in the scene image is l 1 *M/L 1 The method comprises the steps of carrying out a first treatment on the surface of the Similarly, the actual distance L between the calibration point 1 and the calibration point 7 2 The pixel distance in the scene image is l 2 The left width of the pixel in the scene image is l 2 *N/L 2 The actual distance L between the calibration point 3 and the calibration point 9 3 The pixel distance in the scene image is l 3 The right width of the pixel of the screen in the scene image is l 3 *N/L 3 The left top vertex of the screen is positioned by the center of the color block, the other three vertices of the screen in the scene image are positioned by the combination of coarse positioning and fine positioning, and the position of the screen in the scene image is positioned accurately; when the subsequent screen does not use nine-point calibration images, the positioning of the screen adopts coarse positioning and real-time positioning, and the position of the screen in the scene image is obtained in real time by combining the parameters of fine positioning;
mapping relation between scene image and fine positioning screen:
after the position of the screen is precisely positioned, a homography image processing algorithm is used for determining the coordinate transformation relation between the screen and the scene image, and the coordinate of the calibration point is mutually transformed in the screen coordinate system and the scene image coordinate system through the coordinate transformation relation, so as to obtain a scene image coordinate system oxy and a screen coordinate system o 1 x 1 y 1 Mapping relation between the two.
4. The method for calibrating line of sight, tracking motion and testing accuracy based on adaptive timing analysis prediction according to claim 1, wherein in the step 3, the eye-controlled nine-point calibration is performed: the subject needs to observe the calibration points appearing at specific positions on the screen according to the system instructions; the coordinates of the calibration points on the screen coordinate system are known and fixed, and the coordinate positions of the calibration points on the screen image coordinate system can be obtained by combining the scene image-screen coordinate transformation relation obtained in the step 2, and the mapping relation between the pupil-cornea reflection vectors and the scene image coordinate system is obtained by mapping the pupil-cornea reflection vectors corresponding to the calibration points and the scene image calibration point coordinates; in the following actual fixation process, firstly, pupil-cornea reflection vectors corresponding to the fixation point are calculated, then, the coordinates of the fixation point on the scene image are calculated through the obtained mapping relation between the pupil-cornea reflection vectors and the scene image, and finally, the position of the fixation point on the display screen is obtained through the foreground image-display screen coordinate transformation relation.
5. The method for calibrating vision, tracking motion and testing accuracy based on adaptive timing analysis prediction according to claim 4, wherein the eye-controlled nine-point calibration comprises the following steps:
(1) Pupil-cornea vector V (V) can be obtained by processing the eye diagram according to pupil-cornea reflection sight line positioning algorithm x ,V y ) Coordinates P (P xi ,P yi ) I=1, 2, …,9, a mapping model between pupil-cornea vector and scene image is established:
unknown a 0 ～a 5 And b 0 ～b 5 Is obtained by eye control type nine-point calibration calculation;
(2) The tester looks at nine calibration points in the screen in turn to obtain pupil-cornea vector V i ＝(V xi ,V yi ),i＝1,2,…,9；
(3) Will V i And coordinates (V) of the gaze point in the screen at each time in the scene image coordinate system xi ,V yi ) Substituting formula (1);
(4) From this, 9 sets of the following equations can be obtained
In order to make the eye control precision high, the minimum value is taken:
for R 2 Obtaining the parameter a by deviator 0 ～a 5 Similarly, let the parameter b 0 ～b 5 ；
So far, a mapping model between pupil-cornea vectors and a scene image is obtained, in the subsequent use, the gaze point coordinates of a scene image coordinate system can be obtained according to the model only by carrying out image processing on each frame of eye pattern, and then the gaze point coordinates under the screen coordinate system are obtained through the coordinate transformation in the step 2.
6. The method for calibrating line of sight, tracking motion and testing accuracy based on adaptive timing analysis prediction according to claim 1, wherein in the step 4, the line of sight positioning accuracy is quantitatively measured: firstly, generating an accuracy test picture which can be a test point or a test line; then sequentially gazing at the test pictures, acquiring coordinates of a gazing point in a screen by system software in the test process, and calculating the vertical distance between eyes and the screen only by a similar triangle principle of imaging characteristics; finally, the line of sight positioning accuracy is calculated in a statistics mode, and then the deviation angle of line of sight positioning is calculated by combining the distance between the target test point coordinates and the gaze point coordinates, and the root mean square error of the deviation angle is calculated as an actual error.
7. The method for calibrating vision, tracking motion and testing accuracy based on adaptive time series analysis and prediction according to claim 6, wherein the method for calculating the vertical distance between eyes and a screen is characterized by comprising the following steps: the resolution of a screen in a scene image is M x N, the actual size of the screen is M x N mm, and the focal length F of a camera is a known quantity, so that the distance D=M x F/M between the screen and the scene camera;
and (3) line-of-sight positioning accuracy statistics and calculation:
the dot represents the actual output point of gaze position coordinates (x 1, y 1) and the target test point coordinates (x 2, y 2) of the system, so the actual distance between the point of gaze position and the target test point is
Thus the deviation angle of the line of sight positioning is α=tan -1 (L/D)；
When accuracy calculation is performed, the root mean square error of the deviation angle needs to be calculated as an actual error, the coordinate sequence of the point of regard is Points, the corresponding deviation angle sequence is P, and the root mean square error of the accuracy can be calculated by the following formula:
8. the method for calibrating line of sight, tracking motion and testing accuracy based on adaptive timing analysis prediction according to claim 1, wherein in step 5, the gaze point coordinate anti-shake mechanism based on the adaptive timing analysis prediction algorithm performs filtering by multiplying the historical gaze point coordinate by a weight, so that the current gaze point coordinate is smooth, and predicts the gaze point coordinate at the next moment; the self-adaptation is reflected in that the weight is automatically adjusted, the gaze point coordinates of the previous m time are stored in a queue mode, whether a target button exists is judged through the gaze point coordinates, if yes, the weight is changed through the error precision calculated in the step 4, and if no, the weight is changed through the intensity of gaze point shake; taking the point-of-regard coordinate at the moment of the maximum jitter degree or the maximum precision error as a wild value to be removed, namely resetting the weight to 0, and then reassigning the weight to improve the weight of other moments with small jitter degree or small precision error;
in an eye movement control software system, the adaptive time sequence analysis prediction algorithm is mainly applied to the following two aspects: firstly, a pupil-cornea reflection sight line positioning algorithm is used for obtaining a fixation point coordinate under a scene image coordinate system; secondly, the gaze point coordinates in the screen coordinate system converted from the scene-screen coordinate system in the step 3 are obtained.
9. The method for calibrating line of sight, tracking motion and testing accuracy based on adaptive timing analysis prediction according to claim 8, wherein the specific changing weight means is as follows:
the intensity of the historical gaze point shake is calculated as follows:
wherein d i-1 Is the distance between the point coordinates of the point of gaze at the moment i-1 and the moment i-2, and so on, d i-m Is the distance between the point of regard coordinates at the moment i-m and the moment i-m-1.
10. The method for calibrating sight, tracking motion and testing accuracy based on self-adaptive time sequence analysis and prediction according to claim 1, wherein in the step 6, eye state detection is performed, edge extraction is performed first, then edge curve fitting is performed to obtain a user eye contour curve, finally eye opening, eye blinking starting, eye closing and eye blinking ending states are selected through eye pixel area selection, eye closing duration and eye blinking times are calculated, and subsequent design of eye movement control instructions is facilitated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310224100.1A CN116382473A (en) | 2023-03-07 | 2023-03-07 | Sight calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310224100.1A CN116382473A (en) | 2023-03-07 | 2023-03-07 | Sight calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116382473A true CN116382473A (en) | 2023-07-04 |
Family
ID=86974087
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310224100.1A Pending CN116382473A (en) | 2023-03-07 | 2023-03-07 | Sight calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116382473A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116820246A (en) * | 2023-07-06 | 2023-09-29 | 上海仙视电子科技有限公司 | Screen adjustment control method and device with self-adaptive visual angle |
-
2023
- 2023-03-07 CN CN202310224100.1A patent/CN116382473A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116820246A (en) * | 2023-07-06 | 2023-09-29 | 上海仙视电子科技有限公司 | Screen adjustment control method and device with self-adaptive visual angle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108427503B (en) | Human eye tracking method and human eye tracking device | |
| EP2975997B1 (en) | System and method for on-axis eye gaze tracking | |
| US9398848B2 (en) | Eye gaze tracking | |
| KR101533686B1 (en) | Apparatus and method for tracking gaze, recording medium for performing the method | |
| Hosp et al. | RemoteEye: An open-source high-speed remote eye tracker: Implementation insights of a pupil-and glint-detection algorithm for high-speed remote eye tracking | |
| US20220100268A1 (en) | Eye tracking device and a method thereof | |
| KR20140125713A (en) | Apparatus and method of gaze tracking based on camera array | |
| WO2023011339A1 (en) | Line-of-sight direction tracking method and apparatus | |
| CN109782902A (en) | A kind of operation indicating method and glasses | |
| Huang et al. | Towards accurate and robust cross-ratio based gaze trackers through learning from simulation | |
| CN103677274A (en) | Interactive projection method and system based on active vision | |
| Cho et al. | Gaze Detection by Wearable Eye‐Tracking and NIR LED‐Based Head‐Tracking Device Based on SVR | |
| CN116382473A (en) | Sight calibration, motion tracking and precision testing method based on self-adaptive time sequence analysis prediction | |
| CN112987910B (en) | Testing method, device, equipment and storage medium of eyeball tracking equipment | |
| Arar et al. | Towards convenient calibration for cross-ratio based gaze estimation | |
| JP5719216B2 (en) | Gaze measurement apparatus and gaze measurement program | |
| JP2003079577A (en) | Visual axis measuring apparatus and method, visual axis measuring program, and recording medium recording the same | |
| CN110275608B (en) | Human eye sight tracking method | |
| CN114740966A (en) | Multi-modal image display control method and system and computer equipment | |
| CN112435347A (en) | E-book reading system and method for enhancing reality | |
| CN112528714A (en) | Single light source-based gaze point estimation method, system, processor and equipment | |
| CN109634431B (en) | Medium-free floating projection visual tracking interaction system | |
| RU2815470C1 (en) | Method for determining direction of view | |
| KR101348903B1 (en) | Cornea radius estimation apparatus using cornea radius estimation algorithm based on geometrical optics for eye tracking and method | |
| JP7423005B2 (en) | Calibration method and device for line of sight measurement using changes in pupil diameter, line of sight measurement device and camera device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |